auronia.co.uk
robots.txt

Robots Exclusion Standard data for auronia.co.uk

Resource Scan

Scan Details

Site Domain auronia.co.uk
Base Domain auronia.co.uk
Scan Status Ok
Last Scan2024-11-09T20:21:31+00:00
Next Scan 2024-12-09T20:21:31+00:00

Last Scan

Scanned2024-11-09T20:21:31+00:00
URL https://auronia.co.uk/robots.txt
Redirect https://www.auronia.co.uk/robots.txt
Redirect Domain www.auronia.co.uk
Redirect Base auronia.co.uk
Domain IPs 104.26.6.72, 104.26.7.72, 172.67.69.139, 2606:4700:20::681a:648, 2606:4700:20::681a:748, 2606:4700:20::ac43:458b
Redirect IPs 104.26.6.72, 104.26.7.72, 172.67.69.139, 2606:4700:20::681a:648, 2606:4700:20::681a:748, 2606:4700:20::ac43:458b
Response IP 172.67.69.139
Found Yes
Hash 12885fe1badde6a0eca001b8c3f9554b37c78ad7e7cdbf1abc6f7b7f24b60a01
SimHash 2b30337b4efa

Groups

googlebot

Rule Path
Allow /*?utm_source=google&utm_medium=surfaces&utm_campaign=feed_clicks&utm_content=surfaces_across_google

*

Rule Path
Disallow /*?
Disallow /*%26
Allow /*?p=
Disallow /*?p=*&
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /pkginfo/
Disallow /report/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /vendor/

googlebot

Rule Path
Disallow /media/catalog/attribute/images/*
Disallow /media//amasty/shopby/option_images/*
Disallow /media/amasty/shopby/option_images/*
Allow /customer/section/load/
Disallow /index.php/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalogsearch/
Disallow /checkout/
Disallow /control/
Disallow /contacts/
Disallow /customer/
Disallow /customize/
Disallow /newsletter/
Disallow /review/
Disallow /sendfriend/
Disallow /wishlist/
Disallow /composer.json
Disallow /composer.lock
Disallow /CONTRIBUTING.md
Disallow /CONTRIBUTOR_LICENSE_AGREEMENT.html
Disallow /COPYING.txt
Disallow /Gruntfile.js
Disallow /LICENSE.txt
Disallow /LICENSE_AFL.txt
Disallow /package.json
Disallow /RELEASE_NOTES.txt
Disallow /*?*product_list_mode=
Disallow /*?*product_list_order=
Disallow /*?*product_list_limit=
Disallow /*?*product_list_dir=
Disallow /*?SID=
Disallow /*.php$
Disallow /*.svn$
Disallow /*.git$
Disallow /*.idea$
Disallow /*.sql$
Disallow /*.zip$
Disallow /*.gz$
Disallow /*.tgz$
Disallow /*.sample$
Disallow /configurator3d/

seokicks

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

majesticseo

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

xovi

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

onpagebot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

search17

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

slysearch

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

pixray-seeker

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

lb-spider

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

htdig

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

edisterbot

Rule Path
Disallow /

swebot

Rule Path
Disallow /

picmole

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yeti-mobile

Rule Path
Disallow /

pagepeeker

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

netestatenecrawler

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

comodosslchecker

Rule Path
Disallow /

comodo-certificates-spider

Rule Path
Disallow /

gonzo

Rule Path
Disallow /

schrein

Rule Path
Disallow /

afiliaswebminingtool

Rule Path
Disallow /

suggybot

Rule Path
Disallow /

bdbrandprotect

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

updownerbot

Rule Path
Disallow /

lex

Rule Path
Disallow /

contentcrawler

Rule Path
Disallow /

dcpbot

Rule Path
Disallow /

kaloogabot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

icjobs

Rule Path
Disallow /

obot

Rule Path
Disallow /

webmastercoffee

Rule Path
Disallow /

qualidator

Rule Path
Disallow /

webinator

Rule Path
Disallow /

scooter

Rule Path
Disallow /

thunderstone

Rule Path
Disallow /

larbin

Rule Path
Disallow /

opidoobot

Rule Path
Disallow /

ips-agent

Rule Path
Disallow /

tineye

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

unister

Rule Path
Disallow /

reverseget

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

Comments

  • ALL LIVE - robots folder
  • Noindex Filters not for google
  • Directories
  • Images
  • Paths (clean URLs)
  • Files
  • Do not index pages that are sorted or filtered.
  • Do not index session ID
  • GIT, SVN directory and dump files
  • Configurator3d
  • SEO Spiders

Warnings

  • 2 invalid lines.
  • `noindex` is not a known field.