business-hours.com.au
robots.txt

Robots Exclusion Standard data for business-hours.com.au

Resource Scan

Scan Details

Site Domain business-hours.com.au
Base Domain business-hours.com.au
Scan Status Ok
Last Scan2024-11-13T10:04:43+00:00
Next Scan 2024-11-20T10:04:43+00:00

Last Scan

Scanned2024-11-13T10:04:43+00:00
URL https://business-hours.com.au/robots.txt
Redirect https://www.business-hours.com.au/robots.txt
Redirect Domain www.business-hours.com.au
Redirect Base business-hours.com.au
Domain IPs 195.154.31.74
Redirect IPs 195.154.31.74
Response IP 195.154.31.74
Found Yes
Hash 991c507a576cdea169e5f81e9ce54b1706271d63e7d8dae2ba45030ee050fa7d
SimHash d85cc180d1f3

Groups

*

Rule Path
Disallow /vendor
Disallow /recherche/lieux?type=&ville=
Disallow /index.php/
Disallow /action/
Disallow /liste-ville*
Disallow /liste-type*
Disallow /liste/proximiter*

ahrefsbot
backlinkcrawler
bdbrandprotect
bpimagewalker
ezooms
findlinks
gigabot
httrack
httrack 3.0
ia_archiver
linkwalker
mj12bot/v1.4.3
mj12bot
net vampire
python-urllib
rogerbot
sogou web spider
sosospider
spbot
updownerbot
semalt.com
robothumb.com
trustpilot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.business-hours.com.au/xml/au.xml

Comments

  • Crawlers

Warnings

  • 1 invalid line.