amray.com
robots.txt

Robots Exclusion Standard data for amray.com

Resource Scan

Scan Details

Site Domain amray.com
Base Domain amray.com
Scan Status Ok
Last Scan2024-06-06T03:37:20+00:00
Next Scan 2024-07-06T03:37:20+00:00

Last Scan

Scanned2024-06-06T03:37:20+00:00
URL http://amray.com/robots.txt
Domain IPs 2607:f790:fff8:700::226, 74.126.5.226
Response IP 74.126.5.226
Found Yes
Hash 446e1ffe00e3ed8ed73fe641e5a1326d20137032ce3e91488b4bb7db07a274ae
SimHash 814403c11b17

Groups

*

Rule Path
Disallow /searches
Disallow /dlist
Disallow /whois

googlebot

Rule Path
Disallow /snaps
Disallow /free-web-stuff
Disallow /searches
Allow /

yahoo slurp

Rule Path
Disallow /free-web-stuff
Disallow /snaps
Disallow /searches
Allow /

yahoo-mmcrawler

Rule Path
Disallow /snaps
Disallow /searches
Allow /

yandex bot

Rule Path
Disallow /snaps
Disallow /searches
Disallow /

robot

Rule Path
Disallow /

bot

Rule Path
Disallow /

spider

Rule Path
Disallow /

java

Rule Path
Disallow /

nutch

Rule Path
Disallow /

voyager

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /