1.andrew0668.com
robots.txt

Robots Exclusion Standard data for 1.andrew0668.com

Resource Scan

Scan Details

Site Domain 1.andrew0668.com
Base Domain andrew0668.com
Scan Status Ok
Last Scan2025-06-03T05:35:31+00:00
Next Scan 2025-06-10T05:35:31+00:00

Last Scan

Scanned2025-06-03T05:35:31+00:00
URL http://1.andrew0668.com/robots.txt
Redirect http://1.andrew0668.com:8080/robots.txt
Domain IPs 112.118.195.126
Response IP 112.118.195.126
Found Yes
Hash 22ab27db5ceb2ebfe1da2bd36dc4506f26f6e3a6636ac6bf986bba8d5cc78377
SimHash e84696527d9a

Groups

googlebot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

mediapartners-google

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

googlebot

Rule Path
Disallow /home/andrew/public_html/fcgi-bin/wish

googlebot

Rule Path
Disallow /home/andrew/public_html/fcgi-bin/wishcomi

mediapartners-google

Rule Path
Disallow /home/andrew/public_html/fcgi-bin/wish

mediapartners-google

Rule Path
Disallow /home/andrew/public_html/fcgi-bin/wishcomi

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 300

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 300

yahoobot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 300

yahooseeker

Rule Path
Disallow

Other Records

Field Value
crawl-delay 300

yahoo-mmcrawler

Rule Path
Disallow

Other Records

Field Value
crawl-delay 300

msnbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 300

baiduspider

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 300

yodaobot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 9900

rabot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

naverbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

pjspide

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

robocrawl

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

felix

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

kapito

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

nzexplorer

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

ia_archiver

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

scooter

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

jeeves askjeeve

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

aspseek

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

atomz

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

bbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

dittospyder

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

architextspider

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

infoseek

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

look.com

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

lycos

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

dotbot

Rule Path
Disallow /

*

Rule Path
Disallow /home/andrew/public_html/fcgi-bin/wishcomi/

Other Records

Field Value
crawl-delay 600

*

Rule Path
Disallow /home/andrew/public_html/fcgi-bin/wish/

Other Records

Field Value
crawl-delay 600

*

Rule Path
Disallow /

*

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600