cihai123.com
robots.txt

Robots Exclusion Standard data for cihai123.com

Resource Scan

Scan Details

Site Domain cihai123.com
Base Domain cihai123.com
Scan Status Ok
Last Scan2024-06-21T18:01:26+00:00
Next Scan 2024-07-21T18:01:26+00:00

Last Scan

Scanned2024-06-21T18:01:26+00:00
URL http://cihai123.com/robots.txt
Redirect http://www.cihai123.com/robots.txt
Redirect Domain www.cihai123.com
Redirect Base cihai123.com
Domain IPs 8.218.54.52
Redirect IPs 8.218.54.52
Response IP 8.218.54.52
Found Yes
Hash 68cdff85e697c62d86b81a0f379264b1338c2f7f6c4b31ff1bcf0b45e2aac6bf
SimHash 023ed7223b53

Groups

googlebot

Rule Path
Disallow /

googlebot-mobile

Rule Path
Disallow /

yahoo! slurp

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

custo

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

jakarta commons-httpclient

Rule Path
Disallow /

yandex bot

Rule Path
Disallow /

suspected bot masqurading as mozilla

Rule Path
Disallow /

a php script

Rule Path
Disallow /

java (often spam bot)

Rule Path
Disallow /

phantom

Rule Path
Disallow /

nutch

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

web core

Rule Path
Disallow /

bubing

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /