tvdiary.net
robots.txt

Robots Exclusion Standard data for tvdiary.net

Resource Scan

Scan Details

Site Domain tvdiary.net
Base Domain tvdiary.net
Scan Status Ok
Last Scan2024-09-25T02:25:43+00:00
Next Scan 2024-10-02T02:25:43+00:00

Last Scan

Scanned2024-09-25T02:25:43+00:00
URL https://tvdiary.net/robots.txt
Redirect https://www.tvdiary.net/robots.txt
Redirect Domain www.tvdiary.net
Redirect Base tvdiary.net
Domain IPs 209.159.146.219
Redirect IPs 209.159.146.219
Response IP 209.159.146.219
Found Yes
Hash 0dd9195208e2315280346a51b6250613b7cae882bc2f32b8a75ffb8deb68b292
SimHash 2840db8356b5

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

slurp

Rule Path
Allow /

msnbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /

yahoo-blogs

Rule Path
Allow /

yahoo-mmcrawler

Rule Path
Allow /

teoma

Rule Path
Allow /

Comments

  • disallow all
  • but allow only important bots