saintsreport.com
robots.txt

Robots Exclusion Standard data for saintsreport.com

Resource Scan

Scan Details

Site Domain saintsreport.com
Base Domain saintsreport.com
Scan Status Ok
Last Scan2024-05-17T13:18:42+00:00
Next Scan 2024-05-24T13:18:42+00:00

Last Scan

Scanned2024-05-17T13:18:42+00:00
URL https://saintsreport.com/robots.txt
Domain IPs 170.249.221.141
Response IP 170.249.221.141
Found Yes
Hash 639190bf3f7af412d9db7e9b2df94ac33dd4d1c68dcb4f0a7f06b447daf26cb3
SimHash 214a54038595

Groups

googlebot

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

msnbot

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

bingbot

Rule Path
Disallow

slurp

Rule Path
Disallow

yahoo! slurp

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

yahoo-blogs/v3.9

Rule Path
Disallow

ia_archiver

Rule Path
Disallow

gigabot

Rule Path
Disallow /

robozilla

Rule Path
Disallow /

nutch

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

psbot

Rule Path
Disallow /

clickagy

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

sogou

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

yandex

Rule Path
Disallow /

*

Rule Path
Disallow
Disallow /cgi-bin/
Disallow /conversations/
Disallow /forums/infraction-discussion-board.28/
Disallow /forums/admins-and-mods-board.18/
Disallow /admin.php/
Disallow /login/
Disallow /andrus/

Other Records

Field Value
crawl-delay 20

Other Records

Field Value
sitemap https://saintsreport.com/sitemap.xml

Comments

  • robots.txt generated by atozseotools.com