restthecase.com
robots.txt

Robots Exclusion Standard data for restthecase.com

Resource Scan

Scan Details

Site Domain restthecase.com
Base Domain restthecase.com
Scan Status Ok
Last Scan2025-09-30T12:12:36+00:00
Next Scan 2025-10-07T12:12:36+00:00

Last Scan

Scanned2025-09-30T12:12:36+00:00
URL https://restthecase.com/robots.txt
Domain IPs 104.21.69.230, 172.67.215.109, 2606:4700:3030::ac43:d76d, 2606:4700:3036::6815:45e6
Response IP 104.21.69.230
Found Yes
Hash 7a13fc8de6623b7960acfa329e1d09b29fdcdeee40d4306ab0ec37d8f25d7d78
SimHash 61ac6ad1d7f1

Groups

*

Rule Path
Disallow /*?page=

googlebot
bingbot

Rule Path
Allow /
Allow /

googlebot-news

Rule Path
Allow /

oai-searchbot
chatgpt-user
perplexitybot
firecrawlagent
andibot
exabot
phindbot
youbot

Rule Path
Allow /

Other Records

Field Value
sitemap https://restthecase.com/sitemap-blog.xml
sitemap https://restthecase.com/sitemap-news.xml
sitemap https://restthecase.com/sitemap-lawyer.xml

Comments

  • Default: Allow all crawlers
  • Major search engines – allow
  • Explicitly allow everything else
  • Google News Bot
  • Sitemap declarations
  • Friendly AI bots – allow