demstate.net
robots.txt

Robots Exclusion Standard data for demstate.net

Resource Scan

Scan Details

Site Domain demstate.net
Base Domain demstate.net
Scan Status Ok
Last Scan2026-04-02T09:23:58+00:00
Next Scan 2026-04-09T09:23:58+00:00

Last Scan

Scanned2026-04-02T09:23:58+00:00
URL https://demstate.net/robots.txt
Redirect https://demstate.com/robots.txt
Redirect Domain demstate.com
Redirect Base demstate.com
Domain IPs 104.21.82.58, 172.67.153.207, 2606:4700:3033::6815:523a, 2606:4700:3033::ac43:99cf
Redirect IPs 104.21.49.38, 172.67.141.71, 2606:4700:3034::6815:3126, 2606:4700:3037::ac43:8d47
Response IP 172.67.141.71
Found Yes
Hash f734d2bb2d586dbea40d8b2ac308f51b326ff4c7bd2002757a4f21d90cb42ddf
SimHash 20015ac59491

Groups

*

Rule Path
Allow /

googlebot

Rule Path
Allow /
Disallow /admin/
Disallow /auth/

Other Records

Field Value
sitemap https://demstate.com/sitemap-v2.xml
sitemap https://demstate.com/news-sitemap-v2.xml
sitemap https://demstate.com/video-sitemap-v2.xml

Comments

  • Important crawl directives for Google
  • Block admin and API routes from crawling