devbrains.tn
robots.txt

Robots Exclusion Standard data for devbrains.tn

Resource Scan

Scan Details

Site Domain devbrains.tn
Base Domain devbrains.tn
Scan Status Ok
Last Scan2025-10-04T11:13:01+00:00
Next Scan 2025-10-11T11:13:01+00:00

Last Scan

Scanned2025-10-04T11:13:01+00:00
URL https://devbrains.tn/robots.txt
Domain IPs 104.21.93.147, 172.67.211.32, 2606:4700:3031::ac43:d320, 2606:4700:3036::6815:5d93
Response IP 172.67.211.32
Found Yes
Hash 877dda3f0b4d8f4910fa5b7bf36ef8a0ee05cfd730c5474c9f0be45421c99e02
SimHash 49009a358453

Groups

*

Rule Path
Disallow /login
Disallow /register
Disallow /profile/edit
Disallow /terms-of-service
Disallow /privacy
Disallow /forum/ask

ia_archiver

Rule Path
Allow /privacy
Allow /terms-of-service
Disallow /

Other Records

Field Value
sitemap https://devbrains.tn/sitemap.xml

Comments

  • Sitemap
  • exclude login pages
  • Block Wayback Machine