Log In Sign Up

drivethrurpg.com
robots.txt

Robots Exclusion Standard data for drivethrurpg.com

Resource Scan

Scan Details

Site Domain drivethrurpg.com
Base Domain drivethrurpg.com
Scan Status Ok
Last Scan4/12/2025, 8:40:57 AM
Next Scan 5/12/2025, 8:40:57 AM

Last Scan

Scanned4/12/2025, 8:40:57 AM
URL https://drivethrurpg.com/robots.txt
Redirect https://www.drivethrurpg.com/robots.txt
Redirect Domain www.drivethrurpg.com
Redirect Base drivethrurpg.com
Domain IPs 18.216.116.235
Redirect IPs 3.132.125.192, 3.141.61.194, 3.21.182.152
Response IP 3.132.125.192
Found Yes
Hash 6af9730263fa1a7ad456e9ad6e09c64babda7669c3643fdd050c6c1b5f677a6c
SimHash aa78954feef6

Groups

*

Rule Path
Allow /
Back to top

Comments

  • Our production robots.txt file; this robots.txt is extremely permissive because the damage that
  • can be done to the site by excluding important bots that drive our SEO rankings far outweights
  • the impact of any given misbehaving bots we are currently aware of. Future exclusions may be
  • added as needed.
  • It is for sites that have taken over a top level domain.
Back to top