Log In Sign Up

py.md
robots.txt

Robots Exclusion Standard data for py.md

Resource Scan

Scan Details

Site Domain py.md
Base Domain py.md
Scan Status Ok
Last Scan2026-03-10T21:25:14+00:00
Next Scan 2026-03-17T21:25:14+00:00

Last Scan

Scanned2026-03-10T21:25:14+00:00
URL https://py.md/robots.txt
Domain IPs 104.21.23.84, 172.67.209.217, 2606:4700:3037::6815:1754, 2606:4700:3037::ac43:d1d9
Response IP 172.67.209.217
Found Yes
Hash 63f680082b9d24a5341486542baf4272e857bda7ac5026867ccbe9de72938759
SimHash ec485a10be01

Groups

*

Rule Path
Allow /
Disallow /admin
Disallow /settings
Disallow /api/
Disallow /login
Disallow /logout
Disallow /create-admin
Disallow /reset-password
Disallow /verify
Disallow /verify-email
Disallow /stats
Disallow /banned
Disallow /404
Disallow /confirm-*
Disallow /add-*
Disallow /link/edit/
Disallow /admin/link/edit/
Disallow /create-user
Disallow /get-report-email
Disallow /get-support-email
Disallow /*?*
Disallow /*/stats
Disallow /*/protected
Disallow /*%2B
Allow /
Allow /about
Allow /sponsors
Allow /privacy
Allow /terms
Allow /report
Allow /docs
Allow /blog
Allow /status
Allow /expired
Back to top

Other Records

Field Value
sitemap /sitemap.xml
Back to top

Comments

  • Block admin and private areas
  • Block HTMX partials and admin functions
  • Block dynamic/user generated content that shouldn't be indexed
  • Allow important public pages
  • Sitemap location
Back to top