pythonicway.com
robots.txt

Robots Exclusion Standard data for pythonicway.com

Resource Scan

Scan Details

Site Domain pythonicway.com
Base Domain pythonicway.com
Scan Status Ok
Last Scan2025-08-11T00:58:43+00:00
Next Scan 2025-09-10T00:58:43+00:00

Last Scan

Scanned2025-08-11T00:58:43+00:00
URL https://pythonicway.com/robots.txt
Domain IPs 104.21.8.188, 172.67.157.208, 2606:4700:3033::ac43:9dd0, 2606:4700:3037::6815:8bc
Response IP 104.21.8.188
Found Yes
Hash 3a3b9f3c67b722526dfc8c23762f54e33cf5c2eef1be623aafdd6d5c231ca0b3
SimHash 884e5ff7edc1

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /private/
Disallow /temp/
Allow /static/
Allow /css/
Allow /js/
Allow /*.css
Allow /*.js
Allow /*.svg
Allow /*.png
Allow /*.jpg
Allow /*.jpeg
Allow /*.webp
Allow /*.ico

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://pythonicway.com/sitemap.xml

Comments

  • Sitemap
  • Crawl-delay for respectful crawling
  • Block access to admin areas (if any in the future)
  • Allow all static assets