Log In Sign Up

mcjoe21.com
robots.txt

Robots Exclusion Standard data for mcjoe21.com

Resource Scan

Scan Details

Site Domain mcjoe21.com
Base Domain mcjoe21.com
Scan Status Ok
Last Scan2025-06-10T20:53:27+00:00
Next Scan 2025-06-24T20:53:27+00:00

Last Scan

Scanned2025-06-10T20:53:27+00:00
URL https://mcjoe21.com/robots.txt
Domain IPs 2001:8d8:100f:f000::286, 217.160.0.225
Response IP 217.160.0.225
Found Yes
Hash 1256225f31546d0680dc81a92bf4b27a50859e99cb2f7be6cd3644b73d3624ef
SimHash 082cdcc48bda

Groups

*
ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

ccbot

Rule Path
Disallow /
Disallow */cache/ionos-performance/
Back to top

Other Records

Field Value
sitemap https://mcjoe21.com/sitemap.xml
Back to top

Comments

  • Block archive.org bots
  • Block Common Crawl (CCBot)
  • More info: https://commoncrawl.org/big-picture/frequently-asked-questions/
Back to top