Log In Sign Up

covid19.atmago.com
robots.txt

Robots Exclusion Standard data for covid19.atmago.com

Resource Scan

Scan Details

Site Domain covid19.atmago.com
Base Domain atmago.com
Scan Status Ok
Last Scan2024-11-16T14:49:02+00:00
Next Scan 2024-11-30T14:49:02+00:00

Last Scan

Scanned2024-11-16T14:49:02+00:00
URL https://covid19.atmago.com/robots.txt
Domain IPs 76.76.21.164, 76.76.21.98
Response IP 76.76.21.164
Found Yes
Hash e43b923f8470fe20d799705a7dfccfac77793c828e93044a06615497694ed1e3
SimHash 084cc9d27771

Groups

*

Rule Path
Allow /
Disallow /pr/*

mj12bot
ahrefsbot
semrushbot
mauibot
dotbot

Rule Path
Disallow /
Back to top

Comments

  • start disallowed routes
  • pr site belum
  • end disallowed routes
  • Sitemaps index
  • not yet
  • Block bad bots
  • https://studiofreya.com/2018/11/15/how-to-stop-bots-from-crawling-your-website
  • https://www.simtechdev.com/blog/good-and-bad-bots-to-control-to-save-server-resources-and-improve-performance/
Back to top