Log In Sign Up

agile1.com
robots.txt

Robots Exclusion Standard data for agile1.com

Resource Scan

Scan Details

Site Domain agile1.com
Base Domain agile1.com
Scan Status Ok
Last Scan2025-05-16T09:56:42+00:00
Next Scan 2025-06-15T09:56:42+00:00

Last Scan

Scanned2025-05-16T09:56:42+00:00
URL https://agile1.com/robots.txt
Domain IPs 44.229.107.247
Response IP 44.229.107.247
Found Yes
Hash 9463662e7d06a133262cf86b4fbb87ec6257d47117877a8c5eb121eaf331bfd4
SimHash d937e800c0d2

Groups

gptbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /
Back to top

Comments

  • Block gptbot from the entire site
  • Block bingbot from a specific subdirectory
  • Block all other bots from a different subdirectory
  • User-agent: *
  • Disallow: /
Back to top