Log In Sign Up

ag.agile1.com
robots.txt

Robots Exclusion Standard data for ag.agile1.com

Resource Scan

Scan Details

Site Domain ag.agile1.com
Base Domain agile1.com
Scan Status Ok
Last Scan2025-11-28T08:12:26+00:00
Next Scan 2025-12-28T08:12:26+00:00

Last Scan

Scanned2025-11-28T08:12:26+00:00
URL https://ag.agile1.com/robots.txt
Domain IPs 34.212.250.221, 44.253.215.100
Response IP 34.212.250.221
Found Yes
Hash 835957a0a26e215c0b5c323e485c7a43970aecace2f9393aa07bc1cbe28bf1de
SimHash d167cc10c013

Groups

gptbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

gecko

Rule Path
Disallow /

microsoftpreview

Rule Path
Disallow /

microsoft-webdav-miniredir

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

skypeuripreview

Rule Path
Disallow /

skype

Rule Path
Disallow /
Back to top

Comments

  • Block gptbot from the entire site
  • Block all other bots from a different subdirectory
  • User-agent: *
  • Disallow: /
Back to top