Log In Sign Up

gator.io
robots.txt

Robots Exclusion Standard data for gator.io

Resource Scan

Scan Details

Site Domain gator.io
Base Domain gator.io
Scan Status Ok
Last Scan2025-10-19T14:35:49+00:00
Next Scan 2025-11-18T14:35:49+00:00

Last Scan

Scanned2025-10-19T14:35:49+00:00
URL https://gator.io/robots.txt
Domain IPs 44.227.170.53, 54.149.226.195
Response IP 54.149.226.195
Found Yes
Hash a3f4ddfe075345a6745dd689b27cf7ff9cc5ab43326780028a5b02f3aeb3739a
SimHash c21bb080440a

Groups

rogerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

yandexrenderresourcesbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Allow /images

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

wpmu dev broken link checker spider

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

semanticscholarbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

*

Rule Path
Disallow /*.aspx
Disallow /reset
Disallow /login
Disallow /register
Disallow /signup

Other Records

Field Value
crawl-delay 10
Back to top