disneyinstitute.com
robots.txt

Robots Exclusion Standard data for disneyinstitute.com

Resource Scan

Scan Details

Site Domain disneyinstitute.com
Base Domain disneyinstitute.com
Scan Status Ok
Last Scan2024-08-30T00:01:16+00:00
Next Scan 2024-09-29T00:01:16+00:00

Last Scan

Scanned2024-08-30T00:01:16+00:00
URL https://disneyinstitute.com/robots.txt
Redirect https://www.disneyinstitute.com/robots.txt
Redirect Domain www.disneyinstitute.com
Redirect Base disneyinstitute.com
Domain IPs 52.84.229.119, 52.84.229.53, 52.84.229.56, 52.84.229.95
Redirect IPs 118.215.83.222
Response IP 118.215.83.222
Found Yes
Hash 312bb68c4feb4d7b8ef9af82ea9ed15b84cb7cc0ea6334e8e15ace93dde58d4e
SimHash c01cdc50c511

Groups

*

Rule Path
Allow /
Disallow /admin/
Disallow /di-admin/
Disallow /certain-confirmation
Disallow /enterprise
Disallow /account/
Disallow /app_code/
Disallow /aspnet_client/
Disallow /AgeGate/
Disallow /Bin/
Disallow /Master/
Disallow /Search/
Disallow /Survey/
Disallow /swf/
Disallow /UserControls/
Disallow /*?CMP
Disallow /*?s_kwcid
Disallow /*?keyword_id
Disallow /*?f
Disallow /*?t
Disallow /*?ReturnURL
Disallow /*?aspxerrorpath
Disallow /DI-hero_Resources/*
Disallow /flvVideo/*
Disallow /videos/*
Disallow /_disid/
Disallow /_html/
Disallow /dashboard/
Disallow /documents/

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.disneyinstitute.com/sitemap.xml

Comments

  • GRP-48421 Blocking AI Crawlers