www.uk.insight.com
robots.txt

Robots Exclusion Standard data for www.uk.insight.com

Resource Scan

Scan Details

Site Domain www.uk.insight.com
Base Domain insight.com
Scan Status Ok
Last Scan2025-12-30T14:23:20+00:00
Next Scan 2026-01-29T14:23:20+00:00

Last Scan

Scanned2025-12-30T14:23:20+00:00
URL https://www.uk.insight.com/robots.txt
Redirect https://uk.insight.com/robots.txt
Redirect Domain uk.insight.com
Redirect Base insight.com
Domain IPs 199.181.77.92
Redirect IPs 23.50.81.110
Response IP 104.69.158.211
Found Yes
Hash c9a53398b1436031d9a3dc6f6c86d771c4e72f6d591a456b03652bbba5bea216
SimHash 0f3899d3e553

Groups

*

Rule Path
Disallow /*?*
Allow /*.html
Disallow /en_US/search*.html
Disallow /insightweb/
Disallow /flytrap/
Disallow /content/dam/insight-web/*/solutions/service-provider/microsite/assets/
Disallow /content/dam/insight-web/*/pdfs/
Disallow /content/dam/insight/*/
Disallow /content/dam/global/*/pdfs/
Allow /insightweb/*.css$
Allow /*?qtype=
Allow /*?pq=
Allow /*?identifier=shopping
Allow /*?partnermessage

gptbot
chatgpt-user
googlebot
google-extended
anthropic-ai
bingbot
perplexitybot
youbot

Rule Path
Disallow

ccbot
facebookbot
neevaai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://uk.insight.com/sitemap.xml

Comments

  • Robots.txt for Insight.com
  • Default rules
  • Allowed AI crawlers
  • Blocked crawlers