icslearn.co.uk
robots.txt

Robots Exclusion Standard data for icslearn.co.uk

Resource Scan

Scan Details

Site Domain icslearn.co.uk
Base Domain icslearn.co.uk
Scan Status Ok
Last Scan2025-08-17T18:53:06+00:00
Next Scan 2025-09-16T18:53:06+00:00

Last Scan

Scanned2025-08-17T18:53:06+00:00
URL https://icslearn.co.uk/robots.txt
Redirect https://www.icslearn.co.uk/robots.txt
Redirect Domain www.icslearn.co.uk
Redirect Base icslearn.co.uk
Domain IPs 104.26.2.90, 104.26.3.90, 172.67.68.14, 2606:4700:20::681a:25a, 2606:4700:20::681a:35a, 2606:4700:20::ac43:440e
Redirect IPs 104.26.2.90, 104.26.3.90, 172.67.68.14, 2606:4700:20::681a:25a, 2606:4700:20::681a:35a, 2606:4700:20::ac43:440e
Response IP 104.26.2.90
Found Yes
Hash 77e651ae0c73d86bfa6e3d0420c950ea04a9af4bf14603c86f7d86fac219f0f6
SimHash f10a60619604

Groups

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /
Allow /DependencyHandler.axd?
Disallow /aspnet_client/
Disallow /bin/
Disallow /config/
Disallow /data/
Disallow /install/
Disallow /macroScripts/
Disallow /masterpages/
Disallow /umbraco/
Disallow /backoffice/
Disallow /umbraco_client/
Disallow /usercontrols/
Disallow /xslt/
Disallow /contactform/
Disallow /Search/
Disallow /404/
Disallow /checkout/
Disallow /orderconfirmation/
Disallow /site-general-settings/
Disallow /landing-pages/*
Disallow /lp/*
Disallow /thank-you/
Disallow /thank-you-competition/
Disallow /thank-you-for-your-enquiry/

Other Records

Field Value
sitemap https://www.icslearn.co.uk/sitemap_index.xml