cipd.co.uk
robots.txt

Robots Exclusion Standard data for cipd.co.uk

Resource Scan

Scan Details

Site Domain cipd.co.uk
Base Domain cipd.co.uk
Scan Status Ok
Last Scan2025-09-04T09:40:35+00:00
Next Scan 2025-10-04T09:40:35+00:00

Last Scan

Scanned2025-09-04T09:40:35+00:00
URL https://cipd.co.uk/robots.txt
Redirect https://www.cipd.co.uk/robots.txt
Redirect Domain www.cipd.co.uk
Redirect Base cipd.co.uk
Domain IPs 217.114.94.2
Redirect IPs 104.18.32.221, 172.64.155.35, 2606:4700:440d::ac40:9b23, 2a06:98c1:3101::6812:20dd
Response IP 172.64.155.35
Found Yes
Hash 039198d290d0a63d827b3b434cf325f9f4127a685c1b87b7abba396b68bc41c6
SimHash 711459108d93

Groups

*

Rule Path
Allow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.cipd.org/sitemap.xml