capedge.com
robots.txt

Robots Exclusion Standard data for capedge.com

Resource Scan

Scan Details

Site Domain capedge.com
Base Domain capedge.com
Scan Status Ok
Last Scan2024-10-28T14:55:31+00:00
Next Scan 2024-11-04T14:55:31+00:00

Last Scan

Scanned2024-10-28T14:55:31+00:00
URL https://capedge.com/robots.txt
Redirect https://capedge.com/robots.txt/
Domain IPs 13.35.185.83, 13.35.185.84, 13.35.185.89, 13.35.185.92, 2600:9000:2085:1600:3:aab1:f0c0:93a1, 2600:9000:2085:2e00:3:aab1:f0c0:93a1, 2600:9000:2085:400:3:aab1:f0c0:93a1, 2600:9000:2085:6200:3:aab1:f0c0:93a1, 2600:9000:2085:6c00:3:aab1:f0c0:93a1, 2600:9000:2085:a200:3:aab1:f0c0:93a1, 2600:9000:2085:cc00:3:aab1:f0c0:93a1, 2600:9000:2085:fc00:3:aab1:f0c0:93a1
Response IP 13.35.238.22
Found Yes
Hash 07c08987fce82ced13c09a63f0095116f2a793d7fc85456a84913ad859066b49
SimHash 181c9453d590

Groups

*

Rule Path
Disallow /institutional/nport
Disallow /institutional/nport/*

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

hubspot crawler 1.0

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

Other Records

Field Value
sitemap https://capedge.com/sitemap-index.xml