cdn.experts-exchange.com
robots.txt

Robots Exclusion Standard data for cdn.experts-exchange.com

Resource Scan

Scan Details

Site Domain cdn.experts-exchange.com
Base Domain experts-exchange.com
Scan Status Ok
Last Scan2025-05-14T12:12:53+00:00
Next Scan 2025-06-13T12:12:53+00:00

Last Scan

Scanned2025-05-14T12:12:53+00:00
URL https://cdn.experts-exchange.com/robots.txt
Redirect https://www.experts-exchange.com/robots.jsp
Redirect Domain www.experts-exchange.com
Redirect Base experts-exchange.com
Domain IPs 104.22.4.165, 104.22.5.165, 172.67.36.241, 2606:4700:10::6816:4a5, 2606:4700:10::6816:5a5, 2606:4700:10::ac43:24f1
Redirect IPs 104.22.4.165, 104.22.5.165, 172.67.36.241, 2606:4700:10::6816:4a5, 2606:4700:10::6816:5a5, 2606:4700:10::ac43:24f1
Response IP 104.22.4.165
Found Yes
Hash 3f5199804055673cbbf9b5ac587fec9988eb623340eb31af44df5a6f8d9bdb50
SimHash f21c967a8eb0

Groups

*

Rule Path
Disallow /login.jsp
Disallow /checkout.jsp
Disallow /signup.jsp
Disallow /forgotPassword.jsp
Disallow /searchResults.jsp
Disallow /advancedSearch.jsp
Disallow /search.jsp
Disallow /cookiePage.jsp
Disallow /viewCodeSnippet.jsp
Disallow /getCaptcha/
Disallow /cdn-cgi/
Disallow /async/
Disallow /async/nb.jsp
Disallow /*.async$
Disallow /*.php$
Disallow /*.asp$
Disallow /*.aspx$
Disallow /*.cgi$
Disallow /questions/_next/
Allow /questions/_next/static/css/*

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

moatbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

omgili

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.experts-exchange.com/siteMapXML.jsp