Log In Sign Up

humblebundle-a.akamaihd.net
robots.txt

Robots Exclusion Standard data for humblebundle-a.akamaihd.net

Resource Scan

Scan Details

Site Domain humblebundle-a.akamaihd.net
Base Domain akamaihd.net
Scan Status Ok
Last Scan2025-02-17T04:03:29+00:00
Next Scan 2025-03-19T04:03:29+00:00

Last Scan

Scanned2025-02-17T04:03:29+00:00
URL https://humblebundle-a.akamaihd.net/robots.txt
Redirect https://cdn.humblebundle.com/robots.txt
Redirect Domain cdn.humblebundle.com
Redirect Base humblebundle.com
Domain IPs 23.45.207.202, 23.45.207.205
Redirect IPs 104.18.39.232, 172.64.148.24
Response IP 104.18.39.232
Found Yes
Hash bde1d824cab25f67cbbe28958da54f64f6b5aad5acffaae599e55f8f6b58eefe
SimHash 701c59c0e113

Groups

*

Rule Path
Disallow /?key*
Disallow /?s=thanks
Disallow /emailhelper
Disallow /delete-key
Disallow /download-lister
Disallow /store/product/*
Disallow /user/associate
Disallow /user/signup-complete
Disallow /widget/v2/*
Disallow /return-paypal
Disallow /return-billing-agreement-paypal
Disallow /return/
Disallow /*_escaped_fragment_%3Dsystem-requirements

amazonbot
applebot
applebot-extended
bytespider
ccbot
chatgpt-user
claude-web
claudebot
diffbot
facebookbot
gptbot
google-extended
httrack
nutch
oai-searchbot
offline explorer
perplexitybot
scrapy
youbot
anthropic-ai
cohere-ai
omgili

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.humblebundle.com/sitemap.xml
Back to top

Comments

  • Blocking known AI content scrapers
Back to top

Warnings

  • 1 invalid line.
Back to top