heroforge.com
robots.txt

Robots Exclusion Standard data for heroforge.com

Resource Scan

Scan Details

Site Domain heroforge.com
Base Domain heroforge.com
Scan Status Ok
Last Scan2025-09-13T10:01:16+00:00
Next Scan 2025-10-13T10:01:16+00:00

Last Scan

Scanned2025-09-13T10:01:16+00:00
URL https://heroforge.com/robots.txt
Redirect https://www.heroforge.com/static/robots.txt
Redirect Domain www.heroforge.com
Redirect Base heroforge.com
Domain IPs 104.18.30.195, 104.18.31.195, 2606:4700::6812:1ec3, 2606:4700::6812:1fc3
Redirect IPs 104.18.30.195, 104.18.31.195, 2606:4700::6812:1ec3, 2606:4700::6812:1fc3
Response IP 104.18.31.195
Found Yes
Hash 9e20201eba80c8cedf90751778ce444c4fd689fa340c0bbf0fefdaba2c7a22c7
SimHash 40d88bd0a0dc

Groups

googlebot

Rule Path
Disallow /static
Allow /static/img/feed/
Allow /static/img/logo/

googlebot-image

Rule Path
Disallow /static
Allow /static/img/feed/
Allow /static/img/logo/

meta-externalagent

Rule Path
Disallow /static
Allow /static/img/feed/
Allow /static/img/logo/

bingbot

Rule Path
Disallow /static
Allow /static/img/feed/
Allow /static/img/logo/

bingpreview

Rule Path
Disallow /static
Allow /static/img/feed/
Allow /static/img/logo/

*

Rule Path
Disallow /static
Allow /static/img/logo/

gptbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

Comments

  • robotstxt.org/
  • Explicitly block AI scrapers