audioheritage.org
robots.txt

Robots Exclusion Standard data for audioheritage.org

Resource Scan

Scan Details

Site Domain audioheritage.org
Base Domain audioheritage.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-01-20T06:34:20+00:00
Next Scan 2026-01-27T06:34:20+00:00

Last Successful Scan

Scanned2025-12-20T05:52:24+00:00
URL https://audioheritage.org/robots.txt
Domain IPs 104.21.88.61, 172.67.173.81, 2606:4700:3035::6815:583d, 2606:4700:3037::ac43:ad51
Response IP 172.67.173.81
Found Yes
Hash 12d5adf372424e45a7d45adb08f11c369c2d9329beb421de98f43e93c2411f86
SimHash 521c5850a402

Groups

gptbot

Rule Path
Disallow /

*

Rule Path
Disallow /issue.php*

*

Rule Path
Disallow /verify.php*

*

Rule Path
Disallow /r.php*

ahrefsbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

peer39_crawler
peer39_crawler/1.0

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://audioheritage.org/html/site-map/site-map.htm