universalcompendium.com
robots.txt

Robots Exclusion Standard data for universalcompendium.com

Resource Scan

Scan Details

Site Domain universalcompendium.com
Base Domain universalcompendium.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-10T09:02:42+00:00
Next Scan 2025-11-08T09:02:42+00:00

Last Successful Scan

Scanned2025-04-13T08:53:55+00:00
URL https://universalcompendium.com/robots.txt
Domain IPs 104.225.208.26
Response IP 104.225.208.26
Found Yes
Hash 5c109305b50d0eda127280d92e5d897757ec66de4b6e659b8de3adfd5dfdeb32
SimHash 08048072cfb2

Groups

*

Rule Path
Allow /

mozilla/5.0 (compatible; amznkassocbot/4.0)

Rule Path
Allow /

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /Sitemap%3Ahttp%3A//universalcompendium.com/sitemap.xml