crma.org
robots.txt

Robots Exclusion Standard data for crma.org

Resource Scan

Scan Details

Site Domain crma.org
Base Domain crma.org
Scan Status Ok
Last Scan2026-01-20T07:21:07+00:00
Next Scan 2026-02-19T07:21:07+00:00

Last Scan

Scanned2026-01-20T07:21:07+00:00
URL https://crma.org/robots.txt
Redirect https://www.crma.org/robots.txt
Redirect Domain www.crma.org
Redirect Base crma.org
Domain IPs 3.130.152.106, 3.143.123.39, 3.143.49.207
Redirect IPs 3.130.152.106, 3.143.123.39, 3.143.49.207
Response IP 3.143.49.207
Found Yes
Hash d9abf067a32ef57f5ba76d412b14fb9915a79ac16a829f5565211dd6ad2e8e1b
SimHash 6e0cd4d88097

Groups

*

Rule Path
Disallow /application/attributes
Disallow /application/authentication
Disallow /application/bootstrap
Disallow /application/config
Disallow /application/controllers
Disallow /application/elements
Disallow /application/helpers
Disallow /application/jobs
Disallow /application/languages
Disallow /application/mail
Disallow /application/models
Disallow /application/page_types
Disallow /application/single_pages
Disallow /application/tools
Disallow /application/views
Disallow /Content/*
Disallow *.aspx
Disallow *.axd

python-requests/2.28.1

Rule Path
Disallow /

fidget-spinner-bot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.crma.org/sitemap.xml