csm.dev
robots.txt
Robots Exclusion Standard data for csm.dev
Resource Scan
Scan Details
| Site Domain | csm.dev |
| Base Domain | csm.dev |
| Scan Status | Ok |
| Last Scan | 2025-04-16T21:42:57+00:00 |
| Next Scan | 2025-04-23T21:42:57+00:00 |
Last Scan
| Scanned | 2025-04-16T21:42:57+00:00 |
| URL | https://csm.dev/robots.txt |
| Domain IPs | 104.21.71.232, 172.67.172.19, 2606:4700:3035::ac43:ac13, 2606:4700:3037::6815:47e8 |
| Response IP | 172.67.172.19 |
| Found | Yes |
| Hash | 8a76a569608cb48f7a42804a15b9fb8aae9513aa798f1dcfd355865d09d541e7 |
| SimHash | 443fc4e3ef9b |
Groups
petalbot
aspiegelbot
ahrefsbot
semrushbot
dotbot
mauibot
mj12bot
yandexbot
yandexcalendar
mail.ru_bot
blexbot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /whats-new/ |
| Disallow | /account/ |
| Disallow | /attachments/ |
| Disallow | /goto/ |
| Disallow | /posts/ |
| Disallow | /login/ |
| Disallow | /search/ |
| Disallow | /admin.php |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |