themuse.com
robots.txt
Robots Exclusion Standard data for themuse.com
Resource Scan
Scan Details
| Site Domain | themuse.com |
| Base Domain | themuse.com |
| Scan Status | Ok |
| Last Scan | 2024-11-12T07:56:35+00:00 |
| Next Scan | 2024-11-19T07:56:35+00:00 |
Last Scan
| Scanned | 2024-11-12T07:56:35+00:00 |
| URL | https://themuse.com/robots.txt |
| Redirect | https://www.themuse.com/robots.txt |
| Redirect Domain | www.themuse.com |
| Redirect Base | themuse.com |
| Domain IPs | 104.18.105.113, 104.18.106.113, 2606:4700::6812:6971, 2606:4700::6812:6a71 |
| Redirect IPs | 104.18.105.113, 104.18.106.113, 2606:4700::6812:6971, 2606:4700::6812:6a71 |
| Response IP | 104.18.105.113 |
| Found | Yes |
| Hash | 6e710b4b8db6ac2e2f49181f1ce9acc5652cc746a4881e135df21bce71c48c1f |
| SimHash | 79058271cfc7 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /clients/ |
| Disallow | /dashboard/ |
| Allow | /clients/*/media/* |
| Allow | /static/* |
| Disallow | /profiles/*/embed |
| Disallow | /profiles/*/embed?* |
| Disallow | /profiles/*/modules/*/embed |
| Disallow | /profiles/*/framed |
| Disallow | /profiles/*/framed?* |
| Disallow | /profiles/*/modules/*/framed |
| Disallow | /companies/*/embed |
| Disallow | /companies/*/embed?* |
| Disallow | /companies/*/modules/*/embed |
| Disallow | /companies/*/framed |
| Disallow | /companies/*/framed?* |
| Disallow | /companies/*/modules/*/framed |
| Disallow | /job/redirect/* |
| Disallow | /jobs/*/*?*framed* |
| Disallow | /vendor/* |
| Disallow | /tracking/amp* |
| Disallow | /api/users* |
| Disallow | /cdn-cgi/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.themuse.com/sitemap.xml |