comicvine.com
robots.txt
Robots Exclusion Standard data for comicvine.com
Resource Scan
Scan Details
| Site Domain | comicvine.com |
| Base Domain | comicvine.com |
| Scan Status | Ok |
| Last Scan | 2024-11-13T22:10:38+00:00 |
| Next Scan | 2024-11-20T22:10:38+00:00 |
Last Scan
| Scanned | 2024-11-13T22:10:38+00:00 |
| URL | https://comicvine.com/robots.txt |
| Redirect | https://comicvine.gamespot.com/robots.txt |
| Redirect Domain | comicvine.gamespot.com |
| Redirect Base | gamespot.com |
| Domain IPs | 199.232.208.194, 199.232.212.194 |
| Redirect IPs | 199.232.208.194, 199.232.212.194 |
| Response IP | 199.232.44.194 |
| Found | Yes |
| Hash | e73f6588e148d7f374940221cbf20a2aaf91ceb723c746ff19e06398304d175e |
| SimHash | d35c985163a0 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /api/$ |
| Disallow | /api/* |
| Disallow | /notifications/ |
| Disallow | /search/ |
| Disallow | *login%3D* |
| Disallow | /videos/embed |
| Disallow | /videos/feed/ |
| Disallow | /wiki/moderation/ |
| Disallow | /postRender |
| Disallow | /forums/*/flag/ |
| Disallow | /forums/*/delete/ |
| Disallow | /forums/*/edit/ |
| Disallow | /forums/*/lock/ |
| Disallow | /forums/*/anchor/ |
| Disallow | /forums/*/add-to-favorite/ |
| Disallow | /forums/*/best-answer/*/ |
| Disallow | /jsonsearch/ |
| Disallow | /chat/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Comments