comicvine.gamespot.com
robots.txt
Robots Exclusion Standard data for comicvine.gamespot.com
Resource Scan
Scan Details
| Site Domain | comicvine.gamespot.com |
| Base Domain | gamespot.com |
| Scan Status | Ok |
| Last Scan | 2024-05-30T19:22:13+00:00 |
| Next Scan | 2024-06-06T19:22:13+00:00 |
Last Scan
| Scanned | 2024-05-30T19:22:13+00:00 |
| URL | https://comicvine.gamespot.com/robots.txt |
| Domain IPs | 199.232.208.194, 199.232.212.194 |
| Response IP | 199.232.44.194 |
| Found | Yes |
| Hash | 8c903436128799ea035de74ce1439620735de99b2a3611d5fc676e3d9ee04a5f |
| SimHash | c74498c56307 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /api/$ |
| Disallow | /api/* |
| Disallow | /notifications/ |
| Disallow | /search/ |
| Disallow | *login%3D* |
| Disallow | /videos/embed |
| Disallow | /videos/feed/ |
| Disallow | /wiki/moderation/ |
| Disallow | /postRender |
| Disallow | /forums/*/flag/ |
| Disallow | /forums/*/delete/ |
| Disallow | /forums/*/edit/ |
| Disallow | /forums/*/lock/ |
| Disallow | /forums/*/anchor/ |
| Disallow | /forums/*/add-to-favorite/ |
| Disallow | /forums/*/best-answer/*/ |
| Disallow | /jsonsearch/ |
| Disallow | /chat/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
Comments