watchmojo.com
robots.txt
Robots Exclusion Standard data for watchmojo.com
Resource Scan
Scan Details
| Site Domain | watchmojo.com |
| Base Domain | watchmojo.com |
| Scan Status | Ok |
| Last Scan | 2024-11-12T21:42:33+00:00 |
| Next Scan | 2024-11-19T21:42:33+00:00 |
Last Scan
| Scanned | 2024-11-12T21:42:33+00:00 |
| URL | https://watchmojo.com/robots.txt |
| Redirect | https://www.watchmojo.com/robots.txt |
| Redirect Domain | www.watchmojo.com |
| Redirect Base | watchmojo.com |
| Domain IPs | 192.124.249.169 |
| Redirect IPs | 192.124.249.169 |
| Response IP | 192.124.249.169 |
| Found | Yes |
| Hash | fa33b1aed382c91094ceeb89977f7808eaa19c2764c1644aeeaffed7a7f447b4 |
| SimHash | 484628a0c420 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /$ |
| Allow | /sitemap/ |
| Allow | /video/ |
| Allow | /articles/ |
| Allow | /suggest/ |
| Allow | /play/ |
| Allow | /search/ |
| Allow | /amp/ |
| Allow | /exclusive/ |
| Allow | /categories/ |
| Allow | /shows/ |
| Allow | /tag/ |
| Allow | /author/ |
| Allow | /article-list/ |
| Allow | /blog/ |
| Allow | /getmojo/ |
| Allow | /es/ |
| Allow | /fr/ |
| Allow | /msmojo/ |
| Disallow | /my/ |
| Disallow | /mobile/ |
| Disallow | /trivia/ |
| Disallow | /migrate/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.watchmojo.com/sitemap/suggest |
| sitemap | https://www.watchmojo.com/sitemap/suggest/1 |
| sitemap | https://www.watchmojo.com/sitemap/suggest/2 |
| sitemap | https://www.watchmojo.com/sitemap/suggest/3 |
| sitemap | https://www.watchmojo.com/sitemap/suggest/4 |
| sitemap | https://www.watchmojo.com/sitemap/suggest/5 |
| sitemap | https://www.watchmojo.com/sitemap/suggest/6 |
| sitemap | https://www.watchmojo.com/sitemap/suggest/7 |
| sitemap | https://www.watchmojo.com/sitemap/suggest/8 |
| sitemap | https://www.watchmojo.com/sitemap/suggest/9 |
| sitemap | https://www.watchmojo.com/sitemap/suggest/10 |
| sitemap | https://www.watchmojo.com/sitemap/video |
| sitemap | https://www.watchmojo.com/sitemap/blog |
| sitemap | https://www.watchmojo.com/sitemap/trivia |
| sitemap | https://www.watchmojo.com/sitemap/article |