mgid.com
robots.txt
Robots Exclusion Standard data for mgid.com
Resource Scan
Scan Details
| Site Domain | mgid.com |
| Base Domain | mgid.com |
| Scan Status | Ok |
| Last Scan | 2024-10-15T12:59:10+00:00 |
| Next Scan | 2024-11-14T12:59:10+00:00 |
Last Scan
| Scanned | 2024-10-15T12:59:10+00:00 |
| URL | https://mgid.com/robots.txt |
| Redirect | https://www.mgid.com/robots.txt |
| Redirect Domain | www.mgid.com |
| Redirect Base | mgid.com |
| Domain IPs | 64.58.116.132, 64.58.116.142 |
| Redirect IPs | 104.19.129.76, 104.19.130.76, 104.19.131.76, 104.19.132.76, 104.19.133.76, 2606:4700:1::6813:814c, 2606:4700:1::6813:824c, 2606:4700:1::6813:834c, 2606:4700:1::6813:844c, 2606:4700:1::6813:854c |
| Response IP | 104.19.129.76 |
| Found | Yes |
| Hash | fb251632d47cc3e474303dd2b547a40f635bc22009c1b9b4b22033b08aa60999 |
| SimHash | e1448f27fa90 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search |
| Disallow | /redirect |
| Disallow | /news |
| Disallow | /rnews |
| Disallow | /tnews |
| Disallow | /ghits |
| Disallow | /pnews |
| Disallow | /graphql |
| Disallow | /webinars |
| Disallow | /translator |
| Disallow | /dpd |
| Disallow | /landing/cpc/push-for-advertisers/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.mgid.com/sitemap.xml |
Warnings
- `host` is not a known field.