contagiousmedia.com
robots.txt
Robots Exclusion Standard data for contagiousmedia.com
Resource Scan
Scan Details
Site Domain | contagiousmedia.com |
Base Domain | contagiousmedia.com |
Scan Status | Ok |
Last Scan | 2024-10-28T17:26:49+00:00 |
Next Scan | 2024-11-04T17:26:49+00:00 |
Last Scan
Scanned | 2024-10-28T17:26:49+00:00 |
URL | https://contagiousmedia.com/robots.txt |
Redirect | https://www.buzzfeed.com/robots.txt |
Redirect Domain | www.buzzfeed.com |
Redirect Base | buzzfeed.com |
Domain IPs | 13.35.210.112, 13.35.210.2, 13.35.210.23, 13.35.210.74 |
Redirect IPs | 151.101.130.114, 151.101.194.114, 151.101.2.114, 151.101.66.114 |
Response IP | 199.232.46.114 |
Found | Yes |
Hash | 36b0dd7e2efa604a68a37097e01ac63fe17566c8ac3737799a7c1debdfe46e25 |
SimHash | 313a5d518c81 |
Groups
msnbot
Rule | Path |
---|---|
Disallow | /*.xml$ |
Disallow | /buzz/*.xml$ |
Disallow | /category/*.xml$ |
Disallow | /mobile/ |
Disallow | *?s=mobile |
Disallow | *?s=bpage-next |
Disallow | *?s=lightbox |
Disallow | /fabordrab/ |
Disallow | /bfmp/ |
Disallow | /buzzfeed/ |
Disallow | /contest |
Disallow | /contests |
Disallow | /plugin/ |
Disallow | /embed/ |
Disallow | /_comments/ |
Disallow | /bookmarks/ |
Disallow | /drafts/ |
Other Records
Field | Value |
---|---|
crawl-delay | 120 |
*
Rule | Path |
---|---|
Disallow | /buzz/*.xml$ |
Disallow | /category/*.xml$ |
Disallow | /mobile/ |
Disallow | *?s=bpage-next |
Disallow | *?s=lightbox |
Disallow | *?s=feedpager |
Disallow | /fabordrab/ |
Disallow | /bfmp/ |
Disallow | /buzzfeed/ |
Disallow | /contest |
Disallow | /contests |
Disallow | /_ga/ |
Disallow | /static/ |
Disallow | /dashboard/ |
Disallow | /plugin/ |
Disallow | /api/ |
Disallow | /buzzfeed/api/ |
Disallow | /embed/ |
Disallow | /_comments/ |
Disallow | /bookmarks/ |
Disallow | /drafts/ |
Disallow | /bpage-preview/ |
Disallow | /cms_preview_ui/ |
Disallow | /cms_preview_ui_static/ |
Disallow | /search/ |
Other Records
Field | Value |
---|---|
sitemap | https://www.buzzfeed.com/sitemap/asis.xml |
sitemap | https://www.buzzfeed.com/sitemap/buzzfeed.xml |
sitemap | https://www.buzzfeed.com/sitemap/tasty.xml |
sitemap | https://www.buzzfeed.com/sitemap/video.xml |
sitemap | https://www.buzzfeed.com/sitemap/shopping.xml |
sitemap | https://www.buzzfeed.com/sitemap/topics.xml |
sitemap | https://www.buzzfeed.com/sitemap/travel.xml |
sitemap | https://www.buzzfeed.com/sitemap/news.xml |
sitemap | https://www.buzzfeed.com/sitemap/buzzfeed-community.xml |
sitemap | https://www.buzzfeed.com/sitemap/editions.xml |
sitemap | https://www.buzzfeed.com/webstories/sitemap.xml |