nypost.com
robots.txt
Robots Exclusion Standard data for nypost.com
Resource Scan
Scan Details
| Site Domain | nypost.com |
| Base Domain | nypost.com |
| Scan Status | Ok |
| Last Scan | 2024-05-15T00:25:00+00:00 |
| Next Scan | 2024-05-22T00:25:00+00:00 |
Last Scan
| Scanned | 2024-05-15T00:25:00+00:00 |
| URL | https://nypost.com/robots.txt |
| Domain IPs | 192.0.66.32 |
| Response IP | 192.0.66.32 |
| Found | Yes |
| Hash | 87a17741e815aecc249f3f08370b302eee9585c520dd65fdaa5ff71e161c9f04 |
| SimHash | 3ee9ca4464c0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Allow | /wp-admin/admin-ajax.php |
| Disallow | /account/ |
| Allow | /account/subscribe |
| Disallow | /5850/ |
*
| Rule | Path |
|---|---|
| Disallow | /search/ |
| Disallow | /horoscope/ |
| Disallow | /*?x= |
| Disallow | /*?action= |
| Disallow | /*%26action%3D |
| Disallow | /*?adid= |
| Disallow | /*%26adid%3D |
| Disallow | /*?bitrate= |
| Disallow | /*%26bitrate%3D |
| Disallow | /*?channel= |
| Disallow | /*%26channel%3D |
| Disallow | /*?clipid= |
| Disallow | /*%26clipid%3D |
| Disallow | /*?contentCollection= |
| Disallow | /*%26contentCollection%3D |
| Disallow | /*?contentId= |
| Disallow | /*%26contentId%3D |
| Disallow | /*?ei= |
| Disallow | /*%26ei%3D |
| Disallow | /*?format= |
| Disallow | /*%26format%3D |
| Disallow | /*?height= |
| Disallow | /*%26height%3D |
| Disallow | /*?iframe= |
| Disallow | /*%26iframe%3D |
| Disallow | /*?mediaId= |
| Disallow | /*%26mediaId%3D |
| Disallow | /*?module= |
| Disallow | /*%26module%3D |
| Disallow | /*?monthnum= |
| Disallow | /*%26monthnum%3D |
| Disallow | /*?pgtype= |
| Disallow | /*%26pgtype%3D |
| Disallow | /*?preview= |
| Disallow | /*%26preview%3D |
| Disallow | /*?priority= |
| Disallow | /*%26priority%3D |
| Disallow | /*?ref= |
| Disallow | /*%26ref%3D |
| Disallow | /*?referrer= |
| Disallow | /*%26referrer%3D |
| Disallow | /*?sa= |
| Disallow | /*%26sa%3D |
| Disallow | /*?section= |
| Disallow | /*%26section%3D |
| Disallow | /*?share= |
| Disallow | /*%26share%3D |
| Disallow | /*?TB_iframe= |
| Disallow | /*%26TB_iframe%3D |
| Disallow | /*?TBL_Prod100Page= |
| Disallow | /*%26TBL_Prod100Page%3D |
| Disallow | /*?theme_preview= |
| Disallow | /*%26theme_preview%3D |
| Disallow | /*?usg= |
| Disallow | /*%26usg%3D |
| Disallow | /*?ved= |
| Disallow | /*%26ved%3D |
| Disallow | /*?version= |
| Disallow | /*%26version%3D |
| Disallow | /*?VID= |
| Disallow | /*%26VID%3D |
| Disallow | /*?vxBitrate= |
| Disallow | /*%26vxBitrate%3D |
| Disallow | /*?vxChannel= |
| Disallow | /*%26vxChannel%3D |
| Disallow | /*?vxClipId= |
| Disallow | /*%26vxClipId%3D |
| Disallow | /*?vxSiteId= |
| Disallow | /*%26vxSiteId%3D |
| Disallow | /*?width= |
| Disallow | /*%26width%3D |
| Disallow | /*?year= |
| Disallow | /*%26year%3D |
| Disallow | /*.xml?uuid= |
| Disallow | /*?post_type=feedback&p= |
| Disallow | /*?wptouch_preview_theme= |
| Disallow | /*%26gt%3B |
| Disallow | /?attachment_id= |
| Disallow | /affiliate/ |
| Disallow | /ai_log/ |
| Disallow | /*?*pg99.asia* |
| Disallow | /*%26*pg99.asia* |
| Disallow | /*?*gg155.cn* |
| Disallow | /*%26*gg155.cn* |
| Disallow | /*?*To66.Asia* |
| Disallow | /*%26*To66.Asia* |
| Disallow | /tag/credible/ |
| Disallow | /personal-finance/ |
| Disallow | /banking/ |
| Disallow | /credit-cards/ |
| Disallow | /loans/ |
| Disallow | /personal-loans/ |
| Disallow | /refinance-student-loans/ |
| Disallow | /student-loans/ |
| Disallow | /mortgages/ |
| Disallow | /home-equity/ |
| Disallow | /mortgage-refinance/ |
*
| Rule | Path |
|---|---|
| Disallow | /new-york-post-instagram-feed/ |
| Disallow | /new-york-post-tiktok-feed/ |
| Disallow | /alexa-new-york-post-instagram-feed/ |
| Disallow | /alexa-new-york-post-tiktok-feed/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://nypost.com/news-sitemap.xml |
| sitemap | https://nypost.com/news-sitemap.xml |
| sitemap | https://nypost.com/sitemap-502.xml |
| sitemap | https://nypost.com/sitemap-1865.xml |
| sitemap | https://nypost.com/sitemap-1999.xml |
| sitemap | https://nypost.com/sitemap-2000.xml |
| sitemap | https://nypost.com/sitemap-2001.xml |
| sitemap | https://nypost.com/sitemap-2002.xml |
| sitemap | https://nypost.com/sitemap-2003.xml |
| sitemap | https://nypost.com/sitemap-2004.xml |
| sitemap | https://nypost.com/sitemap-2005.xml |
| sitemap | https://nypost.com/sitemap-2006.xml |
| sitemap | https://nypost.com/sitemap-2007.xml |
| sitemap | https://nypost.com/sitemap-2008.xml |
| sitemap | https://nypost.com/sitemap-2009.xml |
| sitemap | https://nypost.com/sitemap-2010.xml |
| sitemap | https://nypost.com/sitemap-2011.xml |
| sitemap | https://nypost.com/sitemap-2012.xml |
| sitemap | https://nypost.com/sitemap-2013.xml |
| sitemap | https://nypost.com/sitemap-2014.xml |
| sitemap | https://nypost.com/sitemap-2015.xml |
| sitemap | https://nypost.com/sitemap-2016.xml |
| sitemap | https://nypost.com/sitemap-2017.xml |
| sitemap | https://nypost.com/sitemap-2018.xml |
| sitemap | https://nypost.com/sitemap-2019.xml |
| sitemap | https://nypost.com/sitemap-2020.xml |
| sitemap | https://nypost.com/sitemap-2021.xml |
| sitemap | https://nypost.com/sitemap-2022.xml |
| sitemap | https://nypost.com/sitemap-2023.xml |
| sitemap | https://nypost.com/sitemap-2024.xml |
| sitemap | https://nypost.com/sitemap-nypost-section.xml |
| sitemap | https://nypost.com/sitemap-nypost-post_tag.xml |
| sitemap | https://nypost.com/sitemap-nypost-authors.xml |
| sitemap | https://nypost.com/sitemap-nypost-pages.xml |
| sitemap | https://nypost.com/sitemap-nypost-sportstat.xml |
Comments