pagesix.com
robots.txt
Robots Exclusion Standard data for pagesix.com
Resource Scan
Scan Details
Site Domain | pagesix.com |
Base Domain | pagesix.com |
Scan Status | Ok |
Last Scan | 2024-10-05T15:59:50+00:00 |
Next Scan | 2024-10-12T15:59:50+00:00 |
Last Scan
Scanned | 2024-10-05T15:59:50+00:00 |
URL | https://pagesix.com/robots.txt |
Domain IPs | 192.0.66.32 |
Response IP | 192.0.66.32 |
Found | Yes |
Hash | 6dc66c07c9c874fbd85c913a6f805871877668fa65cc58d0e2f5a05f9f803534 |
SimHash | 0e31ce0444e2 |
Groups
*
Rule | Path |
---|---|
Disallow | /wp-admin/ |
Allow | /wp-admin/admin-ajax.php |
Disallow | /account/ |
Allow | /account/subscribe |
Disallow | /5850/ |
*
Rule | Path |
---|---|
Disallow | /search/ |
Disallow | /horoscope/ |
Disallow | /*?x= |
Disallow | /*?action= |
Disallow | /*%26action%3D |
Disallow | /*?adid= |
Disallow | /*%26adid%3D |
Disallow | /*?bitrate= |
Disallow | /*%26bitrate%3D |
Disallow | /*?channel= |
Disallow | /*%26channel%3D |
Disallow | /*?clipid= |
Disallow | /*%26clipid%3D |
Disallow | /*?contentCollection= |
Disallow | /*%26contentCollection%3D |
Disallow | /*?contentId= |
Disallow | /*%26contentId%3D |
Disallow | /*?ei= |
Disallow | /*%26ei%3D |
Disallow | /*?format= |
Disallow | /*%26format%3D |
Disallow | /*?height= |
Disallow | /*%26height%3D |
Disallow | /*?iframe= |
Disallow | /*%26iframe%3D |
Disallow | /*?mediaId= |
Disallow | /*%26mediaId%3D |
Disallow | /*?module= |
Disallow | /*%26module%3D |
Disallow | /*?monthnum= |
Disallow | /*%26monthnum%3D |
Disallow | /*?pgtype= |
Disallow | /*%26pgtype%3D |
Disallow | /*?preview= |
Disallow | /*%26preview%3D |
Disallow | /*?priority= |
Disallow | /*%26priority%3D |
Disallow | /*?ref= |
Disallow | /*%26ref%3D |
Disallow | /*?referrer= |
Disallow | /*%26referrer%3D |
Disallow | /*?sa= |
Disallow | /*%26sa%3D |
Disallow | /*?section= |
Disallow | /*%26section%3D |
Disallow | /*?share= |
Disallow | /*%26share%3D |
Disallow | /*?TB_iframe= |
Disallow | /*%26TB_iframe%3D |
Disallow | /*?TBL_Prod100Page= |
Disallow | /*%26TBL_Prod100Page%3D |
Disallow | /*?theme_preview= |
Disallow | /*%26theme_preview%3D |
Disallow | /*?usg= |
Disallow | /*%26usg%3D |
Disallow | /*?ved= |
Disallow | /*%26ved%3D |
Disallow | /*?version= |
Disallow | /*%26version%3D |
Disallow | /*?VID= |
Disallow | /*%26VID%3D |
Disallow | /*?vxBitrate= |
Disallow | /*%26vxBitrate%3D |
Disallow | /*?vxChannel= |
Disallow | /*%26vxChannel%3D |
Disallow | /*?vxClipId= |
Disallow | /*%26vxClipId%3D |
Disallow | /*?vxSiteId= |
Disallow | /*%26vxSiteId%3D |
Disallow | /*?width= |
Disallow | /*%26width%3D |
Disallow | /*?year= |
Disallow | /*%26year%3D |
Disallow | /*.xml?uuid= |
Disallow | /*?post_type=feedback&p= |
Disallow | /*?wptouch_preview_theme= |
Disallow | /*%26gt%3B |
Disallow | /?attachment_id= |
Disallow | /affiliate/ |
Disallow | /ai_log/ |
Disallow | /*?*pg99.asia* |
Disallow | /*%26*pg99.asia* |
Disallow | /*?*gg155.cn* |
Disallow | /*%26*gg155.cn* |
Disallow | /*?*To66.Asia* |
Disallow | /*%26*To66.Asia* |
*
Rule | Path |
---|---|
Disallow | /tag/credible/ |
Disallow | /personal-finance/ |
Disallow | /page-six-instagram-feed/ |
Disallow | /page-six-tiktok-feed/ |
Other Records
Field | Value |
---|---|
sitemap | https://pagesix.com/news-sitemap.xml |
sitemap | https://pagesix.com/sitemap-1999.xml |
sitemap | https://pagesix.com/sitemap-2000.xml |
sitemap | https://pagesix.com/sitemap-2001.xml |
sitemap | https://pagesix.com/sitemap-2002.xml |
sitemap | https://pagesix.com/sitemap-2003.xml |
sitemap | https://pagesix.com/sitemap-2004.xml |
sitemap | https://pagesix.com/sitemap-2005.xml |
sitemap | https://pagesix.com/sitemap-2006.xml |
sitemap | https://pagesix.com/sitemap-2007.xml |
sitemap | https://pagesix.com/sitemap-2008.xml |
sitemap | https://pagesix.com/sitemap-2009.xml |
sitemap | https://pagesix.com/sitemap-2010.xml |
sitemap | https://pagesix.com/sitemap-2011.xml |
sitemap | https://pagesix.com/sitemap-2012.xml |
sitemap | https://pagesix.com/sitemap-2013.xml |
sitemap | https://pagesix.com/sitemap-2014.xml |
sitemap | https://pagesix.com/sitemap-2015.xml |
sitemap | https://pagesix.com/sitemap-2016.xml |
sitemap | https://pagesix.com/sitemap-2017.xml |
sitemap | https://pagesix.com/sitemap-2018.xml |
sitemap | https://pagesix.com/sitemap-2019.xml |
sitemap | https://pagesix.com/sitemap-2020.xml |
sitemap | https://pagesix.com/sitemap-2021.xml |
sitemap | https://pagesix.com/sitemap-2022.xml |
sitemap | https://pagesix.com/sitemap-2023.xml |
sitemap | https://pagesix.com/sitemap-2024.xml |
sitemap | https://pagesix.com/sitemap-nypost-section.xml |
sitemap | https://pagesix.com/sitemap-nypost-post_tag.xml |
sitemap | https://pagesix.com/sitemap-nypost-authors.xml |
sitemap | https://pagesix.com/sitemap-nypost-pages.xml |
Comments