herbshook.com
robots.txt

Robots Exclusion Standard data for herbshook.com

Resource Scan

Scan Details

Site Domain herbshook.com
Base Domain herbshook.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-04-30T16:09:59+00:00
Next Scan 2024-06-29T16:09:59+00:00

Last Successful Scan

Scanned2021-10-05T22:12:20+00:00
URL http://herbshook.com/robots.txt
Redirect https://www.jpost.com/robots.txt
Redirect Domain www.jpost.com
Redirect Base jpost.com
Found Yes
Hash 4b1612c1b16bd455e173b13f007c430d3c2091161c64ac51bdf4d7ec48700390
SimHash 7d145800f551

Groups

*
*
twitterbot

Rule Path
Allow /kabbalah
Allow /cybertech
Disallow /trackback/
Disallow /*search%3Bquery*
Disallow /comments/
Disallow /advertserve/shim.html

Other Records

Field Value
sitemap https://www.jpost.com/GoogleNewsSiteMap/GNSiteMap
sitemap https://www.jpost.com/jpgooglesitemap/sitemapindex.xml