Log In Sign Up

test.post-gazette.com
robots.txt

Robots Exclusion Standard data for test.post-gazette.com

Resource Scan

Scan Details

Site Domain test.post-gazette.com
Base Domain post-gazette.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonRequest timed out.
Last Scan2025-11-04T16:08:27+00:00
Next Scan 2026-01-03T16:08:27+00:00

Last Successful Scan

Scanned2025-08-13T00:40:26+00:00
URL https://test.post-gazette.com/robots.txt
Domain IPs 69.164.218.238
Response IP 69.164.218.238
Found Yes
Hash 765c9812f7e3bffca52655ba2cf322784cdf207f7860a510fc392117a3ff9a22
SimHash 5345c70f6112

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow
Disallow /*.pdf

googlebot-mobile

Rule Path
Disallow

bingbot

Rule Path
Disallow /

msnbot

Rule Path
Disallow

slurp

Rule Path
Disallow

bingpreview

Rule Path
Disallow

*

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://test.post-gazette.com/sitemapindex.xml
sitemap https://test.post-gazette.com/sitemap/000/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/001/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/002/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/003/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/004/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/005/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/006/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/007/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/008/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/009/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/010/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/011/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/012/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/013/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/014/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/015/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/016/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/017/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/018/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/019/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/020/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/021/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/022/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/023/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/024/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/025/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/026/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/027/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/028/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/029/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/030/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/031/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/032/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/033/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/034/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/035/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/036/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/037/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/038/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/039/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/040/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/041/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/042/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/043/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/044/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/045/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/046/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/047/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/048/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/049/sitemap_books.xml
sitemap https://test.post-gazette.com/sitemap/050/sitemap_books.xml
Back to top