houstonpublicmedia.org
robots.txt
Robots Exclusion Standard data for houstonpublicmedia.org
Resource Scan
Scan Details
| Site Domain | houstonpublicmedia.org |
| Base Domain | houstonpublicmedia.org |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-08-23T17:51:22+00:00 |
| Next Scan | 2024-10-22T17:51:22+00:00 |
Last Successful Scan
| Scanned | 2024-07-01T17:49:56+00:00 |
| URL | https://houstonpublicmedia.org/robots.txt |
| Redirect | https://www.houstonpublicmedia.org/robots.txt |
| Redirect Domain | www.houstonpublicmedia.org |
| Redirect Base | houstonpublicmedia.org |
| Domain IPs | 34.210.194.27 |
| Redirect IPs | 2600:9000:23d1:2e00:3:f03:a940:93a1, 2600:9000:23d1:800:3:f03:a940:93a1, 2600:9000:23d1:8600:3:f03:a940:93a1, 2600:9000:23d1:a400:3:f03:a940:93a1, 2600:9000:23d1:ac00:3:f03:a940:93a1, 2600:9000:23d1:c000:3:f03:a940:93a1, 2600:9000:23d1:ca00:3:f03:a940:93a1, 2600:9000:23d1:f800:3:f03:a940:93a1, 65.9.112.55, 65.9.112.66, 65.9.112.73, 65.9.112.84 |
| Response IP | 3.160.246.20 |
| Found | Yes |
| Hash | 262a2ea9281ec25d52b73dd9b1ae60797958c58649679b03817a87b14e9a311c |
| SimHash | 58215911c1b5 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /ProdStage |
| Disallow | /wp-admin/ |
| Disallow | /wp/wp-admin/ |
| Disallow | /wp/wp-admin/admin-ajax.php |
| Disallow | /videos/ |
| Disallow | /pages/ |
| Disallow | /news/awards/ |
| Disallow | /support/studio-society/members/ |
| Disallow | /support/affinity-council/members/ |
| Disallow | /wp-json/ |
amazonbot
anthropic-ai
applebot-extended
awariorssbot
awariosmartbot
bytespider
ccbot
chatgpt-user
claudebot
claude-web
cohere-ai
dataforseobot
diffbot
facebookbot
friendlycrawler
google-extended
googleother
gptbot
img2dataset
imagesiftbot
magpie-crawler
meltwater
omgili
omgilibot
peer39_crawler
peer39_crawler/1.0
perplexitybot
piplbot
scoop.it
seekr
youbot
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.houstonpublicmedia.org/sitemap.xml |
| sitemap | https://www.houstonpublicmedia.org/newssitemap.xml |
Comments