news.at
robots.txt

Robots Exclusion Standard data for news.at

Resource Scan

Scan Details

Site Domain news.at
Base Domain news.at
Scan Status Ok
Last Scan2024-05-23T20:31:31+00:00
Next Scan 2024-05-30T20:31:31+00:00

Last Scan

Scanned2024-05-23T20:31:31+00:00
URL https://news.at/robots.txt
Redirect https://www.news.at/robots.txt
Redirect Domain www.news.at
Redirect Base news.at
Domain IPs 185.51.8.103
Redirect IPs 185.150.8.220, 194.182.181.219
Response IP 194.182.181.219
Found Yes
Hash 46fe89281693f3a12c85a079a61b0d1c24ba261bdbd56d8b66fbf2e12962f039
SimHash 6f01886de3bb

Groups

bingbot

Rule Path
Disallow /ajax/
Disallow /cgi-bin/

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /ajax/
Disallow /cgi-bin/

Other Records

Field Value
sitemap https://www.news.at/googlenews.xml
sitemap https://www.news.at/sitemap-topics.xml
sitemap https://www.news.at/sitemap-ressorts.xml
sitemap https://www.news.at/sitemap-articles-index.xml