mobile.news.com.au
robots.txt
Robots Exclusion Standard data for mobile.news.com.au
Resource Scan
Scan Details
| Site Domain | mobile.news.com.au |
| Base Domain | news.com.au |
| Scan Status | Ok |
| Last Scan | 2024-11-14T14:38:27+00:00 |
| Next Scan | 2024-11-21T14:38:27+00:00 |
Last Scan
| Scanned | 2024-11-14T14:38:27+00:00 |
| URL | http://mobile.news.com.au/robots.txt |
| Redirect | https://www.news.com.au/robots.txt |
| Redirect Domain | www.news.com.au |
| Redirect Base | news.com.au |
| Domain IPs | 23.49.60.137, 23.49.60.185 |
| Redirect IPs | 23.36.48.116, 2600:1413:b000:384::ebe, 2600:1413:b000:389::ebe, 2600:1413:b000:38b::ebe, 2600:1413:b000:38d::ebe, 2600:1413:b000:390::ebe |
| Response IP | 23.54.56.122 |
| Found | Yes |
| Hash | 7286a784ee07dbd5f3e14ad51d3a097ec6652d2bd6858ba739bc6eb4427622f4 |
| SimHash | 1120c80aa953 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /*/comments-* |
| Disallow | /*/print/ |
| Disallow | /*SIT* |
| Disallow | /*.swf |
| Disallow | /printpage/ |
| Disallow | /double-rainbow |
| Disallow | /double-rainbow/ |
| Disallow | /*-simplify/* |
| Disallow | /it-test-only/ |
| Disallow | /enewsletters/ |
| Disallow | /search?q=* |
| Disallow | */rss$ |
| Disallow | */rss2/$ |
| Disallow | */feed$ |
| Disallow | /*/coupons/visit/* |
| Disallow | /app-route/ |
| Disallow | /app/ |
| Disallow | /finance/business/stockhead/news/*/news-story/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.news.com.au/sitemap.xml |
| sitemap | https://www.news.com.au/news-sitemap.xml |
| sitemap | https://www.news.com.au/coupons/sitemap.xml |
| sitemap | https://www.news.com.au/video-sitemap.xml |
Comments