zdnet.com.au
robots.txt
Robots Exclusion Standard data for zdnet.com.au
Resource Scan
Scan Details
| Site Domain | zdnet.com.au |
| Base Domain | zdnet.com.au |
| Scan Status | Ok |
| Last Scan | 2024-05-16T21:35:28+00:00 |
| Next Scan | 2024-05-23T21:35:28+00:00 |
Last Scan
| Scanned | 2024-05-16T21:35:28+00:00 |
| URL | https://zdnet.com.au/robots.txt |
| Redirect | https://www.zdnet.com/robots.txt |
| Redirect Domain | www.zdnet.com |
| Redirect Base | zdnet.com |
| Domain IPs | 34.149.132.124 |
| Redirect IPs | 199.232.194.154, 199.232.198.154, 2a04:4e42:4c::666, 2a04:4e42:4d::666 |
| Response IP | 146.75.42.154 |
| Found | Yes |
| Hash | fab3ebbc7cdf8d3e1f116e3f5f3e188d5874a1da4b30f06ba38ef43de6125efa |
| SimHash | 281abc408917 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /user/* |
| Disallow | /members/ |
| Disallow | /members/newsletters/ |
| Disallow | /members/alerts/add/ |
| Disallow | /search/ |
| Disallow | *Xhr* |
| Disallow | */xhr* |
| Disallow | */ajax/* |
| Disallow | */fly/*/bundles/flyjs/* |
| Disallow | */libs/* |
| Disallow | */version%21libs/* |
| Disallow | /.well-known/* |
| Disallow | /index.php/* |
| Disallow | *?beta=* |
| Disallow | *?ftag=* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.zdnet.com/sitemaps/news.xml |
| sitemap | https://www.zdnet.com/sitemaps/topics.xml |
| sitemap | https://www.zdnet.com/sitemaps/article/index.xml |
| sitemap | https://www.zdnet.com/sitemaps/gallery/index.xml |
| sitemap | https://www.zdnet.com/sitemaps/video/index.xml |
| sitemap | https://www.zdnet.com/sitemaps/review/index.xml |
| sitemap | https://www.zdnet.com/sitemaps/blogs.xml |
Comments