mtv.co.nz
robots.txt

Robots Exclusion Standard data for mtv.co.nz

Resource Scan

Scan Details

Site Domain mtv.co.nz
Base Domain mtv.co.nz
Scan Status Failed
Failure StageFetching resource.
Failure ReasonRequest timed out.
Last Scan2024-05-21T05:08:31+00:00
Next Scan 2024-07-20T05:08:31+00:00

Last Successful Scan

Scanned2024-03-23T02:52:36+00:00
URL https://mtv.co.nz/robots.txt
Redirect https://www.mtv.com.au/robots.txt
Redirect Domain www.mtv.com.au
Redirect Base mtv.com.au
Domain IPs 34.213.106.51, 54.68.182.72
Redirect IPs 23.203.76.71, 2600:1413:1:59e::955, 2600:1413:1:5a0::955
Response IP 104.103.145.116
Found Yes
Hash da4b61cf0c3a33c32f2fbbcca01bb343da464c8485c24ea2b61d109cacbf4747
SimHash 4802c824cc13

Groups

*

Rule Path
Disallow /api/

petalbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.mtv.com.au/xmlsitemap/video
sitemap https://www.mtv.com.au/xmlsitemap/episode
sitemap https://www.mtv.com.au/xmlsitemap/season
sitemap https://www.mtv.com.au/xmlsitemap/photogallery
sitemap https://www.mtv.com.au/xmlsitemap/news
sitemap https://www.mtv.com.au/xmlsitemap/show