mtv.fi
robots.txt

Robots Exclusion Standard data for mtv.fi

Resource Scan

Scan Details

Site Domain mtv.fi
Base Domain mtv.fi
Scan Status Ok
Last Scan2024-05-03T22:58:42+00:00
Next Scan 2024-05-10T22:58:42+00:00

Last Scan

Scanned2024-05-03T22:58:42+00:00
URL https://mtv.fi/robots.txt
Redirect https://www.mtv.fi:443/robots.txt
Redirect Domain www.mtv.fi
Redirect Base mtv.fi
Domain IPs 13.49.130.253, 13.50.123.140, 16.16.17.81
Redirect IPs 13.226.210.15, 13.226.210.2, 13.226.210.77, 13.226.210.81
Response IP 18.165.171.81
Found Yes
Hash c04741f6995e3d70970b59d0b0171af10fef4e6efebe15f0a2335a0cb14bb691
SimHash 3a555841a513

Groups

*

Rule Path
Disallow /rss_feeds

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.mtv.fi/sitemap.xml