ibnlokmat.tv
robots.txt

Robots Exclusion Standard data for ibnlokmat.tv

Resource Scan

Scan Details

Site Domain ibnlokmat.tv
Base Domain ibnlokmat.tv
Scan Status Ok
Last Scan2024-06-20T21:47:50+00:00
Next Scan 2024-07-20T21:47:50+00:00

Last Scan

Scanned2024-06-20T21:47:50+00:00
URL http://www.ibnlokmat.tv/robots.txt
Redirect https://lokmat.news18.com/robots.txt
Redirect Domain lokmat.news18.com
Redirect Base news18.com
Domain IPs 2600:1413:1::6011:4812, 2600:1413:1::7d38:db22, 72.247.127.187, 72.247.127.192
Redirect IPs 23.54.58.50, 2600:1413:5000:181::3379, 2600:1413:5000:18d::3379
Response IP 23.54.58.50
Found Yes
Hash 42f983f6aa823ee55e7c778adfc4d0fe861123077c3cceb13588fb297124e3ed
SimHash 7a3f9052a551

Groups

*

Rule Path
Allow /
Disallow /events/general-election-2019/
Disallow /jw_player/
Disallow /uc/
Disallow /1039154/
Disallow /category/video/page-*
Disallow /category/video/page-*/
Disallow /mutual-funds/
Disallow /insurance/
Disallow /auto/
Disallow /amp/assets/.jpeg/$
Disallow /astrology/horoscope/tag/
Disallow /astrology/horoscope/category/
Disallow /search/*
Disallow /archives/*/page-*
Disallow /category/*
Disallow /amp/astrology/horoscope/*
Disallow /astrology/horoscope/*

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

mazbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://lokmat.news18.com/commonfeeds/v1/lok/sitemap/today
sitemap https://lokmat.news18.com/commonfeeds/v1/lok/sitemap-index.xml
sitemap https://lokmat.news18.com/commonfeeds/v1/lok/sitemap/google-news.xml
sitemap https://lokmat.news18.com/commonfeeds/v1/lok/sitemap-image-index.xml
sitemap https://lokmat.news18.com/commonfeeds/v1/lok/sitemap-video-index.xml
sitemap https://lokmat.news18.com/commonfeeds/v1/lok/sitemap/webstories-sitemap-index.xml