kannada.news18.com
robots.txt

Robots Exclusion Standard data for kannada.news18.com

Resource Scan

Scan Details

Site Domain kannada.news18.com
Base Domain news18.com
Scan Status Ok
Last Scan2024-09-28T05:37:16+00:00
Next Scan 2024-10-12T05:37:16+00:00

Last Scan

Scanned2024-09-28T05:37:16+00:00
URL https://kannada.news18.com/robots.txt
Domain IPs 23.50.89.253, 2600:1413:1:98a::3393
Response IP 104.69.47.169
Found Yes
Hash 6b546d2b406e9447991591723b28be48e13b5dbf4df29e021d1d3442386980a4
SimHash 440dc958a585

Groups

*

Rule Path
Allow /
Disallow /events/general-election-2019/
Disallow /uc/
Disallow /marketing/
Disallow /uc-election-widget/
Disallow /tag/%*/
Disallow /tag/*/page-*/
Disallow /tag/*/photogallery/*
Disallow /tag/*/videos/*
Disallow /tag/*/news/*
Disallow /byline/%*/
Disallow /byline/*/photogallery/page-*/
Disallow /byline/*/videos/page-*/
Disallow /byline/*/news/page-*/
Disallow /byline/*htmlpage-*html/page-*/
Disallow /byline/*htmlpage-*/page-*/
Disallow /byline/*htmlpage-*/photogallery/
Disallow /byline/*/page-*/tag/*/
Disallow /byline/*/page-*/$
Disallow /amp/tag/%*/
Disallow /amp/tag/*/page-*/
Disallow /amp/tag/*/photogallery/*
Disallow /amp/tag/*/videos/*
Disallow /amp/tag/*/news/*
Disallow /amp/*page-1page-1*
Disallow /amp/entertainment/page-*/
Disallow /videos/uncategorized/page-*/
Disallow /videos/trending/page-*/
Disallow /amp/videos/*.html/page-*/
Disallow /amp/videos/page-*/
Disallow /news/page-*/
Disallow /videos/*.html/page-*/
Disallow /amp/videos/national-international/page-*/
Disallow /amp/state/page-*/
Disallow /agency/news18-kannada/page-*/privacy-policy/
Disallow /photogallery/*/page-*
Disallow /tag/*/privacy-policy/
Disallow /tag/*/ram-mandir/
Disallow /photogallery/*/page-*/
Disallow /*html/1000
Disallow /amp/videos/*/privacy-policy/
Disallow /amp/*/entertainmentpage-*/page-*/
Disallow /amp/tag/*/privacy-policy/
Disallow /amp/*/page-*/
Disallow /cricketnext/
Disallow /videos/*/ram-mandir/
Disallow /videos/*htmlpage-*page-*page-*page-*/page-*/
Disallow /ampnews/
Disallow /amp/assembly-election-2019/
Disallow /assembly-election-2019/
Disallow /*page-1page-*
Disallow /scorecard/
Disallow /virudhunagarpage-1
Disallow /lok-sabha-election-2019/
Disallow /assembly-elections-2018/
Disallow /vod/
Disallow /astrology/horoscope/
Disallow /amp/astrology/horoscope/
Disallow /articles-sitemap
Disallow /image-sitemap
Disallow /videos-sitemap
Disallow /assembly-elections-2022/
Disallow /assembly-elections-2023/
Disallow /assembly-elections-march-2022/
Disallow /*/page-*/privacy-policy/
Disallow /*/page-*/$
Disallow /states-by-election-result-2018/
Disallow /election-result-2013/
Disallow /chikmagaluru/
Disallow /shimoga/
Disallow /embed/
Disallow /budget-2021/
Disallow /devanagere/
Disallow /manipur/
Disallow /year-ender-2019/
Disallow /1
Disallow /2
Disallow /3
Disallow /4
Disallow /5
Disallow /6
Disallow /7
Disallow /8
Disallow /9
Disallow /0

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

mazbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

img2dataset

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://kannada.news18.com/commonfeeds/v1/kan/sitemap/today
sitemap https://kannada.news18.com/commonfeeds/v1/kan/sitemap-index.xml
sitemap https://kannada.news18.com/commonfeeds/v1/kan/sitemap/google-news.xml
sitemap https://kannada.news18.com/commonfeeds/v1/kan/sitemap-image-index.xml
sitemap https://kannada.news18.com/commonfeeds/v1/kan/sitemap-video-index.xml
sitemap https://kannada.news18.com/commonfeeds/v1/kan/sitemap/webstories-sitemap-index.xml