files.prokerala.com
robots.txt

Robots Exclusion Standard data for files.prokerala.com

Resource Scan

Scan Details

Site Domain files.prokerala.com
Base Domain prokerala.com
Scan Status Ok
Last Scan2024-05-10T10:33:33+00:00
Next Scan 2024-06-09T10:33:33+00:00

Last Scan

Scanned2024-05-10T10:33:33+00:00
URL https://files.prokerala.com/robots.txt
Domain IPs 96.17.180.182, 96.17.180.184
Response IP 23.59.168.168
Found Yes
Hash 2edf04c67097c0ed170268ea6d94e19c0acc6740679ed8d37e70057de8d124b2
SimHash cc3ef960a43b

Groups

googlebot-news

Rule Path
Allow /news/
Allow /astrology/
Allow /festivals/
Allow /$
Allow /?amp=1$
Disallow /

mediapartners-google

Rule Path
Allow /
Disallow

adsbot-google

Rule Path
Disallow

proximic

Rule Path
Allow /
Disallow

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

moget

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

scooperbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /
Allow /news/

yandex

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yeti

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

*

Rule Path
Allow /
Allow /res/images/
Disallow /account/
Disallow /api/
Disallow /astrology/location-search.php
Disallow /astrology/*panchang*/*.pdf$
Disallow /calendar/*.pdf$
Disallow /automobile/image-update.php
Disallow /automobile/news/
Disallow /automobile/search_details.php
Disallow /banking/ajax-search.php
Disallow /downloads/downloadfile.php
Disallow /education/imgs/email/
Disallow /general/calendar/date.php
Disallow /general/calendar/mal-date.php
Disallow /general/calendar/pdf/
Disallow /general/recipes/recipe.php
Disallow /how-to/call/loc_search.php
Disallow /movies/articles/
Disallow /res/
Disallow /redirect/
Disallow /search/
Disallow /share/
Disallow /travel/airports/search.php
Disallow /travel/indian-railway/search.php
Disallow /travel/hotels/ajax_search.php
Disallow /travel/hotels/imgs/
Disallow /travel/airlines/destination_search.php
Disallow /travel/airlines/airline_search.php
Disallow /travel/search.php
Disallow /what-is/meaning-of/notfound/
Disallow /*/cron/

Other Records

Field Value
sitemap https://www.prokerala.com/banking/sitemap-index.xml.gz
sitemap https://www.prokerala.com/bikes/sitemap-index.xml.gz
sitemap https://www.prokerala.com/news/photos/sitemap-index.xml.gz
sitemap https://www.prokerala.com/pincode/sitemap-index.xml.gz
sitemap https://www.prokerala.com/sitemap/daily.xml.gz
sitemap https://www.prokerala.com/videos/sitemap-index.xml.gz

Comments

  • AI Crawlers

Warnings

  • 2 invalid lines.