prokerala.net
robots.txt

Robots Exclusion Standard data for prokerala.net

Resource Scan

Scan Details

Site Domain prokerala.net
Base Domain prokerala.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-11-01T10:10:21+00:00
Next Scan 2025-01-30T10:10:21+00:00

Last Successful Scan

Scanned2023-10-09T05:43:43+00:00
URL https://prokerala.net/robots.txt
Redirect https://www.prokerala.com/robots.txt
Redirect Domain www.prokerala.com
Redirect Base prokerala.com
Domain IPs 104.21.59.150, 172.67.179.190, 2606:4700:3030::6815:3b96, 2606:4700:3030::ac43:b3be
Redirect IPs 96.17.180.182, 96.17.180.184
Response IP 184.28.235.59
Found Yes
Hash 2edf04c67097c0ed170268ea6d94e19c0acc6740679ed8d37e70057de8d124b2
SimHash cc3ef960a43b

Groups

googlebot-news

Rule Path
Allow /news/
Allow /astrology/
Allow /festivals/
Allow /$
Allow /?amp=1$
Disallow /

mediapartners-google

Rule Path
Allow /
Disallow

adsbot-google

Rule Path
Disallow

proximic

Rule Path
Allow /
Disallow

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

moget

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

scooperbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /
Allow /news/

yandex

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yeti

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

*

Rule Path
Allow /
Allow /res/images/
Disallow /account/
Disallow /api/
Disallow /astrology/location-search.php
Disallow /astrology/*panchang*/*.pdf$
Disallow /calendar/*.pdf$
Disallow /automobile/image-update.php
Disallow /automobile/news/
Disallow /automobile/search_details.php
Disallow /banking/ajax-search.php
Disallow /downloads/downloadfile.php
Disallow /education/imgs/email/
Disallow /general/calendar/date.php
Disallow /general/calendar/mal-date.php
Disallow /general/calendar/pdf/
Disallow /general/recipes/recipe.php
Disallow /how-to/call/loc_search.php
Disallow /movies/articles/
Disallow /res/
Disallow /redirect/
Disallow /search/
Disallow /share/
Disallow /travel/airports/search.php
Disallow /travel/indian-railway/search.php
Disallow /travel/hotels/ajax_search.php
Disallow /travel/hotels/imgs/
Disallow /travel/airlines/destination_search.php
Disallow /travel/airlines/airline_search.php
Disallow /travel/search.php
Disallow /what-is/meaning-of/notfound/
Disallow /*/cron/

Other Records

Field Value
sitemap https://www.prokerala.com/banking/sitemap-index.xml.gz
sitemap https://www.prokerala.com/bikes/sitemap-index.xml.gz
sitemap https://www.prokerala.com/news/photos/sitemap-index.xml.gz
sitemap https://www.prokerala.com/pincode/sitemap-index.xml.gz
sitemap https://www.prokerala.com/sitemap/daily.xml.gz
sitemap https://www.prokerala.com/videos/sitemap-index.xml.gz

Comments

  • AI Crawlers

Warnings

  • 2 invalid lines.