vahak.in
robots.txt

Robots Exclusion Standard data for vahak.in

Resource Scan

Scan Details

Site Domain vahak.in
Base Domain vahak.in
Scan Status Ok
Last Scan2024-09-23T11:32:01+00:00
Next Scan 2024-10-07T11:32:01+00:00

Last Scan

Scanned2024-09-23T11:32:01+00:00
URL https://vahak.in/robots.txt
Redirect https://www.vahak.in/robots.txt
Redirect Domain www.vahak.in
Redirect Base vahak.in
Domain IPs 76.76.21.21
Redirect IPs 76.76.21.9, 76.76.21.93
Response IP 76.76.21.61
Found Yes
Hash 55b424cc857246fe1ff8cb9dd72e3b6609d301fec21c077d5aa3b76f5853b8c8
SimHash 951648400ff8

Groups

adsbot-google

Rule Path
Disallow

googlebot

Rule Path
Disallow
Allow /blogs/*

*

Rule Path
Disallow /ac/*
Disallow /premium/*
Disallow /privacy-policy
Disallow /terms-and-conditions
Disallow /*-terms-and-conditions
Disallow /*-terms-and-condition

bytespider

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

filterdb.iss.net/crawler

Rule Path
Disallow /

headlesschrome

Rule Path
Disallow /

europarchive.org

Rule Path
Disallow /

duckduckbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.vahak.in/sitemap.xml