diyagupta.in
robots.txt

Robots Exclusion Standard data for diyagupta.in

Resource Scan

Scan Details

Site Domain diyagupta.in
Base Domain diyagupta.in
Scan Status Ok
Last Scan2025-06-10T07:38:03+00:00
Next Scan 2025-07-10T07:38:03+00:00

Last Scan

Scanned2025-06-10T07:38:03+00:00
URL https://diyagupta.in/robots.txt
Domain IPs 2a02:4780:84:d56d:4a23:a70d:a796:8eb7, 84.32.84.213
Response IP 191.101.228.8
Found Yes
Hash d473d32a621a7af1a384d84ac48add04858fae23b9f3411032c4e180ab25d81a
SimHash 1900d0e34e13

Groups

*

Rule Path
Allow /

googlebot

Rule Path
Allow /

slurp

Rule Path
Allow /

msnbot

Rule Path
Allow /

ia_archiver

Rule Path
Allow /

scrubby

Rule Path
Allow /

baiduspider

Rule Path
Allow /

httrack
netcaptor
offline explorer
spiderku/0.9
steeler
webcopier v3.3
webcopier v3.2a
webcopier
webcawler
web downloader/4.9
web downloader/5.8
webgather 3.0
webstripper/2.56
webzip/3.65
webzip
wget
zao
zeus 2.6

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.diyagupta.in/sitemap.xml