diyagupte.com
robots.txt

Robots Exclusion Standard data for diyagupte.com

Resource Scan

Scan Details

Site Domain diyagupte.com
Base Domain diyagupte.com
Scan Status Ok
Last Scan2024-06-11T01:48:17+00:00
Next Scan 2024-07-11T01:48:17+00:00

Last Scan

Scanned2024-06-11T01:48:17+00:00
URL https://diyagupte.com/robots.txt
Redirect https://www.diyagupte.com/robots.txt
Redirect Domain www.diyagupte.com
Redirect Base diyagupte.com
Domain IPs 2a02:4780:84:5298:97cd:ed9e:9d5a:f81c, 84.32.84.222
Redirect IPs 2a02:4780:39:5034:4c27:6971:dbcb:b57c, 84.32.84.57
Response IP 93.127.196.175
Found Yes
Hash 0b7a455c2b9847968481b55a40f8085bb5a77af5e4d5537e189a6e385c02028f
SimHash 2900d0e74e93

Groups

*

Rule Path
Allow /

googlebot

Rule Path
Allow /

slurp

Rule Path
Allow /

msnbot

Rule Path
Allow /

ia_archiver

Rule Path
Allow /

scrubby

Rule Path
Allow /

baiduspider

Rule Path
Allow /

httrack
netcaptor
offline explorer
spiderku/0.9
steeler
webcopier v3.3
webcopier v3.2a
webcopier
webcawler
web downloader/4.9
web downloader/5.8
webgather 3.0
webstripper/2.56
webzip/3.65
webzip
wget
zao
zeus 2.6

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.diyagupte.com/sitemap.xml