womenguide.in
robots.txt

Robots Exclusion Standard data for womenguide.in

Resource Scan

Scan Details

Site Domain womenguide.in
Base Domain womenguide.in
Scan Status Ok
Last Scan2025-10-16T22:36:56+00:00
Next Scan 2025-10-23T22:36:56+00:00

Last Scan

Scanned2025-10-16T22:36:56+00:00
URL https://womenguide.in/robots.txt
Redirect https://www.womenguide.in/robots.txt
Redirect Domain www.womenguide.in
Redirect Base womenguide.in
Domain IPs 104.21.20.199, 172.67.194.31, 2606:4700:3030::6815:14c7, 2606:4700:3037::ac43:c21f
Redirect IPs 104.21.20.199, 172.67.194.31, 2606:4700:3030::6815:14c7, 2606:4700:3037::ac43:c21f
Response IP 172.67.194.31
Found Yes
Hash ca0f664ec3f66c9bc60f106b916f0db52b1d3facc09f3f70584ef935ae53f20b
SimHash 60544983c516

Groups

googlebot

Rule Path
Allow

googlebot-image

Rule Path
Allow

googlebot-mobile

Rule Path
Allow

*

Rule Path
Disallow /amp/

msnbot

Rule Path
Disallow

slurp

Rule Path
Disallow

teoma

Rule Path
Disallow

gigabot

Rule Path
Disallow

robozilla

Rule Path
Disallow

nutch

Rule Path
Disallow

ia_archiver

Rule Path
Disallow

baiduspider

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

psbot

Rule Path
Disallow

yahoo-blogs/v3.9

Rule Path
Disallow

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

kangaroo bot

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

cohere-training-data-crawler

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

webzio-extended

Rule Path
Disallow /

youbot

Rule Path
Disallow /