dresseye.me
robots.txt

Robots Exclusion Standard data for dresseye.me

Resource Scan

Scan Details

Site Domain dresseye.me
Base Domain dresseye.me
Scan Status Ok
Last Scan2026-02-21T05:57:28+00:00
Next Scan 2026-03-23T05:57:28+00:00

Last Scan

Scanned2026-02-21T05:57:28+00:00
URL https://dresseye.me/robots.txt
Domain IPs 34.225.213.201, 52.70.217.31
Response IP 34.225.213.201
Found Yes
Hash 1b7905dda768c190d75eaea0f6a5a6e9dc2598218de12068121d521fead2d9fc
SimHash fd56d9f6cb88

Groups

petalbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

claudebot
claude-user

Rule Path
Disallow /

anthropicbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

yandexbot
yandex*

Rule Path
Disallow /

aliyun*

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

bytespider*
zoominfobot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

applebot-extended*
ccbot*
diffbot*
friendlycrawler*
google-extended*
gptbot*
image2dataset*
imagesiftbot*
meta-externalagent*
timpibot*

Rule Path
Disallow /

ahrefsbot*
barkrowler*
blexbot*
dataforseobot
mj12bot*

Rule Path
Disallow /

awariobot*
bw/1.1*
dalvik/2.1.0*
dataprovider*
go-http-client*
googleother*
ioncrawl*
isscyberriskcrawler*
java*
mozlila*
news-please*
orbbot*
peer39_crawler*
python-requests*
scrapy*
velenpublicwebcrawler*
wp_is_mobile*
zoominfobot*

Rule Path
Disallow /

censysinspect*
expanse*
internet-measurement*

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap http://www.dresseye.com/sitemap.xml

Warnings

  • 4 invalid lines.