Log In Sign Up

zeppy.io
robots.txt

Robots Exclusion Standard data for zeppy.io

Resource Scan

Scan Details

Site Domain zeppy.io
Base Domain zeppy.io
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-07-19T14:32:19+00:00
Next Scan 2024-10-17T14:32:19+00:00

Last Successful Scan

Scanned2023-11-30T12:31:25+00:00
URL https://zeppy.io/robots.txt
Domain IPs 104.26.10.142, 104.26.11.142, 172.67.71.211, 2606:4700:20::681a:a8e, 2606:4700:20::681a:b8e, 2606:4700:20::ac43:47d3
Response IP 104.26.10.142
Found Yes
Hash 91f7332d01672e28ff926ce9f175ef82d27ba963878e27e82bfc2c7033b2f0ab
SimHash 48545444c7f8

Groups

ia_archiver

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

coccocbot

Rule Path
Disallow /

adbeat_bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

clickagy

Rule Path
Disallow /

yandexmetrika

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

cukbot

Rule Path
Disallow /

yandexmobilebot

Rule Path
Disallow /

leikibot

Rule Path
Disallow /

yandexmobilebot

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

robots

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

yesupbot

Rule Path
Disallow /

who.is bot

Rule Path
Disallow /

ssearch_bot

Rule Path
Disallow /

mbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

sogou

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 80
Back to top

Warnings

  • 2 invalid lines.
Back to top