Log In Sign Up

cafecityguide.website
robots.txt

Robots Exclusion Standard data for cafecityguide.website

Resource Scan

Scan Details

Site Domain cafecityguide.website
Base Domain cafecityguide.website
Scan Status Ok
Last Scan2025-07-12T07:42:07+00:00
Next Scan 2025-07-19T07:42:07+00:00

Last Scan

Scanned2025-07-12T07:42:07+00:00
URL https://cafecityguide.website/robots.txt
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.32.1
Found Yes
Hash 191af56c528c43a95158ce68292c195e06fc3e6a7feff4adb8f8adf0189dfc65
SimHash 18344d4ae68b

Groups

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

alexibot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

xenu’s

Rule Path
Disallow /

xenu’s link sleuth 1.1c

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

clickagy intelligence bot v2

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Back to top