www3.clustrmaps.com
robots.txt

Robots Exclusion Standard data for www3.clustrmaps.com

Resource Scan

Scan Details

Site Domain www3.clustrmaps.com
Base Domain clustrmaps.com
Scan Status Ok
Last Scan2026-01-10T22:04:41+00:00
Next Scan 2026-02-09T22:04:41+00:00

Last Scan

Scanned2026-01-10T22:04:41+00:00
URL https://www3.clustrmaps.com/robots.txt
Redirect https://clustrmaps.com/robots.txt
Redirect Domain clustrmaps.com
Redirect Base clustrmaps.com
Domain IPs 104.20.22.84, 172.66.172.34, 2606:4700:10::6814:1654, 2606:4700:10::ac42:ac22
Redirect IPs 104.20.22.84, 172.66.172.34, 2606:4700:10::6814:1654, 2606:4700:10::ac42:ac22
Response IP 172.66.172.34
Found Yes
Hash 8e87852bc840cbb32e80d67f6ac70cf846039e8a401cf378ccb60bba155820db
SimHash 0b0b802677bb

Groups

*

Rule Path
Disallow /website_directory
Disallow /map_v2.png
Disallow /map_v3.png
Disallow /a/hm/
Disallow /a/jx/
Disallow /bl/tools/r
Disallow /bl/tools/bv
Disallow /bl/opt-out
Disallow /c/
Disallow /persons/i/
Disallow /bv/
Disallow /details/
Disallow /ajax/
Disallow /amp/
Disallow /person/jx/
Disallow /fips/
Disallow /widget_call_home.js
Disallow /globe_call_home.js

mj12bot

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

garlikcrawler/1.1 (http://garlik.com/, crawler@garlik.com)

Rule Path
Disallow /

linguee

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

Warnings

  • `host` is not a known field.