www2.clustrmaps.com
robots.txt

Robots Exclusion Standard data for www2.clustrmaps.com

Resource Scan

Scan Details

Site Domain www2.clustrmaps.com
Base Domain clustrmaps.com
Scan Status Ok
Last Scan2025-05-13T22:48:41+00:00
Next Scan 2025-06-12T22:48:41+00:00

Last Scan

Scanned2025-05-13T22:48:41+00:00
URL https://www2.clustrmaps.com/robots.txt
Redirect https://clustrmaps.com/robots.txt
Redirect Domain clustrmaps.com
Redirect Base clustrmaps.com
Domain IPs 104.22.72.194, 104.22.73.194, 172.67.43.119, 2606:4700:10::6816:48c2, 2606:4700:10::6816:49c2, 2606:4700:10::ac43:2b77
Redirect IPs 104.22.72.194, 104.22.73.194, 172.67.43.119, 2606:4700:10::6816:48c2, 2606:4700:10::6816:49c2, 2606:4700:10::ac43:2b77
Response IP 104.22.73.194
Found Yes
Hash 8e87852bc840cbb32e80d67f6ac70cf846039e8a401cf378ccb60bba155820db
SimHash 0b0b802677bb

Groups

*

Rule Path
Disallow /website_directory
Disallow /map_v2.png
Disallow /map_v3.png
Disallow /a/hm/
Disallow /a/jx/
Disallow /bl/tools/r
Disallow /bl/tools/bv
Disallow /bl/opt-out
Disallow /c/
Disallow /persons/i/
Disallow /bv/
Disallow /details/
Disallow /ajax/
Disallow /amp/
Disallow /person/jx/
Disallow /fips/
Disallow /widget_call_home.js
Disallow /globe_call_home.js

mj12bot

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

garlikcrawler/1.1 (http://garlik.com/, crawler@garlik.com)

Rule Path
Disallow /

linguee

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

Warnings

  • `host` is not a known field.