vogons.org
robots.txt

Robots Exclusion Standard data for vogons.org

Resource Scan

Scan Details

Site Domain vogons.org
Base Domain vogons.org
Scan Status Ok
Last Scan2024-05-26T01:56:55+00:00
Next Scan 2024-06-25T01:56:55+00:00

Last Scan

Scanned2024-05-26T01:56:55+00:00
URL https://vogons.org/robots.txt
Redirect https://www.vogons.org/robots.txt
Redirect Domain www.vogons.org
Redirect Base vogons.org
Domain IPs 2600:3c03:e000:397::1, 69.164.212.194
Redirect IPs 2600:3c03:e000:397::1, 69.164.212.194
Response IP 69.164.212.194
Found Yes
Hash 00c6b856e43c685be3c7cdd0881c47c5a503e3c1d6aa55efcf3b4eeb760d3d9c
SimHash d070f7227231

Groups

ahrefsbot

Rule Path
Disallow /

linkdex

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

linkdexbot/2.0

Rule Path
Disallow /

linkdexbot/2.1

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

linkfluence.com

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

yeti

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yak/1.0

Rule Path
Disallow /

yak

Rule Path
Disallow /

proximic

Rule Path
Disallow /