sitestory.dk
robots.txt

Robots Exclusion Standard data for sitestory.dk

Resource Scan

Scan Details

Site Domain sitestory.dk
Base Domain sitestory.dk
Scan Status Ok
Last Scan2024-09-30T16:40:33+00:00
Next Scan 2024-10-07T16:40:33+00:00

Last Scan

Scanned2024-09-30T16:40:33+00:00
URL https://sitestory.dk/robots.txt
Domain IPs 178.20.216.31
Response IP 178.20.216.31
Found Yes
Hash ff2848fcf449d28bf6975b4b40a67b8a026bac125707b4945592c7093d30f39a
SimHash 480af1808bf3

Groups

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexmedia

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

zillacrawler

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

kalooga

Rule Path
Disallow /

popscreenbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

netseer

Rule Path
Disallow /

proximic

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

psbot

Rule Path
Disallow /

obot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

imspider

Rule Path
Disallow /

yeti

Rule Path
Disallow /

beetlebot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

zookabot

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/
Disallow /error/