limburgsetaal.org
robots.txt

Robots Exclusion Standard data for limburgsetaal.org

Resource Scan

Scan Details

Site Domain limburgsetaal.org
Base Domain limburgsetaal.org
Scan Status Ok
Last Scan2025-09-15T05:56:09+00:00
Next Scan 2025-09-22T05:56:09+00:00

Last Scan

Scanned2025-09-15T05:56:09+00:00
URL https://limburgsetaal.org/robots.txt
Domain IPs 2a00:d640:d640:9999::2eeb:288b, 46.235.40.139
Response IP 46.235.40.139
Found Yes
Hash a51a9606218494f549ba48f3412fe14431a56912b156f44e351d4f6e6e28737a
SimHash 61846c4808f2

Groups

twiceler

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

ieautodiscovery

Rule Path
Disallow /

henrythemiragorobot

Rule Path
Disallow /

psbot

Rule Path
Disallow /

psycheclone

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

moozilla

Rule Path
Disallow /

seekbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

msfrontpage

Rule Path
Disallow /

gonzo1

Rule Path
Disallow /

gonzo2

Rule Path
Disallow /

francis/2.0

Rule Path
Disallow /

wwwster/1.4

Rule Path
Disallow /

gigabot/2.0

Rule Path
Disallow /

seekbot/1.0

Rule Path
Disallow /

nutchcvs/0.7.1

Rule Path
Disallow /

voyager/1.0

Rule Path
Disallow /

msfrontpage/3.0

Rule Path
Disallow /

snapbot/1.0

Rule Path
Disallow /

kopernikus t-online email 3.x

Rule Path
Disallow /

yahooseeker/m1a1-r2d2

Rule Path
Disallow /

kopernikus

Rule Path
Disallow /

wget/1.10.1

Rule Path
Disallow /

java/1.4.1_04

Rule Path
Disallow /

gonzo2[d]

Rule Path
Disallow /

gonzo1[p]

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

*

Rule Path
Disallow /admin/
Disallow /contrib/
Disallow /doc/
Disallow /lib/
Disallow /modules/
Disallow /plugins/
Disallow /scripts/
Disallow /tmp/