learninfreedom.org
robots.txt

Robots Exclusion Standard data for learninfreedom.org

Resource Scan

Scan Details

Site Domain learninfreedom.org
Base Domain learninfreedom.org
Scan Status Ok
Last Scan2025-05-16T16:39:18+00:00
Next Scan 2025-06-15T16:39:18+00:00

Last Scan

Scanned2025-05-16T16:39:18+00:00
URL https://learninfreedom.org/robots.txt
Domain IPs 216.92.69.31
Response IP 216.92.69.31
Found Yes
Hash 61d3441008bfec9a0a7a50e107a62fe5f983bdcf51b8745b67921ee187248bba
SimHash a00af8d4abb8

Groups

anawave
emailcollector
emailsiphon
emailwolf
extractorpro
flashsite
go-get-it
grab-a-site
hotcargo
httploader
memoweb
nearsite
netattache
parasite
radview
radview/httploader
second site
secondsite
sitesnagger
spidybot
teleport
teleport pro
visual web
visualweb
wbi_client
webcompass
webcopy
webdownloader
webretriever
websnake
webvcr
webwhacker
webzip
wget
twiceler

Rule Path
Disallow /

*

Rule Path
Disallow /admi
Disallow /guest
Disallow /cgi
Disallow /respons
Disallow /log
Disallow /error
Disallow /suggest
Disallow /comment
Disallow /improv
Disallow /new
Disallow /dag
Disallow /stu
Disallow /proj
Disallow /year

Other Records

Field Value
sitemap http://learninfreedom.org/sitemap.xml