learninfreedom.org
robots.txt
Robots Exclusion Standard data for learninfreedom.org
Resource Scan
Scan Details
Site Domain | learninfreedom.org |
Base Domain | learninfreedom.org |
Scan Status | Ok |
Last Scan | 2025-05-16T16:39:18+00:00 |
Next Scan | 2025-06-15T16:39:18+00:00 |
Last Scan
Scanned | 2025-05-16T16:39:18+00:00 |
URL | https://learninfreedom.org/robots.txt |
Domain IPs | 216.92.69.31 |
Response IP | 216.92.69.31 |
Found | Yes |
Hash | 61d3441008bfec9a0a7a50e107a62fe5f983bdcf51b8745b67921ee187248bba |
SimHash | a00af8d4abb8 |
Groups
anawave
emailcollector
emailsiphon
emailwolf
extractorpro
flashsite
go-get-it
grab-a-site
hotcargo
httploader
memoweb
nearsite
netattache
parasite
radview
radview/httploader
second site
secondsite
sitesnagger
spidybot
teleport
teleport pro
visual web
visualweb
wbi_client
webcompass
webcopy
webdownloader
webretriever
websnake
webvcr
webwhacker
webzip
wget
twiceler
Rule | Path |
---|---|
Disallow | / |
*
Rule | Path |
---|---|
Disallow | /admi |
Disallow | /guest |
Disallow | /cgi |
Disallow | /respons |
Disallow | /log |
Disallow | /error |
Disallow | /suggest |
Disallow | /comment |
Disallow | /improv |
Disallow | /new |
Disallow | /dag |
Disallow | /stu |
Disallow | /proj |
Disallow | /year |
Other Records
Field | Value |
---|---|
sitemap | http://learninfreedom.org/sitemap.xml |