courses.fromminstitute.org
robots.txt

Robots Exclusion Standard data for courses.fromminstitute.org

Resource Scan

Scan Details

Site Domain courses.fromminstitute.org
Base Domain fromminstitute.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-15T06:19:31+00:00
Next Scan 2025-10-15T06:19:31+00:00

Last Successful Scan

Scanned2025-08-09T06:23:50+00:00
URL https://courses.fromminstitute.org/robots.txt
Domain IPs 162.159.128.53, 162.159.138.52
Response IP 162.159.128.53
Found Yes
Hash 323badad2c90b66a83c34d93d6f94889adcd8bbf56301f42d2629d56a54fe2e7
SimHash ba14f0003b99

Groups

botify
spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

mj12bot

Rule Path
Disallow /

*

Rule Path
Disallow /search?
Disallow /list?
Disallow /sign_in?
Disallow /sign_up?
Disallow /*?*filters=*
Disallow /*comments
Disallow /spaces/13197311*
Disallow /spaces/13262265*
Disallow /spaces/13263250*
Disallow /spaces/13872335*
Disallow /spaces/13262123*
Disallow /spaces/13263414*
Disallow /spaces/13866424*
Disallow /spaces/14101898*
Disallow /spaces/13872608*
Disallow /spaces/13864332*
Disallow /spaces/13864268*
Disallow /spaces/13864265*
Disallow /spaces/13864232*
Disallow /spaces/13263300*
Disallow /spaces/13872104*
Disallow /spaces/13197323*
Disallow /spaces/13895017*
Disallow /spaces/19634678*
Disallow /spaces/13262747*
Disallow /spaces/13872597*
Disallow /spaces/13864234*
Disallow /spaces/13872096*
Disallow /spaces/13872336*
Disallow /spaces/13263580*
Disallow /spaces/13864219*
Disallow /spaces/13864271*
Disallow /spaces/13262366*
Disallow /spaces/13864267*
Disallow /spaces/13864225*
Disallow /spaces/14489438*
Disallow /spaces/13197325*
Disallow /spaces/13872629*
Disallow /spaces/13863112*
Disallow /spaces/13864222*
Disallow /spaces/13872102*
Disallow /spaces/16597390*
Disallow /spaces/13197315*
Disallow /communities/

Other Records

Field Value
sitemap https://courses.fromminstitute.org/sitemap.xml