mimentor.org
robots.txt

Robots Exclusion Standard data for mimentor.org

Resource Scan

Scan Details

Site Domain mimentor.org
Base Domain mimentor.org
Scan Status Ok
Last Scan2024-11-11T05:39:44+00:00
Next Scan 2024-11-25T05:39:44+00:00

Last Scan

Scanned2024-11-11T05:39:44+00:00
URL https://www.mimentor.org/robots.txt
Domain IPs 162.159.128.53, 162.159.138.52
Response IP 162.159.138.52
Found Yes
Hash a4aa8a2589763cf20e339835e450e22d908b615287bf300eddc0355e568cef52
SimHash 2ad9c841ab93

Groups

botify
spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

mj12bot

Rule Path
Disallow /

*

Rule Path
Disallow /search?
Disallow /list?
Disallow /sign_in?
Disallow /sign_up?
Disallow /*?*filters=*
Disallow /*comments
Disallow /spaces/280915*
Disallow /spaces/16636795*
Disallow /spaces/11251228*
Disallow /spaces/5734407*
Disallow /spaces/235914*
Disallow /spaces/1838043*
Disallow /spaces/1407774*
Disallow /spaces/427607*
Disallow /spaces/659335*
Disallow /spaces/499167*
Disallow /spaces/4748268*
Disallow /spaces/342656*
Disallow /spaces/221004*
Disallow /spaces/659342*
Disallow /spaces/659349*
Disallow /spaces/276524*
Disallow /spaces/343519*
Disallow /spaces/762819*
Disallow /spaces/827107*
Disallow /spaces/2329573*
Disallow /spaces/7643687*
Disallow /spaces/11288418*
Disallow /spaces/7661383*
Disallow /spaces/225683*
Disallow /spaces/827072*
Disallow /spaces/593855*
Disallow /spaces/11519823*
Disallow /spaces/2518187*
Disallow /spaces/1658687*
Disallow /spaces/724946*
Disallow /communities/

Other Records

Field Value
sitemap https://www.mimentor.org/sitemap.xml