refmasters.app
robots.txt

Robots Exclusion Standard data for refmasters.app

Resource Scan

Scan Details

Site Domain refmasters.app
Base Domain refmasters.app
Scan Status Ok
Last Scan2025-10-03T11:12:28+00:00
Next Scan 2025-10-17T11:12:28+00:00

Last Scan

Scanned2025-10-03T11:12:28+00:00
URL https://refmasters.app/robots.txt
Domain IPs 147.185.161.77, 147.185.161.78
Response IP 147.185.161.77
Found Yes
Hash a4f0743563750b1611c65222c2fe08bc209bc7c30be19b7f3ac8da6cd77a05c9
SimHash fc0558121238

Groups

botify
spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

mj12bot

Rule Path
Disallow /

*

Rule Path
Disallow /search?
Disallow /list?
Disallow /sign_in?
Disallow /sign_up?
Disallow /*?*filters=*
Disallow /*comments
Disallow /spaces/17209622*
Disallow /spaces/20054299*
Disallow /spaces/17991349*
Disallow /spaces/20409724*
Disallow /spaces/19599186*
Disallow /spaces/15886652*
Disallow /spaces/20619137*
Disallow /spaces/18357076*
Disallow /spaces/18543220*
Disallow /spaces/17048141*
Disallow /spaces/20954385*
Disallow /spaces/15870005*
Disallow /spaces/15968782*
Disallow /spaces/14327436*
Disallow /spaces/16280455*
Disallow /spaces/16314085*
Disallow /spaces/17116189*
Disallow /spaces/17188649*
Disallow /spaces/16858427*
Disallow /spaces/14654528*
Disallow /spaces/17243728*
Disallow /spaces/18556379*
Disallow /spaces/17054522*
Disallow /spaces/19316447*
Disallow /spaces/18502733*
Disallow /spaces/19743300*
Disallow /spaces/18880917*
Disallow /spaces/19888548*
Disallow /spaces/16511161*
Disallow /spaces/17195215*
Disallow /spaces/17109667*
Disallow /spaces/14573842*
Disallow /spaces/16256371*
Disallow /spaces/16104940*
Disallow /spaces/15905500*
Disallow /spaces/17345012*
Disallow /spaces/15977061*
Disallow /spaces/21204431*
Disallow /spaces/16502575*
Disallow /spaces/20413615*
Disallow /spaces/20055323*
Disallow /spaces/21065880*
Disallow /spaces/19984968*
Disallow /spaces/18459805*
Disallow /spaces/20727221*
Disallow /spaces/21250781*
Disallow /spaces/19903667*
Disallow /spaces/18522898*
Disallow /spaces/16666866*
Disallow /spaces/20039297*
Disallow /spaces/20191250*
Disallow /communities/

Other Records

Field Value
sitemap https://refmasters.app/sitemap.xml