brainscape.com
robots.txt

Robots Exclusion Standard data for brainscape.com

Resource Scan

Scan Details

Site Domain brainscape.com
Base Domain brainscape.com
Scan Status Ok
Last Scan2024-10-10T19:57:38+00:00
Next Scan 2024-10-24T19:57:38+00:00

Last Scan

Scanned2024-10-10T19:57:38+00:00
URL https://brainscape.com/robots.txt
Redirect https://www.brainscape.com/robots.txt
Redirect Domain www.brainscape.com
Redirect Base brainscape.com
Domain IPs 35.172.97.227, 50.16.106.173
Redirect IPs 35.172.97.227, 50.16.106.173
Response IP 50.16.106.173
Found Yes
Hash 6709fdf43d1176499e4f275ea0a5ab3fd6ff5e0df658c686d3ffeaaf3a18685e
SimHash 925c7403f202

Groups

*

Rule Path
Disallow /cart/
Disallow /images/
Disallow /learn/search*
Disallow /market_screens_thumbs/
Disallow /market_screens/
Disallow /password_reset*
Disallow /profiles/
Disallow /sign_in*
Disallow /sign_up*
Disallow /terms*
Disallow /blog/category/
Disallow /blog/tag/

semrushbot
siteauditbot
rogerbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.brainscape.com/sitemaps/sitemap.xml
sitemap https://www.brainscape.com/academy/sitemap-posts.xml
sitemap https://www.brainscape.com/academy/sitemap-tags.xml

Comments

  • blocking bad bots