Log In Sign Up

juniperresearch.com
robots.txt

Robots Exclusion Standard data for juniperresearch.com

Resource Scan

Scan Details

Site Domain juniperresearch.com
Base Domain juniperresearch.com
Scan Status Ok
Last Scan2024-09-14T05:00:31+00:00
Next Scan 2024-10-14T05:00:31+00:00

Last Scan

Scanned2024-09-14T05:00:31+00:00
URL https://juniperresearch.com/robots.txt
Redirect https://www.juniperresearch.com/robots.txt
Redirect Domain www.juniperresearch.com
Redirect Base juniperresearch.com
Domain IPs 192.124.249.169
Redirect IPs 192.124.249.169
Response IP 192.124.249.169
Found Yes
Hash f62d59419905aef78710660f005d11ca3b1bef98e5d6016b3c925cdc1dd22c34
SimHash 09365b71c393

Groups

*

Rule Path
Disallow /umbraco/
Disallow /umbraco/*

*

Rule Path
Disallow /App_Data/
Disallow /App_Plugins/
Disallow /bin/
Disallow /config/
Disallow /css/
Disallow /js/

*

Rule Path
Disallow /web.config

*

Rule Path
Allow /sitemap.xml

dotbot

Rule Path
Disallow /

mj12bot.

Rule Path
Disallow /

turnitinbot.

Rule Path
Disallow /

baiduspider
baiduspider
baiduspider+

Rule Path
Disallow /

sogou

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.juniperresearch.com/sitemap.xml
sitemap https://www.juniperresearch.com/sitemap.xml
Back to top

Comments

  • Disallow crawling of Umbraco backoffice
  • Disallow crawling of specific folders
  • Disallow crawling of specific files
  • Allow crawling of sitemap
  • Sitemap path
  • Disallow bots
Back to top