Log In Sign Up

envirosafetyproducts.com
robots.txt

Robots Exclusion Standard data for envirosafetyproducts.com

Resource Scan

Scan Details

Site Domain envirosafetyproducts.com
Base Domain envirosafetyproducts.com
Scan Status Ok
Last Scan2024-11-14T11:00:35+00:00
Next Scan 2024-11-28T11:00:35+00:00

Last Scan

Scanned2024-11-14T11:00:35+00:00
URL https://envirosafetyproducts.com/robots.txt
Redirect https://www.envirosafetyproducts.com/robots.txt
Redirect Domain www.envirosafetyproducts.com
Redirect Base envirosafetyproducts.com
Domain IPs 23.227.38.65
Redirect IPs 23.227.38.74, 2620:127:f00f:e::
Response IP 23.227.38.74
Found Yes
Hash f7dd03ea447cf759b6b51c6ea1ece5a6e11683dc136ac079f5e0b285b2262c17
SimHash 2f115b4266d8

Groups

*

Rule Path
Disallow /catalogsearch/
Disallow /review/
Disallow /static/*/frontend/Sm/market_child/en_US
Disallow /recommendations/products
Disallow /cartquickpro/catalog_product/
Disallow /service/catalog/product/view
Disallow /customer/account/login/referer/
Disallow /review/product/list/
Disallow /checkout/cart/
Disallow /media/*.jpg
Disallow /wpm*
Disallow /services/*
Disallow *.atom

*

Rule Path
Disallow /a/downloads/-/*
Disallow /admin
Disallow /cart
Disallow /orders
Disallow /checkouts/
Disallow /checkout
Disallow /67121283328/checkouts
Disallow /67121283328/orders
Disallow /carts
Disallow /account
Disallow /collections/*sort_by*
Disallow /*/collections/*sort_by*
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow */collections/*filter*%26*filter*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*?*oseid=*
Disallow /*preview_theme_id*
Disallow /*preview_script_id*
Disallow /policies/
Disallow /*/policies/
Disallow /*/*?*ls=*&ls=*
Disallow /*/*?*ls%3D*%3Fls%3D*
Disallow /*/*?*ls%3D*%3Fls%3D*
Disallow /search
Disallow /apple-app-site-association
Disallow /.well-known/shopify/monorail
Disallow /cdn/wpm/*.js
Disallow /recommendations/products
Disallow /*/recommendations/products

adsbot-google

Rule Path
Disallow /checkouts/
Disallow /checkout
Disallow /carts
Disallow /orders
Disallow /67121283328/checkouts
Disallow /67121283328/orders
Disallow /*?*oseid=*
Disallow /*preview_theme_id*
Disallow /*preview_script_id*
Disallow /cdn/wpm/*.js

nutch

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /a/downloads/-/*
Disallow /admin
Disallow /cart
Disallow /orders
Disallow /checkouts/
Disallow /checkout
Disallow /67121283328/checkouts
Disallow /67121283328/orders
Disallow /carts
Disallow /account
Disallow /collections/*sort_by*
Disallow /*/collections/*sort_by*
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow */collections/*filter*%26*filter*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*?*oseid=*
Disallow /*preview_theme_id*
Disallow /*preview_script_id*
Disallow /policies/
Disallow /*/policies/
Disallow /*/*?*ls=*&ls=*
Disallow /*/*?*ls%3D*%3Fls%3D*
Disallow /*/*?*ls%3D*%3Fls%3D*
Disallow /search
Disallow /apple-app-site-association
Disallow /.well-known/shopify/monorail
Disallow /cdn/wpm/*.js

Other Records

Field Value
crawl-delay 10

ahrefssiteaudit

Rule Path
Disallow /a/downloads/-/*
Disallow /admin
Disallow /cart
Disallow /orders
Disallow /checkouts/
Disallow /checkout
Disallow /67121283328/checkouts
Disallow /67121283328/orders
Disallow /carts
Disallow /account
Disallow /collections/*sort_by*
Disallow /*/collections/*sort_by*
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow /*/collections/*%2B*
Disallow */collections/*filter*%26*filter*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*/blogs/*%2B*
Disallow /*?*oseid=*
Disallow /*preview_theme_id*
Disallow /*preview_script_id*
Disallow /policies/
Disallow /*/policies/
Disallow /*/*?*ls=*&ls=*
Disallow /*/*?*ls%3D*%3Fls%3D*
Disallow /*/*?*ls%3D*%3Fls%3D*
Disallow /search
Disallow /apple-app-site-association
Disallow /.well-known/shopify/monorail
Disallow /cdn/wpm/*.js

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

semrushbot
pricespider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

msnbot-media
msnbot
bingbot-media
bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

duckduckbot
slurp
facebot
exabot
ia_archiver

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

ahrefsbot
ahrefssiteaudit
rogerbot
yandex
yandexbot
baidu
baiduspider
sogou
ninjacrawler

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.envirosafetyproducts.com/sitemap.xml
sitemap https://www.envirosafetyproducts.com/sitemap.xml
sitemap https://www.envirosafetyproducts.com/sitemap.xml
sitemap https://www.envirosafetyproducts.com/sitemap.xml
Back to top

Comments

  • Enable robots.txt rules for all crawlers
  • Adding Sitemaps for crawlers
  • Do not crawl seach pages and not-SEO optimized catalog links
  • Disallow: /catalog/product_compare/
  • Disallow: /catalog/category/view/
  • Disallow: /catalog/product
  • Disallow: /blogs/*/tagged/
  • Disallow: /productalert/add/price/
  • Disallow: /favouritelist/index/addfavouritelist/
  • Disallow: /blog/category/
  • Disallow: /brands/index/view/id/
  • Disallow: /blog/*/amp
  • Disallow: /catalog/product/view/
  • Disallow: /sendfriend/product/send/id/
  • Disallow: */page_action/aggregate
  • crawl delay
  • crawl delay for bing search
  • crawl delay
  • Block unwanted bot crawlers
Back to top