Log In Sign Up

tenaquip.com
robots.txt

Robots Exclusion Standard data for tenaquip.com

Resource Scan

Scan Details

Site Domain tenaquip.com
Base Domain tenaquip.com
Scan Status Ok
Last Scan2024-09-22T15:34:12+00:00
Next Scan 2024-10-22T15:34:12+00:00

Last Scan

Scanned2024-09-22T15:34:12+00:00
URL https://tenaquip.com/robots.txt
Redirect https://www.tenaquip.com/robots.txt
Redirect Domain www.tenaquip.com
Redirect Base tenaquip.com
Domain IPs 104.18.16.59, 104.18.17.59, 2606:4700::6812:103b, 2606:4700::6812:113b
Redirect IPs 104.18.16.59, 104.18.17.59, 2606:4700::6812:103b, 2606:4700::6812:113b
Response IP 104.18.16.59
Found Yes
Hash d81e1ffa83abaeb87f2d8a2a51b7e13689a97934eda1bb5a1d71220421dd3cec
SimHash 733cc0e1cc18

Groups

ravencrawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

alphabot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

velenpublicwebcrawler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

amazonbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

bingbot
msnbot
msnbot-media
adidxbot
bingpreview
linkedinbot
adroll dynamic creative feed parser bot (delight@adroll.com)

Rule Path
Allow /category/*?*
Allow /categorie/*?*
Allow /brand/*?*
Allow /marque/*?*

Other Records

Field Value
crawl-delay 60

semrushbot-si
semrushbot
siteauditbot-desktop
siteauditbot-mobile

Rule Path
Allow /product/*?*
Allow /produit/*?*

Other Records

Field Value
crawl-delay 30

storebot-google

Rule Path
Allow /shop/*?*
Allow /magasin/*?*

Other Records

Field Value
crawl-delay 60

*

Rule Path
Disallow /category/*?*
Disallow /categorie/*?*
Disallow /brand/*?*
Disallow /marque/*?*
Disallow /campaign/*?*
Disallow /campagne/*?*
Disallow /account/login?*
Disallow /rechercher
Disallow /search
Allow /*?page=*

Other Records

Field Value
crawl-delay 60
Back to top

Other Records

Field Value
sitemap /tenaquip/sitemap/sitemap-index.xml
Back to top

Comments

  • Deny all query params
  • Deny search pages entirely
  • Allow navigation pages
  • Allow refinement by manufacturer
  • UPDATE 04-29-24, DON't ALLOW MANUFACT REFINE
  • Allow: /*?manufacturer=*
  • Allow: /*?fabricant=*
Back to top