Log In Sign Up

parktool.com
robots.txt

Robots Exclusion Standard data for parktool.com

Resource Scan

Scan Details

Site Domain parktool.com
Base Domain parktool.com
Scan Status Ok
Last Scan2024-09-25T00:05:04+00:00
Next Scan 2024-10-09T00:05:04+00:00

Last Scan

Scanned2024-09-25T00:05:04+00:00
URL https://parktool.com/robots.txt
Redirect https://www.parktool.com/en-us/robots.txt
Redirect Domain www.parktool.com
Redirect Base parktool.com
Domain IPs 50.226.78.59
Redirect IPs 50.226.78.59
Response IP 50.226.78.59
Found Yes
Hash 24cb676c91b153b71d38e1500248d6fbb3cf2b277b37c77da3129f1dc7ece34e
SimHash 501c091274a0

Groups

anthropic-ai

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

brightedge crawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

censysinspect

Rule Path
Disallow /

expanse

Rule Path
Disallow /

internetmeasurement

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

adbeat

Rule Path
Disallow /

bw/1.1

Rule Path
Disallow /

bw/1.2

Rule Path
Disallow /

ezoicbot

Rule Path
Disallow /

geedoproductsearch

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

wf search/nutch

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

*

Rule Path
Disallow /craft
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/
Disallow /login/*
Disallow /search*
Disallow /school/locator?*
Disallow /library/*
Disallow /product/docs?*
Disallow /wta/*
Disallow /assets/img/form*
Disallow /assets/img/media*
Allow /cart/order
Disallow /cart*
Allow /user/register
Disallow /user*
Disallow *.rss
Back to top

Other Records

Field Value
sitemap https://www.parktool.com/en-us/sitemaps-1-sitemap.xml
sitemap https://www.parktool.com/en-int/sitemaps-1-sitemap.xml
Back to top

Comments

  • robots.txt for https://www.parktool.com/en-us/
  • live
  • block all AI bots
  • block SEO crawlers
  • security scanners
  • search engine crawlers
  • other bots and scrapers
  • don't allow web crawlers to index cpresources/ or vendor/
Back to top