jpcycles.com
robots.txt

Robots Exclusion Standard data for jpcycles.com

Resource Scan

Scan Details

Site Domain jpcycles.com
Base Domain jpcycles.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-07-24T02:13:49+00:00
Next Scan 2024-10-22T02:13:49+00:00

Last Successful Scan

Scanned2023-09-29T00:57:37+00:00
URL https://jpcycles.com/robots.txt
Redirect https://www.jpcycles.com/robots.txt
Redirect Domain www.jpcycles.com
Redirect Base jpcycles.com
Domain IPs 104.18.38.128, 172.64.149.128, 2606:4700:4400::6812:2680, 2606:4700:4400::ac40:9580
Redirect IPs 104.18.38.128, 172.64.149.128, 2606:4700:4400::6812:2680, 2606:4700:4400::ac40:9580
Response IP 172.64.149.128
Found Yes
Hash aebc9b21b49c3d1b1a1f613908816ba88c987adfd11616d360862f2dc6845c6c
SimHash 6496f330d6b6

Groups

*

Rule Path
Allow /
Disallow /account
Disallow /cart
Disallow /checkout
Disallow /search
Disallow /api/
Disallow /auth/identity/callback
Disallow /auth/identity?state=*
Disallow /wish-list
Disallow /feedback$
Disallow /vehicle_selector
Disallow /fitment_notes/
Disallow /mobile_landing*
Disallow /videos
Allow /*.js*
Allow /*.css*
Allow /*.jpg*
Allow /*.gif*
Allow /*.png*

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

msnbot-media

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1.0

Other Records

Field Value
sitemap https://www.jpcycles.com/sitemap.xml