Log In Sign Up

shipparts.com
robots.txt

Robots Exclusion Standard data for shipparts.com

Resource Scan

Scan Details

Site Domain shipparts.com
Base Domain shipparts.com
Scan Status Ok
Last Scan2024-09-30T16:33:46+00:00
Next Scan 2024-10-30T16:33:46+00:00

Last Scan

Scanned2024-09-30T16:33:46+00:00
URL https://shipparts.com/robots.txt
Redirect https://www.shipparts.com/robots.txt
Redirect Domain www.shipparts.com
Redirect Base shipparts.com
Domain IPs 8.210.195.187
Redirect IPs 8.210.195.187
Response IP 8.210.195.187
Found Yes
Hash ed0685c1384e7b72e03e56d790612e613d44efae6d36c92ac7ec22d865feb09c
SimHash 5036184f912e

Groups

*

Rule Path
Disallow /bin/

acoonbot

Rule Path
Disallow /

apachebench

Rule Path
Disallow /

applebot

Rule Path
Disallow /

asktbfxtv

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

borneobot

Rule Path
Disallow /

chinasospider

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

connotate.com

Rule Path
Disallow /

contentgrabber.com

Rule Path
Disallow /

coolpadwebkit

Rule Path
Disallow /

crawldaddy

Rule Path
Disallow /

curl

Rule Path
Disallow /

cyotek.com

Rule Path
Disallow /

dexi.io

Rule Path
Disallow /

digext

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

edisterbot

Rule Path
Disallow /

elefent

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

feeddemon

Rule Path
Disallow /

feedly

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

httpclient

Rule Path
Disallow /

httrack.com

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

indy

Rule Path
Disallow /

jaunty

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

jorgee

Rule Path
Disallow /

library

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

obot

Rule Path
Disallow /

octoparse.com

Rule Path
Disallow /

parsehub.com

Rule Path
Disallow /

perl

Rule Path
Disallow /

psbot

Rule Path
Disallow /

purebot

Rule Path
Disallow /

python

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

scrapinghub.com

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

sourceforge.net

Rule Path
Disallow /

spbot

Rule Path
Disallow /

spiderman

Rule Path
Disallow /

spinn3r.com

Rule Path
Disallow /

swebot

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

turnitinbot-agent

Rule Path
Disallow /

uipath.com

Rule Path
Disallow /

universalfeedparser

Rule Path
Disallow /

wangidspider

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

webdup

Rule Path
Disallow /

webharvy.com

Rule Path
Disallow /

wget

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wsanalyzer

Rule Path
Disallow /

xbfmozilla

Rule Path
Disallow /

xenu

Rule Path
Disallow /

yottaa

Rule Path
Disallow /

yunguance

Rule Path
Disallow /

yyspider

Rule Path
Disallow /

zmeu

Rule Path
Disallow /
Back to top

Comments

  • robots.txt for https://www.shipparts.com
Back to top

Warnings

  • 2 invalid lines.
Back to top