myprimarket.com
robots.txt

Robots Exclusion Standard data for myprimarket.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	myprimarket.com
Base Domain	myprimarket.com
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Server returned a server error.
Last Scan	2024-10-01T06:23:24+00:00
Next Scan	2024-10-31T06:23:24+00:00

Last Successful Scan

Scanned	2024-08-10T11:43:41+00:00
URL	https://myprimarket.com/robots.txt
Redirect	https://www.myprimarket.com/robots.txt
Redirect Domain	www.myprimarket.com
Redirect Base	myprimarket.com
Domain IPs	104.21.77.44, 172.67.204.120, 2606:4700:3030::6815:4d2c, 2606:4700:3037::ac43:cc78
Redirect IPs	104.21.77.44, 172.67.204.120, 2606:4700:3030::6815:4d2c, 2606:4700:3037::ac43:cc78
Response IP	172.67.204.120
Found	Yes
Hash	e70fdd20c708a6a6d90386cb9cfe6e6ed7e6f4fd110b7b50c2c9371e1b5602a5
SimHash	621e537bced4

Groups

*

Rule	Path
Disallow	/add-to-cart%3D
Disallow	/add_to_wishlist%3D
Disallow	/wp-admin/
Allow	/wp-admin/admin-ajax.php
Disallow	/thanks/
Disallow	/*page/
Disallow	/author/
Disallow	/category/
Disallow	/cart/
Disallow	/checkout/
Disallow	/my-account/
Disallow	/portfolio_page/
Disallow	/orders/
Disallow	/search
Disallow	/wishlist/

Rule

Path

Disallow

/*add-to-cart%3D*

Disallow

/*add_to_wishlist%3D*

Disallow

/wp-admin/

Allow

/wp-admin/admin-ajax.php

Disallow

/thanks/

Disallow

/*page/

Disallow

/author/

Disallow

/category/

Disallow

/cart/

Disallow

/checkout/

Disallow

/my-account/

Disallow

/portfolio_page/

Disallow

/orders/

Disallow

/search

Disallow

/wishlist/

googlebot

Rule	Path
Disallow

Rule

Path

Disallow

Other Records

Field	Value
crawl-delay	5

Field

Value

crawl-delay

googlebot-image

Rule	Path
Disallow

Rule

Path

Disallow

googlebot-mobile

Rule	Path
Allow	/

Rule

Path

Allow

googlebot-news

Rule	Path
Allow	/

Rule

Path

Allow

googlebot-video

Rule	Path
Allow	/

Rule

Path

Allow

storebot-google

Rule	Path
Allow	/

Rule

Path

Allow

adsbot-google

Rule	Path
Allow	/

Rule

Path

Allow

bingbot

Rule	Path
Allow	/

Rule

Path

Allow

msnbot

Rule	Path
Allow	/

Rule

Path

Allow

msnbot-media

Rule	Path
Allow	/

Rule

Path

Allow

adldxbot

Rule	Path
Allow	/

Rule

Path

Allow

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

duggmirror

Rule	Path
Disallow	/

Rule

Path

Disallow

googlebot

Rule	Path
Disallow	/*.inc$
Disallow	/*.cgi$
Disallow	/*.wmv$
Disallow	/*.ico$
Disallow	/*.opml$
Disallow	/*.shtml$
Disallow	/*.cgi$
Disallow	/*.xhtml$

Rule

Path

Disallow

/*.inc$

Disallow

/*.cgi$

Disallow

/*.wmv$

Disallow

/*.ico$

Disallow

/*.opml$

Disallow

/*.shtml$

Disallow

/*.cgi$

Disallow

/*.xhtml$

mediapartners-google*

Rule	Path
Disallow
Allow	/*

Rule

Path

Disallow

Allow

yandexbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

slurp

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

duckduckbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

baiduspider

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

baiduspider-image

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

baiduspider-mobile

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

baiduspider-news

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

baiduspider-video

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

sogou

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

soso spider

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

exabot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

sogou spider

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

seekportbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

facebook external hit

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	1

Field

Value

crawl-delay

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefs

Rule	Path
Disallow	/

Rule

Path

Disallow

majestic

Rule	Path
Disallow	/

Rule

Path

Disallow

rogerbot

Rule	Path
Disallow	/

Rule

Path

Disallow

babbar.tech

Rule	Path
Disallow	/

Rule

Path

Disallow

babbar

Rule	Path
Disallow	/

Rule

Path

Disallow

geedobot

Rule	Path
Disallow	/

Rule

Path

Disallow

ninja

Rule	Path
Disallow	/

Rule

Path

Disallow

ninjabot

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

zoominfobot

Rule	Path
Disallow	/

Rule

Path

Disallow

aspiegelbot

Rule	Path
Disallow	/

Rule

Path

Disallow

colocrossing

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-sa

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule

Path

Disallow

sentibot

Rule

Path

Disallow

ccbot

Rule

Path

Disallow

asterias

Rule

Path

Disallow

backdoorbot/1.0

Rule

Path

Disallow

black hole

Rule

Path

Disallow

blowfish/1.0

Rule

Path

Disallow

botalot

Rule

Path

Disallow

builtbottough

Rule

Path

Disallow

bullseye/1.0

Rule

Path

Disallow

bunnyslippers

Rule

Path

Disallow

bhcbot

Rule

Path

Disallow

cegbfeieh

Rule

Path

Disallow

cheesebot

Rule

Path

Disallow

cherrypicker

Rule

Path

Disallow

cherrypickerelite/1.0

Rule

Path

Disallow

cherrypickerse/1.0

Rule

Path

Disallow

copyrightcheck

Rule

Path

Disallow

cosmos

Rule

Path

Disallow

crescent

Rule

Path

Disallow

crescent internet toolpak http ole control v.1.0

Rule

Path

Disallow

demandbase

Rule

Path

Disallow

cliqzbot

Rule

Path

Disallow

botscout

Rule

Path

Disallow

dittospyder

Rule

Path

Disallow

emailcollector

Rule

Path

Disallow

emailsiphon

Rule

Path

Disallow

emailwolf

Rule

Path

Disallow

erocrawler

Rule

Path

Disallow

extractorpro

Rule

Path

Disallow

foobot

Rule

Path

Disallow

harvest/1.5

Rule

Path

Disallow

hloader

Rule

Path

Disallow

httplib

Rule

Path

Disallow

humanlinks

Rule

Path

Disallow

infonavirobot

Rule

Path

Disallow

jennybot

Rule

Path

Disallow

kenjin spider

Rule

Path

Disallow

keyword density/0.9

Rule

Path

Disallow

lexibot

Rule

Path

Disallow

libweb/clshttp

Rule

Path

Disallow

linkextractorpro

Rule

Path

Disallow

linkscan/8.1a unix

Rule

Path

Disallow

linkwalker

Rule

Path

Disallow

lnspiderguy

Rule

Path

Disallow

lwp-trivial

Rule

Path

Disallow

lwp-trivial/1.34

Rule

Path

Disallow

mata hari

Rule

Path

Disallow

microsoft url control - 5.01.4511

Rule

Path

Disallow

microsoft url control - 6.00.8169

Rule

Path

Disallow

miixpc

Rule

Path

Disallow

miixpc/4.2

Rule

Path

Disallow

mister pix

Rule

Path

Disallow

moget

Rule

Path

Disallow

moget/2.1

Rule

Path

Disallow

mozilla/4

Rule

Path

Disallow

surdotlybot

Rule

Path

Disallow

tweetmemebot/4.0

Rule

Path

Disallow

mozilla/4.0 (compatible; msie 4.0; windows 98)

Rule

Path

Disallow

mozilla/4.0 (compatible; msie 4.0; windows nt)

Rule

Path

Disallow

mozilla/4.0 (compatible; msie 4.0; windows xp)

Rule

Path

Disallow

mozilla/4.0 (compatible; msie 4.0; windows 2000)

Rule

Path

Disallow

golabbot

Rule

Path

Disallow

cliqzbot

Rule

Path

Disallow

netants

Rule

Path

Disallow

nicerspro

Rule

Path

Disallow

offline explorer

Rule

Path

Disallow

openfind

Rule

Path

Disallow

openfind data gathere

Rule

Path

Disallow

propowerbot/2.14

Rule

Path

Disallow

prowebwalker

Rule

Path

Disallow

queryn metasearch

Rule

Path

Disallow

repomonkey

Rule

Path

Disallow

repomonkey bait & tackle/v1.01

Rule

Path

Disallow

rma

Rule

Path

Disallow

sitesnagger

Rule

Path

Disallow

spankbot

Rule

Path

Disallow

spanner

Rule

Path

Disallow

suzuran

Rule

Path

Disallow

szukacz/1.4

Rule

Path

Disallow

teleport

Rule

Path

Disallow

teleportpro

Rule

Path

Disallow

telesoft

Rule

Path

Disallow

the intraformant

Rule

Path

Disallow

thenomad

Rule

Path

Disallow

tighttwatbot

Rule

Path

Disallow

titan

Rule

Path

Disallow

getintent

Rule

Path

Disallow

gigabot

Rule

Path

Disallow

ltx71

Rule

Path

Disallow

tocrawl/urldispatcher

Rule

Path

Disallow

true_robot

Rule

Path

Disallow

true_robot/1.0

Rule

Path

Disallow

turingos

Rule

Path

Disallow

hybridbot

Rule

Path

Disallow

proximic

Rule

Path

Disallow

grapeshotcrawler

Rule

Path

Disallow

mail.ru_bot/2.0

Rule

Path

Disallow

urly warning

Rule

Path

Disallow

vci

Rule

Path

Disallow

vci webviewer vci webviewer win32

Rule

Path

Disallow

web image collector

Rule

Path

Disallow

webauto

Rule

Path

Disallow

webbandit

Rule

Path

Disallow

webbandit/3.50

Rule

Path

Disallow

webcopier

Rule

Path

Disallow

webenhancer

Rule

Path

Disallow

webmasterworldforumbot

Rule

Path

Disallow

websauger

Rule

Path

Disallow

website quester

Rule

Path

Disallow

webster pro

Rule

Path

Disallow

webstripper

Rule

Path

Disallow

webzip

Rule

Path

Disallow

webzip/4.0

Rule

Path

Disallow

wget

Rule

Path

Disallow

wget/1.5.3

Rule

Path

Disallow

wget/1.6

Rule

Path

Disallow

www-collector-e

Rule

Path

Disallow

xenu&

Product

Comment

xenu&

039;s

Rule

Path

Disallow

xenu&

Product

Comment

xenu&

039;s Link Sleuth 1.1c

Rule

Path

Disallow

zeus

Rule

Path

Disallow

zeus 32297 webster pro v2.9 win32

Rule

Path

Disallow

spbot

Rule

Path

Disallow

tweetmemebot

Rule

Path

Disallow

xovibot

Rule

Path

Disallow

femtosearchbot

Rule

Path

Disallow

yoozbot

Rule

Path

Disallow

awariorssbot

Rule

Path

Disallow

linkdexbot

Rule

Path

Disallow

kraken/0.1

Rule

Path

Disallow

openhosebot/2.1

Rule

Path

Disallow

mediatoolkitbot

Rule

Path

Disallow

grapeshot

Rule

Path

Disallow

obot/2.3.1

Rule

Path

Disallow

vagabondo/4.0

Rule

Path

Disallow

deusu

Rule

Path

Disallow

deepcrawl

Rule

Path

Disallow

safednsbot

Rule

Path

Disallow

seznambot/3.2

Rule

Path

Disallow

plukkie

Rule

Path

Disallow

spbot/4.4.2

Rule

Path

Disallow

flipboardproxy/1.1

Rule

Path

Disallow

focuseekbot

Rule

Path

Disallow

blexbot

Rule

Path

Disallow

queryseekerspider

Rule

Path

Disallow

smtbot

Rule

Path

Disallow

everyonesocialbot

Rule

Path

Disallow

ranksonicbot

Rule

Path

Disallow

ranksonicsiteauditor

Rule

Path

Disallow

bubing

Rule

Path

Disallow

voltron

Rule

Path

Disallow

linguee

Rule

Path

Disallow

crmnlcrawlagent

Rule

Path

Disallow

blp_bbot

Rule

Path

Disallow

blp_bbot/0.1

Rule

Path

Disallow

blp_bbot/0.1

Rule

Path

Disallow

dataforseobot

Rule

Path

Disallow

dataforseobot/1.0

Rule

Path

Disallow

bytespider

Rule

Path

Disallow

amazonbot

Rule

Path

Disallow

Other Records

Field

Value

sitemap

https://www.myprimarket.com/sitemap_index.xml

sitemap

https://www.myprimarket.com/product-sitemap1.xml

sitemap

https://www.myprimarket.com/product-sitemap2.xml

sitemap

https://www.myprimarket.com/product-sitemap3.xml

sitemap

https://www.myprimarket.com/product-sitemap4.xml

sitemap

https://www.myprimarket.com/product-sitemap5.xml

sitemap

https://www.myprimarket.com/product-sitemap6.xml

sitemap

https://www.myprimarket.com/product-sitemap7.xml

sitemap

https://www.myprimarket.com/he/sitemap_index.xml

sitemap

https://www.myprimarket.com/de/sitemap_index.xml

sitemap

https://www.myprimarket.com/ar/sitemap_index.xml

sitemap

https://www.myprimarket.com/be/sitemap_index.xml

sitemap

https://www.myprimarket.com/bg/sitemap_index.xml

sitemap

https://www.myprimarket.com/cs/sitemap_index.xml

sitemap

https://www.myprimarket.com/nl/sitemap_index.xml

sitemap

https://www.myprimarket.com/et/sitemap_index.xml

sitemap

https://www.myprimarket.com/fi/sitemap_index.xml

sitemap

https://www.myprimarket.com/id/sitemap_index.xml

sitemap

https://www.myprimarket.com/ga/sitemap_index.xml

sitemap

https://www.myprimarket.com/kk/sitemap_index.xml

sitemap

https://www.myprimarket.com/ko/sitemap_index.xml

sitemap

https://www.myprimarket.com/lt/sitemap_index.xml

sitemap

https://www.myprimarket.com/mk/sitemap_index.xml

sitemap

https://www.myprimarket.com/ms/sitemap_index.xml

sitemap

https://www.myprimarket.com/ro/sitemap_index.xml

sitemap

https://www.myprimarket.com/sk/sitemap_index.xml

sitemap

https://www.myprimarket.com/sl/sitemap_index.xml

sitemap

https://www.myprimarket.com/es/sitemap_index.xml

sitemap

https://www.myprimarket.com/sv/sitemap_index.xml

sitemap

https://www.myprimarket.com/th/sitemap_index.xml

sitemap

https://www.myprimarket.com/uk/sitemap_index.xml

sitemap

https://www.myprimarket.com/no/sitemap_index.xml

sitemap

https://www.myprimarket.com/cy/sitemap_index.xml

sitemap

https://www.myprimarket.com/ru/sitemap_index.xml

sitemap

https://www.myprimarket.com/tr/sitemap_index.xml

sitemap

https://www.myprimarket.com/vi/sitemap_index.xml

sitemap

https://www.myprimarket.com/sr/sitemap_index.xml

sitemap

https://www.myprimarket.com/pt/sitemap_index.xml

sitemap

https://www.myprimarket.com/pl/sitemap_index.xml

sitemap

https://www.myprimarket.com/lv/sitemap_index.xml

sitemap

https://www.myprimarket.com/lb/sitemap_index.xml

sitemap

https://www.myprimarket.com/ja/sitemap_index.xml

sitemap

https://www.myprimarket.com/it/sitemap_index.xml

sitemap

https://www.myprimarket.com/is/sitemap_index.xml

sitemap

https://www.myprimarket.com/hu/sitemap_index.xml

sitemap

https://www.myprimarket.com/hi/sitemap_index.xml

sitemap

https://www.myprimarket.com/el/sitemap_index.xml

sitemap

https://www.myprimarket.com/ka/sitemap_index.xml

sitemap

https://www.myprimarket.com/fr/sitemap_index.xml

sitemap

https://www.myprimarket.com/da/sitemap_index.xml

sitemap

https://www.myprimarket.com/hr/sitemap_index.xml

sitemap

https://www.myprimarket.com/bs/sitemap_index.xml

sitemap

https://www.myprimarket.com/az/sitemap_index.xml

sitemap

https://www.myprimarket.com/hy/sitemap_index.xml

sitemap

https://www.myprimarket.com/sq/sitemap_index.xml

sitemap

https://www.myprimarket.com/af/sitemap_index.xml

Comments

disallow archiving site
disable duggmirror
disallow all files ending with these extensions
Google AdSense
Slow down a bit
And block bots that keep hitting us too hard / do not obey the disallow/nofollows:
SEO by Myprimarket.com

myprimarket.comrobots.txt

Resource Scan

Scan Details

Last Successful Scan

Groups

*

googlebot

Other Records

googlebot-image

googlebot-mobile

googlebot-news

googlebot-video

storebot-google

adsbot-google

bingbot

msnbot

msnbot-media

adldxbot

ia_archiver

duggmirror

googlebot

mediapartners-google*

yandexbot

Other Records

slurp

Other Records

duckduckbot

Other Records

baiduspider

Other Records

baiduspider-image

Other Records

baiduspider-mobile

Other Records

baiduspider-news

Other Records

baiduspider-video

Other Records

sogou

Other Records

soso spider

Other Records

exabot

Other Records

sogou spider

Other Records

seekportbot

Other Records

facebook external hit

Other Records

pinterest

Other Records

dotbot

ahrefsbot

ahrefs

majestic

rogerbot

babbar.tech

babbar

geedobot

ninja

ninjabot

petalbot

zoominfobot

aspiegelbot

colocrossing

mj12bot

semrushbot-sa

semrushbot

sentibot

ccbot

asterias

backdoorbot/1.0

black hole

blowfish/1.0

botalot

builtbottough

bullseye/1.0

bunnyslippers

bhcbot

myprimarket.com
robots.txt

repomonkey bait & tackle/v1.01