machine-solution.com
robots.txt

Robots Exclusion Standard data for machine-solution.com

Resource Scan

Scan Details

Site Domain machine-solution.com
Base Domain machine-solution.com
Scan Status Ok
Last Scan2024-06-10T09:10:24+00:00
Next Scan 2024-07-10T09:10:24+00:00

Last Scan

Scanned2024-06-10T09:10:24+00:00
URL https://machine-solution.com/robots.txt
Domain IPs 104.16.39.93, 104.16.40.93, 104.16.41.93, 104.16.42.93, 104.16.43.93, 2606:4700::6810:275d, 2606:4700::6810:285d, 2606:4700::6810:295d, 2606:4700::6810:2a5d, 2606:4700::6810:2b5d
Response IP 104.16.42.93
Found Yes
Hash 30e172e04766bff3696bc1139b1cb46fba4a5ec902705aa8caa70560deaff299
SimHash ba708953cc44

Groups

*

Rule Path Comment
Disallow /product/ -
Disallow /store/adm Crawlers do not need access to your console, so this rule disallows all console pages for crawlers
Disallow /store/shopCa Crawlers cannot add items to a shopping cart, so prevent them from viewing the cart page
Disallow /store/WriteR Crawlers cannot submit product reviews, so there is no need for them to view this page. The content of product reviews is on a separate page that is allowed
Disallow /store/addtoc is used in embedded commerce and allows items to be added to the cart. There is no content on this page and therefore crawlers do not need to see it.
Disallow /store/OnePageCh generally contains no inherent SEO value, and crawlers cannot add items to their cart to be able to access this page, therefore it is disallowed
Disallow /store/checko This page contains no content
Disallow /*Attrib%3D If you use attributes, these rules allow you to avoid duplicate content warnings when customers filter by specific attributes.
Disallow /*?Attrib= If you use attributes, these rules allow you to avoid duplicate content warnings when customers filter by specific attributes.
Disallow *attribs%3D If you use attributes, these rules allow you to avoid duplicate content warnings when customers filter by specific attributes.
Disallow *Attribs%3D If you use attributes, these rules allow you to avoid duplicate content warnings when customers filter by specific attributes.

*

Rule Path
Disallow /sitemap.gz

googlebot

Rule Path
Allow /Shared/Themes/

mozilla/5.0 (compatible; ahrefsbot/5.0; +http://ahrefs.com/robot/)

Rule Path
Disallow /

addthis

Rule Path
Disallow /

addthis.com

Rule Path
Disallow /

baiduspider*

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

exabot*

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

yandex

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

ru_bot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

niki-bot

Rule Path
Disallow /

urlappendbot

Rule Path
Disallow /

merchantcentricbot

Rule Path
Disallow /

bitlybot

Rule Path
Disallow /

bitlybot/2.0

Rule Path
Disallow /

mozilla/5.0 (compatible; linkdexbot/2.2; +http://www.linkdex.com/bots/)

Rule Path
Disallow /

loadtimebot

Rule Path
Disallow /

nerdybot

Rule Path
Disallow /

seegnifybot/1.0.0

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

komodiabot

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

superpagesurlverifybot/1.0

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

openlinkprofiler

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

mozilla/5.0 (compatible; pmoz.info odp link checker; +http://pmoz.info/doc/botinfo.htm)

Rule Path
Disallow /

mozilla/5.0 (compatible; mfgpagesbot/2.1; http://www.mfgpages.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; topicwaybot/2.1; http://www.topicway.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; openfosbot/2.1; http://www.openfos.com)

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

mozilla/5.0 (compatible; duckduckgo-favicons-bot/1.0; +http://duckduckgo.com)

Rule Path
Disallow /

gimme60bot

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

betabot

Rule Path
Disallow /

scrapy/0.24.4

Rule Path
Disallow /

xenu link sleuth/1.3.8

Rule Path
Disallow /

python-urllib/2.7

Rule Path
Disallow /

contextad bot

Rule Path
Disallow /

claritydailybot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

orangebot

Rule Path
Disallow /

mozilla/5.0 (compatible; spbot/4.4.2; +http://openlinkprofiler.org/bot )

Rule Path
Disallow /

go 1.1 package http

Rule Path
Disallow /

java/1.7.0_03

Rule Path
Disallow /

java/1.7.0_60-ea

Rule Path
Disallow /

java/1.7.0_75

Rule Path
Disallow /

hubspot marketing grader

Rule Path
Disallow /

mozilla/4.0 (compatible; blog search;)

Rule Path
Disallow /

Warnings

  • 2 invalid lines.