Log In Sign Up

solidamerica.com
robots.txt

Robots Exclusion Standard data for solidamerica.com

Resource Scan

Scan Details

Site Domain solidamerica.com
Base Domain solidamerica.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-12T19:53:29+00:00
Next Scan 2025-02-10T19:53:29+00:00

Last Successful Scan

Scanned2023-10-20T16:27:08+00:00
URL https://solidamerica.com/robots.txt
Domain IPs 198.23.51.26
Response IP 198.23.51.26
Found Yes
Hash d0e6b1632320fe52ece08b3921857997934f223907c43ff7af6f9da47219206e
SimHash 4af0042010ee

Groups

*

Rule Path
Allow /
Disallow /oc-admin/
Disallow /cgi-bin/
Disallow /*oc-admin
Disallow /user/login
Disallow /search/
Disallow /search

googlebot

Rule Path
Allow /
Disallow /oc-admin/
Disallow /cgi-bin/
Disallow /*oc-admin
Disallow /user/login
Disallow /search/
Disallow /search

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

dall-e

Rule Path
Disallow /

vertex ai

Rule Path
Disallow /

mediapartners-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

yandexbot

Rule Path
Disallow /

bingbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /

msnbot

Rule Path
Allow /

slurp

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /
Disallow /admin
Disallow /ajax/view/

mozilla/5.0 (compatible; neevabot/1.0; +https://neeva.com/neevabot)

Rule Path
Disallow /

awariosmartbot/1.0 (+https://awario.com/bots.html; bots@awario.com)

Rule Path
Disallow /

mozilla/5.0 (compatible; seekport crawler; http://seekport.com/)

Rule Path
Disallow /

mozilla/5.0 (compatible; sphereup; http://www.sphereup.com/)

Rule Path
Disallow /

linkedinbot

Rule Path
Disallow /
Disallow /admin
Disallow /ajax/view/

proximic

Rule Path
Disallow

*

Rule Path
Disallow

slurp

Rule Path
Allow /
Disallow /admin

ahrefsbot

Rule Path
Disallow /
Disallow /admin

semrushbot

Rule Path
Disallow /
Disallow /admin

semrushbot-sa

Rule Path
Disallow /
Disallow /admin

facebot

Rule Path
Disallow /
Disallow /admin

semrushbot

Rule Path
Disallow /

*
ahrefsbot

Rule Path
Disallow /

unisterbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

seodiver

Rule Path
Disallow /

spbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

meanpathbot

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

obot

Rule Path
Disallow /

fr-crawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

cloudservermarketspider

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

lipperhey-kaus-australis

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

spiderbot

Rule Path
Disallow /

linkstats

Rule Path
Disallow /

jobboersebot

Rule Path
Disallow /

iccrawler

Rule Path
Disallow /

plista

Rule Path
Disallow /

domain re-animator bot

Rule Path
Disallow /

lipperhey-kaus-australis

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

um-ic

Rule Path
Disallow /

mindupbot

Rule Path
Disallow /

sg-orbiter

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

kraken

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

openhosebot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

thumbsniper

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

implisensebot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

python/3.5 aiohttp

Rule Path
Disallow /

toweya.com

Rule Path
Disallow /

netestate

Rule Path
Disallow /

bubing

Rule Path
Disallow /

linguee

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

indeedbot

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

gosign-security-crawler

Rule Path
Disallow /

siteliner

Rule Path
Disallow /

sabsimbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

df bot 1.0

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.solidamerica.com/imagesmap.xml
sitemap https://www.solidamerica.com/sitemap-large.xml
sitemap https://www.solidamerica.com/googlesitemap.xml
sitemap https://www.solidamerica.com/sitemap.xml
sitemap https://www.solidamerica.com/imagesite1map.xml
sitemap https://www.solidamerica.com/imagessitemapfromgenerator.xml
sitemap https://jobshiring.solidamerica.com/post-sitemap.xml
sitemap https://jobshiring.solidamerica.com/page-sitemap.xml
sitemap https://jobshiring.solidamerica.com/job_listing-sitemap.xml
sitemap https://jobshiring.solidamerica.com/category-sitemap.xml
sitemap https://jobshiring.solidamerica.com/post_tag-sitemap.xml
sitemap https://www.solidamerica.com/cutegiftshop/sitemap_index.xml
sitemap https://www.solidamerica.com/cutegiftshop/post-sitemap.xml
sitemap https://www.solidamerica.com/cutegiftshop/page-sitemap.xml
sitemap https://www.solidamerica.com/cutegiftshop/post_tag-sitemap.xml
sitemap https://www.solidamerica.com/cutegiftshop/product-sitemap.xml
sitemap https://www.solidamerica.com/cutegiftshop/product_tag-sitemap2.xml
sitemap https://www.solidamerica.com/cutegiftshop/category-sitemap.xml
sitemap https://www.solidamerica.com/cutegiftshop/product_tag-sitemap.xml
sitemap https://www.solidamerica.com/cutegiftshop/product_cat-sitemap.xml
Back to top

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK
  • Slow down bots
  • Backlink Analysis
  • Bot der Leipziger Unister Holding GmbH
  • http://moz.com/products
  • http://www.searchmetrics.com
  • http://www.majestic12.co.uk/projects/dsearch/mj12bot.php
  • http://www.domaintools.com/webmasters/surveybot.php
  • http://www.seodiver.com/bot
  • http://openlinkprofiler.org/bot
  • http://www.wotbox.com/bot/
  • http://www.opensiteexplorer.org/dotbot
  • http://moz.com/researchtools/ose/dotbot
  • http://www.meanpath.com/meanpathbot.html
  • http://www.backlinktest.com/crawler.html
  • http://www.brandwatch.com/magpie-crawler/
  • http://filterdb.iss.net/crawler/
  • http://webmeup-crawler.com
  • https://megaindex.com/crawler
  • http://www.cloudservermarket.com
  • http://www.trendiction.de/de/publisher/bot
  • http://www.exalead.com
  • http://www.career-x.de/bot.html
  • https://www.lipperhey.com/en/about/
  • https://www.lipperhey.com/en/about/
  • https://turnitin.com/robot/crawlerinfo.html
  • http://help.coccoc.com/
  • ubermetrics-technologies.com
  • datenbutler.de
  • http://searchgears.de/uber-uns/crawling-faq.html
  • http://commoncrawl.org/faq/
  • https://www.qwant.com/
  • http://linkfluence.net/
  • http://www.botje.com/plukkie.htm
  • https://www.safedns.com/searchbot
  • http://www.haosou.com/help/help_3_2.html
  • http://www.haosou.com/help/help_3_2.html
  • http://www.moz.com/dp/rogerbot
  • http://www.openhose.org/bot.html
  • http://www.screamingfrog.co.uk/seo-spider/
  • http://thumbsniper.com
  • http://www.radian6.com/crawler
  • http://cliqz.com/company/cliqzbot
  • https://www.aihitdata.com/about
  • http://www.trendiction.com/en/publisher/bot
  • http://warebay.com/bot.html
  • http://www.website-datenbank.de/
  • http://law.di.unimi.it/BUbiNG.html
  • http://www.linguee.com/bot; bot@linguee.com
  • https://www.semrush.com/bot/
  • www.sentibot.eu
  • http://velen.io
  • https://moz.com/help/guides/moz-procedures/what-is-rogerbot
  • http://www.garlik.com
  • https://www.gosign.de/typo3-extension/typo3-sicherheitsmonitor/
  • http://www.siteliner.com/bot
  • https://sabsim.com
  • http://ltx71.com/
Back to top

Warnings

  • 8 invalid lines.
Back to top