Log In Sign Up

smartsd.com
robots.txt

Robots Exclusion Standard data for smartsd.com

Resource Scan

Scan Details

Site Domain smartsd.com
Base Domain smartsd.com
Scan Status Ok
Last Scan2024-09-23T08:38:50+00:00
Next Scan 2024-10-07T08:38:50+00:00

Last Scan

Scanned2024-09-23T08:38:50+00:00
URL https://smartsd.com/robots.txt
Domain IPs 31.171.200.170
Response IP 31.171.200.170
Found Yes
Hash c5192cd6d083269aa41446abb5f53a180b44915a365f66e0bf7ea24cb55fe9a0
SimHash 721853402d32

Groups

petalbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

cincraw

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

netseer

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

getintent crawler

Rule Path
Disallow /

bidswitchbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

femtosearchbot

Rule Path
Disallow /

owler

Rule Path
Disallow /

tracemyfile

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

safednsbot

Rule Path
Disallow /

hybridbot

Rule Path
Disallow /

feedly

Rule Path
Disallow /

feedburner

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

theoldreader.com

Rule Path
Disallow /

semantic-visions.com

Rule Path
Disallow /

proximic

Rule Path
Disallow /

weborama-fetcher

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

ias_crawler

Rule Path
Disallow /
Back to top

Comments

  • robots.txt for https://smartsd.com
  • Huawei PetalBot knows where to go.
  • Yandex this is for you.
  • China Chincraw stop poking.
  • http://www.trendiction.com/en/publisher/bot
  • http://www.grapeshot.co.uk/crawler.php
  • http://www.semrush.com/bot.html
  • http://www.semrush.com/bot.html
  • http://mj12bot.com
  • http://www.netseer.com/crawler.html
  • http://www.pinterest.com/bot.html
  • http://getintent.com/bot.html
Back to top