Log In Sign Up

auto-tests.com
robots.txt

Robots Exclusion Standard data for auto-tests.com

Resource Scan

Scan Details

Site Domain auto-tests.com
Base Domain auto-tests.com
Scan Status Ok
Last Scan2024-06-09T04:05:13+00:00
Next Scan 2024-07-09T04:05:13+00:00

Last Scan

Scanned2024-06-09T04:05:13+00:00
URL https://auto-tests.com/robots.txt
Domain IPs 128.204.218.48
Response IP 128.204.218.48
Found Yes
Hash 24b107bf81de3cb536d2a653a1d2c35402575be60c4692c8795cb55c30dc5f2f
SimHash 921c690281a3

Groups

*

Rule Path
Disallow /statics/norob/
Disallow /categories.php
Disallow /tests/
Disallow /*category
Disallow /*backgrounds
Disallow /admin/
Disallow /langs/
Disallow /wa/
Disallow /search.php
Disallow /sperrmi.php
Disallow /sperrmic.php
Disallow /login.php

ia_archiver

Rule Path
Disallow /

lipperhey

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

domaincrawler/1.0

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

linguee

Rule Path
Disallow /

exabot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

yottaamonitor

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

docomo

Rule Path
Disallow /

bspider

Rule Path
Disallow /

psbot

Rule Path
Disallow /

ng

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandex bot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

spbot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

bixolabs

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

purebot

Rule Path
Disallow /

yeti/1.0

Rule Path
Disallow /

yeti

Rule Path
Disallow /

docomo

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

sbider

Rule Path
Disallow /

geohasher

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

flightdeckreportsbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

yottaamonitor

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

lmspider

Rule Path
Disallow /

speedy

Rule Path
Disallow /

lmspider

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

aboundexbot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

crystalsemanticsbot

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

r6_commentreader

Rule Path
Disallow /

motor

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

my nutch spider

Rule Path
Disallow /

swebot

Rule Path
Disallow /

comodospider

Rule Path
Disallow /

integromedb

Rule Path
Disallow /
Back to top

Warnings

  • 4 invalid lines.
Back to top