autovaruosad.ee
robots.txt

Robots Exclusion Standard data for autovaruosad.ee

Resource Scan

Scan Details

Site Domain autovaruosad.ee
Base Domain autovaruosad.ee
Scan Status Ok
Last Scan2025-09-29T23:51:59+00:00
Next Scan 2025-10-06T23:51:59+00:00

Last Scan

Scanned2025-09-29T23:51:59+00:00
URL https://autovaruosad.ee/robots.txt
Domain IPs 104.21.9.231, 172.67.161.99, 2606:4700:3033::ac43:a163, 2606:4700:3035::6815:9e7
Response IP 104.21.9.231
Found Yes
Hash 1212c1f294d4a88d7ebd83f8aa1fefa0749d0db60a44b63504922c3f62c1486c
SimHash 714cc4608158

Groups

*

Rule Path
Disallow /api
Disallow /admin
Disallow /vhid
Disallow /ru/vhod
Disallow /vidnovlennya-parolyu
Disallow /ru/vostanovlenie-paralja
Disallow /zareyestruvatisya
Disallow /ru/zaregistrirovatsya
Disallow /obrane
Disallow /ru/izbrannoe
Disallow /porivnanna
Disallow /ru/sravnenie
Disallow /koshuk
Disallow /ru/korzina
Disallow /*product_search?request=
Disallow /*filter%5Bcriteria%5D
Disallow /*filter%5Bmanu_names%5D
Disallow %5B%5D%3D
Disallow *utm*%3D
Allow /*.css
Allow /*.js
Allow /*.jpg
Allow /*.jpeg
Allow /*.png
Allow /*.gif
Allow /*.svg
Allow /*.pdf
Allow /*.webp

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 7200

appcodescrawler

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

fatbot 2.0

Rule Path
Disallow /

ahc/1.0

Rule Path
Disallow /

ahc/2.0

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

sogou blog

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

sogou news spider

Rule Path
Disallow /

sogou orion spider

Rule Path
Disallow /

sogou spider2

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

bluechipbacklinks

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

Other Records

Field Value
sitemap https://autovaruosad.ee/sitemap.xml