auto-dsk.com
robots.txt

Robots Exclusion Standard data for auto-dsk.com

Resource Scan

Scan Details

Site Domain auto-dsk.com
Base Domain auto-dsk.com
Scan Status Ok
Last Scan2025-09-26T02:27:53+00:00
Next Scan 2025-10-03T02:27:53+00:00

Last Scan

Scanned2025-09-26T02:27:53+00:00
URL https://auto-dsk.com/robots.txt
Domain IPs 2604:2dc0:100:1c5f::1, 51.81.106.95
Response IP 51.81.106.95
Found Yes
Hash d9e53eab642e0a81683c3bc486ec6bb567ee2197d8a600d35d44b11f6e91a9a2
SimHash eb0c485a92e9

Groups

*

Rule Path
Disallow /admin/
Disallow /cert/
Disallow /data/
Disallow /includes/
Disallow /install/
Disallow /languages/
Disallow /plugins/
Disallow /templates/
Disallow /upgrade/
Disallow /api/
Disallow /js/
Disallow /affiche.php
Disallow /captcha.php
Disallow /comment.php
Disallow /cycle_image.php
Disallow /goods_script.php
Disallow /receive.php
Disallow /region.php
Disallow /respond.php
Disallow /feed.php
Disallow /gallery.php
Disallow /*goods_type*
Disallow /*order%3D*
Disallow /*sort%3D*
Disallow /*cat_id%3D*
Disallow /*display%3D*
Disallow /c-1-koren*
Disallow /c-2-shini*
Disallow /c-3-diski*
Disallow /*-page-*
Disallow /flow.php
Disallow /404.html
Allow /*.css$
Allow /*.js$
Allow /images/
Allow /favicon.ico

mj12bot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 120

yandex

Rule Path
Disallow /

spbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 40

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

Other Records

Field Value
sitemap https://auto-dsk.com/sitemaps.xml
sitemap https://auto-dsk.com/sitemaps_shop.xml