machineryline.ee
robots.txt

Robots Exclusion Standard data for machineryline.ee

Resource Scan

Scan Details

Site Domain machineryline.ee
Base Domain machineryline.ee
Scan Status Ok
Last Scan2024-11-18T08:17:54+00:00
Next Scan 2024-11-25T08:17:54+00:00

Last Scan

Scanned2024-11-18T08:17:54+00:00
URL https://machineryline.ee/robots.txt
Domain IPs 144.76.229.22
Response IP 144.76.229.22
Found Yes
Hash f21fffceaf7213209b08731e3f33ea5a5e97ef68b20f4576d423234513cb3055
SimHash c22b52e5a6d7

Groups

*

Rule Path
Disallow */version
Disallow */resizable
Disallow */focusable
Disallow */controlgroup
Disallow */checkboxradio
Disallow */mouse
Disallow */draggable
Disallow */button
Disallow /big-photos/
Disallow /api/
Disallow /autopl_map.php
Disallow /ban_stat.php
Disallow /banner_preview.php
Disallow /captcha/
Disallow /change-locale/
Disallow /change-measurements/
Disallow /charts/
Disallow /compare/
Disallow /components/
Disallow /contact-with-dealer/
Disallow /dealer_registration.php
Disallow /dealers-pages/
Disallow /dealers/map/
Disallow /delivery/tw/
Disallow /dev/
Disallow /e/
Disallow /email_form.php
Disallow /export/
Disallow /header-menu/
Disallow /image_val.php
Disallow /includes/
Disallow /index-menu/
Disallow /login/
Disallow /message-backup-controller/
Disallow /ni/
Disallow /order/
Disallow /orig_pages/
Disallow /payment/
Disallow /performatives.php
Disallow /performatives.php
Disallow /place_ad.php
Disallow /prices_example.php
Disallow /print-pdf/
Disallow /registration
Disallow /registration/
Disallow /rss/
Disallow /sales-history.php
Disallow /sales-history/
Disallow /sales-search-history/
Disallow /sales/
Disallow /search.php
Disallow /search/
Disallow /search/suggest/
Disallow /search_text.php
Disallow /secure_code.php
Disallow /share/
Disallow /stat/
Disallow /subscribe.php
Disallow /test-work/
Disallow /track-mail/
Disallow /translate.php
Disallow /translations/
Disallow /u.php
Disallow /ubl/index.php?adv_code=
Disallow /user_question/
Disallow /switchLocale/
Disallow /rchat/
Disallow /start
Disallow /user/messages/
Disallow /vehicle-inspection/
Disallow /vehicle-leasing/
Disallow /vehicle-service-request/
Disallow /place-for-free/
Disallow /feedback/
Disallow /impressum-d/
Disallow /vehicles-reviews/create/
Disallow /ni/user-menu-links/
Disallow /send-to-messenger/
Disallow /my/
Disallow /stock.php
Disallow /ru*/version
Disallow /ru*/resizable
Disallow /ru*/focusable
Disallow /ru*/controlgroup
Disallow /ru*/checkboxradio
Disallow /ru*/mouse
Disallow /ru*/draggable
Disallow /ru*/button
Disallow /ru/big-photos/
Disallow /ru/api/
Disallow /ru/autopl_map.php
Disallow /ru/ban_stat.php
Disallow /ru/banner_preview.php
Disallow /ru/captcha/
Disallow /ru/change-locale/
Disallow /ru/change-measurements/
Disallow /ru/charts/
Disallow /ru/compare/
Disallow /ru/components/
Disallow /ru/contact-with-dealer/
Disallow /ru/dealer_registration.php
Disallow /ru/dealers-pages/
Disallow /ru/dealers/map/
Disallow /ru/delivery/tw/
Disallow /ru/dev/
Disallow /ru/e/
Disallow /ru/email_form.php
Disallow /ru/export/
Disallow /ru/header-menu/
Disallow /ru/image_val.php
Disallow /ru/includes/
Disallow /ru/index-menu/
Disallow /ru/login/
Disallow /ru/message-backup-controller/
Disallow /ru/ni/
Disallow /ru/order/
Disallow /ru/orig_pages/
Disallow /ru/payment/
Disallow /ru/performatives.php
Disallow /ru/performatives.php
Disallow /ru/place_ad.php
Disallow /ru/prices_example.php
Disallow /ru/print-pdf/
Disallow /ru/registration
Disallow /ru/registration/
Disallow /ru/rss/
Disallow /ru/sales-history.php
Disallow /ru/sales-history/
Disallow /ru/sales-search-history/
Disallow /ru/sales/
Disallow /ru/search.php
Disallow /ru/search/
Disallow /ru/search/suggest/
Disallow /ru/search_text.php
Disallow /ru/secure_code.php
Disallow /ru/share/
Disallow /ru/stat/
Disallow /ru/subscribe.php
Disallow /ru/test-work/
Disallow /ru/track-mail/
Disallow /ru/translate.php
Disallow /ru/translations/
Disallow /ru/u.php
Disallow /ru/ubl/index.php?adv_code=
Disallow /ru/user_question/
Disallow /ru/switchLocale/
Disallow /ru/rchat/
Disallow /ru/start
Disallow /ru/user/messages/
Disallow /ru/vehicle-inspection/
Disallow /ru/vehicle-leasing/
Disallow /ru/vehicle-service-request/
Disallow /ru/place-for-free/
Disallow /ru/feedback/
Disallow /ru/impressum-d/
Disallow /ru/vehicles-reviews/create/
Disallow /ru/ni/user-menu-links/
Disallow /ru/send-to-messenger/
Disallow /my/
Disallow /ru/stock.php

baiduspider
baiduspider-image
baiduspider-video
baiduspider-news
baiduspider-favo
baiduspider-cpro
baiduspider-ads

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

petabot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

claudebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

facebookexternalhit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

imagesiftbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap https://machineryline.ee/sitemap.xml