tomas.by
robots.txt

Robots Exclusion Standard data for tomas.by

Resource Scan

Scan Details

Site Domain tomas.by
Base Domain tomas.by
Scan Status Ok
Last Scan2025-09-26T12:41:16+00:00
Next Scan 2025-10-03T12:41:16+00:00

Last Scan

Scanned2025-09-26T12:41:16+00:00
URL https://tomas.by/robots.txt
Domain IPs 185.66.70.42
Response IP 185.66.70.42
Found Yes
Hash 546dfc70729a2b73970fb67de5dd0b45f34e63da78949567f33fe0b3cdf88e87
SimHash 018cf9a38378

Groups

*

Rule Path
Disallow *?*
Disallow *%26*
Disallow */t/*/g-*
Disallow */go/*
Disallow */goals/*
Disallow */poisk/
Disallow */search/
Disallow */statistic/*
Disallow */files/temp/
Disallow */registration/*
Disallow */stats/
Disallow */cabinet_c/*
Disallow */cabinet/*
Disallow */filters/*
Disallow */materials/*/*/
Disallow */botcheck/main/
Disallow *.pdf
Disallow */f/*
Disallow */i/*
Disallow */cds/*
Disallow */track-message/*
Disallow */track-emailing/*
Disallow */activate_email/*
Disallow */ac/*
Disallow */map-address/*
Disallow */ajax/*
Disallow */cart/*
Disallow */track-image/
Disallow */admin/*
Disallow */faq/*
Disallow /cs/*
Allow */inc/*
Allow */lego/*
Allow */pics/*
Allow */local_files/*
Allow */files/*
Allow */go/$
Allow */frontend/*

googlebot

Rule Path
Disallow *?*
Disallow *%26*
Disallow */t/*/g-*
Disallow */go/*
Disallow */goals/*
Disallow */poisk/
Disallow */search/
Disallow */statistic/*
Disallow */files/temp/
Disallow */registration/*
Disallow */stats/
Disallow */cabinet_c/*
Disallow */cabinet/*
Disallow */filters/*
Disallow */materials/*/*/
Disallow */botcheck/main/
Disallow *.pdf
Disallow */f/*
Disallow */i/*
Disallow */cds/*
Disallow */track-message/*
Disallow */track-emailing/*
Disallow */activate_email/*
Disallow */ac/*
Disallow */map-address/*
Disallow */ajax/*
Disallow */cart/*
Disallow */track-image/
Disallow */admin/*
Disallow */faq/*
Disallow /cs/*
Allow */p/*/?e=1&i=*
Allow */inc/*
Allow */lego/*
Allow */pics/*
Allow */local_files/*
Allow */files/*
Allow */go/$
Allow */frontend/*

yandex

Rule Path
Disallow *?*
Disallow *%26*
Disallow */t/*/g-*
Disallow */go/*
Disallow */goals/*
Disallow */poisk/
Disallow */search/
Disallow */statistic/*
Disallow */files/temp/
Disallow */registration/*
Disallow */stats/
Disallow */cabinet_c/*
Disallow */cabinet/*
Disallow */filters/*
Disallow */materials/*/*/
Disallow */botcheck/main/
Disallow *.pdf
Disallow */f/*
Disallow */i/*
Disallow */cds/*
Disallow */track-message/*
Disallow */track-emailing/*
Disallow */activate_email/*
Disallow */ac/*
Disallow */map-address/*
Disallow */ajax/*
Disallow */cart/*
Disallow */track-image/
Disallow */admin/*
Disallow */faq/*
Disallow /cs/*
Disallow /brest/k/filter-*
Disallow /vitebsk/k/filter-*
Disallow /gomel/k/filter-*
Disallow /grodno/k/filter-*
Disallow /mogilev/k/filter-*
Disallow /orsha/k/filter-*
Disallow /borisov/k/filter-*
Disallow /baranovichi/k/filter-*
Disallow /pinsk/k/filter-*
Disallow /bobruysk/k/filter-*
Allow */inc/*
Allow */lego/*
Allow */pics/*
Allow */local_files/*
Allow */files/*
Allow */go/$
Allow */frontend/*

bubing

Rule Path
Disallow /

yadirectbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

yandexdirect

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

mail.ru

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Warnings

  • `clean-param` is not a known field.