b2b.ostrovit.com
robots.txt

Robots Exclusion Standard data for b2b.ostrovit.com

Resource Scan

Scan Details

Site Domain b2b.ostrovit.com
Base Domain ostrovit.com
Scan Status Ok
Last Scan2024-10-05T13:28:51+00:00
Next Scan 2024-11-04T13:28:51+00:00

Last Scan

Scanned2024-10-05T13:28:51+00:00
URL https://b2b.ostrovit.com/robots.txt
Domain IPs 5.149.167.181
Response IP 5.149.167.181
Found Yes
Hash 0e666560c273f85f43a38bf965563ef270894cf3da37de7fd2f44f231035c5c5
SimHash 1352d518843b

Groups

*

Rule Path
Disallow /*?rec=*
Disallow /*%26rec%3D*

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

fyberspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

twengabot-discover

Rule Path
Disallow /

twengabot/2.0

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

twengabot-2.0

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

*

Rule Path
Disallow /en/basketchange.html
Disallow /pl/basketchange.html
Disallow /ru/basketchange.html
Disallow /en/basketedit.html
Disallow /pl/basketedit.html
Disallow /ru/basketedit.html
Disallow /en/basket-onchange.html
Disallow /pl/basket-onchange.html
Disallow /ru/basket-onchange.html
Disallow /en/client-address.html
Disallow /pl/client-address.html
Disallow /ru/client-address.html
Disallow /en/client-addresses.html
Disallow /pl/client-addresses.html
Disallow /ru/client-addresses.html
Disallow /en/client-new.html
Disallow /pl/client-new.html
Disallow /ru/client-new.html
Disallow /en/client-orders.html
Disallow /pl/client-orders.html
Disallow /ru/client-orders.html
Disallow /en/client-save.html
Disallow /pl/client-save.html
Disallow /ru/client-save.html
Disallow /en/client-files.html
Disallow /pl/client-files.html
Disallow /ru/client-files.html
Disallow /en/download-attachment.html
Disallow /pl/download-attachment.html
Disallow /ru/download-attachment.html
Disallow /en/login.html
Disallow /pl/login.html
Disallow /ru/login.html
Disallow /en/loginedit.html
Disallow /pl/loginedit.html
Disallow /ru/loginedit.html
Disallow /en/loginonce.html
Disallow /pl/loginonce.html
Disallow /ru/loginonce.html
Disallow /en/membership-card.html
Disallow /pl/membership-card.html
Disallow /ru/membership-card.html
Disallow /en/order1.html
Disallow /pl/order1.html
Disallow /ru/order1.html
Disallow /en/order2.html
Disallow /pl/order2.html
Disallow /ru/order2.html
Disallow /en/order3.html
Disallow /pl/order3.html
Disallow /ru/order3.html
Disallow /en/ordercancel.html
Disallow /pl/ordercancel.html
Disallow /ru/ordercancel.html
Disallow /en/orderconfirm.html
Disallow /pl/orderconfirm.html
Disallow /ru/orderconfirm.html
Disallow /en/orderdetails.html
Disallow /pl/orderdetails.html
Disallow /ru/orderdetails.html
Disallow /en/order-document.html
Disallow /pl/order-document.html
Disallow /ru/order-document.html
Disallow /en/order-newpayment.html
Disallow /pl/order-newpayment.html
Disallow /ru/order-newpayment.html
Disallow /en/order-payment.html
Disallow /pl/order-payment.html
Disallow /ru/order-payment.html
Disallow /en/order-postauction.html
Disallow /pl/order-postauction.html
Disallow /ru/order-postauction.html
Disallow /en/order-wrappers.html
Disallow /pl/order-wrappers.html
Disallow /ru/order-wrappers.html
Disallow /en/payments-confirm.html
Disallow /pl/payments-confirm.html
Disallow /ru/payments-confirm.html
Disallow /en/products-bought.html
Disallow /pl/products-bought.html
Disallow /ru/products-bought.html
Disallow /en/products-requests.html
Disallow /pl/products-requests.html
Disallow /ru/products-requests.html
Disallow /en/przelew.html
Disallow /pl/przelew.html
Disallow /ru/przelew.html
Disallow /en/rebate-code.html
Disallow /pl/rebate-code.html
Disallow /ru/rebate-code.html
Disallow /en/rma-add.html
Disallow /pl/rma-add.html
Disallow /ru/rma-add.html
Disallow /en/rma-list.html
Disallow /pl/rma-list.html
Disallow /ru/rma-list.html
Disallow /en/settings.html
Disallow /pl/settings.html
Disallow /ru/settings.html
Disallow /en/checkup.html
Disallow /pl/checkup.html
Disallow /ru/checkup.html
Disallow /en/rma-open.html
Disallow /pl/rma-open.html
Disallow /ru/rma-open.html
Disallow /en/returns-open.html
Disallow /pl/returns-open.html
Disallow /ru/returns-open.html
Disallow /en/return.html
Disallow /pl/return.html
Disallow /ru/return.html
Disallow /en/affiliate-resources.html
Disallow /pl/affiliate-resources.html
Disallow /ru/affiliate-resources.html
Disallow /edi/
Disallow /ajax/
Disallow query_id%3D*

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://b2b.ostrovit.com/sitemap.xml.gz

Comments

  • Pages with rec parameter - IAI Recommendation System
  • Automatically banned scanners and crawlers section
  • Section end

Warnings

  • 2 invalid lines.