obsessive.com
robots.txt

Robots Exclusion Standard data for obsessive.com

Resource Scan

Scan Details

Site Domain obsessive.com
Base Domain obsessive.com
Scan Status Ok
Last Scan2024-09-15T09:10:25+00:00
Next Scan 2024-10-15T09:10:25+00:00

Last Scan

Scanned2024-09-15T09:10:25+00:00
URL https://obsessive.com/robots.txt
Domain IPs 5.149.161.96
Response IP 5.149.161.96
Found Yes
Hash 79164bcec2801ce182c3a3fc5f9af6c89f551b93a1d28bf0c6029693416778ee
SimHash 725c9431089f

Groups

*

Rule Path
Disallow /*?rec=*
Disallow /*%26rec%3D*

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

metajobbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

fyberspider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

twengabot-discover

Rule Path
Disallow /

twengabot/2.0

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

twengabot-2.0

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

*

Rule Path
Disallow /en/basketchange.html
Disallow /it/basketchange.html
Disallow /en/basketedit.html
Disallow /it/basketedit.html
Disallow /en/basket-onchange.html
Disallow /it/basket-onchange.html
Disallow /en/client-address.html
Disallow /it/client-address.html
Disallow /en/client-addresses.html
Disallow /it/client-addresses.html
Disallow /en/client-new.html
Disallow /it/client-new.html
Disallow /en/client-orders.html
Disallow /it/client-orders.html
Disallow /en/client-save.html
Disallow /it/client-save.html
Disallow /en/client-files.html
Disallow /it/client-files.html
Disallow /en/download-attachment.html
Disallow /it/download-attachment.html
Disallow /en/login.html
Disallow /it/login.html
Disallow /en/loginedit.html
Disallow /it/loginedit.html
Disallow /en/loginonce.html
Disallow /it/loginonce.html
Disallow /en/membership-card.html
Disallow /it/membership-card.html
Disallow /en/order1.html
Disallow /it/order1.html
Disallow /en/order2.html
Disallow /it/order2.html
Disallow /en/order3.html
Disallow /it/order3.html
Disallow /en/ordercancel.html
Disallow /it/ordercancel.html
Disallow /en/orderconfirm.html
Disallow /it/orderconfirm.html
Disallow /en/orderdetails.html
Disallow /it/orderdetails.html
Disallow /en/order-document.html
Disallow /it/order-document.html
Disallow /en/order-newpayment.html
Disallow /it/order-newpayment.html
Disallow /en/order-payment.html
Disallow /it/order-payment.html
Disallow /en/order-postauction.html
Disallow /it/order-postauction.html
Disallow /en/order-wrappers.html
Disallow /it/order-wrappers.html
Disallow /en/payments-confirm.html
Disallow /it/payments-confirm.html
Disallow /en/products-bought.html
Disallow /it/products-bought.html
Disallow /en/products-requests.html
Disallow /it/products-requests.html
Disallow /en/przelew.html
Disallow /it/przelew.html
Disallow /en/rebate-code.html
Disallow /it/rebate-code.html
Disallow /en/rma-add.html
Disallow /it/rma-add.html
Disallow /en/rma-list.html
Disallow /it/rma-list.html
Disallow /en/settings.html
Disallow /it/settings.html
Disallow /en/checkup.html
Disallow /it/checkup.html
Disallow /en/rma-open.html
Disallow /it/rma-open.html
Disallow /en/returns-open.html
Disallow /it/returns-open.html
Disallow /en/return.html
Disallow /it/return.html
Disallow /en/affiliate-resources.html
Disallow /it/affiliate-resources.html
Disallow /edi/
Disallow /ajax/
Disallow query_id%3D*

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://obsessive.com/sitemap.xml.gz

Comments

  • Pages with rec parameter - IAI Recommendation System
  • Automatically banned scanners and crawlers section
  • Section end

Warnings

  • 2 invalid lines.