tobi.com
robots.txt

Robots Exclusion Standard data for tobi.com

Resource Scan

Scan Details

Site Domain tobi.com
Base Domain tobi.com
Scan Status Ok
Last Scan2025-06-28T10:26:30+00:00
Next Scan 2025-07-28T10:26:30+00:00

Last Scan

Scanned2025-06-28T10:26:30+00:00
URL https://tobi.com/robots.txt
Redirect https://www.tobi.com/robots.txt
Redirect Domain www.tobi.com
Redirect Base tobi.com
Domain IPs 104.26.4.12, 104.26.5.12, 172.67.71.22, 2606:4700:20::681a:40c, 2606:4700:20::681a:50c, 2606:4700:20::ac43:4716
Redirect IPs 104.26.4.12, 104.26.5.12, 172.67.71.22, 2606:4700:20::681a:40c, 2606:4700:20::681a:50c, 2606:4700:20::ac43:4716
Response IP 104.26.4.12
Found Yes
Hash 7f61414d4269906f773ae396064abf5a98a6aaa22e8db1385cf4250c6bacfd70
SimHash 73375e568f40

Groups

*

Rule Path
Disallow /a/
Disallow */account/
Disallow /analytics/
Disallow */breadcrumbs/
Disallow /cart/
Disallow /checkout/
Disallow /confirm_email/
Disallow /comment/
Disallow /e/
Disallow /feed/catalog
Disallow /i/
Disallow /images/
Disallow */login
Disallow */likes
Disallow /men
Disallow /ping/
Disallow */product_collection_widget/
Disallow /post_signup/
Disallow */related_products/
Disallow /store/
Disallow /store_country/
Disallow */trending_now/
Disallow /mobile_ping/
Disallow /javascript_error/
Disallow /path/
Disallow */shop/
Disallow /marketing/
Disallow /color_image/
Disallow /likes_service/
Disallow */search
Disallow /checkout
Disallow /sg/
Disallow /ie/
Disallow /dk/
Disallow /nl/
Disallow /no/
Disallow /se/

nutch

Rule Path
Disallow /

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

Comments

  • Disallow: /*?*
  • Disallow: /*?*=*=
  • Disallow: /*?color_id=*&
  • Disallow: /*?page=*&
  • Allow: /*?page=
  • Allow: /*?color_id=
  • Version 20240711