m.lightinthebox.com
robots.txt

Robots Exclusion Standard data for m.lightinthebox.com

Resource Scan

Scan Details

Site Domain m.lightinthebox.com
Base Domain lightinthebox.com
Scan Status Ok
Last Scan2024-11-14T21:52:20+00:00
Next Scan 2024-11-28T21:52:20+00:00

Last Scan

Scanned2024-11-14T21:52:20+00:00
URL https://m.lightinthebox.com/robots.txt
Domain IPs 125.56.219.17, 23.32.29.96
Response IP 125.56.219.17
Found Yes
Hash 45cdb458b288d0ba5d4ca211f268b34ee1eb55c922f4f58c2a3aeb1e4c86fd3f
SimHash c563d7fc8e33

Groups

*

Rule Path
Disallow /cache/
Disallow /api/
Disallow /plugins/
Disallow /newproducttags/
Disallow /ns/
Disallow /*/ns/
Allow /*%26litb_from%3Dpaid_adwords_shopping
Allow /*%26litb_from%3Dbing_shopping
Disallow */knowledge-base/
Disallow */r/term-of-use.html
Disallow */r/privacy.html
Disallow */partners.html
Disallow */r/testimonials.html
Disallow */r/contact-us.html
Disallow */qa/*_c
Disallow */html/FAQ.html
Disallow */html/newsletter-2012-07-13_es.html
Disallow */html/Color-Charts-es.html
Disallow */html/All-You-Need-To-Know_es.html
Disallow /es/productvote/294705-332259
Disallow /hr/dropship.html
Disallow */html/Shoes_Fit_Guide_it.html
Disallow */html/Faucet_Buying_Guide_es.html
Disallow */html/litb_2013-12-10_EN.html
Disallow /es/productvote/225962-256468
Disallow */list/

pinterest/0.2 (+https://www.pinterest.com/)

Rule Path
Allow /

almaden

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

axmo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

booch

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

downloader

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

expired domain sleuth

Rule Path
Disallow /

franklin locator

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

grub

Rule Path
Disallow /

hughcrawler

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

lcabotaccept

Rule Path
Disallow /

iconsurf

Rule Path
Disallow /

iltrovatore-setaccio

Rule Path
Disallow /

indy library

Rule Path
Disallow /

iupui

Rule Path
Disallow /

kittiecentral

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

larbin

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

metatagrobot

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

nextgensearch

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

oracle ultra search

Rule Path
Disallow /

peerbot

Rule Path
Disallow /

pictureofinternet

Rule Path
Disallow /

plantynet

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

scspider

Rule Path
Disallow /

soft411

Rule Path
Disallow /

spider.acont.de

Rule Path
Disallow /

sqworm

Rule Path
Disallow /

ssm agent

Rule Path
Disallow /

tamu

Rule Path
Disallow /

theusefulbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

tutorial crawler

Rule Path
Disallow /

tutorgig

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webzip

Rule Path
Disallow /

zipppbot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wget

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

mozdex

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

Other Records

Field Value
sitemap https://m.lightinthebox.com/sitemap.xml