lexpansion.com
robots.txt

Robots Exclusion Standard data for lexpansion.com

Resource Scan

Scan Details

Site Domain lexpansion.com
Base Domain lexpansion.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-04-05T08:56:05+00:00
Next Scan 2024-07-04T08:56:05+00:00

Last Successful Scan

Scanned2023-02-18T05:47:09+00:00
URL http://lexpansion.com/robots.txt
Redirect https://www.lexpress.fr/robots.txt
Redirect Domain www.lexpress.fr
Redirect Base lexpress.fr
Domain IPs 95.131.136.80
Redirect IPs 2600:1413:b000:13::b857:c195, 2600:1413:b000:13::b857:c19c, 72.247.81.121, 72.247.81.146
Response IP 42.99.140.219
Found Yes
Hash 23bea9a4ef0d54d8b72be992ba3039d422dcff45140ace0a0886b5aa3f84e9bf
SimHash ca4dda040366

Groups

*

Rule Path
Disallow /tiny/
Disallow /includes
Disallow /imgs
Disallow /imgstat

tunitinbot

Rule Path
Disallow /

converacrawler

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

digimind

Rule Path
Disallow /

knowings d

Rule Path
Disallow /

kbcrawl

Rule Path
Disallow /

wget

Rule Path
Disallow /

amisoftware

Rule Path
Disallow /

newzbin

Rule Path
Disallow /

ask n read

Rule Path
Disallow /

qwam content intelligence

Rule Path
Disallow /

zite

Rule Path
Disallow /

flipboard

Rule Path
Disallow /

flipboardproxy

Rule Path
Disallow /

youmag

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.lexpress.fr/arc/outboundfeeds/sitemap-news.xml
sitemap https://www.lexpress.fr/arc/outboundfeeds/sitemap.xml