lexpansion.lexpress.fr
robots.txt

Robots Exclusion Standard data for lexpansion.lexpress.fr

Resource Scan

Scan Details

Site Domain lexpansion.lexpress.fr
Base Domain lexpress.fr
Scan Status Ok
Last Scan2024-05-16T07:34:06+00:00
Next Scan 2024-05-23T07:34:06+00:00

Last Scan

Scanned2024-05-16T07:34:06+00:00
URL https://lexpansion.lexpress.fr/robots.txt
Domain IPs 23.52.171.128, 23.52.171.130, 2600:1413:b000:14::b857:c146, 2600:1413:b000:14::b857:c147
Response IP 42.99.140.219
Found Yes
Hash f22c4ac4887dde2a4dd5abe23abb61169877a0fd713a709ca709f0b8fcf16100
SimHash 486dda040346

Groups

*

Rule Path
Disallow /tiny/
Disallow /includes
Disallow /imgs
Disallow /imgstat
Disallow */pf/api/
Disallow /.well-known/assetlinks.json

tunitinbot

Rule Path
Disallow /

converacrawler

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

digimind

Rule Path
Disallow /

knowings d

Rule Path
Disallow /

kbcrawl

Rule Path
Disallow /

wget

Rule Path
Disallow /

amisoftware

Rule Path
Disallow /

newzbin

Rule Path
Disallow /

ask n read

Rule Path
Disallow /

qwam content intelligence

Rule Path
Disallow /

zite

Rule Path
Disallow /

flipboard

Rule Path
Disallow /

flipboardproxy

Rule Path
Disallow /

youmag

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.lexpress.fr/arc/outboundfeeds/sitemap-news.xml
sitemap https://www.lexpress.fr/arc/outboundfeeds/sitemap.xml