Log In Sign Up

sudouest.fr
robots.txt

Robots Exclusion Standard data for sudouest.fr

Resource Scan

Scan Details

Site Domain sudouest.fr
Base Domain sudouest.fr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-20T22:06:38+00:00
Next Scan 2025-11-18T22:06:38+00:00

Last Successful Scan

Scanned2024-10-25T20:52:11+00:00
URL https://sudouest.fr/robots.txt
Redirect https://www.sudouest.fr:443/robots.txt
Redirect Domain www.sudouest.fr
Redirect Base sudouest.fr
Domain IPs 45.223.122.231, 45.223.97.231
Redirect IPs 45.223.107.231
Response IP 45.223.107.231
Found Yes
Hash 1787486e8b115a641b73c8367f614ab204d41319d6890a16ce6cbd2a3e3e74c0
SimHash 24bf53f1c435

Groups

adsbot-google
adsbot-google-mobile
apis-google
applebot
bingbot
bnf.fr_bot
exabot
googlebot
googlebot-image
googlebot-mobile
googlebot-news
googlebot-video
googlebot_nauxeo
ia_archiver
mediapartners-google
proximic
slurp
storebot-google
twitterbot
upday
sistrix
beopbot
weborama-fetcher
flipboard
flipboardproxy

Rule Path
Disallow /idalgo/
Disallow /carte/
Disallow /campub/
Disallow /recherche/
Disallow /legacy/
Disallow /_profiler/
Disallow /sar_topics/
Disallow /sar_als/
Disallow /reagir/
Disallow /blocks/article/
Disallow /block/article/
Disallow /styles-2018/
Disallow /www/

meltawer

Rule Path
Disallow /

digimind

Rule Path
Disallow /

knowings

Rule Path
Disallow /

sindup

Rule Path
Disallow /

cision

Rule Path
Disallow /

talkwater

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

converacrawler

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

kbcrawl

Rule Path
Disallow /

amisoftware

Rule Path
Disallow /

newzbin

Rule Path
Disallow /

ask n read

Rule Path
Disallow /

qwam content intelligence

Rule Path
Disallow /

zite

Rule Path
Disallow /

youmag

Rule Path
Disallow /

synthesio

Rule Path
Disallow /

trendybuzz

Rule Path
Disallow /

spotter

Rule Path
Disallow /

scoop.it

Rule Path
Disallow /

linkfluence

Rule Path
Disallow /

augure

Rule Path
Disallow /

corporama

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Allow /$
Disallow /*

k2spider

Rule Path
Disallow /

libwww

Rule Path
Disallow /

wget

Rule Path
Disallow /

adequat

Rule Path
Disallow /

adequat-systems

Rule Path
Disallow /

auramundi

Rule Path
Disallow /

coexel

Rule Path
Disallow /

ellisphere

Rule Path
Disallow /

leadbox

Rule Path
Disallow /

mention

Rule Path
Disallow /

moreover

Rule Path
Disallow /

mytwip

Rule Path
Disallow /

opinion-tracker

Rule Path
Disallow /

proxem

Rule Path
Disallow /

score3

Rule Path
Disallow /

trendeo

Rule Path
Disallow /

vecteurplus

Rule Path
Disallow /

verticalsearch

Rule Path
Disallow /

vsw

Rule Path
Disallow /

winello

Rule Path
Disallow /

fetch

Rule Path
Disallow /

infoseek

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

asknread.com

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

argus

Rule Path
Disallow /

augure

Rule Path
Disallow /

bloodhound

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

cydralspider

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

downloadexpress

Rule Path
Disallow /

edd

Rule Path
Disallow /

eureka

Rule Path
Disallow /

europresse

Rule Path
Disallow /

explore

Rule Path
Disallow /

factiva

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

gammaspider

Rule Path
Disallow /

httrack

Rule Path
Disallow /

indexer

Rule Path
Disallow /

kantar

Rule Path
Disallow /

larbin

Rule Path
Disallow /

linko

Rule Path
Disallow /

manageo

Rule Path
Disallow /

mediacompil

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

newscan-online

Rule Path
Disallow /

npbot

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

pimptrain

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

raven

Rule Path
Disallow /

readability.com

Rule Path
Disallow /

sindup

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

tunitinbot

Rule Path
Disallow /

up2news

Rule Path
Disallow /

verif

Rule Path
Disallow /

wapspider

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webzinger

Rule Path
Disallow /

webzip

Rule Path
Disallow /

wget

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.sudouest.fr/sitemap-news.xml
sitemap https://www.sudouest.fr/sitemap.xml
Back to top

Comments

  • Allowed search engines directives
  • Sitemaps
  • Algolia-Crawler-Verif: 484AF3B08B7789C7
Back to top

Warnings

  • 4 invalid lines.
Back to top