Log In Sign Up

m.challenges.fr
robots.txt

Robots Exclusion Standard data for m.challenges.fr

Resource Scan

Scan Details

Site Domain m.challenges.fr
Base Domain challenges.fr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-04-03T13:13:25+00:00
Next Scan 2024-07-02T13:13:25+00:00

Last Successful Scan

Scanned2023-11-13T12:50:21+00:00
URL https://m.challenges.fr/robots.txt
Redirect https://www.challenges.fr/robots.txt
Redirect Domain www.challenges.fr
Redirect Base challenges.fr
Domain IPs 18.161.111.123, 18.161.111.129, 18.161.111.23, 18.161.111.85, 2600:9000:23d1:1c00:2:81cb:91c0:93a1, 2600:9000:23d1:4200:2:81cb:91c0:93a1, 2600:9000:23d1:6000:2:81cb:91c0:93a1, 2600:9000:23d1:6c00:2:81cb:91c0:93a1, 2600:9000:23d1:8200:2:81cb:91c0:93a1, 2600:9000:23d1:aa00:2:81cb:91c0:93a1, 2600:9000:23d1:ca00:2:81cb:91c0:93a1, 2600:9000:23d1:de00:2:81cb:91c0:93a1
Redirect IPs 108.138.189.51, 108.138.189.60, 108.138.189.68, 108.138.189.72, 2600:9000:248c:3a00:5:2ce0:f480:93a1, 2600:9000:248c:3c00:5:2ce0:f480:93a1, 2600:9000:248c:6000:5:2ce0:f480:93a1, 2600:9000:248c:6600:5:2ce0:f480:93a1, 2600:9000:248c:a00:5:2ce0:f480:93a1, 2600:9000:248c:ac00:5:2ce0:f480:93a1, 2600:9000:248c:ca00:5:2ce0:f480:93a1, 2600:9000:248c:cc00:5:2ce0:f480:93a1
Response IP 18.155.129.42
Found Yes
Hash cd7f993d30e43ffc060b1633603a225f37a25d479b9a950ef0db960cbd6805a7
SimHash 5907de2ac392

Groups

*

Rule Path
Disallow /redis
Disallow /comments
Disallow /shareCount
Disallow /automobile/mag
Disallow /*.asp
Disallow /*/breve.html
Disallow /*/article.html
Disallow /*/article_p*.html
Disallow /*.php
Disallow /search/*
Disallow /*/commentaires
Disallow /*.rss
Disallow /*.atom
Disallow /espace-debat/utilisateur/*

riddler

Rule Path
Disallow /

*

Rule Path
Disallow /_Incapsula_Resource?*
Back to top

Other Records

Field Value
sitemap https://www.challenges.fr/sitemap.news.xml
sitemap https://www.challenges.fr/sitemap.xml
sitemap https://www.challenges.fr/sitemap-images.xml
Back to top