lematin.ma
robots.txt

Robots Exclusion Standard data for lematin.ma

Resource Scan

Scan Details

Site Domain lematin.ma
Base Domain lematin.ma
Scan Status Ok
Last Scan2024-10-29T00:28:49+00:00
Next Scan 2024-11-05T00:28:49+00:00

Last Scan

Scanned2024-10-29T00:28:49+00:00
URL https://lematin.ma/robots.txt
Domain IPs 104.26.14.64, 104.26.15.64, 172.67.69.186, 2606:4700:20::681a:e40, 2606:4700:20::681a:f40, 2606:4700:20::ac43:45ba
Response IP 104.26.15.64
Found Yes
Hash 56e002d6f35a5f5d6a2684569c0b1c4fec0fafed29f980bedc54c65f44768911
SimHash 380cd019c9ed

Groups

*

Rule Path
Disallow /ajax/*
Disallow /ajax*
Disallow /print*
Disallow /getRelatedArticles*
Disallow /getMostReadArticles*
Disallow /article_count/*
Disallow /get-menu-header*
Disallow /article.php*
Disallow /login-mgt
Disallow /widget/*
Disallow */page/*
Disallow /_beta/
Disallow /_lematin/
Disallow /api/*
Disallow /js/
Disallow /files/
Disallow /apis/
Disallow /supplement/economie/2009/Credit-conso_Les-menages-a-bout-de-souffle/Politique-commerciale_La-bonne-voie-oui-maisa-euro/
Disallow /devise-historique/
Disallow /cinema/apiseances/
Disallow /ccgm/inscription.html
Disallow /journal/2019/c-parti-4-jours-musique-judeo-arabe/325523.html%C2%A0%C2%A0/
Disallow /article/plus-video/?cat=cis&before=
Disallow /ccgm-inscription.html
Disallow /post-question.html?t=
Disallow /article/plus-video/?cat=siam&before=
Disallow /ajax/autocomplate/?mot=
Disallow /post-question.html
Disallow /edition-electronique/*?*
Disallow /archives/maroc/*/*?*
Disallow /archives/Maroc/*/*?*
Disallow /journal/*/*/*.html*?*
Disallow /express/*/*/*.html*?*
Disallow /*/*/*/*/*/*.html*?*
Disallow /reader/abonnement/*?*
Disallow /cinema/apiactualement
Disallow /pharmacie-signalinfo/
Disallow /evenement/colloque-fef.html
Disallow /ar/sports/interviews
Disallow /api/amp/derniere-heure/*?*
Disallow /article/plus-video/?cat=afriquedeveloppement&before=
Disallow /article/plus-video/?cat=logismed&before=
Disallow /service-emailing.html?email=
Disallow /cdn-cgi/l/email-protection
Disallow /site/auth/?authclient=google
Disallow /service-emailing.html
Disallow /cinema/apihome/

twitterbot

Rule Path
Allow /images

facebookexternalhit

Rule Path
Allow /images

Comments

  • Certain social media sites are whitelisted to allow crawlers to access page markup when links to /images are shared.