moteur.ma
robots.txt

Robots Exclusion Standard data for moteur.ma

Resource Scan

Scan Details

Site Domain moteur.ma
Base Domain moteur.ma
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-17T03:58:12+00:00
Next Scan 2024-12-16T03:58:12+00:00

Last Successful Scan

Scanned2023-12-26T10:55:25+00:00
URL https://moteur.ma/robots.txt
Redirect https://www.moteur.ma/robots.txt
Redirect Domain www.moteur.ma
Redirect Base moteur.ma
Domain IPs 51.254.199.4
Redirect IPs 51.254.199.4
Response IP 51.254.199.4
Found Yes
Hash 5b16e6aad48baab76c5ce0358a4be63c58cce404d75d2a7a85846c39ff0ffe31
SimHash ea54ce20f618

Groups

*

Rule Path
Disallow /fr/ads/
Disallow /ar/ads/
Disallow /fr/neuf/get_version/
Disallow /ar/neuf/get_version/
Disallow /fr/neuf/viewsup/
Disallow /ar/neuf/viewsup/

facebookbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 9

bingbot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

slurp

Rule Path
Disallow /

yahoo! slurp

Rule Path
Disallow /

wiederfreibot/1.0

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot/1.0

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

exabot

Rule Path
Disallow /

pathdefender

Rule Path
Disallow /

goodzer

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

yandex

Rule Path
Disallow /

bspider

Rule Path
Disallow /

voltron

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

linkdexbot/2.0

Rule Path
Disallow /

getintentcrawler

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

vegebot

Rule Path
Disallow /

vegi bot

Rule Path
Disallow /

adbeat_bot

Rule Path
Disallow /

jooblebot

Rule Path
Disallow /