testmonjob.fr
robots.txt

Robots Exclusion Standard data for testmonjob.fr

Resource Scan

Scan Details

Site Domain testmonjob.fr
Base Domain testmonjob.fr
Scan Status Ok
Last Scan2024-11-16T04:18:42+00:00
Next Scan 2024-11-23T04:18:42+00:00

Last Scan

Scanned2024-11-16T04:18:42+00:00
URL https://testmonjob.fr/robots.txt
Domain IPs 109.234.161.224
Response IP 109.234.161.224
Found Yes
Hash b6f4e9911dc19e88bbb9fccb7e7d473700f2d8ba4affdc229bf89fb69bd8f5b6
SimHash 5907dc326ab1

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-login.php
Allow /wp-admin/admin-ajax.php

awariosmartbot
ahrefsbot
aspiegelbot
blexbot
barkrowler
dotbot
mj12bot
mauibot
nimbostratus-bot
petalbot
semrushbot
seznambot
sogou
serpstatbot
trendiction
criteobot
twitterbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 180

nerdybot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.testmonjob.fr/sitemaps.xml

Warnings

  • 1 invalid line.