cadeboite.fr
robots.txt

Robots Exclusion Standard data for cadeboite.fr

Resource Scan

Scan Details

Site Domain cadeboite.fr
Base Domain cadeboite.fr
Scan Status Ok
Last Scan2024-09-28T04:58:57+00:00
Next Scan 2024-10-05T04:58:57+00:00

Last Scan

Scanned2024-09-28T04:58:57+00:00
URL https://cadeboite.fr/robots.txt
Domain IPs 185.98.131.195
Response IP 185.98.131.195
Found Yes
Hash afc42a07633ca1cb37d4c6c5880c888a74b7c043624ccbd68fac2de443971339
SimHash 21125983c6a7

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /trackback
Disallow /feed
Disallow /comments
Disallow /category/*/*
Disallow */trackback
Disallow */comments
Disallow /archives/
Disallow /type/
Allow /tag/*/*
Disallow /author/
Disallow /*.php$
Disallow /*.inc$
Disallow /*?*
Disallow /*?
Disallow /pdf/
Disallow *.pdf
Disallow /wpostahs-slider-category/
Disallow /timeline_slider_post/

googlebot-mobile

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

mediapartners-google*

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

ninjabot

Rule Path
Allow /

tineye/1.1

Rule Path
Disallow /

yahoo-mmcrawler

Rule Path
Disallow

voilabot

Rule Path
Disallow /*.jpg$
Disallow /*.jpeg$
Disallow /*.gif$
Disallow /*.png$

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

msnbot-media

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

teoma

Rule Path
Disallow /*.jpg$
Disallow /*.jpeg$
Disallow /*.gif$
Disallow /*.png$

ia_archiver
duggmirror
sitecheck.internetseer.com
zealbot
msiecrawler
sitesnagger
webstripper
webcopier
fetch
offline explorer
teleport
teleportpro
webzip
linko
httrack
microsoft.url.control
xenu
larbin
libwww
zyborg
download ninja
wget
grub-client

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.cadeboite.fr/sitemap.xml