croisierenet.com
robots.txt

Robots Exclusion Standard data for croisierenet.com

Resource Scan

Scan Details

Site Domain croisierenet.com
Base Domain croisierenet.com
Scan Status Ok
Last Scan2024-09-19T06:50:27+00:00
Next Scan 2024-10-19T06:50:27+00:00

Last Scan

Scanned2024-09-19T06:50:27+00:00
URL https://croisierenet.com/robots.txt
Redirect https://www.croisierenet.com/robots.txt
Redirect Domain www.croisierenet.com
Redirect Base croisierenet.com
Domain IPs 13.33.30.111, 13.33.30.19, 13.33.30.43, 13.33.30.44
Redirect IPs 13.35.18.16, 13.35.18.78, 13.35.18.86, 13.35.18.87
Response IP 13.35.18.16
Found Yes
Hash 448cfe562bf84818825ee38e745092d4632e20e31c4a0c08aa73d11d2fb67c4f
SimHash 0ec3dd98e708

Groups

*

Rule Path
Disallow /
Disallow /ajax/affinage.php
Disallow /ajax/dateJackpot.php
Disallow /ajax/fiche.php
Disallow /ajax/jourferie.php
Disallow /ajax/moteur.php
Disallow /ajax/similaires.php
Disallow /ajax_jourferie.php
Disallow /ajax_moteur.php
Disallow /ajax_moteur_affinage.php
Disallow /ajax_moteurBateau.php
Disallow /ajax_moteurCompagnie.php
Disallow */action?
Disallow /search?
Disallow /search_resa?
Disallow */package-*
Disallow /fp-*
Disallow */tag/*

googlebot
googlebot-image
mediapartners-google
googlebot-news
googlebot-video
google-extended
adsbot-google
adsbot-google-mobile
adsbot-google-mobile-apps
bingbot
adidxbot
duckduckbot
slurp
oncrawl
qwantify
pinterest
ahrefsbot
ahrefssiteaudit
semrushbot
ahrefssiteaudit
barkrowler
chatgpt-user
gptbot

Rule Path
Allow /
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow *package2.html
Disallow /comp_101/
Disallow /ajax/
Disallow /cron/
Disallow /listeV4.php
Disallow /liste-produits/
Disallow */tag/*
Disallow /fp-*
Disallow */blog-voyage/?s=*

feedfetcher-google
googleproducer

Rule Path
Allow /********************
Allow /*.jpg************
Allow /*.jpeg************
Allow /*.png************
Allow /*.gif************
Allow /*.webp************
Allow /*.svg************
Allow /*.tif************
Allow /*.mp4***********
Allow /*.css******************
Allow /*.js****************
Allow /*.pdf***************

Other Records

Field Value
sitemap https://www.croisierenet.com/sitemaps/fr/sitemap-index.xml

Comments

  • Block access to all UA by default
  • Disallow miscellaneous page pattern
  • Allow access to specific UA
  • Disallow Blog URL pattern
  • Allow Discover
  • Allow images and special medias