cruceros.cl
robots.txt

Robots Exclusion Standard data for cruceros.cl

Resource Scan

Scan Details

Site Domain cruceros.cl
Base Domain cruceros.cl
Scan Status Ok
Last Scan2024-05-09T22:12:28+00:00
Next Scan 2024-06-08T22:12:28+00:00

Last Scan

Scanned2024-05-09T22:12:28+00:00
URL https://cruceros.cl/robots.txt
Redirect https://www.cruceros.cl/robots.txt
Redirect Domain www.cruceros.cl
Redirect Base cruceros.cl
Domain IPs 52.84.229.109, 52.84.229.37, 52.84.229.38, 52.84.229.57
Redirect IPs 13.225.4.116, 13.225.4.2, 13.225.4.33, 13.225.4.6
Response IP 13.225.4.6
Found Yes
Hash 2c7eaa1b783af8179657b225545a7181f4cb4dd6f613965f66f404537d7d0db4
SimHash 0ec3fe90e718

Groups

*

Rule Path
Disallow /
Disallow /ajax/affinage.php
Disallow /ajax/dateJackpot.php
Disallow /ajax/fiche.php
Disallow /ajax/jourferie.php
Disallow /ajax/moteur.php
Disallow /ajax/similaires.php
Disallow /ajax_jourferie.php
Disallow /ajax_moteur.php
Disallow /ajax_moteur_affinage.php
Disallow /ajax_moteurBateau.php
Disallow /ajax_moteurCompagnie.php
Disallow */action?
Disallow /search?
Disallow /search_resa?
Disallow */package-*
Disallow /fp-*
Disallow */tag/*

googlebot
googlebot-image
mediapartners-google
googlebot-news
googlebot-video
google-extended
adsbot-google
adsbot-google-mobile
adsbot-google-mobile-apps
bingbot
adidxbot
duckduckbot
slurp
oncrawl
qwantify
pinterest
ahrefsbot
ahrefssiteaudit
semrushbot
ahrefssiteaudit
barkrowler
chatgpt-user
gptbot

Rule Path
Allow /
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow *package2.html
Disallow /comp_101/
Disallow /ajax/
Disallow /cron/
Disallow /listeV4.php
Disallow /liste.php
Disallow /liste-produits/
Allow /*.jpg************
Allow /*.jpeg************
Allow /*.png************
Allow /*.gif************
Allow /*.webp************
Allow /*.svg************
Allow /*.tif************
Allow /*.mp4***********
Allow /*.css******************
Allow /*.js****************
Allow /*.pdf***************

Other Records

Field Value
sitemap https://www.cruceros.cl/sitemaps/cl/sitemap-index.xml

Comments

  • Block access to all UA by default
  • Disallow miscellaneous page pattern
  • Allow access to specific UA
  • Disallow Blog URL pattern
  • Allow images and special medias