crucero.com.ar
robots.txt

Robots Exclusion Standard data for crucero.com.ar

Resource Scan

Scan Details

Site Domain crucero.com.ar
Base Domain crucero.com.ar
Scan Status Ok
Last Scan2024-09-17T06:13:10+00:00
Next Scan 2024-10-17T06:13:10+00:00

Last Scan

Scanned2024-09-17T06:13:10+00:00
URL https://crucero.com.ar/robots.txt
Redirect https://www.crucero.com.ar/robots.txt
Redirect Domain www.crucero.com.ar
Redirect Base crucero.com.ar
Domain IPs 13.227.254.121, 13.227.254.21, 13.227.254.38, 13.227.254.89
Redirect IPs 52.84.229.115, 52.84.229.36, 52.84.229.62, 52.84.229.64
Response IP 52.84.229.36
Found Yes
Hash 98d09938bcb96053cc290c0b84ec1c32420a1e591333c189ae3080a23d392acc
SimHash 0e43fe90e708

Groups

*

Rule Path
Disallow /
Disallow /ajax/affinage.php
Disallow /ajax/dateJackpot.php
Disallow /ajax/fiche.php
Disallow /ajax/jourferie.php
Disallow /ajax/moteur.php
Disallow /ajax/similaires.php
Disallow /ajax_jourferie.php
Disallow /ajax_moteur.php
Disallow /ajax_moteur_affinage.php
Disallow /ajax_moteurBateau.php
Disallow /ajax_moteurCompagnie.php
Disallow */action?
Disallow /search?
Disallow /search_resa?
Disallow */package-*
Disallow /fp-*
Disallow */tag/*

googlebot
googlebot-image
mediapartners-google
googlebot-news
googlebot-video
google-extended
adsbot-google
adsbot-google-mobile
adsbot-google-mobile-apps
bingbot
adidxbot
duckduckbot
slurp
oncrawl
qwantify
pinterest
ahrefsbot
ahrefssiteaudit
semrushbot
ahrefssiteaudit
barkrowler
chatgpt-user
gptbot

Rule Path
Allow /
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow *package2.html
Disallow /comp_101/
Disallow /ajax/
Disallow /cron/
Disallow /listeV4.php
Disallow /liste.php
Disallow /liste-produits/
Allow /*.jpg************
Allow /*.jpeg************
Allow /*.png************
Allow /*.gif************
Allow /*.webp************
Allow /*.svg************
Allow /*.tif************
Allow /*.mp4***********
Allow /*.css******************
Allow /*.js****************
Allow /*.pdf***************

Other Records

Field Value
sitemap https://www.crucero.com.ar/sitemaps/ar/sitemap-index.xml

Comments

  • Block access to all UA by default
  • Disallow miscellaneous page pattern
  • Allow access to specific UA
  • Disallow Blog URL pattern
  • Allow images and special medias