Log In Sign Up

destinia.com
robots.txt

Robots Exclusion Standard data for destinia.com

Resource Scan

Scan Details

Site Domain destinia.com
Base Domain destinia.com
Scan Status Ok
Last Scan2024-09-20T12:17:52+00:00
Next Scan 2024-09-27T12:17:52+00:00

Last Scan

Scanned2024-09-20T12:17:52+00:00
URL https://destinia.com/robots.txt
Domain IPs 104.26.6.74, 104.26.7.74, 172.67.70.190, 2606:4700:20::681a:64a, 2606:4700:20::681a:74a, 2606:4700:20::ac43:46be
Response IP 104.26.7.74
Found Yes
Hash 075b4426879fa8aef8f58e6f17bc0853556360bbb56e853830f7f7083412f9ca
SimHash 404a90400eb8

Groups

*

Rule Path
Disallow /online/
Disallow /hotel_comments/
Disallow /www/r.php
Disallow /www/b.php
Disallow /www/ajax/index.php
Disallow /www/yahoo/get_hotels.php
Disallow /www/boletines
Disallow /www/rss
Disallow /www/localizacion.php
Disallow /www/customer.php
Disallow /www/hotel_details
Disallow /*hotelsearch_checkin_day
Allow /www/viajes_espacio
Allow /web/black-friday
Disallow /*web/tmp
Disallow /*_priceasc
Disallow /*_pricedesc
Disallow /*_nameasc
Disallow /*_namedesc
Disallow /*_categoryasc
Disallow /*_categorydesc
Disallow /*_ptsasc
Disallow /*_distancedesc
Disallow /*_distanceasc
Disallow /*_servicesdesc
Disallow /*_servicesasc
Disallow /*_ptsdesc
Disallow /*gr%3D
Disallow /*forecasts

*

Rule Path
Disallow /guides/wp-includes/
Allow /guides/wp-admin/admin-ajax.php
Disallow /guides/wp-admin/
Disallow /guides/trackback/
Disallow /guides/wp-content/plugins/
Disallow /guides/wp-content/themes/
Disallow /guides/wp-content/cache/
Allow /guides/wp-content/*.css
Allow /guides/wp-content/*.less
Allow /guides/wp-content/*.js
Allow /guides/wp-content/*.jpg
Allow /guides/wp-content/*.jpeg
Allow /guides/wp-content/*.png
Allow /guides/wp-content/*.webp
Allow /guides/wp-content/*.webm
Disallow /guides/wp-login.php
Disallow /guides/wp-register.php
Disallow /guides/xmlrpc.php
Disallow /guides/cgi-bin/
Disallow /guides/?s=
Disallow /guides/search
Disallow /guides/archives/
Disallow /guides/index.php
Disallow /guides/*.php$
Disallow /guides/*.inc$
Disallow /guides/trackback/
Disallow /guides/*/trackback/
Disallow /guides/page/
Disallow /guides/tag/
Disallow /guides/category/
Disallow /guides/feed/
Disallow /guides/comments/feed/
Disallow /guides/*?replytocom
Allow /guides/wp-content/uploads/
Disallow /guides/*.git$
Disallow /guides/*.sql$
Disallow /guides/*.tgz$
Disallow /guides/*.gz$
Disallow /guides/*.tar$
Disallow /guides/*.svn$
Disallow /guides/*.bz2$
Disallow /guides/*.log$

gptbot

Rule Path
Disallow /
Allow /m/faqs
Allow /m/about_us

Other Records

Field Value
crawl-delay 10

anthropic-ai

Rule Path
Disallow /
Allow /m/faqs
Allow /m/about_us

Other Records

Field Value
crawl-delay 10

google-extended

Rule Path
Disallow /
Allow /m/faqs
Allow /m/about_us

Other Records

Field Value
crawl-delay 10

claude-web

Rule Path
Disallow /
Allow /m/faqs
Allow /m/about_us

Other Records

Field Value
crawl-delay 10

claudebot

Rule Path
Disallow /
Allow /m/faqs
Allow /m/about_us

Other Records

Field Value
crawl-delay 10

perplexitybot

Rule Path
Disallow /
Allow /m/faqs
Allow /m/about_us

Other Records

Field Value
crawl-delay 10

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claritybot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /

findxbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

httrack

Rule Path
Disallow /

libwww

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

megaindex.ru/

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

memorybot

Rule Path
Disallow /

mj12

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 50

panscient.com

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

seznam

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

smrjbot

Rule Path
Disallow /

webcopier

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://destinia.com/sitemap-index-es.xml.gz
sitemap https://destinia.com/sitemap-index-en.xml.gz
sitemap https://destinia.com/sitemap-index-pt.xml.gz
sitemap https://destinia.com/sitemap-index-pt-br.xml.gz
sitemap https://destinia.com/sitemap-index-fr.xml.gz
sitemap https://destinia.com/sitemap-index-it.xml.gz
sitemap https://destinia.com/sitemap-index-de.xml.gz
sitemap https://destinia.com/sitemap-index-ar.xml.gz
sitemap https://destinia.com/sitemap-index-ca.xml.gz
sitemap https://destinia.com/sitemap-index-da.xml.gz
sitemap https://destinia.com/sitemap-index-he.xml.gz
sitemap https://destinia.com/sitemap-index-hu.xml.gz
sitemap https://destinia.com/sitemap-index-ja.xml.gz
sitemap https://destinia.com/sitemap-index-ko.xml.gz
sitemap https://destinia.com/sitemap-index-nl.xml.gz
sitemap https://destinia.com/sitemap-index-pl.xml.gz
sitemap https://destinia.com/sitemap-index-ro.xml.gz
sitemap https://destinia.com/sitemap-index-ru.xml.gz
sitemap https://destinia.com/sitemap-index-sv.xml.gz
sitemap https://destinia.com/sitemap-index-tr.xml.gz
sitemap https://destinia.com/sitemap-index-zh.xml.gz
sitemap https://destinia.com/sitemap-image-index-es.xml.gz
sitemap https://destinia.com/sitemap-image-index-en.xml.gz
sitemap https://destinia.com/sitemap-image-index-pt.xml.gz
sitemap https://destinia.com/sitemap-image-index-pt-br.xml.gz
sitemap https://destinia.com/sitemap-image-index-fr.xml.gz
sitemap https://destinia.com/sitemap-image-index-it.xml.gz
sitemap https://destinia.com/sitemap-image-index-de.xml.gz
sitemap https://destinia.com/sitemap-image-index-ar.xml.gz
sitemap https://destinia.com/sitemap-image-index-ca.xml.gz
sitemap https://destinia.com/sitemap-image-index-da.xml.gz
sitemap https://destinia.com/sitemap-image-index-he.xml.gz
sitemap https://destinia.com/sitemap-image-index-hu.xml.gz
sitemap https://destinia.com/sitemap-image-index-ja.xml.gz
sitemap https://destinia.com/sitemap-image-index-ko.xml.gz
sitemap https://destinia.com/sitemap-image-index-nl.xml.gz
sitemap https://destinia.com/sitemap-image-index-pl.xml.gz
sitemap https://destinia.com/sitemap-image-index-ro.xml.gz
sitemap https://destinia.com/sitemap-image-index-ru.xml.gz
sitemap https://destinia.com/sitemap-image-index-sv.xml.gz
sitemap https://destinia.com/sitemap-image-index-tr.xml.gz
sitemap https://destinia.com/sitemap-image-index-zh.xml.gz
Back to top

Comments

  • WP Rules
  • Backups
  • AI
  • Blocked Bots
  • Sitemaps
Back to top