Log In Sign Up

vitahaute.com
robots.txt

Robots Exclusion Standard data for vitahaute.com

Resource Scan

Scan Details

Site Domain vitahaute.com
Base Domain vitahaute.com
Scan Status Ok
Last Scan2026-03-16T18:50:54+00:00
Next Scan 2026-03-23T18:50:54+00:00

Last Scan

Scanned2026-03-16T18:50:54+00:00
URL https://vitahaute.com/robots.txt
Domain IPs 50.87.140.201
Response IP 50.87.140.201
Found Yes
Hash 56bdd084cc893f753243ba4b2f8b8f2381c3472957bde1330c14585d7bdf91ac
SimHash 7982da4664bb

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-includes/
Disallow /.git
Disallow /.env
Disallow /vendor/
Disallow /cgi-bin/
Disallow /readme.html
Disallow /license.txt
Disallow /?s=
Disallow /search/
Disallow /tag/
Disallow /author/
Disallow /date/
Disallow /attachment/
Disallow /comments/feed/
Disallow /category/*/*/feed/
Disallow /*?replytocom=
Disallow /*?utm_*
Disallow /*?fbclid=
Disallow /*?gclid=
Disallow /*?tracking=
Disallow /*?filter=
Disallow /*?sort=
Disallow /*?share=
Disallow /*?attachment_id=
Disallow /*?wordfence
Disallow /*?post_type=thirstylink*
Allow /wp-content/uploads/
Allow /wp-content/plugins/
Allow /wp-content/themes/
Allow /wp-includes/js/
Allow /*.jpg$
Allow /*.jpeg$
Allow /*.png$
Allow /*.gif$
Allow /page/

Other Records

Field Value
crawl-delay 5

ahrefsbot
mj12bot
semrushbot
dotbot
blexbot
megaindex
screaming frog seo spider
rogerbot
exabot
yeti
sistrix
wget
python-urllib
masscan
nimbostratus-bot
sitebot
seznambot
domaincrawler
linkpadbot
proximic
speedyspider
caphyon
feedfinder
baiduspider
cliqzbot
domainstatsbot
gigabot
httrack
scrapy
twengabot
turnitinbot
spinn3r
voilabot
yahoo! slurp
yodaobot
zoombot
semrushbot-sa
seekport crawler
bdcbot
openlinkprofiler
dotbot/1.0
ahrefsbot/6.1

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://vitahaute.com/sitemap_index.xml
Back to top

Comments

  • This virtual robots.txt file was created by the Virtual Robots.txt WordPress plugin: https://www.wordpress.org/plugins/pc-robotstxt/
  • Ultimate Super-Max Robots.txt for vitahaute.com
  • Protects WordPress, blocks aggressive bots, keeps SEO-friendly
  • Generated for maximum security + indexing efficiency
  • --- ads.txt placeholder ---
  • No programmatic ads currently in use
  • Prevents search engines from logging /ads.txt errors
  • --- Global crawl rules ---
  • Protect core WordPress areas
  • Block sensitive/developer files
  • Block thin content & low-value pages
  • Block unnecessary parameters
  • Allow essential resources for rendering
  • Allow images
  • Allow paginated pages
  • --- Aggressive / spam bots block ---
  • --- Sitemap ---
Back to top