Log In Sign Up

sandyaveledo.net
robots.txt

Robots Exclusion Standard data for sandyaveledo.net

Resource Scan

Scan Details

Site Domain sandyaveledo.net
Base Domain sandyaveledo.net
Scan Status Ok
Last Scan2025-03-12T15:46:57+00:00
Next Scan 2025-03-19T15:46:57+00:00

Last Scan

Scanned2025-03-12T15:46:57+00:00
URL https://sandyaveledo.net/robots.txt
Domain IPs 104.21.68.114, 172.67.195.6, 2606:4700:3031::6815:4472, 2606:4700:3031::ac43:c306
Response IP 172.67.195.6
Found Yes
Hash d1bb21b16332fb5beaf40bf83eaf8cd3d75ef7fe79b43a40c8be11e8501ccb96
SimHash 6c3849d126b3

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /private/
Disallow /readme.html
Disallow /license.txt
Disallow /search/
Disallow /?s=
Disallow /author/
Disallow
Disallow /tag/
Disallow /wp-json/
Disallow /?rest_route=
Disallow /page/*/?s=
Disallow /*?wordfence_lh=
Allow /*.webp$
Disallow /*?*redirect_to=
Disallow /*?*preview=true

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /*.webp$

googlebot-news

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

badbot

Rule Path
Disallow /

evilscraper

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://sandyaveledo.net/sitemap_index.xml
sitemap https://sandyaveledo.net/sitemap-images.xml
Back to top

Comments

  • OPTIMIZED ROBOTS.TXT FOR SEO, PERFORMANCE, AND SECURITY
  • Site: sandyaveledo.net
  • ---------------------------
  • GENERAL SETTINGS
  • Disallow access to sensitive directories and files
  • Disallow WordPress-specific query parameters
  • Allow .webp images to be indexed
  • Block query parameters often used in unwanted scraping
  • ---------------------------
  • SPECIFIC USER-AGENT RULES
  • Allow Googlebot (general)
  • Allow Googlebot-Image to index .webp images
  • Allow Googlebot for News, Video, and Mobile
  • ---------------------------
  • SITEMAP LOCATION
  • Point to your XML sitemap(s)
  • ---------------------------
  • BLOCK BAD BOTS
  • Example of blocking specific user agents
  • END OF FILE
Back to top