Log In Sign Up

datos-insights.com
robots.txt

Robots Exclusion Standard data for datos-insights.com

Resource Scan

Scan Details

Site Domain datos-insights.com
Base Domain datos-insights.com
Scan Status Ok
Last Scan2025-11-05T03:59:38+00:00
Next Scan 2025-12-05T03:59:38+00:00

Last Scan

Scanned2025-11-05T03:59:38+00:00
URL https://datos-insights.com/robots.txt
Domain IPs 104.26.14.159, 104.26.15.159, 172.67.75.2, 2606:4700:20::681a:e9f, 2606:4700:20::681a:f9f, 2606:4700:20::ac43:4b02
Response IP 172.67.75.2
Found Yes
Hash 87a1971ddff01d3d9ffc306405c328654fc83b8a9b963248a8f3e47788e3eebd
SimHash ed295800e684

Groups

*

Rule Path
Disallow /login/
Disallow /*option%3Dsaml_user_login*
Disallow /app/datos-insights_datoswebsite_1/*

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://datos-insights.com/sitemap_index.xml
Back to top

Comments

  • START YOAST BLOCK
  • ---------------------------
  • Block only the specific problematic authentication URLs
  • Block WordPress admin (standard practice)
  • XML Sitemap location
  • ---------------------------
  • END YOAST BLOCK
Back to top