dhsforyou.com
robots.txt

Robots Exclusion Standard data for dhsforyou.com

Resource Scan

Scan Details

Site Domain dhsforyou.com
Base Domain dhsforyou.com
Scan Status Ok
Last Scan2025-10-21T10:44:54+00:00
Next Scan 2025-11-20T10:44:54+00:00

Last Scan

Scanned2025-10-21T10:44:54+00:00
URL https://dhsforyou.com/robots.txt
Redirect https://www.dtsit.com/robots.txt
Redirect Domain www.dtsit.com
Redirect Base dtsit.com
Domain IPs 104.21.10.139, 172.67.190.104, 2606:4700:3031::6815:a8b, 2606:4700:3031::ac43:be68
Redirect IPs 64.13.162.9
Response IP 64.13.162.9
Found Yes
Hash b4bf2a457277aad51690ef48aaebc4c5aa5852b912beaac0145198366d3a3332
SimHash 2f88485416f2

Groups

*

Rule Path
Allow /
Allow /blog/
Allow /blog/*
Allow /services/it-services/cybersecurity-consultant-services/
Allow /services/it-services/cloud-managed-services/
Allow /services/it-services/remote-it-onsite-support-services/
Allow /services/it-services/co-managed-it/
Allow /services/it-services/business-continuity-services/
Allow /services/it-projects/cloud-migration-consulting/
Allow /services/it-projects/it-office-relocation-services/
Allow /services/it-projects/it-project-management/
Allow /services/consulting-compliance/hipaa-compliance-consulting-services/
Allow /services/consulting-compliance/it-strategy-consulting-services/
Allow /services/consulting-compliance/it-compliance-consulting/
Allow /industries/healthcare/
Allow /industries/manufacturing/
Allow /industries/cpas/
Allow /industries/nonprofits/
Allow /pricing/
Allow /about-us/
Allow /contact-us/
Allow /discovery-session/
Disallow /assessment/
Disallow /assessment/*
Disallow /quizzes/
Disallow /quizzes/*
Disallow /manual/
Disallow /manual/*
Disallow /client-resources/
Disallow /client-resources/*
Disallow /wp-admin/
Disallow /wp-login.php
Disallow /wp-content/plugins/
Disallow /wp-content/themes/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.dtsit.com/sitemap.xml

Comments

  • Explicitly Allow Specific Pages and Directories
  • Explicitly Allow Specific Pages
  • Disallow Specific URLs and Their Subdirectories
  • Specific Blocking for Sensitive Directories
  • Crawl-delay to prevent overwhelming the server