techbehemoths.com
robots.txt
Robots Exclusion Standard data for techbehemoths.com
Resource Scan
Scan Details
| Site Domain | techbehemoths.com |
| Base Domain | techbehemoths.com |
| Scan Status | Ok |
| Last Scan | 2026-01-03T07:02:53+00:00 |
| Next Scan | 2026-02-02T07:02:53+00:00 |
Last Scan
| Scanned | 2026-01-03T07:02:53+00:00 |
| URL | https://techbehemoths.com/robots.txt |
| Domain IPs | 104.26.2.150, 104.26.3.150, 172.67.71.166, 2606:4700:20::681a:296, 2606:4700:20::681a:396, 2606:4700:20::ac43:47a6 |
| Response IP | 104.26.2.150 |
| Found | Yes |
| Hash | a21626971a9afa40c1383b35a07f3c5dc56574519e0765131142014bf8e4f9f5 |
| SimHash | e110cf34a7d2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /pages/ |
| Disallow | /helpers/ |
| Disallow | /email/ |
| Disallow | /dashboard/ |
| Disallow | /company/preview/noactions/ |
| Disallow | /company/join/claim/ |
| Disallow | /valet/confirmation/ |
| Disallow | /project-submission/confirmation/ |
| Disallow | /*?service_name= |
| Disallow | /*?loc_name= |
| Disallow | /*service_url%3D |
| Disallow | /*loc_url%3D |
| Disallow | /safe-url |
| Disallow | /blog/preview/ |
| Disallow | /cdn-cgi/rum |
Other Records
| Field | Value |
|---|---|
| sitemap | https://techbehemoths.com/sitemap.xml |