hawa.jobs
robots.txt
Robots Exclusion Standard data for hawa.jobs
Resource Scan
Scan Details
| Site Domain | hawa.jobs |
| Base Domain | hawa.jobs |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-09-17T03:49:59+00:00 |
| Next Scan | 2024-12-16T03:49:59+00:00 |
Last Successful Scan
| Scanned | 2024-05-20T19:39:39+00:00 |
| URL | https://www.hawa.jobs/robots.txt |
| Domain IPs | 2600:9000:2003:1000:1c:a282:9cc0:93a1, 2600:9000:2003:1800:1c:a282:9cc0:93a1, 2600:9000:2003:5400:1c:a282:9cc0:93a1, 2600:9000:2003:9800:1c:a282:9cc0:93a1, 2600:9000:2003:ba00:1c:a282:9cc0:93a1, 2600:9000:2003:c200:1c:a282:9cc0:93a1, 2600:9000:2003:d400:1c:a282:9cc0:93a1, 2600:9000:2003:dc00:1c:a282:9cc0:93a1, 52.84.229.119, 52.84.229.44, 52.84.229.50, 52.84.229.95 |
| Response IP | 52.84.229.95 |
| Found | Yes |
| Hash | 2b22c869c0da1e094f3a8b69aa2aa78305e333f1282b70310c1cd178ea1c4821 |
| SimHash | 630341484f94 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /admin$ |
| Disallow | /admin/* |
| Disallow | /sa$ |
| Disallow | /sa/* |
| Disallow | /api/* |
| Disallow | /users/auth/* |
| Disallow | /sso/* |
| Disallow | /*?* |
| Disallow | /templates/* |
| Allow | /db_assets/production*?t=* |
| Disallow | /job/*/apply |
| Disallow | /job/*/save_job |
| Disallow | /job/*/unsave_job |
| Disallow | /jobs/*/*/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.hawa.jobs/sitemap.xml |