jobs.recruiter.com
robots.txt
Robots Exclusion Standard data for jobs.recruiter.com
Resource Scan
Scan Details
| Site Domain | jobs.recruiter.com |
| Base Domain | recruiter.com |
| Scan Status | Ok |
| Last Scan | 2024-10-19T12:31:06+00:00 |
| Next Scan | 2024-11-18T12:31:06+00:00 |
Last Scan
| Scanned | 2024-10-19T12:31:06+00:00 |
| URL | https://jobs.recruiter.com/robots.txt |
| Domain IPs | 108.157.254.108, 108.157.254.37, 108.157.254.58, 108.157.254.93 |
| Response IP | 108.157.254.37 |
| Found | Yes |
| Hash | 68fba1ba747f6baa776c822d1cd25c4a0cf82a0b370f94e71de286369c97aea7 |
| SimHash | ad4858018352 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /create-alert?* |
| Disallow | /?* |
| Disallow | /jobs?* |
| Allow | /?nextPageToken* |
| Allow | /jobs?nextPageToken* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://jobs.recruiter.com/jobs-sitemap.xml |