job.id
robots.txt
Robots Exclusion Standard data for job.id
Resource Scan
Scan Details
Site Domain | job.id |
Base Domain | job.id |
Scan Status | Ok |
Last Scan | 2024-09-25T17:12:50+00:00 |
Next Scan | 2024-10-02T17:12:50+00:00 |
Last Scan
Scanned | 2024-09-25T17:12:50+00:00 |
URL | https://job.id/robots.txt |
Domain IPs | 104.21.78.104, 172.67.220.57, 2606:4700:3030::ac43:dc39, 2606:4700:3034::6815:4e68 |
Response IP | 172.67.220.57 |
Found | Yes |
Hash | 123414497a0b53d2ef469586c6758678fef860405aab84f52f0a97ae50f1c784 |
SimHash | 99183341dcf2 |
Groups
*
Rule | Path |
---|---|
Disallow | /password-request |
Disallow | /resend-activation |
Disallow | /shortlist/* |
Other Records
Field | Value |
---|---|
sitemap | https://job.id/sitemap.xml |
Warnings
- 2 invalid lines.
Comments