recruit.net
robots.txt
Robots Exclusion Standard data for recruit.net
Resource Scan
Scan Details
| Site Domain | recruit.net |
| Base Domain | recruit.net |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-06-12T00:49:17+00:00 |
| Next Scan | 2024-06-26T00:49:17+00:00 |
Last Successful Scan
| Scanned | 2024-05-28T00:16:09+00:00 |
| URL | https://recruit.net/robots.txt |
| Redirect | https://www.recruit.net/robots.txt |
| Redirect Domain | www.recruit.net |
| Redirect Base | recruit.net |
| Domain IPs | 104.22.38.219, 104.22.39.219, 172.67.41.26, 2606:4700:10::6816:26db, 2606:4700:10::6816:27db, 2606:4700:10::ac43:291a |
| Redirect IPs | 104.22.38.219, 104.22.39.219, 172.67.41.26, 2606:4700:10::6816:26db, 2606:4700:10::6816:27db, 2606:4700:10::ac43:291a |
| Response IP | 104.22.38.219 |
| Found | Yes |
| Hash | b67a2f94e09297d96aa93d016df0a99baf0bd185a8e11dc6aad368b37fcdf6dc |
| SimHash | dbd5dba2c6c3 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search.html |
| Disallow | /apisearch.html |
| Disallow | /gotojob.html |
| Disallow | /my_jobs.html |
| Disallow | /my_alerts.html |
| Disallow | /my_account.html |
| Disallow | /login.html |
| Disallow | /my_search.html |
| Disallow | /directjob.html |
| Disallow | /sponsorjob.html |
| Disallow | /gotoad.html |
| Disallow | /GetJobAt/http/ |
| Disallow | /recruitnet/get_new_job_count.jsp |
| Disallow | /linkedinlogin_.html |
| Disallow | /facebooklogin_.html |
| Disallow | /linkedinlogin.html |
| Disallow | /advanced_search.html |
| Disallow | /trackerror.html |
| Disallow | /demo/* |
| Disallow | /recruitnet/sg/* |
| Disallow | /recruit/* |
| Disallow | /recruitnet/all/googleonetap.html |