jobsagents.com
robots.txt
Robots Exclusion Standard data for jobsagents.com
Resource Scan
Scan Details
| Site Domain | jobsagents.com |
| Base Domain | jobsagents.com |
| Scan Status | Ok |
| Last Scan | 2025-04-03T08:12:29+00:00 |
| Next Scan | 2025-04-10T08:12:29+00:00 |
Last Scan
| Scanned | 2025-04-03T08:12:29+00:00 |
| URL | https://jobsagents.com/robots.txt |
| Domain IPs | 95.158.143.100 |
| Response IP | 95.158.143.100 |
| Found | Yes |
| Hash | 4d5e17d0d36bcfae316b9db824cc14e9138c467510d6f54d4f41c006ebcd4449 |
| SimHash | e1435e4b39d0 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | *.axd$ | - |
| Disallow | /*.axd$ | - |
| Disallow | /ScriptResource.axd | - |
| Disallow | /WebResource.axd | - |
| Disallow | /*/ScriptResource.axd | - |
| Disallow | /*/WebResource.axd | - |
| Disallow | /*/GenerateXML.ashx | - |
| Disallow | /*/extractjobs.ashx | - |
| Disallow | /*/Keywords.ashx | - |
| Disallow | /redirect | - |
| Disallow | /PayPal.aspx | - |
| Disallow | /IPNHandler.aspx | - |
| Disallow | /sendjobalert.aspx | - |
| Disallow | /NotifyForExpiredAds.aspx | - |
| Disallow | /App_Browsers/ | - |
| Disallow | /App_Code/ | - |
| Disallow | /bin/ | - |
| Disallow | /App_Themes/ | - |
| Disallow | /App_LocalResources/ | - |
| Disallow | /Images/ | - |
| Disallow | /Scripts/ | - |
| Disallow | /Uploads/ | This is an infinite virtual URL space |
Other Records
| Field | Value |
|---|---|
| sitemap | http://www.jobsagents.com/sitemap.xml |
Comments