kentjobs.co.uk
robots.txt
Robots Exclusion Standard data for kentjobs.co.uk
Resource Scan
Scan Details
| Site Domain | kentjobs.co.uk |
| Base Domain | kentjobs.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-09-28T17:20:04+00:00 |
| Next Scan | 2024-10-05T17:20:04+00:00 |
Last Scan
| Scanned | 2024-09-28T17:20:04+00:00 |
| URL | https://kentjobs.co.uk/robots.txt |
| Redirect | https://www.kentjobs.co.uk/robots.txt |
| Redirect Domain | www.kentjobs.co.uk |
| Redirect Base | kentjobs.co.uk |
| Domain IPs | 168.63.98.176 |
| Redirect IPs | 168.63.98.176 |
| Response IP | 168.63.98.176 |
| Found | Yes |
| Hash | b17370d3bd71db16322c2216f342f5b1b096ec0bd2302f0c389044bda47b43de |
| SimHash | 7c5f7a7cbb52 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /Microsoft.WindowsAzure.Caching/ |
| Disallow | /Scripts/ |
| Disallow | /_controls/ |
| Disallow | /_configuration/ |
| Disallow | /_customerrors/ |
| Disallow | /_integrations/ |
| Disallow | /_management/ |
| Disallow | /_rewritten/ |
| Disallow | /holding/ |
| Disallow | /referencehandler.ashx |
| Disallow | /statisticincrementer.ashx |
| Disallow | /vacancyclickincrementer.ashx |
| Disallow | /vacancyviewincrementer.ashx |
| Disallow | /preview.aspx |
| Disallow | /_feeds/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 0.2 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.kentjobs.co.uk/sitemap.xml |