adecco.co.uk
robots.txt
Robots Exclusion Standard data for adecco.co.uk
Resource Scan
Scan Details
| Site Domain | adecco.co.uk |
| Base Domain | adecco.co.uk |
| Scan Status | Ok |
| Last Scan | 2024-09-05T20:18:42+00:00 |
| Next Scan | 2024-10-05T20:18:42+00:00 |
Last Scan
| Scanned | 2024-09-05T20:18:42+00:00 |
| URL | https://adecco.co.uk/robots.txt |
| Redirect | https://www.adecco.co.uk/robots.txt |
| Redirect Domain | www.adecco.co.uk |
| Redirect Base | adecco.co.uk |
| Domain IPs | 13.69.68.63 |
| Redirect IPs | 13.107.246.59, 2620:1ec:bdf::59 |
| Response IP | 13.107.246.59 |
| Found | Yes |
| Hash | 4410c07e4211c9f2290bcbef9727e4094045462e118d74bcfc0a925b1f37424e |
| SimHash | 5244ef41268a |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | /App_Browsers/ | - |
| Disallow | /App_config/ | - |
| Disallow | /App_Data/ | - |
| Disallow | /sitecore | - |
| Disallow | /Sitecore | - |
| Disallow | /sitecore_files/ | - |
| Disallow | /temp/ | - |
| Disallow | /upload/ | - |
| Disallow | /xsl/ | - |
| Disallow | /sitecore*/ | - |
| Disallow | /App_*/ | - |
| Disallow | *cm-adecco-uk.prd.cms* | Disallow pages containing 'cm-adecco-uk.prd.cms' in the URL |
omniexplorer_bot
verticrawlbot
addsearchbot
admantx
ahrefsbot
baiduspider
baiduspider-video
baiduspider-image
changedetection
dotbot
exabot
gozaikbot
grapeshot
ichiro
sogou spider
wesee
wijubot
xovibot
cliqzbot
piplbot
ccbot
jobdiggerspider
slurp
yandex
ia_archiver
aihitbot
barkrowler
bdcbot
blexbot
blp_bbot
brokenlinkcheck.com
buck
cyencebot
domaincrawler
dow jones searchbot
extlinksbot
femtosearchbot
fever
garlikcrawler
gigabot
gobuster
grapeshotcrawler
heritrix
istellabot
jersey
jobkicks
libwww-perl
linkdexbot
linkpadbot
ltx71 - (http://ltx71.com/)
lua-resty-http
lumtelbot
magpie-crawler
magus bot
mail.ru_bot
megaindex.ru
nl-crawler
onpagebot
riddler
scoutjet
scrapy
seekport
seznambot
siteimprove
smtbot
uptimerobot
velenpublicwebcrawler
wget
yacybot
yeti
yisouspider
yunsecuritybot
zoominfobot
| Rule | Path |
|---|---|
| Disallow | / |
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.adecco.co.uk/index-sitemap.xml |
Warnings
- 3 invalid lines.
Comments