tulip.liv.ac.uk
robots.txt
Robots Exclusion Standard data for tulip.liv.ac.uk
Resource Scan
Scan Details
| Site Domain | tulip.liv.ac.uk |
| Base Domain | liv.ac.uk |
| Scan Status | Ok |
| Last Scan | 2024-11-03T09:30:58+00:00 |
| Next Scan | 2024-12-03T09:30:58+00:00 |
Last Scan
| Scanned | 2024-11-03T09:30:58+00:00 |
| URL | https://tulip.liv.ac.uk/robots.txt |
| Domain IPs | 138.253.13.119 |
| Response IP | 138.253.13.119 |
| Found | Yes |
| Hash | ffc16b5e4adf7f024bf90f6f93605d5ef3e0dc506e9929e5c82519ac56ea4434 |
| SimHash | 18ddb44306e6 |
Groups
htdig-3.1.5
harvest-ng
harvest
w3c-checklink
lycos
lycos_spider_(t-rex)
gulliver
scooter
infoseek
ultraseek
fido
netmechanic
architextspider
mirago
slurp
googlebot
google
webcrawler
ia_archiver
gsa-crawler
msnbot
twiceler
| Rule | Path |
|---|---|
| Disallow | /perl/ |
| Disallow | /cgi-bin/ |
| Disallow | /fastcgi/ |
| Disallow | /fcgi-bin/ |
| Disallow | /audio/ |
| Disallow | /icons/ |
biocrawler
| Rule | Path |
|---|---|
| Disallow | /a |
| Disallow | /b |
| Disallow | /ca |
| Disallow | /cb |
| Disallow | /cc |
| Disallow | /cd |
| Disallow | /ce |
| Disallow | /cf |
| Disallow | /cg |
| Disallow | /ch |
| Disallow | /cia |
| Disallow | /cib |
| Disallow | /cic |
| Disallow | /cid |
| Disallow | /cie |
| Disallow | /cif |
| Disallow | /cig |
| Disallow | /cih |
| Disallow | /cii |
| Disallow | /cij |
| Disallow | /cik |
| Disallow | /cim |
| Disallow | /cin |
| Disallow | /cio |
| Disallow | /cip |
| Disallow | /ciq |
| Disallow | /cir |
| Disallow | /cis |
| Disallow | /cit |
| Disallow | /ciu |
| Disallow | /civ |
| Disallow | /ciw |
| Disallow | /cix |
| Disallow | /ciy |
| Disallow | /ciz |
| Disallow | /cj |
| Disallow | /ck |
| Disallow | /cl |
| Disallow | /cm |
| Disallow | /cn |
| Disallow | /co |
| Disallow | /cp |
| Disallow | /cq |
| Disallow | /cr |
| Disallow | /cs |
| Disallow | /ct |
| Disallow | /cu |
| Disallow | /cv |
| Disallow | /cw |
| Disallow | /cx |
| Disallow | /cy |
| Disallow | /cz |
| Disallow | /d |
| Disallow | /e |
| Disallow | /f |
| Disallow | /g |
| Disallow | /h |
| Disallow | /i |
| Disallow | /j |
| Disallow | /k |
| Disallow | /l |
| Disallow | /m |
| Disallow | /n |
| Disallow | /o |
| Disallow | /p |
| Disallow | /q |
| Disallow | /r |
| Disallow | /s |
| Disallow | /t |
| Disallow | /u |
| Disallow | /v |
| Disallow | /w |
| Disallow | /x |
| Disallow | /y |
| Disallow | /z |
| Disallow | /A |
| Disallow | /B |
| Disallow | /C |
| Disallow | /D |
| Disallow | /E |
| Disallow | /F |
| Disallow | /G |
| Disallow | /H |
| Disallow | /I |
| Disallow | /J |
| Disallow | /K |
| Disallow | /L |
| Disallow | /M |
| Disallow | /N |
| Disallow | /O |
| Disallow | /P |
| Disallow | /Q |
| Disallow | /R |
| Disallow | /S |
| Disallow | /T |
| Disallow | /U |
| Disallow | /V |
| Disallow | /W |
| Disallow | /X |
| Disallow | /Y |
| Disallow | /z |
| Disallow | /perl/ |
| Disallow | /cgi-bin/ |
| Disallow | /fastcgi/ |
| Disallow | /fcgi-bin/ |
| Disallow | /audio/ |
| Disallow | /icons/ |
*
| Rule | Path |
|---|---|
| Disallow | / |