Log In Sign Up

agentiz.co.uk
robots.txt

Robots Exclusion Standard data for agentiz.co.uk

Resource Scan

Scan Details

Site Domain agentiz.co.uk
Base Domain agentiz.co.uk
Scan Status Ok
Last Scan2024-06-09T15:13:07+00:00
Next Scan 2024-06-16T15:13:07+00:00

Last Scan

Scanned2024-06-09T15:13:07+00:00
URL https://agentiz.co.uk/robots.txt
Domain IPs 104.21.21.177, 172.67.199.171, 2606:4700:3032::ac43:c7ab, 2606:4700:3033::6815:15b1
Response IP 172.67.199.171
Found Yes
Hash 248b7b6a840900e076736b54b35ed17a54c9d98422d114c1c143dd7d4b16f9dc
SimHash d734024243c1

Groups

psbot
turnitinbot
npbot
petalbot
mj12bot
semrush
crawler_eb_germany
blexbot
grapeshot

Rule Path
Disallow /

baiduspider
haosouspider
sogou
shenma

Rule Path
Allow /$
Allow /cn$
Allow /cn/
Disallow /

yeti
daum

Rule Path
Allow /$
Allow /ko
Allow /ko/
Disallow /

yandex
mail.ru_bot

Rule Path
Allow /$
Allow /ru/
Allow /ru$
Allow /tr/
Allow /tr$
Disallow /

googlebot

Rule Path
Allow /*?q=

*

Rule Path
Allow /
Disallow /*?q=
Disallow /*-xs.jpg$
Disallow /*-xxs.jpg$
Disallow /*-xxxs.jpg$
Disallow /*.webp$
Disallow /*/?type=*
Disallow /*/assets/data/
Disallow /*/assets/json/
Disallow /*/assets/modal/
Disallow /cookie
Disallow /*/policies/cookie
Disallow /*/policies/privacy
Back to top

Comments

  • UK agentiz.co.uk
Back to top

Warnings

  • `host` is not a known field.
Back to top