Log In Sign Up

agentiz.co.uk
robots.txt

Robots Exclusion Standard data for agentiz.co.uk

Resource Scan

Scan Details

Site Domain agentiz.co.uk
Base Domain agentiz.co.uk
Scan Status Ok
Last Scan2024-09-23T15:18:26+00:00
Next Scan 2024-09-30T15:18:26+00:00

Last Scan

Scanned2024-09-23T15:18:26+00:00
URL https://agentiz.co.uk/robots.txt
Domain IPs 104.21.21.177, 172.67.199.171, 2606:4700:3032::ac43:c7ab, 2606:4700:3033::6815:15b1
Response IP 172.67.199.171
Found Yes
Hash 82f288b3ef19eaecc9b3f6f816c98b623b821df3463334a8438d04511ebe3e1d
SimHash 5730124243c1

Groups

psbot
turnitinbot
npbot
petalbot
mj12bot
semrush
crawler_eb_germany
blexbot
grapeshot

Rule Path
Disallow /

baiduspider
haosouspider
sogou
shenma

Rule Path
Allow /$
Allow /cn$
Allow /cn/
Disallow /

yeti
daum

Rule Path
Allow /$
Allow /ko$
Allow /ko/
Disallow /

yandex
mail.ru_bot

Rule Path
Allow /$
Allow /ru$
Allow /ru/
Allow /tr$
Allow /tr/
Disallow /

*

Rule Path
Allow /
Disallow /*-xs.jpg$
Disallow /*-xxs.jpg$
Disallow /*-xxxs.jpg$
Disallow /*/?type=*
Disallow /*/assets/data*
Disallow /*/assets/json*
Disallow /*/assets/modal/
Disallow /*/policies/cookie
Disallow /*/policies/privacy
Disallow /account
Disallow /53914483
Disallow /l_tmp
Back to top

Comments

  • UK agentiz.co.uk
  • User-agent: Googlebot
  • Allow: /*?q=
  • Disallow: /*?q=
  • Disallow: /*.webp$
  • Disallow: /*/account*
Back to top

Warnings

  • `host` is not a known field.
Back to top