Log In Sign Up

agentiz.com
robots.txt

Robots Exclusion Standard data for agentiz.com

Resource Scan

Scan Details

Site Domain agentiz.com
Base Domain agentiz.com
Scan Status Ok
Last Scan2024-09-22T04:51:44+00:00
Next Scan 2024-09-29T04:51:44+00:00

Last Scan

Scanned2024-09-22T04:51:44+00:00
URL https://agentiz.com/robots.txt
Redirect https://www.agentiz.com/robots.txt
Redirect Domain www.agentiz.com
Redirect Base agentiz.com
Domain IPs 104.21.23.178, 172.67.212.117, 2606:4700:3034::ac43:d475, 2606:4700:3036::6815:17b2
Redirect IPs 104.21.23.178, 172.67.212.117, 2606:4700:3034::ac43:d475, 2606:4700:3036::6815:17b2
Response IP 172.67.212.117
Found Yes
Hash b88a4fdebec79c88f906cb2e3256816d19ac7cc9ba5b9e530e2460a255316b36
SimHash 4074862343e3

Groups

*

Rule Path
Allow /
Disallow /g$
Disallow /cookie

psbot
turnitinbot
npbot
petalbot
mj12bot

Rule Path
Disallow /

baiduspider
haosouspider
sogou
shenma

Rule Path
Allow /$
Allow /cn$
Allow /cn/
Disallow /

yeti
daum

Rule Path
Allow /$
Allow /ko
Allow /ko/
Disallow /

yandex
mail.ru_bot

Rule Path
Allow /$
Allow /ru/
Allow /ru$
Allow /tr/
Allow /tr$
Disallow /

seznam

Rule Path
Allow /$
Allow /cs/
Allow /cs$
Allow /en/
Allow /en$
Allow /pl/
Allow /pl$
Allow /ru/
Allow /ru$
Allow /uk/
Allow /uk$
Disallow /
Back to top

Comments

  • www.agentiz.com
Back to top

Warnings

  • `host` is not a known field.
Back to top