tigawa.github.io
robots.txt

Robots Exclusion Standard data for tigawa.github.io

Resource Scan

Scan Details

Site Domain tigawa.github.io
Base Domain tigawa.github.io
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-11-07T16:53:14+00:00
Next Scan 2024-12-07T16:53:14+00:00

Last Successful Scan

Scanned2024-10-09T16:52:32+00:00
URL https://tigawa.github.io/robots.txt
Redirect https://igawa.blog/robots.txt
Redirect Domain igawa.blog
Redirect Base igawa.blog
Domain IPs 185.199.108.153, 185.199.109.153, 185.199.110.153, 185.199.111.153, 2606:50c0:8000::153, 2606:50c0:8001::153, 2606:50c0:8002::153, 2606:50c0:8003::153
Redirect IPs 104.21.85.244, 172.67.212.141, 2606:4700:3031::6815:55f4, 2606:4700:3034::ac43:d48d
Response IP 172.67.212.141
Found Yes
Hash 37d28fca087c6b098d73d2eeb6cc95e51e8cff21d7ec3ce7529d92897b974a6f
SimHash d05c1022b09b

Groups

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

friendly_crawler

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /