dnsdb.io
robots.txt

Robots Exclusion Standard data for dnsdb.io

Resource Scan

Scan Details

Site Domain dnsdb.io
Base Domain dnsdb.io
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2026-01-09T10:51:47+00:00
Next Scan 2026-04-09T10:51:47+00:00

Last Successful Scan

Scanned2023-11-28T22:24:10+00:00
URL https://dnsdb.io/robots.txt
Redirect https://www.dnsdb.io/robots.txt
Redirect Domain www.dnsdb.io
Redirect Base dnsdb.io
Domain IPs 104.21.28.139, 172.67.170.217, 2606:4700:3031::ac43:aad9, 2606:4700:3037::6815:1c8b
Redirect IPs 104.21.28.139, 172.67.170.217, 2606:4700:3031::ac43:aad9, 2606:4700:3037::6815:1c8b
Response IP 172.67.170.217
Found Yes
Hash fc67307c6f422f4141d688850e44e4e47c602f49b93b90ee5ec81669f948ef02
SimHash 981dd74243d3

Groups

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

*

Rule Path
Disallow /zh-cn/search
Disallow /zh-hk/search
Disallow /en-us/search
Disallow /*?*