polkadot.subscan.io
robots.txt

Robots Exclusion Standard data for polkadot.subscan.io

Resource Scan

Scan Details

Site Domain polkadot.subscan.io
Base Domain subscan.io
Scan Status Ok
Last Scan2025-06-05T10:08:09+00:00
Next Scan 2025-07-05T10:08:09+00:00

Last Scan

Scanned2025-06-05T10:08:09+00:00
URL https://polkadot.subscan.io/robots.txt
Domain IPs 104.26.8.57, 104.26.9.57, 172.67.72.8, 2606:4700:20::681a:839, 2606:4700:20::681a:939, 2606:4700:20::ac43:4808
Response IP 172.67.72.8
Found Yes
Hash ee0f9be44ca30602b585008d62d78b400c996f6b95c8c3a2da8da3a3bbcf5500
SimHash 6b0c9843ec13

Groups

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.subscan.io/sitemap.xml
sitemap https://www.subscan.io/sitemap-0.xml
sitemap https://www.subscan.io/server-sitemap.xml

Comments

  • SemrushBot
  • AhrefsBot
  • GPTBot
  • *
  • Host
  • Sitemaps

Warnings

  • `host` is not a known field.