Log In Sign Up

borsen.dk
robots.txt

Robots Exclusion Standard data for borsen.dk

Resource Scan

Scan Details

Site Domain borsen.dk
Base Domain borsen.dk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-04-18T05:52:55+00:00
Next Scan 2024-05-18T05:52:55+00:00

Last Successful Scan

Scanned2024-03-19T11:57:04+00:00
URL https://borsen.dk/robots.txt
Domain IPs 75.2.12.165, 99.83.241.153
Response IP 99.83.241.153
Found Yes
Hash 526fd4b8cfccbeb19387f14dc5eb32781f67dd93a7d7416f01924e90197b7867
SimHash 4115d860afb3

Groups

*

Rule Path
Allow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://borsen.dk/sitemap
Back to top

Comments

  • Search engines
  • Common crawl
  • OpenAI (ChatGPT)
  • OpenAI (ChatGPT realtime search)
  • Anthropic
  • Sitemap
Back to top