Log In Sign Up

dnevnik.bg
robots.txt

Robots Exclusion Standard data for dnevnik.bg

Resource Scan

Scan Details

Site Domain dnevnik.bg
Base Domain dnevnik.bg
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-27T23:39:15+00:00
Next Scan 2024-12-26T23:39:15+00:00

Last Successful Scan

Scanned2024-03-02T23:14:11+00:00
URL https://dnevnik.bg/robots.txt
Redirect https://www.dnevnik.bg/robots.txt
Redirect Domain www.dnevnik.bg
Redirect Base dnevnik.bg
Domain IPs 104.22.12.51, 104.22.13.51, 172.67.23.110, 2606:4700:10::6816:c33, 2606:4700:10::6816:d33, 2606:4700:10::ac43:176e
Redirect IPs 104.22.12.51, 104.22.13.51, 172.67.23.110, 2606:4700:10::6816:c33, 2606:4700:10::6816:d33, 2606:4700:10::ac43:176e
Response IP 104.22.13.51
Found Yes
Hash b4a3a64a613a1375f321998929a85788a2e6cd3e64df63ad6e75d5e7e2a2907f
SimHash f012b11b0737

Groups

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

*

Rule Path
Disallow /show/sendto/*
Disallow /adformats/*
Disallow /biznes/profile/*
Back to top

Other Records

Field Value
sitemap https://www.dnevnik.bg/dnevnik_all0.xml
Back to top

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
  • https://developers.google.com/search/reference/robots_txt?csw=1#url-matching-based-on-path-values
  • https://support.google.com/webmasters/answer/6080548?hl=en
  • https://www.searchenginejournal.com/technical-seo/url-parameter-handling/?amp
  • Used for many other (non-commercial) purposes as well
  • For new training only
  • Not for training, only for user requests
  • Marker for disabling Bard and Vertex AI
  • Speech synthesis only?
  • Multi-purpose, commercial uses; including LLMs
Back to top