diena.lv
robots.txt

Robots Exclusion Standard data for diena.lv

Resource Scan

Scan Details

Site Domain diena.lv
Base Domain diena.lv
Scan Status Ok
Last Scan2024-06-15T01:33:54+00:00
Next Scan 2024-06-22T01:33:54+00:00

Last Scan

Scanned2024-06-15T01:33:54+00:00
URL https://diena.lv/robots.txt
Redirect https://www.diena.lv/robots.txt
Redirect Domain www.diena.lv
Redirect Base diena.lv
Domain IPs 104.21.7.171, 172.67.187.242, 2606:4700:3033::6815:7ab, 2606:4700:3037::ac43:bbf2
Redirect IPs 104.21.7.171, 172.67.187.242, 2606:4700:3033::6815:7ab, 2606:4700:3037::ac43:bbf2
Response IP 172.67.187.242
Found Yes
Hash 985411e11bef439ebe39175e14845b4bf24f83a576b851d65190bbe32a60706c
SimHash 64304d206137

Groups

*

Rule Path
Disallow /admin

Comments

  • www.robotstxt.org/
  • Allow crawling of all content