dailymirror.lk
robots.txt

Robots Exclusion Standard data for dailymirror.lk

Resource Scan

Scan Details

Site Domain dailymirror.lk
Base Domain dailymirror.lk
Scan Status Ok
Last Scan2024-06-12T06:54:58+00:00
Next Scan 2024-06-19T06:54:58+00:00

Last Scan

Scanned2024-06-12T06:54:58+00:00
URL https://dailymirror.lk/robots.txt
Redirect https://www.dailymirror.lk/robots.txt
Redirect Domain www.dailymirror.lk
Redirect Base dailymirror.lk
Domain IPs 104.26.12.192, 104.26.13.192, 172.67.72.186, 2606:4700:20::681a:cc0, 2606:4700:20::681a:dc0, 2606:4700:20::ac43:48ba
Redirect IPs 104.26.12.192, 104.26.13.192, 172.67.72.186, 2606:4700:20::681a:cc0, 2606:4700:20::681a:dc0, 2606:4700:20::ac43:48ba
Response IP 172.67.72.186
Found Yes
Hash e57e138f84a37957d777cf8f0514fc4a45c1690c1b89f813455393148e8c50be
SimHash 6b58d411c295

Groups

*

Rule Path
Allow /

googlebot

Rule Path
Allow /

slurp

Rule Path
Allow /

msnbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3600

Other Records

Field Value
sitemap https://www.dailymirror.lk/sitemaps