lankadeepa.lk
robots.txt

Robots Exclusion Standard data for lankadeepa.lk

Resource Scan

Scan Details

Site Domain lankadeepa.lk
Base Domain lankadeepa.lk
Scan Status Ok
Last Scan2024-06-14T22:34:50+00:00
Next Scan 2024-06-21T22:34:50+00:00

Last Scan

Scanned2024-06-14T22:34:50+00:00
URL https://lankadeepa.lk/robots.txt
Redirect https://www.lankadeepa.lk/robots.txt
Redirect Domain www.lankadeepa.lk
Redirect Base lankadeepa.lk
Domain IPs 104.26.14.44, 104.26.15.44, 172.67.74.210, 2606:4700:20::681a:e2c, 2606:4700:20::681a:f2c, 2606:4700:20::ac43:4ad2
Redirect IPs 104.26.14.44, 104.26.15.44, 172.67.74.210, 2606:4700:20::681a:e2c, 2606:4700:20::681a:f2c, 2606:4700:20::ac43:4ad2
Response IP 172.67.74.210
Found Yes
Hash 060c28755dd37fdcef864d586ee62b4dae3b1879f3836c129f45bdd5b6575b34
SimHash 6a18d933c315

Groups

*

Rule Path
Allow /

googlebot

Rule Path
Allow /

slurp

Rule Path
Allow /

msnbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3600

Other Records

Field Value
sitemap https://www.lankadeepa.lk/sitemaps