valstietis.lt
robots.txt

Robots Exclusion Standard data for valstietis.lt

Resource Scan

Scan Details

Site Domain valstietis.lt
Base Domain valstietis.lt
Scan Status Ok
Last Scan2024-11-14T12:32:40+00:00
Next Scan 2024-11-21T12:32:40+00:00

Last Scan

Scanned2024-11-14T12:32:40+00:00
URL https://valstietis.lt/robots.txt
Redirect https://www.valstietis.lt/robots.txt
Redirect Domain www.valstietis.lt
Redirect Base valstietis.lt
Domain IPs 194.135.86.200
Redirect IPs 194.135.86.200
Response IP 194.135.86.200
Found Yes
Hash 8aded0638b4f52e0940d2666674bf8a7e6f6268737feb555507b70dc2959a71b
SimHash 6100dc544711

Groups

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.valstietis.lt/sitemap/sitemap.xml

Comments

  • Crawl-delay: 10