Log In Sign Up

e-terminai.lt
robots.txt

Robots Exclusion Standard data for e-terminai.lt

Resource Scan

Scan Details

Site Domain e-terminai.lt
Base Domain e-terminai.lt
Scan Status Ok
Last Scan2025-12-28T07:04:42+00:00
Next Scan 2026-01-04T07:04:42+00:00

Last Scan

Scanned2025-12-28T07:04:42+00:00
URL https://e-terminai.lt/robots.txt
Redirect https://www.e-terminai.lt/robots.txt
Redirect Domain www.e-terminai.lt
Redirect Base e-terminai.lt
Domain IPs 45.93.139.28
Redirect IPs 45.93.139.28
Response IP 45.93.139.28
Found Yes
Hash 7d1bb59bef453d702421011ac77c74d12f9ca9b90fb475afae4218ef23756761
SimHash 6d04fad147f6

Groups

googlebot-image

Rule Path
Disallow /

yahoo-mmcrawler

Rule Path
Disallow /

psbot

Rule Path
Disallow /

asterias

Rule Path
Disallow /

badbot

Rule Path
Disallow /

*

Rule Path
Disallow /admin/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.e-terminai.lt/sitemap.xml
Back to top

Comments

  • (Optional) Block Google Image Search from indexing images
  • (Optional) Block Yahoo's multimedia crawler
  • (Optional) Block Picsearch from indexing images
  • Block known scraper bot
  • Block bad bots
  • Allow all agents to crawl all pages except for /admin/
  • (Optional) Add crawl delays for non-Google bots
  • Sitemap location
Back to top