Log In Sign Up

m.livejournal.com
robots.txt

Robots Exclusion Standard data for m.livejournal.com

Resource Scan

Scan Details

Site Domain m.livejournal.com
Base Domain livejournal.com
Scan Status Ok
Last Scan2025-08-05T17:16:48+00:00
Next Scan 2025-08-19T17:16:48+00:00

Last Scan

Scanned2025-08-05T17:16:48+00:00
URL https://m.livejournal.com/robots.txt
Redirect https://www.livejournal.com/robots.txt
Redirect Domain www.livejournal.com
Redirect Base livejournal.com
Domain IPs 81.19.74.0, 81.19.74.1
Redirect IPs 81.19.74.0, 81.19.74.1
Response IP 81.19.74.1
Found Yes
Hash 2c2788645a1587a0c040fc420256414574a1634e91374291f9326168c8655e25
SimHash 691d8c6205e5

Groups

yandex

Rule Path
Allow /
Disallow /allpics.bml
Disallow /update.bml
Disallow /identity
Disallow /login.bml
Disallow /manage
Disallow /poll
Disallow /profile
Disallow /schools
Disallow /todo
Disallow /tools
Disallow /update.bml
Disallow /userinfo.bml
Disallow /users
Allow /ratings/$
Disallow /ratings
Disallow /syn
Disallow /latest
Disallow /ljtimes
Disallow /talkread
Disallow /inbox
Disallow /misc
Disallow /legal
Disallow /checklistposts
Disallow /away
Disallow /rsearch
Disallow /gsearch
Disallow /register.bml
Disallow /delcomment.bml
Disallow /talkscreen.bml
Disallow /give_tokens.bml

spbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

*

Rule Path
Allow /
Disallow /allpics.bml
Disallow /update.bml
Disallow /identity
Disallow /login.bml
Disallow /manage
Disallow /poll
Disallow /profile
Disallow /schools
Disallow /todo
Disallow /tools
Disallow /update.bml
Disallow /userinfo.bml
Disallow /users
Allow /ratings/$
Disallow /ratings
Disallow /syn
Disallow /latest
Disallow /ljtimes
Disallow /talkread
Disallow /inbox
Disallow /misc
Disallow /legal
Disallow /checklistposts
Disallow /away
Disallow /rsearch
Disallow /gsearch
Disallow /register.bml
Disallow /delcomment.bml
Disallow /talkscreen.bml
Disallow /give_tokens.bml
Back to top

Other Records

Field Value
sitemap https://www.livejournal.com/sitemap.xml
Back to top

Comments

  • Blocked journals aren't listed here because robots.txt files
  • can't be above 50k or so, depending on the spider.
  • Instead, blocked journals have HTML inserted in them which
  • should prevent behaved spiders from indexing it.
  • Note that http://username.livejournal.com journals have an
  • autogenerated robots.txt, since it can be small.
Back to top

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.
Back to top