jepcast.com
robots.txt

Robots Exclusion Standard data for jepcast.com

Resource Scan

Scan Details

Site Domain jepcast.com
Base Domain jepcast.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-08-14T02:29:12+00:00
Next Scan 2024-11-12T02:29:12+00:00

Last Successful Scan

Scanned2024-04-17T02:25:32+00:00
URL http://jepcast.com/robots.txt
Redirect https://jep-cast.livejournal.com/robots.txt
Redirect Domain jep-cast.livejournal.com
Redirect Base livejournal.com
Domain IPs 208.113.175.103
Redirect IPs 81.19.74.0, 81.19.74.1
Response IP 81.19.74.0
Found Yes
Hash 9883a20bae077051ce5604d6add008eb285cd1201ac2c61cfe15327b9df62d34
SimHash 6444dad2d631

Groups

*

Rule Path
Allow /data/rss/
Disallow /*.html*mode%3Dreply
Disallow /*.html*replyto
Disallow /data/foaf/
Disallow /tag/
Disallow /friendstimes
Disallow /d0*
Disallow /d1*
Disallow /d2*
Disallow /d3*
Disallow /d4*
Disallow /d5*
Disallow /d6*
Disallow /d7*
Disallow /d8*
Disallow /d9*
Disallow /ratings/users

mediapartners-google*

Rule Path
Allow /

yandex

Rule Path
Allow /
Disallow /data/foaf/
Disallow /ratings/users
Disallow /photo

Other Records

Field Value
crawl-delay 100

googlebot

Rule Path
Allow /
Disallow /ratings/users
Disallow /data/foaf/
Disallow /photo

spbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

Warnings

  • `clean-param` is not a known field.
  • `host` is not a known field.