peepcode.com
robots.txt

Robots Exclusion Standard data for peepcode.com

Resource Scan

Scan Details

Site Domain peepcode.com
Base Domain peepcode.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-07-21T22:35:40+00:00
Next Scan 2024-10-19T22:35:40+00:00

Last Successful Scan

Scanned2021-10-16T13:52:56+00:00
URL http://peepcode.com/robots.txt
Redirect https://www.pluralsight.com/robots.txt
Redirect Domain www.pluralsight.com
Redirect Base pluralsight.com
Found Yes
Hash bfef8192d2842164f79b25b1a8436b9e94e72b7b72abcb6b7c67caabb1e31e81
SimHash 2b3dd0014f71

Groups

*

Rule Path
Disallow /error/
Disallow /training/Player/
Disallow /training/Courses/Transcript/
Disallow /training/courses/transcript/
Disallow /training/courses/Transcript/
Disallow /training/Courses/Transcript/
Disallow /training/Courses/ExerciseFiles/
Disallow /training/Courses/Assessment/
Disallow /training/Login/
Disallow /training/Subscribe/Step1
Disallow /Courses/Transcript/
Disallow /courses/transcript/
Disallow /courses/Transcript/
Disallow /Courses/Transcript/
Disallow /Courses/exercise-files/
Disallow /courses/Exercise-Files/
Disallow /Courses/Exercise-Files/
Disallow /Courses/exercise-files/
Disallow /Courses/Assessment/
Disallow /Courses/assessment/
Disallow /courses/assessment/
Disallow /courses/Assessment/
Disallow /content/dist/fonts/
Disallow /account/
Disallow /training/Error/
Disallow /a/subscribe
Disallow /customer/
Disallow /search
Disallow /newsroom/in-the-news/
Disallow /buy
Disallow /redeemlink
Disallow /content/dam

yandex

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 300

rogerbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 300

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mozilla/4.0+(compatible;+msie+4.01;+windows+nt;+ms+search+6.0+robot)

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

wordpress*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

proximic

Rule Path
Disallow /

maxpoint bot

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

httrack

Rule Path
Disallow /
Disallow /training

ravencrawler

Rule Path
Disallow /
Disallow /training

zumbot*

Rule Path
Disallow /
Disallow /training

sputnikbot

Rule Path
Disallow /

sputnik

Rule Path
Disallow /

sputnikbot/2.3

Rule Path
Disallow /

spbot

Rule Path
Disallow /

spbot/4.0.9

Rule Path
Disallow /

ia_archiver

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

blexbot

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

zing-bottabot/2.0

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 90

crawler4j

Rule Path
Disallow /

diffbot/0.1

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 300

caliperbot/1.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

sogou+web+spider/4.0

Rule Path
Disallow /

bingbot

Rule Path
Disallow /content/

Other Records

Field Value
crawl-delay 1

Warnings

  • 4 invalid lines.