playpaste.com
robots.txt

Robots Exclusion Standard data for playpaste.com

Resource Scan

Scan Details

Site Domain playpaste.com
Base Domain playpaste.com
Scan Status Ok
Last Scan2024-11-14T02:40:12+00:00
Next Scan 2024-11-21T02:40:12+00:00

Last Scan

Scanned2024-11-14T02:40:12+00:00
URL https://playpaste.com/robots.txt
Domain IPs 104.21.66.240, 172.67.165.223, 2606:4700:3032::ac43:a5df, 2606:4700:3037::6815:42f0
Response IP 104.21.66.240
Found Yes
Hash b8f95e751f94c919a13a823d6e85a2847406f62dc09c2208f383257ca0ff3542
SimHash c8d45bb071ca

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /tmp/
Disallow /junk/

screaming frog seo spider

Rule Path
Disallow /

bidswitchbot

Rule Path
Disallow /

w3c_validator

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

validator.nu

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

wget

Rule Path
Disallow /

microsoft office protocol discovery

Rule Path
Disallow /

jigsaw

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

daum

Rule Path
Disallow /

yandeximageresizer

Rule Path
Disallow /

unknown robot identified by \*bot

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

duckduckbot-https

Rule Path
Disallow /

nbot

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

spider

Rule Path
Disallow /

survey

Rule Path
Disallow /

yahoo! slurp

Rule Path
Disallow /

firephp

Rule Path
Disallow /

phantomjs

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

konqueror

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

seek

Rule Path
Disallow /

apache-httpclient

Rule Path
Disallow /

cubot

Rule Path
Disallow /

nautic expo using firefox/1.5

Rule Path
Disallow /

spider

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

ruby script

Rule Path
Disallow /

semanticbot

Rule Path
Disallow /

who.is.bot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

elinks (

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

slackbot-linkexpanding

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

my nutch spider

Rule Path
Disallow /

getintent crawler

Rule Path
Disallow /

phantomjs

Rule Path
Disallow /

ruby script

Rule Path
Disallow /

daum

Rule Path
Disallow /

adidxbot microsoft ad quality control

Rule Path
Disallow /

linkedinbot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

discover

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

konqueror

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

bl.uk_lddc_bot

Rule Path
Disallow /

urllib

Rule Path
Disallow /

velen

Rule Path
Disallow /

slackbot-linkexpanding

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yak-linkfluence

Rule Path
Disallow /

ruby script

Rule Path
Disallow /

wget

Rule Path
Disallow /

semanticbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

firephp

Rule Path
Disallow /

hit

Rule Path
Disallow /

bot\*

Rule Path
Disallow /

velen

Rule Path
Disallow /

wget

Rule Path
Disallow /

seek

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

phantomjs

Rule Path
Disallow /

mail.ru bot

Rule Path
Disallow /

bidswitchbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

bidswitchbot

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

trendiction

Rule Path
Disallow /

elinks

Rule Path
Disallow /

proximic

Rule Path
Disallow /

serendeputy

Rule Path
Disallow /

phantomjs

Rule Path
Disallow /

validator.nu

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

w3c_validator

Rule Path
Disallow /

yahoo! slurp

Rule Path
Disallow /

my nutch spider

Rule Path
Disallow /

survey

Rule Path
Disallow /

fetcher

Rule Path
Disallow /

nbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

crawl

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

links (

Rule Path
Disallow /

msie8 - ( rogue robot )

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

spider

Rule Path
Disallow /

mixrankbot

Rule Path
Disallow /

discover

Rule Path
Disallow /

scanner

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

ng/1

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

daum

Rule Path
Disallow /

blog

Rule Path
Disallow /

xml sitemaps generator

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

serendeputy

Rule Path
Disallow /

my nutch spider

Rule Path
Disallow /

a php script

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

yandex ( catchall )

Rule Path
Disallow /

getintent crawler

Rule Path
Disallow /

cubot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

survey

Rule Path
Disallow /

curl

Rule Path
Disallow /

obot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

link

Rule Path
Disallow /

semanticbot

Rule Path
Disallow /

wget

Rule Path
Disallow /

blog

Rule Path
Disallow /

java ( catchall )

Rule Path
Disallow /

feed

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dalvik

Rule Path
Disallow /

link

Rule Path
Disallow /

links (

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

konqueror

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

perl

Rule Path
Disallow /

checker

Rule Path
Disallow /

checker

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

mixrankbot

Rule Path
Disallow /

validator

Rule Path
Disallow /

semanticbot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

yandeximageresizer

Rule Path
Disallow /

crawl

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

robot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /

crawl

Rule Path
Disallow /

linkcheck

Rule Path
Disallow /

blog

Rule Path
Disallow /

feed

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

hit

Rule Path
Disallow /

discordbot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

bot\*

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

mixrankbot

Rule Path
Disallow /

yak-linkfluence

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

unknown robot identified by bot\*

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

hit

Rule Path
Disallow /

java ( catchall )

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

fetcher

Rule Path
Disallow /

google web preview

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

survey

Rule Path
Disallow /

seek

Rule Path
Disallow /

phantomjs

Rule Path
Disallow /

xenu link sleuth

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

konqueror

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

microsoft office protocol discovery

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

cubot

Rule Path
Disallow /

trendiction

Rule Path
Disallow /

spider

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

feed

Rule Path
Disallow /

checker

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

spider

Rule Path
Disallow /

proximic

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

empty user agent string

Rule Path
Disallow /

survey

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

curl

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

dalvik

Rule Path
Disallow /

apache-httpclient

Rule Path
Disallow /

trendiction

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

velen

Rule Path
Disallow /

bidswitchbot

Rule Path
Disallow /

elinks

Rule Path
Disallow /

proximic

Rule Path
Disallow /

serendeputy

Rule Path
Disallow /

phantomjs

Rule Path
Disallow /

validator.nu

Rule Path
Disallow /

w3c_validator

Rule Path
Disallow /

my nutch spider

Rule Path
Disallow /

survey

Rule Path
Disallow /

fetcher

Rule Path
Disallow /

nbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

crawl

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

links (

Rule Path
Disallow /

msie8 - ( rogue robot )

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

spider

Rule Path
Disallow /

mixrankbot

Rule Path
Disallow /

discover

Rule Path
Disallow /

scanner

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

ng/1

Rule Path
Disallow /

daum

Rule Path
Disallow /

blog

Rule Path
Disallow /

xml sitemaps generator

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

serendeputy

Rule Path
Disallow /

my nutch spider

Rule Path
Disallow /

a php script

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

getintent crawler

Rule Path
Disallow /

cubot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

survey

Rule Path
Disallow /

curl

Rule Path
Disallow /

obot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

link

Rule Path
Disallow /

semanticbot

Rule Path
Disallow /

wget

Rule Path
Disallow /

blog

Rule Path
Disallow /

java ( catchall )

Rule Path
Disallow /

feed

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dalvik

Rule Path
Disallow /

link

Rule Path
Disallow /

links (

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

perl

Rule Path
Disallow /

checker

Rule Path
Disallow /

mixrankbot

Rule Path
Disallow /

validator

Rule Path
Disallow /

semanticbot

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

crawl

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

robot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

*

Rule Path
Disallow /?s=
Disallow /search

*

Rule Path
Disallow /trackback
Disallow /*trackback
Disallow /*trackback*
Disallow /*/trackback

*

Rule Path
Allow /feed/$
Disallow /feed/
Disallow /comments/feed/
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$

Warnings

  • 13 invalid lines.