pivigames.blog
robots.txt

Robots Exclusion Standard data for pivigames.blog

Archived Snapshots

Resource Scan

Scan Details

Site Domain	pivigames.blog
Base Domain	pivigames.blog
Scan Status	Ok
Last Scan	2024-06-05T19:23:53+00:00
Next Scan	2024-06-12T19:23:53+00:00

Last Scan

Scanned	2024-06-05T19:23:53+00:00
URL	https://pivigames.blog/robots.txt
Domain IPs	104.21.17.80, 172.67.175.78, 2606:4700:3030::ac43:af4e, 2606:4700:3035::6815:1150
Response IP	104.21.17.80
Found	Yes
Hash	b69db1f182c5123be09f7b6b537a4bcc77315d5b248d1fc34bada3631c94c9bd
SimHash	a8805b8084fa

Groups

*

Rule	Path
Allow	/wp-content/*.js
Allow	/wp-content/*.css
Allow	/wp-content/litespeed/js/
Allow	/wp-content/litespeed/avatar/
Disallow	/wp-content/themes/
Disallow	/wp-admin/
Disallow	/wp-login.php
Allow	/wp-content/uploads/*
Allow	/wp-includes/*.js
Allow	/wp-includes/*.css
Disallow	/cgi-bin
Disallow	/wp-includes/
Disallow	/*/attachment/
Disallow	/tag/*/page/
Disallow	/tag/*/feed/
Disallow	/page/
Disallow	/comments/
Disallow	/xmlrpc.php
Disallow	/?attachment_id*
Disallow	/tag/
Disallow	/*?

Rule

Path

Allow

/wp-content/*.js

Allow

/wp-content/*.css

Allow

/wp-content/litespeed/js/

Allow

/wp-content/litespeed/avatar/

Disallow

/wp-content/themes/

Disallow

/wp-admin/

Disallow

/wp-login.php

Allow

/wp-content/uploads/*

Allow

/wp-includes/*.js

Allow

/wp-includes/*.css

Disallow

/cgi-bin

Disallow

/wp-includes/

Disallow

/*/attachment/

Disallow

/tag/*/page/

Disallow

/tag/*/feed/

Disallow

/page/

Disallow

/comments/

Disallow

/xmlrpc.php

Disallow

/?attachment_id*

Disallow

/tag/

Disallow

/*?

*

Rule	Path
Disallow	/?s=
Disallow	/search

Rule

Path

Disallow

/?s=

Disallow

/search

*

Rule	Path
Disallow	/trackback
Disallow	/*trackback
Disallow	/trackback
Disallow	/*/trackback

Rule

Path

Disallow

/trackback

Disallow

/*trackback

Disallow

/*trackback*

Disallow

/*/trackback

*

Rule	Path
Allow	/feed/$
Disallow	/feed/
Disallow	/comments/feed/
Disallow	/*/feed/$
Disallow	/*/feed/rss/$
Disallow	/*/trackback/$
Disallow	///feed/$
Disallow	///feed/rss/$
Disallow	///trackback/$
Disallow	///*/feed/$
Disallow	///*/feed/rss/$
Disallow	///*/trackback/$

Rule

Path

Allow

/feed/$

Disallow

/feed/

Disallow

/comments/feed/

Disallow

/*/feed/$

Disallow

/*/feed/rss/$

Disallow

/*/trackback/$

Disallow

/*/*/feed/$

Disallow

/*/*/feed/rss/$

Disallow

/*/*/trackback/$

Disallow

/*/*/*/feed/$

Disallow

/*/*/*/feed/rss/$

Disallow

/*/*/*/trackback/$

noxtrumbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	20

Field

Value

crawl-delay

msnbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	20

Field

Value

crawl-delay

slurp

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	20

Field

Value

crawl-delay

yahoo slurp

Rule	Path
Disallow	/

Rule

Path

Disallow

slurp

Rule	Path
Disallow	/

Rule

Path

Disallow

msiecrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

webcopier

Rule	Path
Disallow	/

Rule

Path

Disallow

httrack

Rule	Path
Disallow	/

Rule

Path

Disallow

microsoft.url.control

Rule	Path
Disallow	/

Rule

Path

Disallow

libwww

Rule	Path
Disallow	/

Rule

Path

Disallow

orthogaffe

Rule	Path
Disallow	/

Rule

Path

Disallow

ubicrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

doc

Rule	Path
Disallow	/

Rule

Path

Disallow

zao

Rule	Path
Disallow	/

Rule

Path

Disallow

sitecheck.internetseer.com

Rule	Path
Disallow	/

Rule

Path

Disallow

zealbot

Rule	Path
Disallow	/

Rule

Path

Disallow

msiecrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

sitesnagger

Rule	Path
Disallow	/

Rule

Path

Disallow

webstripper

Rule	Path
Disallow	/

Rule

Path

Disallow

webcopier

Rule	Path
Disallow	/

Rule

Path

Disallow

fetch

Rule	Path
Disallow	/

Rule

Path

Disallow

offline explorer

Rule	Path
Disallow	/

Rule

Path

Disallow

teleport

Rule	Path
Disallow	/

Rule

Path

Disallow

teleportpro

Rule	Path
Disallow	/

Rule

Path

Disallow

webzip

Rule	Path
Disallow	/

Rule

Path

Disallow

linko

Rule	Path
Disallow	/

Rule

Path

Disallow

httrack

Rule	Path
Disallow	/

Rule

Path

Disallow

microsoft.url.control

Rule	Path
Disallow	/

Rule

Path

Disallow

xenu

Rule	Path
Disallow	/

Rule

Path

Disallow

larbin

Rule	Path
Disallow	/

Rule

Path

Disallow

libwww

Rule	Path
Disallow	/

Rule

Path

Disallow

zyborg

Rule	Path
Disallow	/

Rule

Path

Disallow

download ninja

Rule	Path
Disallow	/

Rule

Path

Disallow

wget

Rule	Path
Disallow	/

Rule

Path

Disallow

grub-client

Rule	Path
Disallow	/

Rule

Path

Disallow

k2spider

Rule	Path
Disallow	/

Rule

Path

Disallow

npbot

Rule	Path
Disallow	/

Rule

Path

Disallow

webreaper

Rule	Path
Disallow	/

Rule

Path

Disallow

survey

Rule	Path
Disallow	/

Rule

Path

Disallow

survey

Rule	Path
Disallow	/

Rule

Path

Disallow

crawl

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot

Rule	Path
Disallow	/

Rule

Path

Disallow

yandexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

yahoo! slurp

Rule	Path
Disallow	/

Rule

Path

Disallow

yahoo! slurp china

Rule

Path

Disallow

pinterestbot

Rule

Path

Disallow

my nutch spider

Rule

Path

Disallow

a php script

Rule

Path

Disallow

applebot

Rule

Path

Disallow

yacybot

Rule

Path

Disallow

empty user agent string

Rule

Path

Disallow

screaming frog seo spider

Rule

Path

Disallow

unknown robot (identified by hit on robots.txt)

Rule

Path

Disallow

facebookexternalhit

Rule

Path

Disallow

yandeximages

Rule

Path

Disallow

telegrambot

Rule

Path

Disallow

nutch ( catchall )

Rule

Path

Disallow

yandexmobilebot

Rule

Path

Disallow

yandexmetrika

Rule

Path

Disallow

domainstatsbot

Rule

Path

Disallow

linkchecker

Rule

Path

Disallow

msiecrawler

Rule

Path

Disallow

lightspeed

Rule

Path

Disallow

bingpreview

Rule

Path

Disallow

bidswitchbot

Rule

Path

Disallow

w3c_validator

Rule

Path

Disallow

bingbot

Rule

Path

Disallow

validator.nu

Rule

Path

Disallow

msnbot-media

Rule

Path

Disallow

wget

Rule

Path

Disallow

microsoft office protocol discovery

Rule

Path

Disallow

jigsaw

Rule

Path

Disallow

barkrowler

Rule

Path

Disallow

daum

Rule

Path

Disallow

yandexbot

Rule

Path

Disallow

yandeximageresizer

Rule

Path

Disallow

duckduckgo-favicons-bot

Rule

Path

Disallow

duckduckbot-https

Rule

Path

Disallow

unknown robot identified by \*bot

Rule

Path

Disallow

semrushbot-si

Rule

Path

Disallow

archive.org_bot

Rule

Path

Disallow

duckduckbot-https

Rule

Path

Disallow

nbot

Rule

Path

Disallow

bit.ly

Rule

Path

Disallow

icc-crawler

Rule

Path

Disallow

spider

Rule

Path

Disallow

survey

Rule

Path

Disallow

yahoo! slurp

Rule

Path

Disallow

firephp

Rule

Path

Disallow

phantomjs

Rule

Path

Disallow

msnbot

Rule

Path

Disallow

mediatoolkitbot

Rule

Path

Disallow

konqueror

Rule

Path

Disallow

seznambot

Rule

Path

Disallow

seek

Rule

Path

Disallow

apache-httpclient

Rule

Path

Disallow

baidu ( catchall )

Rule

Path

Disallow

cubot

Rule

Path

Disallow

nautic expo using firefox/1.5

Rule

Path

Disallow

spider

Rule

Path

Disallow

yisouspider

Rule

Path

Disallow

ruby script

Rule

Path

Disallow

semanticbot

Rule

Path

Disallow

who.is.bot

Rule

Path

Disallow

surdotlybot

Rule

Path

Disallow

msiecrawler

Rule

Path

Disallow

elinks (

Rule

Path

Disallow

barkrowler

Rule

Path

Disallow

slackbot-linkexpanding

Rule

Path

Disallow

sogou web spider

Rule

Path

Disallow

my nutch spider

Rule

Path

Disallow

getintent crawler

Rule

Path

Disallow

phantomjs

Rule

Path

Disallow

ruby script

Rule

Path

Disallow

daum

Rule

Path

Disallow

adidxbot microsoft ad quality control

Rule

Path

Disallow

linkedinbot

Rule

Path

Disallow

surdotlybot

Rule

Path

Disallow

yisouspider

Rule

Path

Disallow

barkrowler

Rule

Path

Disallow

discover

Rule

Path

Disallow

scrapy

Rule

Path

Disallow

ccbot

Rule

Path

Disallow

konqueror

Rule

Path

Disallow

msiecrawler

Rule

Path

Disallow

bl.uk_lddc_bot

Rule

Path

Disallow

urllib

Rule

Path

Disallow

velen

Rule

Path

Disallow

slackbot-linkexpanding

Rule

Path

Disallow

surdotlybot

Rule

Path

Disallow

baiduspider

Rule

Path

Disallow

yak-linkfluence

Rule

Path

Disallow

ruby script

Rule

Path

Disallow

wget

Rule

Path

Disallow

semanticbot

Rule

Path

Disallow

sogou web spider

Rule

Path

Disallow

baiduspider

Rule

Path

Disallow

firephp

Rule

Path

Disallow

hit

Rule

Path

Disallow

bot\*

Rule

Path

Disallow

velen

Rule

Path

Disallow

wget

Rule

Path

Disallow

seek

Rule

Path

Disallow

archive.org_bot

Rule

Path

Disallow

cfnetwork

Rule

Path

Disallow

phantomjs

Rule

Path

Disallow

mail.ru bot

Rule

Path

Disallow

bidswitchbot

Rule

Path

Disallow

ahrefsbot

Rule

Path

Disallow

bidswitchbot

Rule

Path

Disallow

go-http-client

Rule

Path

Disallow

trendiction

Rule

Path

Disallow

elinks

Rule

Path

Disallow

proximic

Rule

Path

Disallow

serendeputy

Rule

Path

Disallow

phantomjs

Rule

Path

Disallow

validator.nu

Rule

Path

Disallow

mojeekbot

Rule

Path

Disallow

w3c_validator

Rule

Path

Disallow

yahoo! slurp

Rule

Path

Disallow

my nutch spider

Rule

Path

Disallow

survey

Rule

Path

Disallow

fetcher

Rule

Path

Disallow

nbot

Rule

Path

Disallow

dotbot

Rule

Path

Disallow

grapeshotcrawler

Rule

Path

Disallow

crawl

Rule

Path

Disallow

ccbot

Rule

Path

Disallow

links (

Rule

Path

Disallow

msie8 - ( rogue robot )

Rule

Path

Disallow

zumbot

Rule

Path

Disallow

spider

Rule

Path

Disallow

mixrankbot

Rule

Path

Disallow

discover

Rule

Path

Disallow

scanner

Rule

Path

Disallow

surdotlybot

Rule

Path

Disallow

ng/1

Rule

Path

Disallow

uptimerobot

Rule

Path

Disallow

daum

Rule

Path

Disallow

blog

Rule

Path

Disallow

xml sitemaps generator

Rule

Path

Disallow

scrapy

Rule

Path

Disallow

serendeputy

Rule

Path

Disallow

my nutch spider

Rule

Path

Disallow

a php script

Rule

Path

Disallow

seznambot

Rule

Path

Disallow

surdotlybot

Rule

Path

Disallow

yandex ( catchall )

Rule

Path

Disallow

getintent crawler

Rule

Path

Disallow

cubot

Rule

Path

Disallow

baiduspider

Rule

Path

Disallow

survey

Rule

Path

Disallow

curl

Rule

Path

Disallow

obot

Rule

Path

Disallow

mj12bot

Rule

Path

Disallow

link

Rule

Path

Disallow

semanticbot

Rule

Path

Disallow

wget

Rule

Path

Disallow

blog

Rule

Path

Disallow

java ( catchall )

Rule

Path

Disallow

feed

Rule

Path

Disallow

blexbot

Rule

Path

Disallow

dalvik

Rule

Path

Disallow

link

Rule

Path

Disallow

links (

Rule

Path

Disallow

surdotlybot

Rule

Path

Disallow

konqueror

Rule

Path

Disallow

yisouspider

Rule

Path

Disallow

perl

Rule

Path

Disallow

checker

Rule

Path

Disallow

checker

Rule

Path

Disallow

pinterestbot

Rule

Path

Disallow

mixrankbot

Rule

Path

Disallow

validator

Rule

Path

Disallow

semanticbot

Rule

Path

Disallow

surdotlybot

Rule

Path

Disallow

yandeximageresizer

Rule

Path

Disallow

crawl

Rule

Path

Disallow

sogou web spider

Rule

Path

Disallow

robot

Rule

Path

Disallow

seznambot

Rule

Path

Disallow

google web preview

Rule

Path

Disallow

seobility

Rule

Path

Disallow

scanner

Rule

Path

Disallow

urllib

Rule

Path

Disallow

slackbot-linkexpanding

Rule

Path

Disallow

msiecrawler

Rule

Path

Disallow

barkrowler

Rule

Path

Disallow

applebot

Rule

Path

Disallow

turnitinbot

Rule

Path

Disallow

rogerbot

Rule

Path

Disallow

*

Rule

Path

Disallow

/?s=

Disallow

/search

*

Rule

Path

Disallow

/trackback

Disallow

/*trackback

Disallow

/*trackback*

Disallow

/*/trackback

*

Rule

Path

Allow

/feed/$

Disallow

/feed/

Disallow

/comments/feed/

Disallow

/*/feed/$

Disallow

/*/feed/rss/$

Disallow

/*/trackback/$

Disallow

/*/*/feed/$

Disallow

/*/*/feed/rss/$

Disallow

/*/*/trackback/$

Disallow

/*/*/*/feed/$

Disallow

/*/*/*/feed/rss/$

Disallow

/*/*/*/trackback/$

Other Records

Field

Value

sitemap

https://pivigames.blog/wp-sitemap.xml

sitemap

https://pivigames.blog/sitemap_index.xml

sitemap

https://pivigames.blog/post-sitemap.xml

sitemap

https://pivigames.blog/page-sitemap.xml

sitemap

https://pivigames.blog/category-sitemap.xml

Comments

START YOAST BLOCK
---------------------------
Bloqueo de las URL dinamicas
Bloqueo de busquedas
Bloqueo de trackbacks
Bloqueo de feeds para crawlers
Ralentizamos algunos bots que se suelen volver locos
Bloqueo de bots y crawlers poco utiles
---------------------------
END YOAST BLOCK

Warnings

8 invalid lines.

pivigames.blogrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

*

*

*

noxtrumbot

Other Records

msnbot

Other Records

slurp

Other Records

yahoo slurp

slurp

msiecrawler

webcopier

httrack

microsoft.url.control

libwww

orthogaffe

ubicrawler

doc

zao

sitecheck.internetseer.com

zealbot

msiecrawler

sitesnagger

webstripper

webcopier

fetch

offline explorer

teleport

teleportpro

webzip

linko

httrack

microsoft.url.control

xenu

larbin

libwww

zyborg

download ninja

wget

grub-client

k2spider

npbot

webreaper

survey

survey

crawl

applebot

yandexbot

yahoo! slurp

yahoo! slurp china

pinterestbot

my nutch spider

a php script

applebot

yacybot

empty user agent string

screaming frog seo spider

unknown robot (identified by hit on robots.txt)

facebookexternalhit

yandeximages

telegrambot

nutch ( catchall )

yandexmobilebot

yandexmetrika

domainstatsbot

linkchecker

msiecrawler

lightspeed

bingpreview

bidswitchbot

w3c_validator

bingbot

validator.nu

pivigames.blog
robots.txt