Log In Sign Up

payap.net
robots.txt

Robots Exclusion Standard data for payap.net

Resource Scan

Scan Details

Site Domain payap.net
Base Domain payap.net
Scan Status Ok
Last Scan2024-10-02T11:12:58+00:00
Next Scan 2024-10-09T11:12:58+00:00

Last Scan

Scanned2024-10-02T11:12:58+00:00
URL https://payap.net/robots.txt
Domain IPs 150.95.219.91
Response IP 150.95.219.91
Found Yes
Hash 03331c811b26bb8eced29e8d327440e93e0bf4ecfe708d21d45b22552a656def
SimHash 331ea95bc9e8

Groups

*

Rule Path
Disallow /administrator/
Disallow /api/
Disallow /cache/
Disallow /calendar/
Disallow /calendar-en/
Disallow /calendar-th/
Disallow /calendar-tw/
Disallow /category/
Disallow /cli/
Disallow /components/
Disallow /component/
Disallow /component/tags
Disallow /component/tags/tag
Disallow /component/tags/tag/?
Disallow /component/users
Disallow /component/jevents
Disallow /component/phocagallery/
Disallow /daitoua/
Disallow /error/
Disallow /images/
Disallow /images/phocagallery/
Disallow /gallery/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /linchii/
Disallow /logs/
Disallow /modules/
Disallow /old2005/
Disallow /plugins/
Disallow /tmp/
Disallow /tag/
Disallow /tags.html*
Disallow /comments
Disallow /comments/category/*/*
Disallow /comments/*/trackback
Disallow /comments/*/
Disallow /comments/*?*
Disallow /comments/*?
Disallow /comments/*.php$
Disallow /comments/*.js$
Disallow /comments/*.inc$
Disallow /comments/*.css$
Disallow /comments/*.gz$
Disallow /comments/*.wmv$
Disallow /comments/*.cgi$
Disallow /comments/*.xhtml
Disallow /?view*
Disallow /?view=*
Disallow /?view=category&id=*
Disallow /?view=category*
Disallow /?view=html*
Disallow /?Itemid=*
Disallow /?tp=*
Disallow /?format=*
Disallow /?rCH2
Disallow *.rCH%3D2

googlebot-image

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

mappy

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

daum

Rule Path
Disallow /

crawler4j

Rule Path
Disallow /

synapse

Rule Path
Disallow /

searchbot1.0

Rule Path
Disallow /

proximic

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

voilabot

Rule Path
Disallow /

megalodon

Rule Path
Disallow /

spbot

Rule Path
Disallow /

accoona-ai-agent

Rule Path
Disallow /

accela bizsearch crawler

Rule Path
Disallow /

aboundex

Rule Path
Disallow /

agentname

Rule Path
Disallow /

agentname/0.1libwww-perl/6.02

Rule Path
Disallow /

arachmo

Rule Path
Disallow /

arks

Rule Path
Disallow /

basichttp

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

becomejpbot

Rule Path
Disallow /

blexbot/1.0

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

casper

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

content crawler

Rule Path
Disallow /

converamultimediacrawler

Rule Path
Disallow /

crazywebcrawler

Rule Path
Disallow /

curl

Rule Path
Disallow /

dcpbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

deepnet explorer

Rule Path
Disallow /

dialogsearch.com bot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

easouspider

Rule Path
Disallow /

ec2linkfinder

Rule Path
Disallow /

eventmachine

Rule Path
Disallow /

exabot

Rule Path
Disallow /

embot-galabuzz/nutch-1.0

Rule Path
Disallow /

ezooms/1.0

Rule Path
Disallow /

e-societyrobot

Rule Path
Disallow /

fastbot crawler beta 2.0

Rule Path
Disallow /

feed crawler

Rule Path
Disallow /

feed parser

Rule Path
Disallow /

feedlybot

Rule Path
Disallow /

flightdeckreportsbot

Rule Path
Disallow /

geohasher

Rule Path
Disallow /

gethtmlw

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

gigablastopensource

Rule Path
Disallow /

gigablastopensource/1

Rule Path
Disallow /

girafabot

Rule Path
Disallow /

gold crawler

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

gslfbot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

iaskspider

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

ichikawakenji

Rule Path
Disallow /

ichikawakenji/nutch-1

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

indy library

Rule Path
Disallow /

influencebot

Rule Path
Disallow /

intelium_bot

Rule Path
Disallow /

iskanie

Rule Path
Disallow /

java

Rule Path
Disallow /

jakarta commons-httpclient

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

libwww

Rule Path
Disallow /

libwww-perl

Rule Path
Disallow /

linkdex.com

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

lipperhey-kaus-australis

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

lssrocketcrawler

Rule Path
Disallow /

lynx

Rule Path
Disallow /

magpierss

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

masagool

Rule Path
Disallow /

mb-sitecrawler

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megalodon.jp

Rule Path
Disallow /

moreoverbot

Rule Path
Disallow /

mrsputnik

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mfcrawler

Rule Path
Disallow /

mvaclient

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

netfront

Rule Path
Disallow /

netfront/3

Rule Path
Disallow /

netscape

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

ninja

Rule Path
Disallow /

nutscrape/1.0

Rule Path
Disallow /

nutchcvs

Rule Path
Disallow /

nutch-1.0-dev

Rule Path
Disallow /

nutch

Rule Path
Disallow /

nutch

Rule Path
Disallow /

noxtrumbot/1.0

Rule Path
Disallow /

outfoxbot

Rule Path
Disallow /

pirst

Rule Path
Disallow /

purebot

Rule Path
Disallow /

pflab

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

page_verifier

Rule Path
Disallow /

pockey

Rule Path
Disallow /

plagger

Rule Path
Disallow /

psbot

Rule Path
Disallow /

qihoobot

Rule Path
Disallow /

qqdownload

Rule Path
Disallow /

qrobot

Rule Path
Disallow /

rganalytics

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

samsung-sgh-e250

Rule Path
Disallow /

sbider

Rule Path
Disallow /

scooter

Rule Path
Disallow /

scidclam

Rule Path
Disallow /

searchbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

setooz

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

simplepie

Rule Path
Disallow /

steeler

Rule Path
Disallow /

stardownloader

Rule Path
Disallow /

stratagems kumo

Rule Path
Disallow /

surdotlybot

Rule Path
Disallow /

surveybot

Rule Path
Disallow /

snapbot

Rule Path
Disallow /

snoopy

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

shim-crawler

Rule Path
Disallow /

snapbot

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sosospider+

Rule Path
Disallow /

solomonobot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

smarty template engine

Rule Path
Disallow /

sgroup crawler

Rule Path
Disallow /

sgroup crawler 1/nutch-1

Rule Path
Disallow /

swebot

Rule Path
Disallow /

testnutch

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

rgspider

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

toread-crawler

Rule Path
Disallow /

libghttp

Rule Path
Disallow /

trackback

Rule Path
Disallow /

tb_send

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

unknown

Rule Path
Disallow /

unwindfetchor

Rule Path
Disallow /

urlresolver

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

voltron

Rule Path
Disallow /

voyager

Rule Path
Disallow /

watchscript

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

webauto

Rule Path
Disallow /

webauto

Rule Path
Disallow /

webcrawler

Rule Path
Disallow /

website explorer

Rule Path
Disallow /

webster pro

Rule Path
Disallow /

webox

Rule Path
Disallow /

webdatacentrebot

Rule Path
Disallow /

wget

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

www-mechanize

Rule Path
Disallow /

www-mechanize/1

Rule Path
Disallow /

wwwc

Rule Path
Disallow /

w_univ_bj_spider

Rule Path
Disallow /

xbfmozilla

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

yodaoice

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

yoofind/yoofind-0.1-dev

Rule Path
Disallow /

zend_http_client

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

zia-httpmirror

Rule Path
Disallow /

nabot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

wget

Rule Path
Disallow /

proxy cache

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

superfeedr

Rule Path
Disallow /

ias_crawler

Rule Path
Disallow

piplbot

Rule Path
Disallow /
Back to top

Comments

  • If the Joomla site is installed within a folder
  • eg www.example.com/joomla/ then the robots.txt file
  • MUST be moved to the site root
  • eg www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to all of the
  • paths.
  • eg the Disallow rule for the /administrator/ folder MUST
  • be changed to read
  • Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • https://www.robotstxt.org/orig.html
  • Disabled all tags - throw 404 so that they get deindexed. google対策.
Back to top

Warnings

  • 2 invalid lines.
Back to top