bisniswah.com
robots.txt

Robots Exclusion Standard data for bisniswah.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	bisniswah.com
Base Domain	bisniswah.com
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Couldn't connect to server.
Last Scan	2024-09-18T06:26:55+00:00
Next Scan	2024-12-17T06:26:55+00:00

Last Successful Scan

Scanned	2024-01-30T01:35:43+00:00
URL	https://bisniswah.com/robots.txt
Domain IPs	104.21.32.130, 172.67.152.16, 2606:4700:3035::6815:2082, 2606:4700:3037::ac43:9810
Response IP	104.21.32.130
Found	Yes
Hash	cef5bc0a28cfb5af537e2546e6bb11bd5379eacaa1b96b3b1c19c856cda5191b
SimHash	3250649aaca5

Groups

*

Rule	Path
Disallow	/wp-admin/
Allow	/wp-admin/admin-ajax.php
Disallow	/preview/
Disallow	//comment-page
Disallow	/*/followers/
Disallow	/*/following/
Disallow	/*/article/
Disallow	/tag//?halaman
Disallow	/wp-content/
Disallow	/wp-includes/
Disallow	/xmlrpc.php
Disallow	/feed/
Disallow	/trackback/

Rule

Path

Disallow

/wp-admin/

Allow

/wp-admin/admin-ajax.php

Disallow

/preview/

Disallow

/*/comment-page*

Disallow

/*/followers/

Disallow

/*/following/

Disallow

/*/article/

Disallow

/tag/*/?halaman*

Disallow

/wp-content/

Disallow

/wp-includes/

Disallow

/xmlrpc.php

Disallow

/feed/

Disallow

/trackback/

a6-indexer

Rule	Path
Disallow	/

Rule

Path

Disallow

ahc

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

aihitbot

Rule	Path
Disallow	/

Rule

Path

Disallow

archive-it

Rule	Path
Disallow	/

Rule

Path

Disallow

archiveteam archivebot

Rule	Path
Disallow	/

Rule

Path

Disallow

aspiegelbot

Rule	Path
Disallow	/

Rule

Path

Disallow

awariosmartbot

Rule	Path
Disallow	/

Rule

Path

Disallow

bananabot

Rule	Path
Disallow	/

Rule

Path

Disallow

barkrowler

Rule	Path
Disallow	/

Rule

Path

Disallow

bdcbot

Rule	Path
Disallow	/

Rule

Path

Disallow

bdfetch

Rule	Path
Disallow	/

Rule

Path

Disallow

betabot

Rule	Path
Disallow	/

Rule

Path

Disallow

blablubb

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

blogpulselive

Rule	Path
Disallow	/

Rule

Path

Disallow

bomborabot

Rule	Path
Disallow	/

Rule

Path

Disallow

bpimagewalker

Rule	Path
Disallow	/

Rule

Path

Disallow

brands-bot

Rule	Path
Disallow	/

Rule

Path

Disallow

brands-bot-logo

Rule	Path
Disallow	/

Rule

Path

Disallow

btcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

bubing

Rule	Path
Disallow	/

Rule

Path

Disallow

bublupbot

Rule	Path
Disallow	/

Rule

Path

Disallow

buck

Rule	Path
Disallow	/

Rule

Path

Disallow

buckyohare

Rule	Path
Disallow	/

Rule

Path

Disallow

builtwith

Rule	Path
Disallow	/

Rule

Path

Disallow

buzzbot

Rule	Path
Disallow	/

Rule

Path

Disallow

bvbot

Rule	Path
Disallow	/

Rule

Path

Disallow

careerbot

Rule	Path
Disallow	/

Rule

Path

Disallow

catchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ccbot

Rule	Path
Disallow	/

Rule

Path

Disallow

checkmarknetwork

Rule	Path
Disallow	/

Rule

Path

Disallow

cincraw

Rule	Path
Disallow	/

Rule

Path

Disallow

clark-crawler2

Rule	Path
Disallow	/

Rule

Path

Disallow

clickagy

Rule	Path
Disallow	/

Rule

Path

Disallow

cliqzbot

Rule	Path
Disallow	/

Rule

Path

Disallow

cms crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

coccocbot-image

Rule	Path
Disallow	/

Rule

Path

Disallow

cocolyzebot

Rule	Path
Disallow	/

Rule

Path

Disallow

compspybot

Rule	Path
Disallow	/

Rule

Path

Disallow

copyright sheriff

Rule	Path
Disallow	/

Rule

Path

Disallow

copyrightcheck

Rule	Path
Disallow	/

Rule

Path

Disallow

cowbot

Rule	Path
Disallow	/

Rule

Path

Disallow

crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

crawler4j

Rule	Path
Disallow	/

Rule

Path

Disallow

crazywebcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

cyotekwebcopy

Rule	Path
Disallow	/

Rule

Path

Disallow

dataforseobot

Rule

Path

Disallow

deepcrawl

Rule

Path

Disallow

deusu

Rule

Path

Disallow

df bot 1.0

Rule

Path

Disallow

discoverybot

Rule

Path

Disallow

doctorhtml

Rule

Path

Disallow

domainstatsbot

Rule

Path

Disallow

domcopbot

Rule

Path

Disallow

doodlebot

Rule

Path

Disallow

dotbot

Rule

Path

Disallow

eventguru

Rule

Path

Disallow

experibot_v1

Rule

Path

Disallow

finbot

Rule

Path

Disallow

findfiles.net

Rule

Path

Disallow

findlinks

Rule

Path

Disallow

findshare

Rule

Path

Disallow

findxbot

Rule

Path

Disallow

followthatpage

Rule

Path

Disallow

fujilabolx2 spider

Rule

Path

Disallow

gigablastopensource

Rule

Path

Disallow

gigabot

Rule

Path

Disallow

gipo-crawler

Rule

Path

Disallow

go-http-client

Rule

Path

Disallow

golabbot

Rule

Path

Disallow

gozaikbot

Rule

Path

Disallow

grapeshotcrawler

Rule

Path

Disallow

heritrix

Rule

Path

Disallow

hexometer

Rule

Path

Disallow

httrack

Rule

Path

Disallow

ia_archiver

Rule

Path

Disallow

icc-crawler

Rule

Path

Disallow

iconsurf

Rule

Path

Disallow

idmarch

Rule

Path

Disallow

internet-structure-research-project-bot

Rule

Path

Disallow

izabee

Rule

Path

Disallow

jobboersebot

Rule

Path

Disallow

kikbot

Rule

Path

Disallow

laserlikebot

Rule

Path

Disallow

linguee

Rule

Path

Disallow

linkfluence

Rule

Path

Disallow

lipperhey spider

Rule

Path

Disallow

madbbot

Rule

Path

Disallow

magpie-crawler

Rule

Path

Disallow

mappy

Rule

Path

Disallow

mauibot

Rule

Path

Disallow

meanpathbot

Rule

Path

Disallow

mediatoolkitbot

Rule

Path

Disallow

megaindex

Rule

Path

Disallow

memorybot

Rule

Path

Disallow

miggibot

Rule

Path

Disallow

mixnodecache

Rule

Path

Disallow

mj12bot

Rule

Path

Disallow

moreover

Rule

Path

Disallow

mtrobot

Rule

Path

Disallow

netmechanic

Rule

Path

Disallow

netpeakcheckerbot

Rule

Path

Disallow

newscuratorbot

Rule

Path

Disallow

nimbostratus

Rule

Path

Disallow

omgili

Rule

Path

Disallow

onalyticabot

Rule

Path

Disallow

openstat

Rule

Path

Disallow

ospscraper

Rule

Path

Disallow

p40bot

Rule

Path

Disallow

pagepeeker

Rule

Path

Disallow

pagesinventory

Rule

Path

Disallow

pdfbot

Rule

Path

Disallow

petalbot

Rule

Path

Disallow

pimeyes.com crawler

Rule

Path

Disallow

pingdom

Rule

Path

Disallow

postpost

Rule

Path

Disallow

promptcloud

Rule

Path

Disallow

prospectb2b

Rule

Path

Disallow

prtg network monitor

Rule

Path

Disallow

psbot

Rule

Path

Disallow

pubmatic

Rule

Path

Disallow

putin

Rule

Path

Disallow

python-requests

Rule

Path

Disallow

repa

Rule

Path

Disallow

riddler

Rule

Path

Disallow

rogerbot

Rule

Path

Disallow

rsiteauditor

Rule

Path

Disallow

safednsbot

Rule

Path

Disallow

salesintelligent

Rule

Path

Disallow

sbl-bot

Rule

Path

Disallow

scrapy

Rule

Path

Disallow

screaming frog seo spider

Rule

Path

Disallow

searchie

Rule

Path

Disallow

seeker

Rule

Path

Disallow

seekport

Rule

Path

Disallow

semanticbot

Rule

Path

Disallow

semrushbot

Rule

Path

Disallow

seokicks

Rule

Path

Disallow

serpstatbot

Rule

Path

Disallow

sitebot

Rule

Path

Disallow

siteexplorer

Rule

Path

Disallow

sitesucker

Rule

Path

Disallow

sogou

Rule

Path

Disallow

spbot

Rule

Path

Disallow

spiderling

Rule

Path

Disallow

sqlmap

Rule

Path

Disallow

steeler

Rule

Path

Disallow

surdotlybot

Rule

Path

Disallow

surveybot

Rule

Path

Disallow

swebot

Rule

Path

Disallow

thumbshots-de-bot

Rule

Path

Disallow

thumbtack-thunderdome

Rule

Path

Disallow

tineye

Rule

Path

Disallow

traackr

Rule

Path

Disallow

tracemyfile

Rule

Path

Disallow

trendictionbot

Rule

Path

Disallow

twmbot

Rule

Path

Disallow

upictobot

Rule

Path

Disallow

uptimebot

Rule

Path

Disallow

urlappendbot

Rule

Path

Disallow

vagabondo

Rule

Path

Disallow

voltron

Rule

Path

Disallow

wangid

Rule

Path

Disallow

wbsearchbot

Rule

Path

Disallow

web sheriff

Rule

Path

Disallow

webbot

Rule

Path

Disallow

webinator

Rule

Path

Disallow

webindex

Rule

Path

Disallow

webreaper

Rule

Path

Disallow

webtechbot

Rule

Path

Disallow

webvac

Rule

Path

Disallow

whatcms

Rule

Path

Disallow

wikido

Rule

Path

Disallow

willybot

Rule

Path

Disallow

wonderbot

Rule

Path

Disallow

xovibot

Rule

Path

Disallow

yak

Rule

Path

Disallow

yandex

Rule

Path

Disallow

Other Records

Field

Value

sitemap

https://bisniswah.com/post-sitemap1.xml

sitemap

https://bisniswah.com/post-sitemap2.xml

sitemap

https://bisniswah.com/post-sitemap3.xml

sitemap

https://bisniswah.com/post-sitemap4.xml

Warnings

4 invalid lines.

bisniswah.comrobots.txt

Resource Scan

Scan Details

Last Successful Scan

Groups

*

a6-indexer

ahc

ahrefsbot

aihitbot

archive-it

archiveteam archivebot

aspiegelbot

awariosmartbot

bananabot

barkrowler

bdcbot

bdfetch

betabot

blablubb

blexbot

blogpulselive

bomborabot

bpimagewalker

brands-bot

brands-bot-logo

btcrawler

bubing

bublupbot

buck

buckyohare

builtwith

buzzbot

bvbot

careerbot

catchbot

ccbot

checkmarknetwork

cincraw

clark-crawler2

clickagy

cliqzbot

cms crawler

coccocbot-image

cocolyzebot

compspybot

copyright sheriff

copyrightcheck

cowbot

crawler

crawler4j

crazywebcrawler

cyotekwebcopy

dataforseobot

deepcrawl

deusu

df bot 1.0

discoverybot

doctorhtml

domainstatsbot

domcopbot

doodlebot

dotbot

eventguru

experibot_v1

finbot

findfiles.net

findlinks

findshare

findxbot

followthatpage

fujilabolx2 spider

gigablastopensource

gigabot

gipo-crawler

go-http-client

golabbot

gozaikbot

grapeshotcrawler

heritrix

bisniswah.com
robots.txt