thibaans.com
robots.txt

Robots Exclusion Standard data for thibaans.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	thibaans.com
Base Domain	thibaans.com
Scan Status	Failed
Failure Stage	Fetching resource.
Failure Reason	Couldn't connect to server.
Last Scan	2025-04-05T03:36:28+00:00
Next Scan	2025-07-04T03:36:28+00:00

Last Successful Scan

Scanned	2022-09-27T14:24:44+00:00
URL	https://thibaans.com/robots.txt
Response IP	103.30.127.29
Found	Yes
Hash	f09be6a22e6a407cde4219bd927ca3098f2ae86755ea96c95645cdd7a537c5e6
SimHash	421c92224180

Groups

*

Rule	Path
Disallow	/wp-admin/
Allow	/wp-admin/admin-ajax.php

Rule

Path

Disallow

/wp-admin/

Allow

/wp-admin/admin-ajax.php

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

seokicks-robot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

adsacomponent

Rule	Path
Disallow	/

Rule

Path

Disallow

aqua_products

Rule	Path
Disallow	/

Rule

Path

Disallow

asterias

Rule	Path
Disallow	/

Rule

Path

Disallow

b2w

Rule	Path
Disallow	/

Rule

Path

Disallow

baidu

Rule	Path
Disallow	/

Rule

Path

Disallow

blo.gs

Rule	Path
Disallow	/

Rule

Path

Disallow

blowfish

Rule	Path
Disallow	/

Rule

Path

Disallow

bot

Rule	Path
Disallow	/

Rule

Path

Disallow

bullseye

Rule	Path
Disallow	/

Rule

Path

Disallow

bunnyslippers

Rule	Path
Disallow	/

Rule

Path

Disallow

cherry pick

Rule	Path
Disallow	/

Rule

Path

Disallow

collect

Rule	Path
Disallow	/

Rule

Path

Disallow

copernic

Rule	Path
Disallow	/

Rule

Path

Disallow

copyrightcheck

Rule	Path
Disallow	/

Rule

Path

Disallow

cosmos

Rule	Path
Disallow	/

Rule

Path

Disallow

crawl

Rule	Path
Disallow	/

Rule

Path

Disallow

crescent

Rule	Path
Disallow	/

Rule

Path

Disallow

cuil

Rule	Path
Disallow	/

Rule

Path

Disallow

domainsdb.net

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

emailwolf

Rule	Path
Disallow	/

Rule

Path

Disallow

exabot

Rule	Path
Disallow	/

Rule

Path

Disallow

extractorpro

Rule	Path
Disallow	/

Rule

Path

Disallow

fairad client

Rule	Path
Disallow	/

Rule

Path

Disallow

freefind

Rule	Path
Disallow	/

Rule

Path

Disallow

getright

Rule	Path
Disallow	/

Rule

Path

Disallow

gigabot

Rule	Path
Disallow	/

Rule

Path

Disallow

grub

Rule	Path
Disallow	/

Rule

Path

Disallow

harvest

Rule	Path
Disallow	/

Rule

Path

Disallow

hatena antenna

Rule	Path
Disallow	/

Rule

Path

Disallow

hloader

Rule	Path
Disallow	/

Rule

Path

Disallow

httplib

Rule	Path
Disallow	/

Rule

Path

Disallow

humanlinks

Rule	Path
Disallow	/

Rule

Path

Disallow

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

iron33

Rule	Path
Disallow	/

Rule

Path

Disallow

jakarta

Rule	Path
Disallow	/

Rule

Path

Disallow

keyword density

Rule	Path
Disallow	/

Rule

Path

Disallow

larbin

Rule	Path
Disallow	/

Rule

Path

Disallow

libweb

Rule	Path
Disallow	/

Rule

Path

Disallow

linkextractorpro

Rule	Path
Disallow	/

Rule

Path

Disallow

linkwalker

Rule	Path
Disallow	/

Rule

Path

Disallow

lwp-trivial

Rule	Path
Disallow	/

Rule

Path

Disallow

majestic-12

Rule

Path

Disallow

mata hari

Rule

Path

Disallow

miixpc

Rule

Path

Disallow

mister pix

Rule

Path

Disallow

moget

Rule

Path

Disallow

naver

Rule

Path

Disallow

netants

Rule

Path

Disallow

netmechanic

Rule

Path

Disallow

nicerspro

Rule

Path

Disallow

nutch

Rule

Path

Disallow

offline explorer

Rule

Path

Disallow

ole control

Rule

Path

Disallow

openfind

Rule

Path

Disallow

openfind data gathere

Rule

Path

Disallow

perman

Rule

Path

Disallow

prowebwalker

Rule

Path

Disallow

pycurl

Rule

Path

Disallow

python-urllib

Rule

Path

Disallow

queryn metasearch

Rule

Path

Disallow

radiation retriever

Rule

Path

Disallow

repomonkey

Rule

Path

Disallow

rma

Rule

Path

Disallow

rogerbot

Rule

Path

Disallow

scan

Rule

Path

Disallow

search

Rule

Path

Disallow

semalt

Rule

Path

Disallow

semalt.com

Rule

Path

Disallow

siphon

Rule

Path

Disallow

sitesnagger

Rule

Path

Disallow

sogou

Rule

Path

Disallow

sootle

Rule

Path

Disallow

soso

Rule

Path

Disallow

spanner

Rule

Path

Disallow

spider

Rule

Path

Disallow

spyder

Rule

Path

Disallow

stanford

Rule

Path

Disallow

suzuran

Rule

Path

Disallow

szukacz

Rule

Path

Disallow

teleport

Rule

Path

Disallow

teleportpro

Rule

Path

Disallow

telesoft

Rule

Path

Disallow

the intraformant

Rule

Path

Disallow

thenomad

Rule

Path

Disallow

tocrawl

Rule

Path

Disallow

turingos

Rule

Path

Disallow

url control

Rule

Path

Disallow

url_spider_pro

Rule

Path

Disallow

urly warning

Rule

Path

Disallow

vci

Rule

Path

Disallow

web image collector

Rule

Path

Disallow

webauto

Rule

Path

Disallow

webbandit

Rule

Path

Disallow

webcopier

Rule

Path

Disallow

webenhancer

Rule

Path

Disallow

webmeup

Rule

Path

Disallow

webmeup.com

Rule

Path

Disallow

websauger

Rule

Path

Disallow

website quester

Rule

Path

Disallow

webster pro

Rule

Path

Disallow

webstripper

Rule

Path

Disallow

webvac

Rule

Path

Disallow

webzip

Rule

Path

Disallow

wget

Rule

Path

Disallow

whowhere

Rule

Path

Disallow

www-collector-e

Rule

Path

Disallow

xenu

Rule

Path

Disallow

yandex

Rule

Path

Disallow

zeus

Rule

Path

Disallow

Other Records

Field

Value

sitemap

https://thibaans.com/post-sitemap.xml

sitemap

https://thibaans.com/page-sitemap.xml

sitemap

https://thibaans.com/category-sitemap.xml

sitemap

https://thibaans.com/post_tag-sitemap.xml

sitemap

https://thibaans.com/author-sitemap.xml

thibaans.comrobots.txt

Resource Scan

Scan Details

Last Successful Scan

Groups

*

ahrefsbot

blexbot

mj12bot

seokicks-robot

semrushbot

adsacomponent

aqua_products

asterias

b2w

baidu

blo.gs

blowfish

bot

bullseye

bunnyslippers

cherry pick

collect

copernic

copyrightcheck

cosmos

crawl

crescent

cuil

domainsdb.net

dotbot

emailwolf

exabot

extractorpro

fairad client

freefind

getright

gigabot

grub

harvest

hatena antenna

hloader

httplib

humanlinks

ia_archiver

iron33

jakarta

keyword density

larbin

libweb

linkextractorpro

linkwalker

lwp-trivial

majestic-12

mata hari

miixpc

mister pix

moget

naver

netants

netmechanic

nicerspro

nutch

offline explorer

ole control

openfind

openfind data gathere

perman

prowebwalker

pycurl

python-urllib

queryn metasearch

radiation retriever

repomonkey

rma

rogerbot

scan

search

semalt

semalt.com

thibaans.com
robots.txt