tahiras.blog
robots.txt

Robots Exclusion Standard data for tahiras.blog

Archived Snapshots

Resource Scan

Scan Details

Site Domain	tahiras.blog
Base Domain	tahiras.blog
Scan Status	Ok
Last Scan	2024-11-14T10:56:26+00:00
Next Scan	2024-11-21T10:56:26+00:00

Last Scan

Scanned	2024-11-14T10:56:26+00:00
URL	https://tahiras.blog/robots.txt
Redirect	https://allgalley.com/robots.txt
Redirect Domain	allgalley.com
Redirect Base	allgalley.com
Domain IPs	192.0.78.177, 192.0.78.234
Redirect IPs	192.0.78.173, 192.0.78.249
Response IP	192.0.78.173
Found	Yes
Hash	e39df56983be5f1dd641ba92a0ad60f0d669fecfbcb79c7941f8822391253f31
SimHash	4304df430173

Groups

*

Rule	Path
Allow	/wp-admin/admin-ajax.php
Allow	//.css
Allow	//.js
Disallow	/wp-admin/
Disallow	/wp-includes/
Disallow	/readme.html
Disallow	/license.txt
Disallow	/xmlrpc.php
Disallow	/wp-login.php
Disallow	/wp-register.php
Disallow	/disclaimer/
Disallow	*?attachment_id=
Disallow	/privacy-policy

Rule

Path

Allow

/wp-admin/admin-ajax.php

Allow

/*/*.css

Allow

/*/*.js

Disallow

/wp-admin/

Disallow

/wp-includes/

Disallow

/readme.html

Disallow

/license.txt

Disallow

/xmlrpc.php

Disallow

/wp-login.php

Disallow

/wp-register.php

Disallow

*/disclaimer/*

Disallow

*?attachment_id=

Disallow

/privacy-policy

googlebot

Rule	Path
Allow	/

Rule

Path

Allow

googlebot-image

Rule	Path
Allow	/wp-content/uploads/

Rule

Path

Allow

/wp-content/uploads/

mediapartners-google

Rule	Path
Allow	/

Rule

Path

Allow

adsbot-google

Rule	Path
Allow	/

Rule

Path

Allow

adsbot-google-mobile

Rule	Path
Allow	/

Rule

Path

Allow

bingbot

Rule	Path
Allow	/

Rule

Path

Allow

msnbot

Rule	Path
Allow	/

Rule

Path

Allow

msnbot-media

Rule	Path
Allow	/wp-content/uploads/

Rule

Path

Allow

/wp-content/uploads/

applebot

Rule	Path
Allow	/

Rule

Path

Allow

yandex

Rule	Path
Allow	/

Rule

Path

Allow

yandeximages

Rule	Path
Allow	/wp-content/uploads/

Rule

Path

Allow

/wp-content/uploads/

slurp

Rule	Path
Allow	/

Rule

Path

Allow

duckduckbot

Rule	Path
Allow	/

Rule

Path

Allow

qwantify

Rule	Path
Allow	/

Rule

Path

Allow

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

archive.org_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

siteexplorer

Rule	Path
Disallow	/

Rule

Path

Disallow

spbot

Rule	Path
Disallow	/

Rule

Path

Disallow

wbsearchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

linkdexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

screaming frog seo spider

Rule	Path
Disallow	/

Rule

Path

Disallow

netestate ne crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

moreover

Rule	Path
Disallow	/

Rule

Path

Disallow

sentibot

Rule	Path
Disallow	/

Rule

Path

Disallow

aboundexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

proximic

Rule	Path
Disallow	/

Rule

Path

Disallow

obot

Rule	Path
Disallow	/

Rule

Path

Disallow

meanpathbot

Rule	Path
Disallow	/

Rule

Path

Disallow

nutch

Rule	Path
Disallow	/

Rule

Path

Disallow

turnitinbot

Rule	Path
Disallow	/

Rule

Path

Disallow

zoominfobot

Rule	Path
Disallow	/

Rule

Path

Disallow

zmeu

Rule	Path
Disallow	/

Rule

Path

Disallow

grapeshot

Rule	Path
Disallow	/

Rule

Path

Disallow

python-requests

Rule	Path
Disallow	/

Rule

Path

Disallow

go-http-client

Rule	Path
Disallow	/

Rule

Path

Disallow

apache-httpclient

Rule	Path
Disallow	/

Rule

Path

Disallow

libwww-perl

Rule	Path
Disallow	/

Rule

Path

Disallow

curl

Rule	Path
Disallow	/

Rule

Path

Disallow

wget

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

*

Rule	Path
Allow	/ads.txt

Rule

Path

Allow

/ads.txt

*

Rule	Path
Allow	/app-ads.txt

Rule

Path

Allow

/app-ads.txt

Other Records

Field	Value
crawl-delay	5

Field

Value

crawl-delay

Comments

Block Bad Bots. "AI recommended setting" by ChatGPT
ChatGPT Bot Blocker - Block ChatGPT Bot from scrapping your content
Allow/Disallow Ads.txt
Allow/Disallow App-ads.txt
This robots.txt file was created by Better Robots.txt (Index & Rank Booster by Pagup) Plugin. https://www.better-robots.com/

tahiras.blogrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

googlebot

googlebot-image

mediapartners-google

adsbot-google

adsbot-google-mobile

bingbot

msnbot

msnbot-media

applebot

yandex

yandeximages

slurp

duckduckbot

qwantify

ia_archiver

archive.org_bot

siteexplorer

spbot

wbsearchbot

linkdexbot

screaming frog seo spider

netestate ne crawler

moreover

sentibot

aboundexbot

proximic

obot

meanpathbot

nutch

turnitinbot

zoominfobot

zmeu

grapeshot

python-requests

go-http-client

apache-httpclient

libwww-perl

curl

wget

gptbot

*

*

Other Records

Comments

tahiras.blog
robots.txt