bestdomainforadsense.blog
robots.txt

Robots Exclusion Standard data for bestdomainforadsense.blog

Archived Snapshots

Resource Scan

Scan Details

Site Domain	bestdomainforadsense.blog
Base Domain	bestdomainforadsense.blog
Scan Status	Ok
Last Scan	2024-11-18T10:15:21+00:00
Next Scan	2024-11-25T10:15:21+00:00

Last Scan

Scanned	2024-11-18T10:15:21+00:00
URL	https://bestdomainforadsense.blog/robots.txt
Domain IPs	2406:da18:9d0:143f:2124:4e9c:36a9:d9de, 52.221.42.138
Response IP	52.221.42.138
Found	Yes
Hash	e153846962bffb5eed14eda7e3ed9224266a851ef56bcc998c4c93b47ddb566f
SimHash	4300df630173

Groups

*

Rule	Path
Allow	/wp-admin/admin-ajax.php
Allow	//.css
Allow	//.js
Disallow	/wp-admin/
Disallow	/wp-includes/
Disallow	/readme.html
Disallow	/license.txt
Disallow	/xmlrpc.php
Disallow	/wp-login.php
Disallow	/wp-register.php
Disallow	/disclaimer/
Disallow	*?attachment_id=
Disallow	/privacy-policy

Rule

Path

Allow

/wp-admin/admin-ajax.php

Allow

/*/*.css

Allow

/*/*.js

Disallow

/wp-admin/

Disallow

/wp-includes/

Disallow

/readme.html

Disallow

/license.txt

Disallow

/xmlrpc.php

Disallow

/wp-login.php

Disallow

/wp-register.php

Disallow

*/disclaimer/*

Disallow

*?attachment_id=

Disallow

/privacy-policy

googlebot-image

Rule	Path
Allow	/wp-content/uploads/

Rule

Path

Allow

/wp-content/uploads/

mediapartners-google

Rule	Path
Allow	/

Rule

Path

Allow

adsbot-google

Rule	Path
Allow	/

Rule

Path

Allow

adsbot-google-mobile

Rule	Path
Allow	/

Rule

Path

Allow

bingbot

Rule	Path
Allow	/

Rule

Path

Allow

msnbot

Rule	Path
Allow	/

Rule

Path

Allow

msnbot-media

Rule	Path
Allow	/wp-content/uploads/

Rule

Path

Allow

/wp-content/uploads/

applebot

Rule	Path
Allow	/

Rule

Path

Allow

yandex

Rule	Path
Allow	/

Rule

Path

Allow

yandeximages

Rule	Path
Allow	/wp-content/uploads/

Rule

Path

Allow

/wp-content/uploads/

slurp

Rule	Path
Allow	/

Rule

Path

Allow

duckduckbot

Rule	Path
Allow	/

Rule

Path

Allow

qwantify

Rule	Path
Allow	/

Rule

Path

Allow

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

archive.org_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

siteexplorer

Rule	Path
Disallow	/

Rule

Path

Disallow

spbot

Rule	Path
Disallow	/

Rule

Path

Disallow

wbsearchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

linkdexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

screaming frog seo spider

Rule	Path
Disallow	/

Rule

Path

Disallow

netestate ne crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

moreover

Rule	Path
Disallow	/

Rule

Path

Disallow

sentibot

Rule	Path
Disallow	/

Rule

Path

Disallow

aboundexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

proximic

Rule	Path
Disallow	/

Rule

Path

Disallow

obot

Rule	Path
Disallow	/

Rule

Path

Disallow

meanpathbot

Rule	Path
Disallow	/

Rule

Path

Disallow

nutch

Rule	Path
Disallow	/

Rule

Path

Disallow

turnitinbot

Rule	Path
Disallow	/

Rule

Path

Disallow

zoominfobot

Rule	Path
Disallow	/

Rule

Path

Disallow

zmeu

Rule	Path
Disallow	/

Rule

Path

Disallow

grapeshot

Rule	Path
Disallow	/

Rule

Path

Disallow

python-requests

Rule	Path
Disallow	/

Rule

Path

Disallow

go-http-client

Rule	Path
Disallow	/

Rule

Path

Disallow

apache-httpclient

Rule	Path
Disallow	/

Rule

Path

Disallow

libwww-perl

Rule	Path
Disallow	/

Rule

Path

Disallow

curl

Rule	Path
Disallow	/

Rule

Path

Disallow

wget

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

*

Rule	Path
Allow	/ads.txt

Rule

Path

Allow

/ads.txt

*

Rule	Path
Allow	/app-ads.txt

Rule

Path

Allow

/app-ads.txt

Other Records

Field	Value
crawl-delay	4

Field

Value

crawl-delay

Comments

Block Bad Bots. "AI recommended setting" by ChatGPT
ChatGPT Bot Blocker - Block ChatGPT Bot from scrapping your content
Allow/Disallow Ads.txt
Allow/Disallow App-ads.txt
This robots.txt file was created by Better Robots.txt (Index & Rank Booster by Pagup) Plugin. https://www.better-robots.com/

bestdomainforadsense.blogrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

googlebot-image

mediapartners-google

adsbot-google

adsbot-google-mobile

bingbot

msnbot

msnbot-media

applebot

yandex

yandeximages

slurp

duckduckbot

qwantify

ia_archiver

archive.org_bot

siteexplorer

spbot

wbsearchbot

linkdexbot

screaming frog seo spider

netestate ne crawler

moreover

sentibot

aboundexbot

proximic

obot

meanpathbot

nutch

turnitinbot

zoominfobot

zmeu

grapeshot

python-requests

go-http-client

apache-httpclient

libwww-perl

curl

wget

gptbot

*

*

Other Records

Comments

bestdomainforadsense.blog
robots.txt