dhofari.com
robots.txt

Robots Exclusion Standard data for dhofari.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	dhofari.com
Base Domain	dhofari.com
Scan Status	Ok
Last Scan	2025-11-21T02:09:07+00:00
Next Scan	2025-12-21T02:09:07+00:00

Last Scan

Scanned	2025-11-21T02:09:07+00:00
URL	https://dhofari.com/robots.txt
Domain IPs	162.0.239.63
Response IP	162.0.239.63
Found	Yes
Hash	f724ca309feef8f7f06489ef3536ae660b0a205be4087dc74eaa462419c1a17a
SimHash	1a925b350b99

Groups

*

Rule	Path
Disallow	/

Rule

Path

Disallow

nbot

Rule	Path
Allow	/

Rule

Path

Allow

bingbot

Rule	Path
Allow	/

Rule

Path

Allow

semrushbot

Rule	Path
Allow	/

Rule

Path

Allow

ahrefsbot

Rule	Path
Allow	/

Rule

Path

Allow

trendiction

Rule	Path
Allow	/

Rule

Path

Allow

googlebot

Rule	Path
Allow	/

Rule

Path

Allow

applebot

Rule	Path
Allow	/

Rule

Path

Allow

yandexbot

Rule	Path
Allow	/

Rule

Path

Allow

link

Rule	Path
Allow	/

Rule

Path

Allow

mj12bot

Rule	Path
Allow	/

Rule

Path

Allow

seek

Rule	Path
Allow	/

Rule

Path

Allow

blexbot

Rule	Path
Allow	/

Rule

Path

Allow

barkrowler

Rule	Path
Allow	/

Rule

Path

Allow

googlebot-image

Rule	Path
Allow	/

Rule

Path

Allow

spider

Rule	Path
Allow	/

Rule

Path

Allow

facebookexternalhit

Rule	Path
Allow	/

Rule

Path

Allow

cfnetwork

Rule	Path
Allow	/

Rule

Path

Allow

dalvik

Rule	Path
Allow	/

Rule

Path

Allow

ccbot

Rule	Path
Allow	/

Rule

Path

Allow

sogou web spider

Rule	Path
Allow	/

Rule

Path

Allow

robot

Rule	Path
Allow	/

Rule

Path

Allow

apache-httpclient

Rule	Path
Allow	/

Rule

Path

Allow

go-http-client

Rule	Path
Allow	/

Rule

Path

Allow

yandeximages

Rule	Path
Allow	/

Rule

Path

Allow

dotbot

Rule	Path
Allow	/

Rule

Path

Allow

baidu ( catchall )

Rule	Path
Allow	/

Rule

Path

Allow

duckduckgo-favicons-bot

Rule	Path
Allow	/

Rule

Path

Allow

mail.ru bot

Rule	Path
Allow	/

Rule

Path

Allow

seznambot

Rule	Path
Allow	/

Rule

Path

Allow

pinterestbot

Rule	Path
Allow	/

Rule

Path

Allow

yandex ( catchall )

Rule	Path
Allow	/

Rule

Path

Allow

zoominfobot

Rule	Path
Allow	/

Rule

Path

Allow

crawl

Rule	Path
Allow	/

Rule

Path

Allow

yisouspider

Rule	Path
Allow	/

Rule

Path

Allow

survey

Rule	Path
Allow	/

Rule

Path

Allow

phantomjs

Rule	Path
Allow	/

Rule

Path

Allow

screaming frog seo spider

Rule	Path
Allow	/

Rule

Path

Allow

linkedinbot

Rule	Path
Allow	/

Rule

Path

Allow

surdotlybot

Rule	Path
Allow	/

Rule

Path

Allow

feedfetcher-google

Rule	Path
Allow	/

Rule

Path

Allow

fetcher

Rule	Path
Allow	/

Rule

Path

Allow

linkdexbot

Rule	Path
Allow	/

Rule

Path

Allow

konqueror

Rule	Path
Allow	/

Rule

Path

Allow

telegrambot

Rule	Path
Allow	/

Rule

Path

Allow

curl

Rule	Path
Allow	/

Rule

Path

Allow

checker

Rule	Path
Allow	/

Rule

Path

Allow

python-urllib

Rule	Path
Allow	/

Rule

Path

Allow

lightspeed

Rule

Path

Allow

java ( catchall )

Rule

Path

Allow

wget

Rule

Path

Allow

yahoo! slurp

Rule

Path

Allow

a php script

Rule

Path

Allow

archive.org_bot

Rule

Path

Allow

Warnings

2 invalid lines.

dhofari.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

nbot

bingbot

semrushbot

ahrefsbot

trendiction

googlebot

applebot

yandexbot

link

mj12bot

seek

blexbot

barkrowler

googlebot-image

spider

facebookexternalhit

cfnetwork

dalvik

ccbot

sogou web spider

robot

apache-httpclient

go-http-client

yandeximages

dotbot

baidu ( catchall )

duckduckgo-favicons-bot

mail.ru bot

seznambot

pinterestbot

yandex ( catchall )

zoominfobot

crawl

yisouspider

survey

phantomjs

screaming frog seo spider

linkedinbot

surdotlybot

feedfetcher-google

fetcher

linkdexbot

konqueror

telegrambot

curl

checker

python-urllib

lightspeed

java ( catchall )

wget

yahoo! slurp

a php script

archive.org_bot

Warnings

dhofari.com
robots.txt