remax.com.py
robots.txt

Robots Exclusion Standard data for remax.com.py

Archived Snapshots

Resource Scan

Scan Details

Site Domain	remax.com.py
Base Domain	remax.com.py
Scan Status	Ok
Last Scan	2025-02-22T22:07:45+00:00
Next Scan	2025-03-24T22:07:45+00:00

Last Scan

Scanned	2025-02-22T22:07:45+00:00
URL	https://remax.com.py/robots.txt
Redirect	https://www.remax.com.py/robots.txt
Redirect Domain	www.remax.com.py
Redirect Base	remax.com.py
Domain IPs	104.21.81.58, 172.67.157.43, 2606:4700:3036::ac43:9d2b, 2606:4700:3037::6815:513a
Redirect IPs	104.21.81.58, 172.67.157.43, 2606:4700:3036::ac43:9d2b, 2606:4700:3037::6815:513a
Response IP	104.21.81.58
Found	Yes
Hash	05cdd568157d42e811e0b05014470909cfb2d4420eeb2db8bf919edeacd4de30
SimHash	7ab5d072e211

Groups

*

Rule	Path
Allow	/
Disallow	/admin/
Disallow	/Admin/
Disallow	/intranet/
Disallow	/Intranet/
Disallow	/test/
Disallow	/stage/
Disallow	/Stage/
Disallow	/sup/
Disallow	/Sup/

Rule

Path

Allow

Disallow

/admin/

Disallow

/Admin/

Disallow

/intranet/

Disallow

/Intranet/

Disallow

/test/

Disallow

/stage/

Disallow

/Stage/

Disallow

/sup/

Disallow

/Sup/

yandex

Rule	Path
Disallow	/

Rule

Path

Disallow

barkrowler

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	15

Field

Value

crawl-delay

oai-searchbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	15

Field

Value

crawl-delay

seekportbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	2

Field

Value

crawl-delay

seekportbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	5

Field

Value

crawl-delay

imagesiftbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	15

Field

Value

crawl-delay

yandexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

haosouspider

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

sogou

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

seekportbot

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	5

Field

Value

crawl-delay

barkrowler

No rules defined. All paths allowed.

Other Records

Field	Value
crawl-delay	10

Field

Value

crawl-delay

yisouspider

Rule	Path
Disallow	/

Rule

Path

Disallow

seznambot

Rule	Path
Disallow	/

Rule

Path

Disallow

linkdexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

steeler

Rule	Path
Disallow	/

Rule

Path

Disallow

spiderling

Rule	Path
Disallow	/

Rule

Path

Disallow

trendiction-bot

Rule	Path
Disallow	/

Rule

Path

Disallow

sogou+web+spider

Rule	Path
Disallow	/

Rule

Path

Disallow

exabot

Rule	Path
Disallow	/

Rule

Path

Disallow

aspiegelbot

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

anthropic-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

ccbot

Rule	Path
Disallow	/

Rule

Path

Disallow

chatgpt-user

Rule	Path
Disallow	/

Rule

Path

Disallow

cohere-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

google-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

Warnings

2 invalid lines.

remax.com.pyrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

yandex

barkrowler

Other Records

oai-searchbot

Other Records

seekportbot

Other Records

seekportbot

Other Records

imagesiftbot

Other Records

yandexbot

haosouspider

dotbot

sogou

mj12bot

seekportbot

Other Records

barkrowler

Other Records

yisouspider

seznambot

linkdexbot

steeler

spiderling

trendiction-bot

sogou+web+spider

exabot

aspiegelbot

petalbot

blexbot

ahrefsbot

gptbot

anthropic-ai

ccbot

chatgpt-user

cohere-ai

google-extended

Warnings

remax.com.py
robots.txt