/.well-known/

Log In Sign Up

befucktube.com
robots.txt

Robots Exclusion Standard data for befucktube.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	befucktube.com
Base Domain	befucktube.com
Scan Status	Ok
Last Scan	2026-02-10T23:08:24+00:00
Next Scan	2026-03-12T23:08:24+00:00

Last Scan

Scanned	2026-02-10T23:08:24+00:00
URL	https://befucktube.com/robots.txt
Domain IPs	104.21.68.158, 172.67.196.169, 2606:4700:3031::6815:449e, 2606:4700:3034::ac43:c4a9
Response IP	104.21.68.158
Found	Yes
Hash	040a10741f7dce7bfdf53a1aeae87a96332b6cbd8d196637f81f319b8d886799
SimHash	091ef502ebb8

Groups

*

Rule

Path

Disallow

/video/*

msnbot

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

2

bingbot

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

2

yandexbot

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

3

baiduspider

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

3

sogou

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

3

semrushbot

Rule

Path

Disallow

/

ahrefsbot

Rule

Path

Disallow

/

Allow

/$

blexbot

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

sistrix

Rule

Path

Disallow

/

ia_archiver

Rule

Path

Disallow

/

proximic

Rule

Path

Disallow

/

siteexplorer

Rule

Path

Disallow

/

semrushbot-sa

Rule

Path

Disallow

/

semrushbot-ba

Rule

Path

Disallow

/

semrushbot-si

Rule

Path

Disallow

/

semrushbot-swa

Rule

Path

Disallow

/

semrushbot-ct

Rule

Path

Disallow

/

semrushbot-bm

Rule

Path

Disallow

/

splitsignalbot

Rule

Path

Disallow

/

Back to top

Warnings

2 invalid lines.

Back to top