/.well-known/

Log In Sign Up

xporn.pro
robots.txt

Robots Exclusion Standard data for xporn.pro

Archived Snapshots

Resource Scan

Scan Details

Site Domain	xporn.pro
Base Domain	xporn.pro
Scan Status	Ok
Last Scan	2025-03-24T05:19:29+00:00
Next Scan	2025-04-23T05:19:29+00:00

Last Scan

Scanned	2025-03-24T05:19:29+00:00
URL	https://xporn.pro/robots.txt
Domain IPs	104.21.71.69, 172.67.143.187, 2606:4700:3034::ac43:8fbb, 2606:4700:3037::6815:4745
Response IP	172.67.143.187
Found	Yes
Hash	1511fabe9cea712cc871626115c881cce6a64898ee2b4d7161b214a7754ad31a
SimHash	535d70c18c19

Groups

blexbot

Rule

Path

Disallow

/

ahrefsbot

Rule

Path

Disallow

/

vagabondo

Rule

Path

Disallow

/

seokicks-robot

Rule

Path

Disallow

/

ia_archiver

Rule

Path

Disallow

/

archive.org_bot

Rule

Path

Disallow

/

special_archiver

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

heritrix

Rule

Path

Disallow

/

netestate ne crawler

Rule

Path

Disallow

/

sistrix

Rule

Path

Disallow

/

wbsearchbot

Rule

Path

Disallow

/

queryseekerspider

Rule

Path

Disallow

/

proximic

Rule

Path

Disallow

/

siteexplorer

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

semrushbot-sa

Rule

Path

Disallow

/

mail.ru

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

30

msnbot

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

7

bingbot

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

7

slurp

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

7

yandex

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

2

Back to top

Other Records

Field

Value

sitemap

https://xporn.pro/sitemap.xml

Back to top

Warnings

2 invalid lines.

Back to top