/.well-known/

Log In Sign Up

pornpervert.com
robots.txt

Robots Exclusion Standard data for pornpervert.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	pornpervert.com
Base Domain	pornpervert.com
Scan Status	Ok
Last Scan	2026-01-11T04:02:57+00:00
Next Scan	2026-02-10T04:02:57+00:00

Last Scan

Scanned	2026-01-11T04:02:57+00:00
URL	https://pornpervert.com/robots.txt
Domain IPs	104.21.46.153, 172.67.140.104, 2606:4700:3033::ac43:8c68, 2606:4700:3035::6815:2e99
Response IP	104.21.46.153
Found	Yes
Hash	505131154d2d2da9dc21622a3f98e947f1280e20207453e8c64a327e8c713bbc
SimHash	505dc1c2c139

Groups

*

Rule

Path

Allow

/

blexbot

Rule

Path

Disallow

/

ahrefsbot

Rule

Path

Disallow

/

vagabondo

Rule

Path

Disallow

/

seokicks-robot

Rule

Path

Disallow

/

ia_archiver

Rule

Path

Disallow

/

archive.org_bot

Rule

Path

Disallow

/

special_archiver

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

special_archiver

Rule

Path

Disallow

/

heritrix

Rule

Path

Disallow

/

netestate ne crawler

Rule

Path

Disallow

/

sistrix

Rule

Path

Disallow

/

wbsearchbot

Rule

Path

Disallow

/

queryseekerspider

Rule

Path

Disallow

/

proximic

Rule

Path

Disallow

/

siteexplorer

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

semrushbot-sa

Rule

Path

Disallow

/

mail.ru

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

30

msnbot

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

7

bingbot

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

7

slurp

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

7

yandex

No rules defined. All paths allowed.

Other Records

Field

Value

crawl-delay

2

Back to top

Warnings

2 invalid lines.

Back to top