/.well-known/

Log In Sign Up

freeporncategories.com
robots.txt

Robots Exclusion Standard data for freeporncategories.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	freeporncategories.com
Base Domain	freeporncategories.com
Scan Status	Ok
Last Scan	2025-03-29T13:23:53+00:00
Next Scan	2025-04-12T13:23:53+00:00

Last Scan

Scanned	2025-03-29T13:23:53+00:00
URL	https://freeporncategories.com/robots.txt
Domain IPs	104.21.71.157, 172.67.146.86, 2606:4700:3030::6815:479d, 2606:4700:3037::ac43:9256
Response IP	172.67.146.86
Found	Yes
Hash	bc6fc88a3e5362e5778e3492593304663c0a90f00e758e3b2d1aaa635a29a199
SimHash	fb1d4d426283

Groups

*

Rule

Path

Disallow

/search*

Disallow

/p/*

Disallow

/v/*

bingbot

Rule

Path

Disallow

/

yandex

Rule

Path

Disallow

/

ahrefsbot

Rule

Path

Disallow

/

semrushbot

Rule

Path

Disallow

/

semrushbot-sa

Rule

Path

Disallow

/

mj12bot

Rule

Path

Disallow

/

siteexplorer

Rule

Path

Disallow

/

xovibot

Rule

Path

Disallow

/

megaindex

Rule

Path

Disallow

/

spbot

Rule

Path

Disallow

/

yisouspider

Rule

Path

Disallow

/

seznambot

Rule

Path

Disallow

/

smtbot

Rule

Path

Disallow

/

mail.ru_bot

Rule

Path

Disallow

/

wbsearchbot

Rule

Path

Disallow

/

blexbot

Rule

Path

Disallow

/

coccocbot-web

Rule

Path

Disallow

/

yeti

Rule

Path

Disallow

/

ccbot

Rule

Path

Disallow

/

proximic

Rule

Path

Disallow

/

comscore

Rule

Path

Disallow

/

gptbot

Rule

Path

Disallow

/

claudebot

Rule

Path

Disallow

/

dotbot

Rule

Path

Disallow

/

applebot-extended

Rule

Path

Disallow

/

deepseekbot

Rule

Path

Disallow

/

Back to top