pnpde.social
robots.txt

Robots Exclusion Standard data for pnpde.social

Archived Snapshots

Resource Scan

Scan Details

Site Domain	pnpde.social
Base Domain	pnpde.social
Scan Status	Ok
Last Scan	2025-08-11T01:17:08+00:00
Next Scan	2025-09-10T01:17:08+00:00

Last Scan

Scanned	2025-08-11T01:17:08+00:00
URL	https://pnpde.social/robots.txt
Domain IPs	157.90.127.188, 2a01:4f8:1c1e:b476::1
Response IP	157.90.127.188
Found	Yes
Hash	13dcbc3e6799fdf84d089f665c7653e16d9123c13bfb41b80be324e63860f1b6
SimHash	12161950f162

Groups

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

anthropic-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

claudebot

Rule	Path
Disallow	/

Rule

Path

Disallow

claude-web

Rule	Path
Disallow	/

Rule

Path

Disallow

yak

Rule	Path
Disallow	/

Rule

Path

Disallow

linkfluence.com

Rule	Path
Disallow	/

Rule

Path

Disallow

ccbot

Rule	Path
Disallow	/

Rule

Path

Disallow

chatgpt-user

Rule	Path
Disallow	/

Rule

Path

Disallow

google-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

imagesiftbot

Rule	Path
Disallow	/

Rule

Path

Disallow

youbot

Rule	Path
Disallow	/

Rule

Path

Disallow

turnitinbot

Rule	Path
Disallow	/

Rule

Path

Disallow

facebookbot

Rule	Path
Disallow	/

Rule

Path

Disallow

omigilibot

Rule	Path
Disallow	/

Rule

Path

Disallow

perplexitybot

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalagent

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

barkrowler

Rule	Path
Disallow	/

Rule

Path

Disallow

bytedance

Rule	Path
Disallow	/

Rule

Path

Disallow

bytespider

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

awariobot
awariorssbot
awariosmartbot

Rule	Path
Disallow	/

Rule

Path

Disallow

magpie-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

dataprovider

Rule	Path
Disallow	/

Rule

Path

Disallow

dataforseobot

Rule	Path
Disallow	/

Rule

Path

Disallow

technicalseodotcom

Rule	Path
Disallow	/

Rule

Path

Disallow

screaming frog seo spider

Rule	Path
Disallow	/

Rule

Path

Disallow

botify

Rule	Path
Disallow	/

Rule

Path

Disallow

oncrawl

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

seokicks-robot

Rule	Path
Disallow	/

Rule

Path

Disallow

netestate ne crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

domainstatsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

pinterestbot

Rule	Path
Disallow	/

Rule

Path

Disallow

omgilibot

Rule	Path
Disallow	/

Rule

Path

Disallow

velenpublicwebcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

vsusearchspider

Rule	Path
Disallow	/

Rule

Path

Disallow

*

Rule	Path
Disallow	/media_proxy/
Disallow	/interact/

Rule

Path

Disallow

/media_proxy/

Disallow

/interact/

Comments

See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
AI
SEO
from https://technicalseo.com/tools/robots-txt/
others

pnpde.socialrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

gptbot

anthropic-ai

claudebot

claude-web

yak

linkfluence.com

ccbot

chatgpt-user

google-extended

imagesiftbot

youbot

turnitinbot

facebookbot

omigilibot

perplexitybot

meta-externalagent

applebot

blexbot

semrushbot

barkrowler

bytedance

bytespider

dotbot

ahrefsbot

awariobotawariorssbotawariosmartbot

magpie-crawler

dataprovider

dataforseobot

technicalseodotcom

screaming frog seo spider

botify

oncrawl

mj12bot

seokicks-robot

netestate ne crawler

domainstatsbot

pinterestbot

omgilibot

velenpublicwebcrawler

vsusearchspider

*

Comments

pnpde.social
robots.txt

awariobot
awariorssbot
awariosmartbot