posfie.com
robots.txt

Robots Exclusion Standard data for posfie.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	posfie.com
Base Domain	posfie.com
Scan Status	Ok
Last Scan	2025-06-17T09:46:37+00:00
Next Scan	2025-06-24T09:46:37+00:00

Last Scan

Scanned	2025-06-17T09:46:37+00:00
URL	https://posfie.com/robots.txt
Domain IPs	52.69.216.56, 54.249.202.98, 54.65.170.102
Response IP	52.69.216.56
Found	Yes
Hash	4474304ced51922705b190976e14d64890c04dcad8fe62a42cca8d5a7a45bcbf
SimHash	7115d140c189

Groups

twitterbot
mediapartners-google
google-display-ads-bot
adsbot-google-mobile
adsbot-google
googlebot

Rule	Path
Allow	/

Rule

Path

Allow

bingbot

Rule	Path
Allow	/

Rule

Path

Allow

Other Records

Field	Value
crawl-delay	30

Field

Value

crawl-delay

feedly

Rule	Path
Allow	/rss
Allow	/

Rule

Path

Allow

/rss

Allow

Other Records

Field	Value
crawl-delay	30

Field

Value

crawl-delay

criteobot

Rule	Path
Disallow	/

Rule

Path

Disallow

slurp

Rule	Path
Disallow	/

Rule

Path

Disallow

megalodon

Rule	Path
Disallow	/

Rule

Path

Disallow

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

ahrefsbot

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot

Rule	Path
Disallow	/

Rule

Path

Disallow

mj12bot

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

chatgpt-user

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

ccbot

Rule	Path
Disallow	/

Rule

Path

Disallow

imagesiftbot

Rule	Path
Disallow	/

Rule

Path

Disallow

claudebot

Rule	Path
Disallow	/

Rule

Path

Disallow

claude-web

Rule	Path
Disallow	/

Rule

Path

Disallow

perplexitybot

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalagent

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalfetcher

Rule	Path
Disallow	/

Rule

Path

Disallow

duckassistbot

Rule	Path
Disallow	/

Rule

Path

Disallow

diffbot

Rule	Path
Disallow	/

Rule

Path

Disallow

cohere-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

oai-searchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

anthropic-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

ai2bot

Rule	Path
Disallow	/

Rule

Path

Disallow

riddler

Rule	Path
Disallow	/

Rule

Path

Disallow

mlbot

Rule	Path
Disallow	/

Rule

Path

Disallow

just-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

duckduckbot

Rule	Path
Disallow	/

Rule

Path

Disallow

sogou web spider

Rule	Path
Disallow	/

Rule

Path

Disallow

rogerbot

Rule	Path
Disallow	/

Rule

Path

Disallow

omgilibot

Rule	Path
Disallow	/

Rule

Path

Disallow

omgili

Rule	Path
Disallow	/

Rule

Path

Disallow

peer39_crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

bytespider

Rule	Path
Disallow	/

Rule

Path

Disallow

ias-or

Rule	Path
Disallow	/

Rule

Path

Disallow

*

Rule	Path
Disallow	/search
Disallow	/create
Disallow	/rss
Disallow	/api

Rule

Path

Disallow

/search

Disallow

/create

Disallow

/rss

Disallow

/api

posfie.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

twitterbotmediapartners-googlegoogle-display-ads-botadsbot-google-mobileadsbot-googlegooglebot

bingbot

Other Records

feedly

Other Records

criteobot

slurp

megalodon

ia_archiver

ahrefsbot

semrushbot

mj12bot

blexbot

gptbot

chatgpt-user

applebot-extended

ccbot

imagesiftbot

claudebot

claude-web

perplexitybot

meta-externalagent

meta-externalfetcher

duckassistbot

diffbot

cohere-ai

oai-searchbot

anthropic-ai

ai2bot

riddler

mlbot

just-crawler

duckduckbot

sogou web spider

rogerbot

omgilibot

omgili

peer39_crawler

bytespider

ias-or

*

posfie.com
robots.txt

twitterbot
mediapartners-google
google-display-ads-bot
adsbot-google-mobile
adsbot-google
googlebot