illicitencounters.com
robots.txt

Robots Exclusion Standard data for illicitencounters.com

Resource Scan

Scan Details

Site Domain illicitencounters.com
Base Domain illicitencounters.com
Scan Status Ok
Last Scan2025-12-09T20:38:45+00:00
Next Scan 2026-01-08T20:38:45+00:00

Last Scan

Scanned2025-12-09T20:38:45+00:00
URL https://illicitencounters.com/robots.txt
Redirect https://www.illicitencounters.com/robots.txt
Redirect Domain www.illicitencounters.com
Redirect Base illicitencounters.com
Domain IPs 52.117.196.121, 52.117.196.125
Redirect IPs 169.62.220.238, 169.62.220.241
Response IP 169.62.220.241
Found Yes
Hash 1165a38b467218e61fc9b8b762ee84d1c5729ade358cd1f53066ec18b486b943
SimHash 72168370ceb1

Groups

*

Rule Path
Allow /

googlebot-image

Rule Path
Disallow /

gptbot

Rule Path
Disallow /member/profile/show/

chatgpt-user

Rule Path
Disallow /member/profile/show/

google-extended

Rule Path
Disallow /member/profile/show/

claude-web

Rule Path
Disallow /member/profile/show/

claudebot

Rule Path
Disallow /member/profile/show/

perplexitybot

Rule Path
Disallow /member/profile/show/

anthropic-ai

Rule Path
Disallow /member/profile/show/

ccbot

Rule Path
Disallow /member/profile/show/

cohere-ai

Rule Path
Disallow /member/profile/show/

openai-crawler

Rule Path
Disallow /member/profile/show/

facebookbot

Rule Path
Disallow /member/profile/show/

bytespider

Rule Path
Disallow /member/profile/show/

amazonbot

Rule Path
Disallow /member/profile/show/

omgilibot

Rule Path
Disallow /member/profile/show/

omnibot

Rule Path
Disallow /member/profile/show/

ai-dev

Rule Path
Disallow /member/profile/show/

scrapingbot

Rule Path
Disallow /member/profile/show/

diffbot

Rule Path
Disallow /member/profile/show/

youbot

Rule Path
Disallow /member/profile/show/

neevaai

Rule Path
Disallow /member/profile/show/

bloombot

Rule Path
Disallow /member/profile/show/

scaleai

Rule Path
Disallow /member/profile/show/

huggingface bot

Rule Path
Disallow /member/profile/show/

applebot-extended

Rule Path
Disallow /member/profile/show/

metaai-bot

Rule Path
Disallow /member/profile/show/

semrushbot

Rule Path
Disallow /member/profile/show/

ahrefsbot

Rule Path
Disallow /member/profile/show/

mj12bot

Rule Path
Disallow /member/profile/show/

petalbot

Rule Path
Disallow /member/profile/show/

dotbot

Rule Path
Disallow /member/profile/show/

blexbot

Rule Path
Disallow /member/profile/show/

barkrowler

Rule Path
Disallow /member/profile/show/

baiduspider

Rule Path
Disallow /member/profile/show/

sogou

Rule Path
Disallow /member/profile/show/

yandexbot

Rule Path
Disallow /member/profile/show/

yandeximages

Rule Path
Disallow /member/profile/show/

yandeximageresizer

Rule Path
Disallow /member/profile/show/

censysinspect

Rule Path
Disallow /member/profile/show/

seznambot

Rule Path
Disallow /member/profile/show/

freefind

Rule Path
Disallow /member/profile/show/

archive.org_bot

Rule Path
Disallow /member/profile/show/