Open Source/LLM 데이터 보안 게이트웨이

Context.dev(보안·컴플라이언스 중심 LLM 데이터 게이트웨이) 아키텍처 및 시스템 설계서

Context Lab 2025. 12. 19. 05:32

Context.dev(보안·컴플라이언스 중심 LLM 데이터 게이트웨이) 아키텍처 및 시스템 설계서

문서 구성

  1. 아키텍처 개요 (요약 다이어그램 + 핵심 컴포넌트)
  2. 상세 컴포넌트 설계 (기능별 책임, 데이터 흐름)
  3. 보안·컴플라이언스 설계 (PII 탐지/익명화, 암호화, 감사, 인증·인가)
  4. 데이터 모델 / 로그 스키마 (Audit / Request / Policy 등 예시)
  5. API 설계 샘플 (엔드포인트, 페이로드, 에러모델)
  6. 인프라·배포·운영 (CI/CD, 모니터링, 스케일링)
  7. ML/PII 탐지 구현 옵션 (오픈소스 vs managed, 성능 트레이드오프)
  8. MVP 우선순위 & 6개월 기술 로드맵 (작업 항목, 산출물)

 

1. 아키텍처 개요

(텍스트 다이어그램)

[Client App / Dev] 

    -> HTTPS -> [API Gateway / Auth (JWT & API Key)]

        -> [Ingress Proxy (Rate limit, WAF)]

            -> [PII Filter & Policy Engine]  --> [Audit Logger (Append-only store)]

            -> [Context Retriever (RAG connectors -> Vector DB)] 

            -> [Context Assembler]

            -> [LLM Proxy (OpenAI / Anthropic / Private LLM)]

            -> [Response Post-Processor (redaction, token scrub)]

            -> [Response -> Client]

            

Background:

- Connector Workers (Sync Notion, Zendesk, DB -> Document store -> Chunker -> Embedding -> Vector DB)

- Admin UI (Policy Editor, Audit Viewer, Dashboard)

- Management Services: RBAC, Billing, Tenant DB (Postgres)

- Secrets & KMS: Vault / Cloud KMS

- Observability: OpenTelemetry -> Prometheus / Grafana / ELK

핵심 아이디어: 모든 LLM 요청은 PII 필터 → 접근 정책 적용 → 감사 로그 기록을 거쳐 LLM에 도달. RAG용 외부 데이터도 이 파이프라인을 통과해 안전화(마스킹/익명화) 후 사용.

 

2. 상세 컴포넌트 설계

2.1 인그레스 레이어

  • API Gateway (AWS API Gateway / Kong / Gloo)

2.2 인증·인가

  • Auth Service
  • 추천: JWT for service calls + API Key for dev usage. Admin/Enterprise는 SSO.

2.3 Ingress Proxy (보안 전처리)

  • TLS termination, IP allowlist, 기본 WAF rules
  • 요청 수집(요약) 후 PII/Policy 검사로 전송

2.4 PII Filter & Policy Engine (핵심)

  • 기능
  • 구성
  • 요구 성능: 100–500 ms 추가 지연 허용 목표 (최적화 필요)

2.5 Context Retriever (RAG Connector)

  • Connectors: Notion, Zendesk, Salesforce, S3, DB (Postgres, MySQL), Confluence, Google Drive
  • Worker architecture:
  • Security: Connectors use per-tenant credentials encrypted by KMS; connector workers run in VPC with private endpoints.

2.6 Context Assembler

  • Given user query:

2.7 LLM Proxy

  • Route to configured LLM provider per tenant (OpenAI/Anthropic/Private LLM)
  • Handle throttling & batching
  • Add usage accounting (token counting)
  • If tenant uses own model (private cloud), support mTLS or VPC peering

2.8 Response Post-Processor

  • Check response for hallucination patterns (confidence heuristics) — optional
  • Final PII sweep (regex) to sanitize any leaks
  • Log redaction events in audit trail

2.9 Audit Logger & Storage

  • Append-only storage (immutable) for every request/response metadata (but not full plaintext by default unless tenant permits)
  • Store:
  • Retention controls per-tenant (compliance)
  • Tamper-evident: store hash chain or sign logs (optionally immutably append to blockchain or WORM storage for high security)

2.10 Admin UI / Developer UX

  • Policy Editor: write rules (allow/deny/mask)
  • Audit Viewer: search by request_id / time / user
  • Connector management: add/remove, status
  • Dashboard: usage, anomaly alerts, billing

 

3. 보안·컴플라이언스 설계

3.1 PII 탐지 및 익명화 원칙

  • 탐지 계층: NER(ML) + deterministic regex + allowlist/blocklist 혼합 방식
  • 마스킹 전략:
  • 사용자 선택: tenant 설정에 따라 full retention(for debugging) or no retention (default privacy-first)

3.2 암호화

  • In-transit: TLS 1.2+ (prefer TLS1.3)
  • At-rest:
  • Key rotation policies and audit for KMS usage

3.3 인증·인가

  • Support:
  • RBAC model: roles (admin, auditor, developer, viewer) with resource-level permissions

3.4 감사 로그(Compliance)

  • Immutable logs with retention and export (CSV / JSON) for auditors
  • Log signing for tamper-evidence
  • SLA + Data Processing Agreement (DPA) templates for enterprise customers
  • Support data subject requests (GDPR Right to Erasure) — delete associated tenant artifacts per policy (with audit trail of deletion request)

3.5 On-Prem / Private Deployment

  • Enterprise option: AWS VPC deployment or fully on-prem Docker/K8s helm chart
  • Provide restricted admin console and local storage
  • Hybrid model: control plane SaaS, data plane in customer VPC (recommended for finance/health)

 

4. 데이터 모델 / 로그 스키마 (예시)

4.1 Audit Log Record (JSON)

{

  "request_id": "uuid",

  "tenant_id": "tenant_123",

  "api_key_id": "key_abc",

  "timestamp": "2025-10-13T08:00:00Z",

  "client_ip": "1.2.3.4",

  "endpoint": "/v1/query",

  "query_text_meta": {

    "hash": "sha256(...)",

    "redacted": true,

    "redaction_summary": [

      {"type":"EMAIL","occurrences":1,"token":"<EMAIL_1>"},

      {"type":"SSN","occurrences":0}

    ]

  },

  "policy_id": "policy_45",

  "policy_decision": "ALLOW_WITH_MASK",

  "retrieved_context": [

    {"source":"zendesk","doc_id":"doc_1","masked":true,"metadata": {"last_updated":"..."}}

  ],

  "llm_provider": "openai-gpt4o",

  "llm_cost_tokens": {"prompt":123, "completion":45},

  "response_meta": {"redacted": true, "redaction_summary":[...]},

  "storage_pointer": "s3://audit/tenant_123/2025/10/13/uuid.json",

  "hash": "sha256_of_record"

}

4.2 RBAC Tables (Postgres)

  • tenants
  • users
  • roles
  • permissions
  • role_bindings (user -> role -> tenant)
  • policies (JSONLogic-like rules)

 

5. API 설계 샘플

5.1 /v1/query (POST)

  • 설명: 안전한 질의 처리 — PII 필터링, context retrieval, LLM proxy, audit logging

요청

POST /v1/query

Authorization: Bearer <JWT or API_KEY>

Content-Type: application/json

 

{

  "tenant_id": "tenant_123",

  "user_id": "user_abc",

  "query": "고객 홍길동의 주문 상태 알려줘",

  "context_sources": ["zendesk","orders_db"],

  "options": {"redaction": "MASK", "response_retention": "NO"}

}

응답 (200)

{

  "request_id": "uuid",

  "status": "ok",

  "response": "홍길동님은 현재 배송중이며, 2025-10-12에 출고되었습니다.",

  "redaction": {"applied": true, "tokens": ["<NAME_1>"]}

}

에러

  • 400: invalid-request
  • 401: unauthorized
  • 403: policy-denied (explain policy_id)
  • 429: rate-limited
  • 500: internal_error (with request_id for debugging)

5.2 /v1/policies (Admin)

  • GET/POST/PUT/DELETE to manage redaction/allowlist rules

 

6. 인프라·배포·운영

6.1 기본 스택 제안

  • Cloud: AWS (GCP도 가능)
  • Compute: EKS (Kubernetes) for core services; Fargate for workers (or ECS)
  • Storage: RDS(Postgres), S3 (artifact + audit), Redis (caching/rate-limits)
  • Vector DB: Pinecone or Weaviate (managed) for MVP; Chroma/Milvus self-host if cost-sensitive
  • LLM Integration: OpenAI + Anthropic initially; support for private models via mTLS
  • Secrets: HashiCorp Vault or AWS KMS + Secrets Manager
  • Observability: OpenTelemetry -> Prometheus + Grafana; ELK/Opensearch for logs
  • CI/CD: GitHub Actions -> Build images -> Deploy via ArgoCD/Flux or GitOps pipelines

6.2 Scaling patterns

  • Make stateless services horizontally scalable (API, proxy, assembler)
  • Connector workers scale based on queue backlog (use SQS/Kafka)
  • Vector DB must be tuned for read QPS (index sharding / replicas)
  • Cache top-k retrieval results for identical queries (TTL short) to save LLM calls

6.3 Reliability & DR

  • Multi-AZ deployments
  • Backups: RDS snapshots, S3 lifecycle
  • Disaster recovery plan and RTO/RPO goals (defined per plan)

 

7. ML / PII 탐지 구현 옵션

7.1 오픈소스 옵션 (빠른 MVP)

  • spaCy Transformer models (NER) fine-tuned for Korean/English
  • Hugging Face models (e.g., xlm-roberta, mDeBERTa) for multilingual NER
  • Deterministic patterns: regex for SSN, emails, phone numbers, credit cards
  • Microsoft Presidio: PII detection framework (templates + analyzers)
  • 장점: 빠름, 비용 낮음. 단점: 초기 정확도 튜닝 필요.

7.2 Managed / Advanced 옵션

  • Use managed NER services or Mistral/HF hosted endpoints for better latency
  • Combine pattern detection + ML ensemble for recall/precision tradeoff
  • Consider differential privacy and token-filtering libraries for stronger guarantees

7.3 성능 고려

  • NER latency: aim <100ms for small text; batch when possible
  • False negatives are critical risk: adopt conservative policy (if unsure, mask)

 

8. MVP 우선순위 & 6개월 기술 로드맵 (구체적 작업)

목표: 3개월 내 동작하는 MVP (PoC-ready), 6개월 내 2개 유료 고객 확보

Month 0-1 (설계 + 초기 개발)

  • 설계 완료: API, RBAC, 정책 포맷, audit schema
  • 기본 infra: EKS cluster, Postgres, Redis, S3 setup
  • Implement: Authentication & API Gateway + basic developer docs (“hello world”)

산출물

  • 작동하는 /v1/query 경로 (단순 proxy)
  • Policy JSON 스펙 문서

Month 2

  • PII Detector v0: spaCy + regex integrated
  • Context Retriever: simple connector for Zendesk + Postgres
  • Audit Logger: write logs to encrypted S3 and index minimal metadata in Postgres
  • Admin UI (very simple): policy upload + audit search

산출물

  • 1개 connector로 RAG flow 통합
  • 기본 Admin UI 데모

Month 3 (MVP release)

  • LLM proxy integration (OpenAI)
  • Redaction end-to-end: query -> mask -> LLM -> post-check -> response
  • Billing: Stripe integration (test mode)
  • PoC onboarding pack & docs

산출물

  • PoC-ready system for devs/agency
  • 1~3 PoC targets 초청 가능

Month 4

  • Harden security: KMS, Vault, WAF tuning
  • Add RBAC + SSO integration
  • Improve NER models (fine-tune on sample data)

Month 5

  • Add Vector DB integration & chunking pipeline for richer RAG
  • Add enterprise features: retention policy UI, export logs
  • Performance tuning & SLA docs

Month 6

  • Onboard 2 paid PoC customers (Starter/Team)
  • Provide enterprise PoC docs for one larger customer (finance/health)
  • Prepare SOC2 Lite checklist / ISO prep start