update worker
This commit is contained in:
meninjar
+250
@@ -0,0 +1,250 @@
|
||||
# 🗺️ Development Plan (Roadmap)
|
||||
|
||||
**Project:** GoPrint Service General (`worker-satusehat`)
|
||||
**Versi:** 1.0
|
||||
**Tanggal:** 2026-05-13
|
||||
|
||||
Peta jalan terurut berdasarkan **dampak vs effort**. Setiap fase punya keluaran yang bisa dirilis berdiri sendiri.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Stabilisasi Production Saat Ini *(1–2 minggu)*
|
||||
|
||||
**Tujuan:** Service yang sekarang sudah jalan (REST + RBAC + KFA Puller) bebas bug kritikal.
|
||||
|
||||
### Langkah Stabilisasi
|
||||
|
||||
1. **Fix `satusehatClient = nil`** ([main.go:235](cmd/api/main.go#L235))
|
||||
- Inisialisasi: `satusehatClient := satusehat.NewClient(cfg)` lalu pass ke `worker.NewManager`.
|
||||
- *Acceptance:* `go run` tidak panic ketika `cfg.SatuSehat.Enabled = true`.
|
||||
|
||||
2. **Pindahkan kredensial worker login ke env**
|
||||
- `WORKER_AUTH_EMAIL`, `WORKER_AUTH_PASSWORD` di `.env` + validation di `config.Validate()`.
|
||||
- Hapus default hardcoded di [worker/auth.go:25-28](internal/worker/auth.go#L25-L28).
|
||||
|
||||
3. **Audit & bersihkan kode mati**
|
||||
- Hapus `patient/migrator.go` jika tidak akan dipakai, atau pindahkan ke folder `archive/` + buang default password.
|
||||
- Hilangkan `truncate()` duplikat: pindah ke `pkg/logger` atau `pkg/utils`.
|
||||
|
||||
4. **Endpoint `/metrics` Prometheus aktif**
|
||||
- Tambahkan handler `promhttp.Handler()` di `routes.go`.
|
||||
- Definisikan counter dasar: `worker_documents_processed_total`, `worker_rate_limit_hits_total`, `satusehat_token_refresh_total`.
|
||||
|
||||
5. **gRPC handler `PermissionHandler` di-uncomment**
|
||||
- Tes via `grpcurl` ke `:9090`.
|
||||
|
||||
**Output:** Tag `v0.9.0-stable`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Aktivasi Worker Klinis *(2–4 minggu)*
|
||||
|
||||
**Tujuan:** Sinkronisasi data obat & radiologi ke SatuSehat jalan otomatis 24/7.
|
||||
|
||||
### Langkah Aktivasi
|
||||
|
||||
1. **Aktifkan worker secara bertahap di [worker.go](internal/worker/worker.go)** dengan urutan dependensi:
|
||||
1. `Encounter` (prasyarat referensi)
|
||||
2. `Medication` (master kode obat lokal → KFA)
|
||||
3. `MedicationRequest`
|
||||
4. `MedicationDispense`
|
||||
5. `ServiceRequest (Radiology)`
|
||||
6. `ImagingStudy`
|
||||
|
||||
2. **Validasi end-to-end per worker:**
|
||||
- Cek `sync_log` (status SUCCESS/FAILED/RATE_LIMITED).
|
||||
- Pastikan FHIR ID kembali & tersimpan di mapping harian.
|
||||
- Monitor log error rate ≤ 1% per jam selama 48 jam soak test.
|
||||
|
||||
3. **Rate-limit config dinamis** — pindahkan konstanta `rateLimitSleep` per worker ke `config.yaml`:
|
||||
|
||||
```yaml
|
||||
satu_sehat:
|
||||
worker:
|
||||
medication: { rate_limit_sleep: 60s, poll_interval: 2s }
|
||||
imagingstudy: { rate_limit_sleep: 60s, poll_interval: 2s }
|
||||
```
|
||||
|
||||
4. **Sync Log Retention Policy**
|
||||
- Tabel `sync_log` bisa membengkak — tambahkan TTL/archive job (retain 90 hari, archive ke S3/MinIO).
|
||||
|
||||
**Output:** Tag `v1.0.0` — release bridging klinis pertama.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Stateless Worker (Multi-Replica Ready) *(2–3 minggu)*
|
||||
|
||||
**Tujuan:** Mendukung deployment Kubernetes / multi-pod tanpa risiko race tracker.
|
||||
|
||||
### Langkah Refactor State
|
||||
|
||||
1. **Tabel `sync_tracker`**
|
||||
|
||||
```sql
|
||||
CREATE TABLE sync_tracker (
|
||||
module_name VARCHAR(64) PRIMARY KEY,
|
||||
last_id BIGINT,
|
||||
last_timestamp TIMESTAMP,
|
||||
updated_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
```
|
||||
|
||||
- Implementasi via row-lock `SELECT ... FOR UPDATE SKIP LOCKED` (Postgres) sehingga hanya satu replica per modul.
|
||||
|
||||
2. **Refactor helper `readLastID()` / `writeTracker()`**
|
||||
- Bikin abstraksi `TrackerStore` di `pkg/tracker/` (Interface: `Get`, `Set` per module name).
|
||||
- Implementasi backend `FileTracker` (lama) & `DBTracker` (baru). Pilih via env `TRACKER_BACKEND=db|file`.
|
||||
|
||||
3. **Migrasi data tracker yang ada**
|
||||
- Script one-shot: baca semua `*.txt` di `internal/worker/satusehat/*/last_*` & `internal/master/kfa/kfa_tracker.txt` → seed ke `sync_tracker`.
|
||||
|
||||
4. **Mapping FHIR ID → Redis hash atau tabel**
|
||||
- `mapperdata/YYYY-MM/YYYY-MM-DD.txt` tidak cocok multi-pod. Pindah ke tabel `fhir_id_map(local_id, fhir_id, resource_type, created_at)`.
|
||||
|
||||
5. **Leader election sederhana untuk worker eksklusif**
|
||||
- Gunakan Redis `SET key NX EX 60` sebagai *lease* untuk worker singleton (KFA Puller, dst).
|
||||
|
||||
**Output:** Tag `v1.1.0` — deployable di Kubernetes dengan ≥ 2 replica.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Event-Driven Architecture *(4–6 minggu)*
|
||||
|
||||
**Tujuan:** Mengurangi polling DB → reactive berdasarkan trigger SIMRS.
|
||||
|
||||
### Langkah Event-Driven
|
||||
|
||||
1. **PostgreSQL LISTEN/NOTIFY**
|
||||
- Trigger pada tabel sumber (mis. `t_pesan_obat` AFTER INSERT) → `NOTIFY pesan_obat_new, NEW.id::text`.
|
||||
- Worker `Medication` subscribe via `pq.NewListener` daripada polling per 2 detik.
|
||||
- Fallback: tetap ada polling tiap N detik sebagai *catch-up* untuk event yang tertinggal.
|
||||
|
||||
2. **Kafka Producer (opsional, untuk multi-microservice)**
|
||||
- Uncomment [main.go:111-113](cmd/api/main.go#L111-L113).
|
||||
- Publish event `fhir.resource.synced` setelah pengiriman sukses untuk konsumsi service lain (audit, BI, dashboard).
|
||||
- Library: `segmentio/kafka-go` (lightweight).
|
||||
|
||||
3. **Dead Letter Queue**
|
||||
- Pesan yang gagal kirim 3× → masuk topik `fhir.dlq` untuk inspeksi manual.
|
||||
|
||||
4. **Webhook callback dari `service-satusehat`**
|
||||
- Jika service downstream support webhook untuk async FHIR commit, langganan untuk update `sync_log.status` lebih cepat.
|
||||
|
||||
**Output:** Tag `v1.2.0`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Worker Coverage Lengkap *(rolling, 1 resource / sprint)*
|
||||
|
||||
**Tujuan:** Mengisi 13 worker stub menjadi production-ready.
|
||||
|
||||
### Urutan Prioritas Worker (berdasarkan kebutuhan pelaporan SatuSehat)
|
||||
|
||||
1. `Condition` (diagnosa)
|
||||
2. `Observation` (vital sign, lab)
|
||||
3. `Procedure` (tindakan medis)
|
||||
4. `DiagnosticReport` (laporan lab/radiologi)
|
||||
5. `AllergyIntolerance`
|
||||
6. `Immunization`
|
||||
7. `Specimen` (lab)
|
||||
8. `ClinicalImpression`
|
||||
9. `EpisodeOfCare`
|
||||
10. `CarePlan`
|
||||
11. `Composition`
|
||||
12. `MedicationStatement`
|
||||
13. `QuestionnaireResponse`
|
||||
|
||||
### Template per Worker
|
||||
|
||||
- Salin pola dari worker yang sudah lengkap (saran: `Medication` untuk transactional, `ServiceRequest` untuk time-based).
|
||||
- Komponen wajib: tracker, rate-limit handler, KFA/master fetch jika perlu, token refresh, sync log, mapping persistence.
|
||||
|
||||
**Output:** Tiap worker = sprint sendiri, release patch (`v1.x.y`).
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Observability & Operasional *(2–3 minggu, paralel)*
|
||||
|
||||
### Langkah Observability
|
||||
|
||||
1. **Grafana Dashboard**
|
||||
- Source: Prometheus `worker_*_total`, `worker_processing_duration_seconds`, `satusehat_response_status_total{code=...}`.
|
||||
- Panel: throughput per worker, error rate, p95 latency, queue depth (lag dari tracker terhadap MAX(id) source).
|
||||
|
||||
2. **Alerting**
|
||||
- Worker tidak progress > 10 menit.
|
||||
- Rate-limit hits > 10/min selama 5 menit.
|
||||
- Token refresh failed > 3 berturut-turut.
|
||||
|
||||
3. **Distributed Tracing (opsional)**
|
||||
- Integrasi OpenTelemetry, trace span dari REST handler → service → repository → API gov't.
|
||||
|
||||
4. **Health & Readiness Probe**
|
||||
- `/health/live`: cek service hidup.
|
||||
- `/health/ready`: cek DB ping, Redis ping, MinIO ping, last token refresh sukses.
|
||||
|
||||
**Output:** Stack monitoring siap pakai (Prometheus + Grafana + Alertmanager).
|
||||
|
||||
---
|
||||
|
||||
## Phase 6 — UI Dashboard *(repository terpisah, ditangani tim frontend)*
|
||||
|
||||
**Tujuan:** Operasional team & admin RS bisa monitor + intervensi tanpa SSH.
|
||||
|
||||
### Fitur Inti Dashboard
|
||||
|
||||
1. **Sync Monitor**
|
||||
- Tabel `sync_log` realtime, filter by status, worker, tanggal.
|
||||
- Re-queue dokumen gagal (button "Retry").
|
||||
|
||||
2. **Tracker Cockpit**
|
||||
- Posisi setiap worker (last_id, last_timestamp) vs. MAX di tabel sumber → lag visual.
|
||||
- Pause / Resume worker via toggle (butuh control-plane API baru: `PUT /api/v1/worker/{name}/state`).
|
||||
|
||||
3. **RBAC Management**
|
||||
- Hirarki halaman dengan checkbox CRUD level per role (konsumsi `GetRolePermissionTree`).
|
||||
- Cache invalidation otomatis lewat REST API existing.
|
||||
|
||||
4. **KFA Browser**
|
||||
- Search produk farmasi yang sudah masuk ke DB lokal.
|
||||
- Trigger ulang puller dari halaman tertentu.
|
||||
|
||||
5. **BPJS Console (opsional)**
|
||||
- Dropdown service VClaim/Antrol → form input → call SDK → tampilkan hasil dekripsi.
|
||||
|
||||
---
|
||||
|
||||
## Phase 7 — Hardening & Compliance *(jangka panjang)*
|
||||
|
||||
- **Audit Trail** lengkap untuk RBAC: siapa, kapan, mengubah apa.
|
||||
- **Field-Level Encryption** untuk data PII (NIK, alamat) yang tersimpan di sync log.
|
||||
- **SLA Report** untuk Kemenkes: persentase data terkirim dalam 1×24 jam.
|
||||
- **Disaster Recovery** runbook: replay dari sync log, re-sync dari ID tertentu.
|
||||
|
||||
---
|
||||
|
||||
## Ringkasan Timeline Indikatif
|
||||
|
||||
| Phase | Durasi | Target Tag | Outcome |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| 0 — Stabilisasi | 1–2 minggu | `v0.9.0` | Bug kritikal fixed, metrics live |
|
||||
| 1 — Aktivasi Klinis | 2–4 minggu | `v1.0.0` | Worker obat & radiologi production |
|
||||
| 2 — Stateless Tracker | 2–3 minggu | `v1.1.0` | Multi-replica safe |
|
||||
| 3 — Event-Driven | 4–6 minggu | `v1.2.0` | LISTEN/NOTIFY + Kafka |
|
||||
| 4 — Worker Lengkap | rolling | `v1.x.y` | 13 resource FHIR aktif |
|
||||
| 5 — Observability | 2–3 minggu | (paralel) | Grafana + alerting |
|
||||
| 6 — UI Dashboard | terpisah | (FE repo) | Operator self-service |
|
||||
| 7 — Hardening | berkelanjutan | — | Compliance & DR |
|
||||
|
||||
---
|
||||
|
||||
## Quick Wins Bulan Ini
|
||||
|
||||
1. **Fix BUG-001** (satusehatClient nil) — 1 jam kerja, dampak besar.
|
||||
2. **Wire `/metrics`** — 30 menit, langsung kelihatan progress di Prometheus.
|
||||
3. **Uncomment Worker `Medication`** + soak test 24 jam staging.
|
||||
4. **Pindahkan password worker ke env** — 1 jam, tutup security finding.
|
||||
|
||||
---
|
||||
|
||||
*Disusun ulang: Tim Engineering GoPrint, 2026-05-13.*
|
||||
Reference in New Issue
Block a user