Skip to content
490 changes: 489 additions & 1 deletion adapter/encryption_admin.go

Large diffs are not rendered by default.

873 changes: 873 additions & 0 deletions adapter/encryption_admin_test.go

Large diffs are not rendered by default.

24 changes: 21 additions & 3 deletions docs/design/2026_05_18_partial_6d_enable_storage_envelope.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

| Field | Value |
|---|---|
| Status | partial — 6D-1 (doc), 6D-2 (startup guards), 6D-3 (capability fan-out helper), 6D-4 (cutover wire + apply dispatch), 6D-5 (storage-layer toggle) shipped; 6D-6 remains |
| Status | partial — 6D-1 (doc), 6D-2 (startup guards), 6D-3 (capability fan-out helper), 6D-4 (cutover wire + apply dispatch), 6D-5 (storage-layer toggle), 6D-6a (EnableStorageEnvelope server method) shipped; 6D-6b (CLI), 6D-6c (main.go wiring + integration test) remain |
| Date | 2026-05-18 |
| Parent design | [`2026_04_29_partial_data_at_rest_encryption.md`](2026_04_29_partial_data_at_rest_encryption.md) |
| Blockers (now satisfied) | 6B (KEK plumbing), 6C-1 / 6C-2 (startup guards), 6C-2d (`ErrSidecarBehindRaftLog` wiring) |
Expand Down Expand Up @@ -45,10 +45,28 @@
pre-6D-6 production wiring keep working unchanged. Operator-
inert until 6D-6 wires both the cipher and the gate in main.go
and exposes the cutover RPC.
- **6D-6a** (EnableStorageEnvelope server method) —
`proto/encryption_admin.proto` adds the `EnableStorageEnvelope`
RPC + `EnableStorageEnvelopeRequest` / `Response` +
`CapabilityVerdict` messages. `adapter/encryption_admin.go`
ships the server method that composes the §3.2 sequence: leader
gate → input validation → sidecar read → bootstrap gate →
idempotent-retry short-circuit (§6.4) → capability fan-out
(6D-3) → propose RotateSubEnableStorageEnvelope through Raft
(6D-4 wire) → post-apply re-read discriminating fresh-success
vs. stale-DEKID race. The 6D-6b CLI and 6D-6c main.go wiring +
integration test slice on top of this server method.

## Open milestones
- **6D-6** — `EnableStorageEnvelope` admin RPC + CLI command +
integration test composing 6D-3 + 6D-4 + 6D-5.

- **6D-6b** — `elastickv-admin enable-storage-envelope` CLI
subcommand that drives the server method end-to-end.
- **6D-6c** — main.go production wiring: cipher + WithEncryption
+ WithStorageEnvelopeGate threaded from the sidecar, plus the
CapabilityFanout closure bound to the live Raft membership
view. End-to-end integration test exercises a single-node
cluster Bootstrap → EnableStorageEnvelope → Put → read-back-
via-envelope.

## 0. Why this doc exists

Expand Down
3 changes: 3 additions & 0 deletions proto/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,6 @@ gen: check-tools
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
admin_forward.proto
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
encryption_admin.proto
313 changes: 284 additions & 29 deletions proto/encryption_admin.pb.go

Large diffs are not rendered by default.

66 changes: 66 additions & 0 deletions proto/encryption_admin.proto
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ service EncryptionAdmin {
rpc RotateDEK (RotateDEKRequest) returns (RotateDEKResponse) {}
rpc RegisterEncryptionWriter (RegisterEncryptionWriterRequest) returns (RegisterEncryptionWriterResponse) {}
rpc ResyncSidecar (ResyncSidecarRequest) returns (ResyncSidecarResponse) {}
rpc EnableStorageEnvelope (EnableStorageEnvelopeRequest) returns (EnableStorageEnvelopeResponse) {}
}

message Empty {}
Expand Down Expand Up @@ -162,3 +163,68 @@ message ResyncSidecarResponse {
// recovering before any DEK exists has nothing to re-derive.
map<uint32, uint32> writer_registry_for_caller = 5;
}

// EnableStorageEnvelopeRequest proposes the §7.1 Phase 1 cutover
// that flips the cluster from cleartext storage writes to §4.1
// envelope writes. Defined in the 6D design doc §3.1; the server
// composes a RotationPayload with SubTag =
// RotateSubEnableStorageEnvelope (0x04) and routes it through the
// default Raft group's leader as a §11.3 0x05 OpRotation entry.
//
// proposer_node_id MUST be non-zero (the §6.1 "not-capable"
// sentinel is rejected at the server boundary, matching the
// existing RotateDEK / BootstrapEncryption posture).
//
// proposer_local_epoch carries the §4.1 16-bit nonce field as
// uint32 (proto3 has no uint16); values above 0xFFFF are
// rejected at the server boundary before any Raft proposal is
// composed. ApplyRotation re-validates at apply time
// (defense-in-depth).
message EnableStorageEnvelopeRequest {
uint64 proposer_node_id = 1;
uint32 proposer_local_epoch = 2; // MUST be <= 0xFFFF on the wire.
}

// EnableStorageEnvelopeResponse reports the outcome of a cutover
// proposal. The §6.4 idempotency contract is encoded as an OK
// status with `was_already_active = true` (NOT AlreadyExists,
// since unary gRPC drops the response body on non-OK status; the
// applied_index must be reachable on the success path).
//
// On a fresh cutover (was_already_active == false), applied_index
// is the Raft index of the entry the leader just proposed and
// waited to apply. On a retried call, applied_index is the
// recorded sidecar.StorageEnvelopeCutoverIndex from the ORIGINAL
// cutover (§6.4) — stable across arbitrary subsequent
// encryption-relevant Raft activity.
//
// capability_summary records which (full_node_id) members were
// probed during the pre-flight gate and what they reported.
// Empty on idempotent retries (was_already_active=true); the
// membership view of the original cutover is not retained.
//
// cutover_index_unknown is the §6.4 defensive fallback: it only
// fires if a sidecar reports StorageEnvelopeActive=true with
// StorageEnvelopeCutoverIndex=0 (operationally impossible under
// normal apply, but hedged against future schema rollback). On
// healthy clusters this stays false. The field is only
// meaningful when was_already_active=true.
message EnableStorageEnvelopeResponse {
uint64 applied_index = 1;
repeated CapabilityVerdict capability_summary = 2;
bool cutover_index_unknown = 3;
bool was_already_active = 4;
}

// CapabilityVerdict is one row of the §4 fan-out summary the
// cutover RPC returns. full_node_id is the route member the leader
// probed; the remaining fields mirror the corresponding member's
// CapabilityReport. A cluster where any verdict has
// encryption_capable=false MUST NOT reach the propose step; the
// summary in the response is the post-hoc record for operators.
message CapabilityVerdict {
uint64 full_node_id = 1;
bool encryption_capable = 2;
string build_sha = 3;
bool sidecar_present = 4;
}
38 changes: 38 additions & 0 deletions proto/encryption_admin_grpc.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading