-
Notifications
You must be signed in to change notification settings - Fork 48
Closed
Description
Parent Issue
Part of #124 (support partitioned table)
Background
In Paimon, ManifestEntry._PARTITION stores partition values as raw BinaryRow binary bytes (Vec<u8>). Currently, BinaryRow (crates/paimon/src/spec/data_file.rs:30-51) is a stub that only stores arity — it cannot hold or parse actual partition data.
To support partitioned table reading, we need BinaryRow to wrap the raw bytes and provide typed accessor methods.
What needs to be done
-
Enhance
BinaryRowto hold actual binary data- Add a
data: Vec<u8>(orBytes) field to back the row - Parse the binary layout: header (8 bytes) + null bit set (aligned) + fixed-length part (8 bytes per field) + variable-length part
- Add a
-
Implement typed getters
is_null_at(pos)— check the null bitget_int(pos)→i32get_long(pos)→i64get_short(pos)→i16get_byte(pos)→i8get_boolean(pos)→boolget_float(pos)→f32get_double(pos)→f64get_string(pos)→&str/String(variable-length, read offset+length from fixed part, then read from variable part)get_binary(pos)→&[u8]
-
Add a constructor from raw bytes
BinaryRow::from_bytes(arity: i32, data: Vec<u8>) -> BinaryRow
-
Unit tests
- Round-trip: construct binary data manually, verify typed getters
- Test null handling
- Test variable-length fields (String, Binary)
Reference
- Java:
BinaryRow - Java:
BinaryRowSerializer
Affected files
crates/paimon/src/spec/data_file.rs—BinaryRowstruct
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels