Document table-level retention and refresh the README for the merged DAOs - #117
Merged
Merged
Conversation
…DAOs Add a Retention / TTL section describing how physical retention is set on the IoTDB side: it is a table property in milliseconds, declared as WITH (TTL=DEFAULT) in the shipped schema and changed with ALTER TABLE ... SET PROPERTIES TTL=<ms>, readable back from information_schema.tables. It also records the Phase-1 limitation that ThingsBoard's per-save ttl argument cannot be a physical-retention directive, because IoTDB retention is table-wide, so the module uses it only for storage data-point accounting. IoTDBTableTtlIT pins both the schema-default and the runtime-ALTER paths against a real IoTDB 2.0.8 container. It asserts the TTL property mechanism only, not physical row eviction, which is asynchronous and compaction-driven and therefore not deterministic in a test. While writing that section the surrounding README turned out to describe the module as it was several changes ago, so this also brings it back in line with the code: time-bucketed aggregation, latest telemetry and the attribute DAO are implemented rather than out of scope, and key discovery now reads DISTINCT keys from both telemetry and the telemetry_latest overlay instead of returning an empty list. The batch findLatestByEntityIds deferral is unchanged and stays documented. Signed-off-by: Zihan Dai <99155080+PDGGK@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Retention / TTL
Physical retention was never documented for this module, even though the shipped schema already declares
telemetrywithWITH (TTL=DEFAULT). This adds a Retention / TTL section to the README covering what an operator actually has to do:TTL=604800000); never-expire is the quoted stringTTL='INF'(the formentity_attributesandtelemetry_latestalready ship with);TTL=DEFAULTinherits the database default. An unquotedTTL=INFis parsed as an identifier and rejected, and any other quoted value — a quoted number ('604800000') or a duration ('7d') — fails withttl value must be 'INF' or a long literal. All four forms were checked against a 2.0.8 container.CREATE TABLE telemetry (...) WITH (TTL=604800000)) or at runtime (ALTER TABLE telemetry SET PROPERTIES TTL=604800000), and read back fromSHOW TABLESor frominformation_schema.tables, where the column has to be quoted as"ttl(ms)".telemetry_lateststaysTTL='INF'deliberately: it holds one row per identity and is the authority for a latest value, so expiring it would drop the latest reading while history remained.It also records a limitation that was implicit in the code and easy to misread: ThingsBoard's
TimeseriesDao.save(..., long ttl)carries a per-data-point TTL, but IoTDB retention can only be expressed per table, so the two cannot be reconciled faithfully. The module uses that argument (withiotdb.defaultTtlMs) only for ThingsBoard's storage data-point accounting, never as an instruction to expire rows.IoTDBTableTtlITpins the operator paths against a realapache/iotdb:2.0.8-standalonecontainer: the schema bootstrap resolvingTTL=DEFAULT, the shippedTTL='INF'onentity_attributesreading back asINF,ALTER TABLE ... SET PROPERTIES TTL=<ms>and back toDEFAULT, and aCREATE TABLE ... WITH (TTL=<ms>)read back exactly. It asserts the TTL property mechanism only — not physical row eviction, which is asynchronous and compaction-driven and so is not deterministic inside a test.README accuracy
Writing that section surfaced that the rest of the README still describes the module as it was several changes ago, so this brings it back in line with the code:
iotdb.ts.experimental-raw-onlyrow now says what the flag really does: the name predates aggregation support and is kept for compatibility, but it is the opt-in for the backend as a whole rather than a raw-only switch.findAllKeysByEntityIdsreadsDISTINCT keyfrom bothtelemetryand thetelemetry_latestoverlay; only the batchfindLatestByEntityIdsdeferral remains, and that bullet is kept.Tests
Full module build is green on JDK 17: 190 unit tests and 54 container integration tests, including the two new TTL tests.