Skip to content

compute, timely-util: tech debt cleanup#36476

Draft
antiguru wants to merge 2 commits into
MaterializeInc:mainfrom
antiguru:tech-debt-cleanup
Draft

compute, timely-util: tech debt cleanup#36476
antiguru wants to merge 2 commits into
MaterializeInc:mainfrom
antiguru:tech-debt-cleanup

Conversation

@antiguru
Copy link
Copy Markdown
Member

@antiguru antiguru commented May 8, 2026

Motivation

Bundle of mechanical cleanups identified during a tech-debt audit of src/compute and src/timely-util (CLU-81). One PR to avoid spam; each item is independent and can be reviewed in isolation by file.

Description

  • timely-util/columnation: add SAFETY comments on every unsafe block in ColumnationStack (copy, clear, two copy_destructured). Project rule: every unsafe needs a SAFETY justification. Invariant: Region::copy returns aliasing items; local: Vec<T> is a tombstone whose Drop glue is suppressed via set_len(0) before drop.

  • timely-util/reclock: document the pusher contract under a # Pusher contract heading. The function returns a Box<dyn Push<...>> that callers must drive — if it is dropped without ever being used, the operator silently deadlocks at the consumer.

  • timely-util/scope_label: re-implement the timely-scope profiling label that was gutted in the timely v4 upgrade (was a no-op with_label(self) -> Self). New design wraps the body of a dataflow in a same-timestamp subgraph installed as a LabelledOperator; each schedule() call runs inside a custom_labels scope tagged timely-scope=<name>. The pre-v4 in-place Scope wrapper is no longer expressible (timely v4 made Scope a concrete struct), so the API changes from let scope = scope.with_label(); to scope.with_label(|scope| { ... }). All seven call sites in compute and storage are migrated. There is one extra subgraph layer per dataflow now, with associated progress-tracking overhead.

  • compute/arrangement: add debug_assert! for the PaddedTrace invariant (padded_since strictly less than the inner trace's logical compaction frontier) at both mutation sites in into_padded and set_logical_compaction. The invariant was previously comment-only and unenforced.

The large line counts in the storage/compute render.rs files are mostly cargo fmt re-indentation from wrapping the dataflow body in a closure; the actual semantic change at each site is one extra level of nesting.

Verification

  • cargo check --workspace passes.
  • cargo clippy --workspace --all-targets -- -D warnings passes.
  • bin/lint rustfmt step passes (other unrelated lint failures: missing buf, trufflehog, protobuf tooling in this environment).

antiguru and others added 2 commits May 8, 2026 16:24
Bundle of mechanical cleanups identified during a tech-debt audit of
src/compute and src/timely-util (Linear: CLU-81).

* timely-util/columnation: add SAFETY comments to all unsafe blocks in
  ColumnationStack (copy, clear, copy_destructured), per project rule.
  The invariant: Region::copy returns aliasing items; local Vec<T> is a
  tombstone whose Drop glue is suppressed via set_len(0) before drop.

* timely-util/reclock: document the pusher contract. The function
  returns a Box<dyn Push<...>> that callers must drive, otherwise the
  operator silently deadlocks.

* timely-util/scope_label: re-implement the timely-scope profiling label
  that was gutted in the timely v4 upgrade. Wraps the body of a
  dataflow in a same-timestamp subgraph installed as a LabelledOperator;
  each schedule() call runs inside a custom_labels scope tagged
  timely-scope=<name>. The v4 in-place Scope wrapper is no longer
  expressible (Scope is now a concrete struct), so the API changes from
  let scope = scope.with_label(); to scope.with_label(|scope| { ... }).
  All seven call sites in compute and storage are updated.

* compute/arrangement: add debug_assert! for the PaddedTrace invariant
  (padded_since < trace logical compaction) at both mutation sites in
  into_padded and set_logical_compaction.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`LabelledOperator` is a private struct; the module-level doc comment
linked to it as if public.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant