Skip to content

emulator: add MZ_METADATA_STORE switch for postgres/foundationdb#35087

Open
antiguru wants to merge 5 commits into
MaterializeInc:mainfrom
antiguru:fdb_emulator_2
Open

emulator: add MZ_METADATA_STORE switch for postgres/foundationdb#35087
antiguru wants to merge 5 commits into
MaterializeInc:mainfrom
antiguru:fdb_emulator_2

Conversation

@antiguru
Copy link
Copy Markdown
Member

Add FoundationDB as an alternative metadata store for the emulator image. Both PostgreSQL and FoundationDB are installed in the base image. The entrypoint reads MZ_METADATA_STORE (default: postgres) to select which backend to start. Set MZ_METADATA_STORE=foundationdb to use FDB.

🤖 Generated with Claude Code

@github-actions
Copy link
Copy Markdown
Contributor

Pre-merge checklist

  • The PR title is descriptive and will make sense in the git log.
  • This PR has adequate test coverage / QA involvement has been duly considered. (trigger-ci for additional test/nightly runs)
  • If this PR includes major user-facing behavior changes, I have pinged the relevant PM to schedule a changelog post.
  • This PR has an associated up-to-date design doc, is a design doc (template), or is sufficiently small to not require a design.
  • If this PR evolves an existing $T ⇔ Proto$T mapping (possibly in a backwards-incompatible way), then it is tagged with a T-proto label.
  • If this PR will require changes to cloud orchestration or tests, there is a companion cloud PR to account for those changes that is tagged with the release-blocker label (example).

@antiguru antiguru marked this pull request as ready for review February 19, 2026 17:00
@antiguru antiguru requested a review from def- February 19, 2026 17:00
Copy link
Copy Markdown
Contributor

@def- def- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How much larger does it make our Docker images to have both Postgres and FoundationDB?
Edit: Just checked: 218 MB -> 322 MB, that's pretty significant for an experiment. I'd propose to add a materialized-fdb image instead for experimentation.

antiguru and others added 5 commits February 24, 2026 17:33
Install FoundationDB client and server alongside PostgreSQL in the
emulator base image. The entrypoint script now reads MZ_METADATA_STORE
(defaulting to "postgres") to select which backend to start and which
URLs to configure. Set MZ_METADATA_STORE=foundationdb to use FDB.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use `foundationdb:?prefix=X` instead of embedding the cluster file path
and legacy `options` parameter. Export FDB_CLUSTER_FILE so the FDB client
discovers the cluster via the environment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Avoid penalizing postgres-only users with the ~100 MB FoundationDB
packages. The postgres image (materialized) no longer includes FDB,
and a new materialized-fdb image provides the FDB-backed variant.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace directory symlink with individual file symlinks, since
mzbuild cannot traverse a symlinked directory.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
materialized-base and materialized-fdb-base duplicated ~40 lines of
shared setup (user creation, apt packages, console, nginx config) and
the console version had already drifted (26.14.0-dev.0 vs 26.13.0-dev.0).
Extract the common parts into materialized-common-base so both
backend-specific bases extend it.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants