Skip to content

fix: populate missing tarball checksums (#84) + init --provider=postgresql (#85) #86

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
renecannao wants to merge 5 commits into
base: master
Choose a base branch
from
Open

fix: populate missing tarball checksums (#84) + init --provider=postgresql (#85) #86

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6c81614
fix: populate missing checksums for 89 tarballs (#84)
renecannao Apr 10, 2026
76c8d9a
feat: dbdeployer init --provider=postgresql (#85)
renecannao Apr 10, 2026
20a9f79
feat: macOS PostgreSQL support via Postgres.app (#85)
renecannao Apr 10, 2026
a1277a1
fix: flatten share/postgresql/ into share/ after Postgres.app extract
renecannao Apr 10, 2026
2bb81a1
fix: use GITHUB_TOKEN for Postgres.app API lookup to avoid rate limit
renecannao Apr 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
125 changes: 125 additions & 0 deletions .github/workflows/integration_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,51 @@ jobs:
cp -a /usr/share/postgresql/${PG_VERSION}/. ~/opt/postgresql/${PG_FULL}/share/
ls ~/opt/postgresql/${PG_FULL}/bin/

- name: Test init --provider=postgresql
run: |
# Run init against isolated dirs, verify it creates them and
# prints the PostgreSQL setup instructions.
OUT=$(./dbdeployer init --provider=postgresql \
--sandbox-binary=/tmp/init-pg-bin \
--sandbox-home=/tmp/init-pg-home \
--skip-shell-completion 2>&1)
echo "$OUT"

# Verify directories were created
[ -d /tmp/init-pg-bin ] || { echo "FAIL: sandbox-binary not created"; exit 1; }
[ -d /tmp/init-pg-home ] || { echo "FAIL: sandbox-home not created"; exit 1; }

# Verify the Linux-specific PostgreSQL setup instructions were printed
echo "$OUT" | grep -q "PostgreSQL binaries are not auto-downloaded on Linux" || {
echo "FAIL: expected Linux PostgreSQL setup instructions in output"
exit 1
}
echo "$OUT" | grep -q "dbdeployer unpack --provider=postgresql" || {
echo "FAIL: expected unpack instructions in output"
exit 1
}

# Verify it didn't try to download a MySQL tarball
echo "$OUT" | grep -q "mysql-" && {
echo "FAIL: output references mysql tarball"
exit 1
}

# Sanity: invalid provider should fail
./dbdeployer init --provider=oracle --dry-run \
--sandbox-binary=/tmp/init-bad \
--sandbox-home=/tmp/init-bad-home 2>&1 | grep -q "unknown provider" || {
echo "FAIL: invalid provider did not error as expected"
exit 1
}

# Cleanup init side-effects so the rest of the job uses its own dirs
rm -rf /tmp/init-pg-bin /tmp/init-pg-home /tmp/init-bad /tmp/init-bad-home
# Restore sandbox defaults so subsequent steps find ~/opt/postgresql
./dbdeployer defaults update sandbox-binary "$HOME/opt/mysql"
./dbdeployer defaults update sandbox-home "$HOME/sandboxes"
echo "OK: dbdeployer init --provider=postgresql works"
Comment on lines +382 to +425
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Add the checksum-completeness gate to CI.

This workflow now covers the PostgreSQL init path, but it still doesn’t implement #84’s requirement to fail PRs when a new non-TiDB tarball is added without a checksum. downloads/tarball_list.json can still regress if it’s edited directly, because the runtime validation in downloads/remote_registry.go is not a PR-time guard.

Suggested CI step 
+      - name: Verify tarball checksums are populated
+        run: |
+          missing=$(jq -r '
+            .Tarballs[]
+            | select((.flavor != "tidb") and (.checksum == null or .checksum == ""))
+            | .name
+          ' downloads/tarball_list.json)
+          [ -z "$missing" ] || {
+            echo "Missing checksums for:"
+            echo "$missing"
+            exit 1
+          }
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/integration_tests.yml around lines 382 - 425, The CI
currently exercises PostgreSQL init but lacks the PR-time gate to reject changes
that add non-TiDB tarballs without checksums; add a new job step in
.github/workflows/integration_tests.yml (near the existing "Test init
--provider=postgresql" step) that runs the checksum-completeness validator
against downloads/tarball_list.json and fails the job if any non-TiDB entry
lacks a checksum; invoke the same validation logic used at runtime (the code in
downloads/remote_registry.go) — e.g., call the repository's checksum validation
command or a small Go script that loads downloads/tarball_list.json and enforces
checksum presence for new tarballs — so PRs are blocked when a missing checksum
is detected.


- name: Test deploy postgresql (single)
run: |
PG_FULL=$(ls ~/opt/postgresql/ | head -1)
Expand Down Expand Up @@ -483,6 +528,86 @@ jobs:
done 2>/dev/null || true
pkill -9 -u "$USER" postgres 2>/dev/null || true

# Test PostgreSQL support on macOS via Postgres.app (#85).
# `dbdeployer init --provider=postgresql` on macOS downloads a Postgres.app
# single-version .dmg, mounts it with hdiutil, and extracts bin/lib/share
# into ~/opt/postgresql/<major>/. This job verifies the full happy path
# end-to-end: init → directory layout → deploy single → SELECT.
postgresql-macos-test:
name: PostgreSQL on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [macos-14, macos-15]
env:
GO111MODULE: on
steps:
- uses: actions/checkout@v4

- uses: actions/setup-go@v5
with:
go-version: '1.22'

- name: Build dbdeployer
run: go build -o dbdeployer .

- name: init --provider=postgresql (downloads Postgres.app)
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Clean slate — no ~/opt/postgresql yet.
rm -rf "$HOME/opt/postgresql" "$HOME/sandboxes-pg-mac"
./dbdeployer init --provider=postgresql \
--sandbox-home="$HOME/sandboxes-pg-mac" \
--skip-shell-completion

# Verify binaries landed under ~/opt/postgresql/<X.Y>/bin
INSTALL_DIR=$(ls -d "$HOME/opt/postgresql"/*/ 2>/dev/null | head -1)
[ -n "$INSTALL_DIR" ] || { echo "FAIL: no version directory under ~/opt/postgresql"; exit 1; }
echo "Install dir: $INSTALL_DIR"
for bin in postgres initdb pg_ctl psql pg_basebackup; do
[ -x "${INSTALL_DIR}bin/$bin" ] || { echo "FAIL: missing $bin at ${INSTALL_DIR}bin/"; exit 1; }
done
"${INSTALL_DIR}bin/postgres" --version
"${INSTALL_DIR}bin/psql" --version

# Check the directory name is in X.Y format
PG_VERSION=$(basename "${INSTALL_DIR%/}")
echo "$PG_VERSION" | grep -Eq '^[0-9]+\.[0-9]+$' || {
echo "FAIL: expected X.Y version dir, got: $PG_VERSION"
exit 1
}

- name: Deploy PostgreSQL single sandbox
run: |
# Discover the X.Y version that init installed
PG_VERSION=$(basename "$(ls -d "$HOME/opt/postgresql"/*/ | head -1 | sed 's:/$::')")
echo "Deploying PostgreSQL $PG_VERSION"
./dbdeployer deploy postgresql "$PG_VERSION" \
--sandbox-home="$HOME/sandboxes-pg-mac"
# Discover the sandbox directory
SBDIR=$(ls -d "$HOME/sandboxes-pg-mac"/pg_sandbox_* | head -1)
echo "Sandbox: $SBDIR"
"$SBDIR/use" -c "SELECT version();"
"$SBDIR/status"

- name: Write + read smoke test
run: |
SBDIR=$(ls -d "$HOME/sandboxes-pg-mac"/pg_sandbox_* | head -1)
"$SBDIR/use" -c "CREATE TABLE mac_smoke(id serial, val text); INSERT INTO mac_smoke(val) VALUES ('hello_from_macos');"
RESULT=$("$SBDIR/use" -c "SELECT val FROM mac_smoke;" 2>&1)
echo "$RESULT"
echo "$RESULT" | grep -q "hello_from_macos" || { echo "FAIL: select did not return inserted row"; exit 1; }

- name: Cleanup
if: always()
run: |
for dir in "$HOME"/sandboxes-pg-mac/pg_sandbox_*; do
[ -d "$dir" ] && bash "$dir/stop" 2>/dev/null || true
done 2>/dev/null || true
pkill -9 -u "$USER" postgres 2>/dev/null || true

# Test InnoDB Cluster topology with MySQL Shell + MySQL Router
innodb-cluster-test:
name: InnoDB Cluster (${{ matrix.mysql-version }})
Expand Down
12 changes: 8 additions & 4 deletions cmd/init.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ func initEnvironment(cmd *cobra.Command, args []string) error {
skipDownloads, _ := flags.GetBool(globals.SkipAllDownloadsLabel)
skipTarballDownload, _ := flags.GetBool(globals.SkipTarballDownloadLabel)
skipCompletion, _ := flags.GetBool(globals.SkipShellCompletionLabel)
provider, _ := flags.GetString(globals.ProviderLabel)

return ops.InitEnvironment(ops.InitOptions{
SandboxBinary: sandboxBinary,
Expand All @@ -39,23 +40,26 @@ func initEnvironment(cmd *cobra.Command, args []string) error {
SkipDownloads: skipDownloads,
SkipTarballDownload: skipTarballDownload,
SkipCompletion: skipCompletion,
Provider: provider,
})
}

var initCmd = &cobra.Command{
Use: "init",
Short: "initializes dbdeployer environment",
Long: `Initializes dbdeployer environment:
Long: `Initializes dbdeployer environment:
* creates $SANDBOX_HOME and $SANDBOX_BINARY directories
* downloads and expands the latest MySQL tarball
* downloads and expands the latest MySQL tarball (default provider)
or prints PostgreSQL setup instructions (--provider=postgresql)
* installs shell completion file`,
RunE: initEnvironment,
}

func init() {
rootCmd.AddCommand(initCmd)
initCmd.PersistentFlags().Bool(globals.SkipAllDownloadsLabel, false, "Do not download any file (skip both MySQL tarball and shell completion file)")
initCmd.PersistentFlags().Bool(globals.SkipTarballDownloadLabel, false, "Do not download MySQL tarball")
initCmd.PersistentFlags().Bool(globals.SkipAllDownloadsLabel, false, "Do not download any file (skip both tarball and shell completion file)")
initCmd.PersistentFlags().Bool(globals.SkipTarballDownloadLabel, false, "Do not download the database tarball")
initCmd.PersistentFlags().Bool(globals.SkipShellCompletionLabel, false, "Do not download shell completion file")
initCmd.PersistentFlags().Bool(globals.DryRunLabel, false, "Show operations but don't run them")
initCmd.PersistentFlags().String(globals.ProviderLabel, globals.ProviderValue, "Database provider (mysql, postgresql)")
}
3 changes: 3 additions & 0 deletions downloads/remote_registry.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,9 @@ func DeleteTarball(tarballs []TarballDescription, tarballName string) ([]Tarball

func CompareTarballChecksum(tarball TarballDescription, fileName string) error {
if tarball.Checksum == "" {
fmt.Fprintf(os.Stderr,
"WARNING: no checksum available for %s — download integrity cannot be verified\n",
tarball.Name)
return nil
}
reCRC := regexp.MustCompile(`(MD5|SHA1|SHA256|SHA512)\s*:\s*(\S+)`)
Expand Down
Loading
Loading