Skip to content

CI: MySQL version matrix (5.7-9.6) and named channels tests #85

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
renecannao merged 4 commits into from
Apr 9, 2026
Merged

CI: MySQL version matrix (5.7-9.6) and named channels tests #85

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8436ee5
Add MySQL version matrix and named channels functional tests (#80, #81)
renecannao Apr 9, 2026
f2d2a0f
Fix CI: link named-channels.md in docs TOC, clean up test
renecannao Apr 9, 2026
fa42679
Fix stuck functional tests: add curl timeouts to all HTTP calls
renecannao Apr 9, 2026
75fe695
Fix named channels test: use correct API endpoint, fix unbound $2
renecannao Apr 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 99 additions & 11 deletions .github/workflows/functional.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ on:
workflow_dispatch:

jobs:
functional:
build:
runs-on: ubuntu-latest
timeout-minutes: 30
timeout-minutes: 10

steps:
- uses: actions/checkout@v4
Expand All @@ -23,12 +23,44 @@ jobs:
go-version: '1.25.7'
cache: true

- name: Build orchestrator (for CLI tests on host)
- name: Build orchestrator
run: go build -o bin/orchestrator ./go/cmd/orchestrator

- name: Start test infrastructure (MySQL + ProxySQL + Orchestrator)
- name: Upload orchestrator binary
uses: actions/upload-artifact@v4
with:
name: orchestrator-binary
path: bin/orchestrator
retention-days: 1

functional-mysql:
runs-on: ubuntu-latest
timeout-minutes: 30
needs: build

strategy:
fail-fast: false
matrix:
mysql_version: ['5.7', '8.0', '8.4', '9.0', '9.2', '9.5', '9.6']

steps:
- uses: actions/checkout@v4

- name: Download orchestrator binary
uses: actions/download-artifact@v4
with:
name: orchestrator-binary
path: bin

- name: Make binary executable
run: chmod +x bin/orchestrator

- name: Start test infrastructure (MySQL + ProxySQL)
working-directory: tests/functional
env:
MYSQL_IMAGE: mysql:${{ matrix.mysql_version }}
run: |
echo "Using MySQL image: $MYSQL_IMAGE"
docker compose up -d mysql1 mysql2 mysql3 proxysql
echo "Waiting for MySQL and ProxySQL to be healthy..."
timeout 120 bash -c '
Expand All @@ -51,10 +83,14 @@ jobs:
' || { echo "Timeout"; docker compose ps; docker compose logs --tail=30; exit 1; }

- name: Setup replication
env:
MYSQL_IMAGE: mysql:${{ matrix.mysql_version }}
run: bash tests/functional/setup-replication.sh

- name: Start orchestrator in Docker network
working-directory: tests/functional
env:
MYSQL_IMAGE: mysql:${{ matrix.mysql_version }}
run: |
docker compose up -d orchestrator
echo "Waiting for orchestrator to be ready..."
Expand All @@ -77,7 +113,62 @@ jobs:
- name: Run failover tests
run: bash tests/functional/test-failover.sh

# ---- PostgreSQL functional tests ----
- name: Run named channels tests
run: bash tests/functional/test-named-channels.sh

- name: Collect orchestrator logs
if: always()
working-directory: tests/functional
env:
MYSQL_IMAGE: mysql:${{ matrix.mysql_version }}
run: |
docker compose logs orchestrator > /tmp/orchestrator-test.log 2>&1 || true

- name: Upload orchestrator logs
if: always()
uses: actions/upload-artifact@v4
with:
name: orchestrator-test-logs-mysql-${{ matrix.mysql_version }}
path: /tmp/orchestrator-test.log

- name: Collect all docker logs on failure
if: failure()
working-directory: tests/functional
env:
MYSQL_IMAGE: mysql:${{ matrix.mysql_version }}
run: docker compose logs > /tmp/docker-compose-logs.txt 2>&1 || true

- name: Upload docker logs on failure
if: failure()
uses: actions/upload-artifact@v4
with:
name: docker-compose-logs-mysql-${{ matrix.mysql_version }}
path: /tmp/docker-compose-logs.txt

- name: Cleanup
if: always()
working-directory: tests/functional
env:
MYSQL_IMAGE: mysql:${{ matrix.mysql_version }}
run: docker compose down -v --remove-orphans 2>/dev/null || true

functional-postgresql:
runs-on: ubuntu-latest
timeout-minutes: 30
needs: build

steps:
- uses: actions/checkout@v4

- name: Download orchestrator binary
uses: actions/download-artifact@v4
with:
name: orchestrator-binary
path: bin

- name: Make binary executable
run: chmod +x bin/orchestrator

- name: Start PostgreSQL containers
working-directory: tests/functional
run: |
Expand Down Expand Up @@ -145,17 +236,14 @@ jobs:
if: always()
working-directory: tests/functional
run: |
docker compose logs orchestrator > /tmp/orchestrator-test.log 2>&1 || true
docker compose logs orchestrator-pg > /tmp/orchestrator-pg-test.log 2>&1 || true

- name: Upload orchestrator logs
if: always()
uses: actions/upload-artifact@v4
with:
name: orchestrator-test-logs
path: |
/tmp/orchestrator-test.log
/tmp/orchestrator-pg-test.log
name: orchestrator-test-logs-postgresql
path: /tmp/orchestrator-pg-test.log

- name: Collect all docker logs on failure
if: failure()
Expand All @@ -166,7 +254,7 @@ jobs:
if: failure()
uses: actions/upload-artifact@v4
with:
name: docker-compose-logs
name: docker-compose-logs-postgresql
path: /tmp/docker-compose-logs.txt

- name: Cleanup
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
- [Topology recovery](topology-recovery.md): recovery process, promotion and hooks.
- [Key-Value stores](kv.md): master discovery for your apps
- [ProxySQL hooks](proxysql-hooks.md): built-in ProxySQL failover integration
- [Named replication channels](named-channels.md): multi-source replication with named channels

#### Operation
- [Observability](observability.md): Prometheus metrics and Kubernetes health endpoints
Expand Down
1 change: 1 addition & 0 deletions docs/toc.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
- [Topology recovery](topology-recovery.md): recovery process, promotion and hooks.
- [Key-Value stores](kv.md): master discovery for your apps
- [ProxySQL hooks](proxysql-hooks.md): built-in ProxySQL failover integration
- [Named replication channels](named-channels.md): multi-source replication with named channels

#### Operation
- [Observability](observability.md): Prometheus metrics and Kubernetes health endpoints
Expand Down
6 changes: 3 additions & 3 deletions tests/functional/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ version: "3.8"

services:
mysql1:
image: mysql:8.4
image: ${MYSQL_IMAGE:-mysql:8.4}
hostname: mysql1
environment:
MYSQL_ROOT_PASSWORD: testpass
Expand All @@ -23,7 +23,7 @@ services:
- mysql1

mysql2:
image: mysql:8.4
image: ${MYSQL_IMAGE:-mysql:8.4}
hostname: mysql2
environment:
MYSQL_ROOT_PASSWORD: testpass
Expand All @@ -47,7 +47,7 @@ services:
- mysql2

mysql3:
image: mysql:8.4
image: ${MYSQL_IMAGE:-mysql:8.4}
hostname: mysql3
environment:
MYSQL_ROOT_PASSWORD: testpass
Expand Down
88 changes: 75 additions & 13 deletions tests/functional/lib.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ pass() {

fail() {
echo " ❌ FAIL: $1"
[ -n "$2" ] && echo " $2"
[ -n "${2:-}" ] && echo " $2"
((FAIL_COUNT++))
}

Expand All @@ -33,7 +33,7 @@ summary() {
test_endpoint() {
local NAME="$1" URL="$2" EXPECT="$3"
local CODE
CODE=$(curl -s -o /dev/null -w "%{http_code}" "$URL" 2>&1)
CODE=$(curl -s --max-time 10 -o /dev/null -w "%{http_code}" "$URL" 2>&1)
if [ "$CODE" = "$EXPECT" ]; then
pass "$NAME (HTTP $CODE)"
else
Expand All @@ -45,7 +45,7 @@ test_endpoint() {
test_body_contains() {
local NAME="$1" URL="$2" EXPECT="$3"
local BODY
BODY=$(curl -s "$URL" 2>&1)
BODY=$(curl -s --max-time 10 "$URL" 2>&1)
if echo "$BODY" | grep -q "$EXPECT"; then
pass "$NAME"
else
Expand All @@ -57,7 +57,7 @@ test_body_contains() {
wait_for_orchestrator() {
echo "Waiting for orchestrator to be ready..."
for i in $(seq 1 30); do
if curl -s -o /dev/null "$ORC_URL/api/clusters" 2>/dev/null; then
if curl -s --max-time 5 -o /dev/null "$ORC_URL/api/clusters" 2>/dev/null; then
echo "Orchestrator ready after ${i}s"
return 0
fi
Expand All @@ -73,35 +73,92 @@ CLUSTER_NAME=""
discover_topology() {
local MASTER_HOST="$1"
echo "Seeding discovery with $MASTER_HOST..."
curl -s "$ORC_URL/api/discover/$MASTER_HOST/3306" > /dev/null
curl -s --max-time 10 "$ORC_URL/api/discover/$MASTER_HOST/3306" > /dev/null

# Also seed replicas directly
curl -s "$ORC_URL/api/discover/mysql2/3306" > /dev/null 2>&1
curl -s "$ORC_URL/api/discover/mysql3/3306" > /dev/null 2>&1
curl -s --max-time 10 "$ORC_URL/api/discover/mysql2/3306" > /dev/null 2>&1
curl -s --max-time 10 "$ORC_URL/api/discover/mysql3/3306" > /dev/null 2>&1

echo "Waiting for topology discovery..."
for i in $(seq 1 60); do
# Get the cluster name dynamically
CLUSTER_NAME=$(curl -s "$ORC_URL/api/clusters" 2>/dev/null | python3 -c "import json,sys; c=json.load(sys.stdin); print(c[0] if c else '')" 2>/dev/null || echo "")
CLUSTER_NAME=$(curl -s --max-time 5 "$ORC_URL/api/clusters" 2>/dev/null | python3 -c "import json,sys; c=json.load(sys.stdin); print(c[0] if c else '')" 2>/dev/null || echo "")
if [ -n "$CLUSTER_NAME" ]; then
local COUNT
COUNT=$(curl -s "$ORC_URL/api/cluster/$CLUSTER_NAME" 2>/dev/null | python3 -c "import json,sys; print(len(json.load(sys.stdin)))" 2>/dev/null || echo "0")
COUNT=$(curl -s --max-time 5 "$ORC_URL/api/cluster/$CLUSTER_NAME" 2>/dev/null | python3 -c "import json,sys; print(len(json.load(sys.stdin)))" 2>/dev/null || echo "0")
if [ "$COUNT" -ge 3 ] 2>/dev/null; then
echo "Full topology discovered (${COUNT} instances, cluster=$CLUSTER_NAME) after ${i}s"
return 0
fi
fi
# Re-seed replicas periodically
if [ "$((i % 10))" = "0" ]; then
curl -s "$ORC_URL/api/discover/mysql2/3306" > /dev/null 2>&1
curl -s "$ORC_URL/api/discover/mysql3/3306" > /dev/null 2>&1
curl -s --max-time 10 "$ORC_URL/api/discover/mysql2/3306" > /dev/null 2>&1
curl -s --max-time 10 "$ORC_URL/api/discover/mysql3/3306" > /dev/null 2>&1
fi
sleep 1
done
echo "WARNING: Cluster=$CLUSTER_NAME, instances=${COUNT:-0} after 60s"
return 1
}

# Detect MySQL major version (e.g., "5.7", "8.0", "8.4", "9.0")
# Caches result in MYSQL_MAJOR_VERSION
MYSQL_MAJOR_VERSION=""
mysql_version() {
if [ -n "$MYSQL_MAJOR_VERSION" ]; then
echo "$MYSQL_MAJOR_VERSION"
return
fi
local FULL
FULL=$(docker compose -f tests/functional/docker-compose.yml exec -T mysql1 \
mysql -uroot -ptestpass -Nse "SELECT VERSION()" 2>/dev/null | tr -d '[:space:]')
MYSQL_MAJOR_VERSION=$(echo "$FULL" | sed -E 's/^([0-9]+\.[0-9]+).*/\1/')
echo "$MYSQL_MAJOR_VERSION"
}

# Check if MySQL version is 5.7 (returns 0 for 5.7, 1 otherwise)
mysql_is_57() {
[ "$(mysql_version)" = "5.7" ]
}

# Return the correct CHANGE REPLICATION SOURCE / CHANGE MASTER command
# Arguments: HOST PORT USER PASSWORD
mysql_change_source_sql() {
local HOST="$1" PORT="$2" USER="$3" PASS="$4"
if mysql_is_57; then
echo "CHANGE MASTER TO MASTER_HOST='${HOST}', MASTER_PORT=${PORT}, MASTER_USER='${USER}', MASTER_PASSWORD='${PASS}', MASTER_AUTO_POSITION=1;"
else
echo "CHANGE REPLICATION SOURCE TO SOURCE_HOST='${HOST}', SOURCE_PORT=${PORT}, SOURCE_USER='${USER}', SOURCE_PASSWORD='${PASS}', SOURCE_AUTO_POSITION=1, GET_SOURCE_PUBLIC_KEY=1;"
fi
}

# Return the correct CHANGE REPLICATION SOURCE / CHANGE MASTER command with FOR CHANNEL
# Arguments: HOST PORT USER PASSWORD CHANNEL
mysql_change_source_channel_sql() {
local HOST="$1" PORT="$2" USER="$3" PASS="$4" CHANNEL="$5"
if mysql_is_57; then
echo "CHANGE MASTER TO MASTER_HOST='${HOST}', MASTER_PORT=${PORT}, MASTER_USER='${USER}', MASTER_PASSWORD='${PASS}', MASTER_AUTO_POSITION=1 FOR CHANNEL '${CHANNEL}';"
else
echo "CHANGE REPLICATION SOURCE TO SOURCE_HOST='${HOST}', SOURCE_PORT=${PORT}, SOURCE_USER='${USER}', SOURCE_PASSWORD='${PASS}', SOURCE_AUTO_POSITION=1, GET_SOURCE_PUBLIC_KEY=1 FOR CHANNEL '${CHANNEL}';"
fi
}

# Return the correct START REPLICA / START SLAVE command
mysql_start_replica_sql() {
if mysql_is_57; then echo "START SLAVE;"; else echo "START REPLICA;"; fi
}

# Return the correct STOP REPLICA / STOP SLAVE command
mysql_stop_replica_sql() {
if mysql_is_57; then echo "STOP SLAVE;"; else echo "STOP REPLICA;"; fi
}

# Return the correct RESET REPLICA ALL / RESET SLAVE ALL command
mysql_reset_replica_all_sql() {
if mysql_is_57; then echo "RESET SLAVE ALL;"; else echo "RESET REPLICA ALL;"; fi
}

# Get ProxySQL servers for a hostgroup
proxysql_servers() {
local HG="$1"
Expand All @@ -120,6 +177,11 @@ mysql_read_only() {
# Get MySQL replication source
mysql_source_host() {
local CONTAINER="$1"
docker compose -f tests/functional/docker-compose.yml exec -T "$CONTAINER" \
mysql -uroot -ptestpass -Nse "SHOW REPLICA STATUS\G" 2>/dev/null | grep "Source_Host" | awk '{print $2}'
if mysql_is_57; then
docker compose -f tests/functional/docker-compose.yml exec -T "$CONTAINER" \
mysql -uroot -ptestpass -Nse "SHOW SLAVE STATUS\G" 2>/dev/null | grep "Master_Host" | awk '{print $2}'
else
docker compose -f tests/functional/docker-compose.yml exec -T "$CONTAINER" \
mysql -uroot -ptestpass -Nse "SHOW REPLICA STATUS\G" 2>/dev/null | grep "Source_Host" | awk '{print $2}'
fi
Comment on lines 177 to +186
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

mysql_source_host() still assumes single-source replication.

On a multi-channel replica, SHOW SLAVE/REPLICA STATUS\G returns one block per channel. This helper will therefore emit every Master_Host/Source_Host it finds, so callers get a concatenated value as soon as mysql3 has both its default channel and extra. Please make this helper channel-aware, or fail fast when multiple channels are present.

🐛 Proposed fix 
 mysql_source_host() {
-    local CONTAINER="$1"
+    local CONTAINER="$1" CHANNEL="${2:-}"
     if mysql_is_57; then
+        local STATUS_SQL="SHOW SLAVE STATUS\\G"
+        [ -n "$CHANNEL" ] && STATUS_SQL="SHOW SLAVE STATUS FOR CHANNEL '${CHANNEL}'\\G"
         docker compose -f tests/functional/docker-compose.yml exec -T "$CONTAINER" \
-            mysql -uroot -ptestpass -Nse "SHOW SLAVE STATUS\G" 2>/dev/null | grep "Master_Host" | awk '{print $2}'
+            mysql -uroot -ptestpass -Nse "$STATUS_SQL" 2>/dev/null |
+            awk -F': ' '$1 == "Master_Host" { print $2; exit }'
     else
+        local STATUS_SQL="SHOW REPLICA STATUS\\G"
+        [ -n "$CHANNEL" ] && STATUS_SQL="SHOW REPLICA STATUS FOR CHANNEL '${CHANNEL}'\\G"
         docker compose -f tests/functional/docker-compose.yml exec -T "$CONTAINER" \
-            mysql -uroot -ptestpass -Nse "SHOW REPLICA STATUS\G" 2>/dev/null | grep "Source_Host" | awk '{print $2}'
+            mysql -uroot -ptestpass -Nse "$STATUS_SQL" 2>/dev/null |
+            awk -F': ' '$1 == "Source_Host" { print $2; exit }'
     fi
 }
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@tests/functional/lib.sh` around lines 177 - 186, The helper
mysql_source_host() currently assumes single-source replication and can return
concatenated hosts for multi-channel replicas; update it to be channel-aware by
adding an optional channel parameter (e.g., channel="$2") and parse the output
of SHOW SLAVE/REPLICA STATUS\G to map Channel (or use absence of Channel for
MySQL 5.7) to its corresponding Master_Host/Source_Host; if a channel is
provided, return only that channel's host, and if no channel is provided but
multiple channels are present, fail fast with a clear error and non-zero exit
code so callers don't receive concatenated values (ensure changes touch
mysql_source_host and its usage contract).

}
Comment on lines 178 to 187
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The mysql_source_host function contains duplicated code for handling MySQL 5.7 and newer versions. You can refactor this to be more DRY by using variables for the version-specific commands and fields.

Suggested change
mysql_source_host() {
local CONTAINER="$1"
docker compose -f tests/functional/docker-compose.yml exec -T "$CONTAINER" \
mysql -uroot -ptestpass -Nse "SHOW REPLICA STATUS\G" 2>/dev/null | grep "Source_Host" | awk '{print $2}'
if mysql_is_57; then
docker compose -f tests/functional/docker-compose.yml exec -T "$CONTAINER" \
mysql -uroot -ptestpass -Nse "SHOW SLAVE STATUS\G" 2>/dev/null | grep "Master_Host" | awk '{print $2}'
else
docker compose -f tests/functional/docker-compose.yml exec -T "$CONTAINER" \
mysql -uroot -ptestpass -Nse "SHOW REPLICA STATUS\G" 2>/dev/null | grep "Source_Host" | awk '{print $2}'
fi
}
mysql_source_host() {
local CONTAINER="$1"
local show_cmd grep_field
if mysql_is_57; then
show_cmd="SHOW SLAVE STATUS\G"
grep_field="Master_Host"
else
show_cmd="SHOW REPLICA STATUS\G"
grep_field="Source_Host"
fi
docker compose -f tests/functional/docker-compose.yml exec -T "$CONTAINER" \
mysql -uroot -ptestpass -Nse "$show_cmd" 2>/dev/null | grep "$grep_field" | awk '{print $2}'
}

2 changes: 1 addition & 1 deletion tests/functional/mysql/master.cnf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@ log-bin=mysql-bin
binlog-format=ROW
gtid-mode=ON
enforce-gtid-consistency=ON
log-replica-updates=ON
log-slave-updates=ON
binlog-row-image=MINIMAL
report-host=mysql1
2 changes: 1 addition & 1 deletion tests/functional/mysql/replica.cnf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@ log-bin=mysql-bin
binlog-format=ROW
gtid-mode=ON
enforce-gtid-consistency=ON
log-replica-updates=ON
log-slave-updates=ON
binlog-row-image=MINIMAL
read-only=ON
2 changes: 1 addition & 1 deletion tests/functional/mysql/replica2.cnf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@ log-bin=mysql-bin
binlog-format=ROW
gtid-mode=ON
enforce-gtid-consistency=ON
log-replica-updates=ON
log-slave-updates=ON
binlog-row-image=MINIMAL
read-only=ON
Loading
Loading