Summary
Since the named-channels merge (#77, #78, #79), all functional tests are broken. Orchestrator starts in the Docker container but cannot save discovered instances to the SQLite backend, causing /api/clusters to return empty results.
Symptoms
- Orchestrator starts successfully and logs "Starting HTTP listener on :3099"
- Discover API calls resolve hostnames (mysql1 → mysql1, sees replica IPs 172.30.0.11, 172.30.0.12)
- But
/api/clusters returns empty ([]) after 60+ seconds
- Only ~30 lines of orchestrator log in 8+ minutes — the continuous discovery loop appears to stop after the first iteration
- No error messages logged
Last working commit
70065f9d (Merge PR #76, issue74-web-ui-refresh) — functional tests passed in 3m42s
First broken commit
e4d5b2ae (Multi-cluster support: manage MySQL and PostgreSQL simultaneously, #73)
Investigation so far
- Schema migration for
provider_type column works correctly on SQLite (verified locally)
- Orchestrator self-elects on local machine (no Docker) without issues
- The discover API handler calls
ReadTopologyInstance which connects to MySQL successfully (hostname resolution + replica IPs appear in logs)
- But instance data never appears in backend DB
- The continuous discovery loop seems to stop/block after the first cycle
- Possibly related to SQLite concurrency with the new
detectProviderType queries running alongside the write path
To reproduce
go build -o bin/orchestrator ./go/cmd/orchestrator
cd tests/functional
docker compose up -d mysql1 mysql2 mysql3 proxysql
bash setup-replication.sh
docker compose up -d orchestrator
# Wait 30s, then:
curl http://localhost:3099/api/clusters # Returns []
docker compose logs orchestrator # Shows only ~30 lines
Impact
All functional CI tests fail on master and all PR branches.
Summary
Since the named-channels merge (#77, #78, #79), all functional tests are broken. Orchestrator starts in the Docker container but cannot save discovered instances to the SQLite backend, causing
/api/clustersto return empty results.Symptoms
/api/clustersreturns empty ([]) after 60+ secondsLast working commit
70065f9d(Merge PR #76, issue74-web-ui-refresh) — functional tests passed in 3m42sFirst broken commit
e4d5b2ae(Multi-cluster support: manage MySQL and PostgreSQL simultaneously, #73)Investigation so far
provider_typecolumn works correctly on SQLite (verified locally)ReadTopologyInstancewhich connects to MySQL successfully (hostname resolution + replica IPs appear in logs)detectProviderTypequeries running alongside the write pathTo reproduce
Impact
All functional CI tests fail on master and all PR branches.