Add an optional platform-finalize publish mode by han-xudong · Pull Request #118 · RoboStack/vinca

han-xudong · 2026-04-08T07:18:59Z

Related to #117.

This proposes an optional deferred publish mode for generated GitHub Actions workflows.

In platform-finalize mode:

build jobs set VINCA_SKIP_UPLOAD=1
each batch uploads its outputs as a workflow artifact
a final per-platform publish job downloads those artifacts and performs the upload only after the required build jobs succeed

This keeps the current behavior as the default and makes the deferred publish flow opt-in.

I also included a small follow-up fix for the Unix artifact upload path so generated workflows use ~/conda-bld/<target> rather than relying on ${{ env.HOME }}.

Validation:

pixi run pytest vinca/test_generate_publish_workflows.py -q
5 passed

han-xudong · 2026-04-08T07:22:31Z

One clarification on scope: although the PR text focuses on GitHub Actions, the commit also updates the Azure generator and adds matching tests there.

I kept those changes in the same PR because the publish model is the same in both generators:

keep the existing behavior as the default
add an opt-in deferred per-platform publish flow
collect batch outputs first, then publish only after the platform completes

If you would prefer to review GitHub Actions and Azure separately, I can split that in a follow-up.

traversaro · 2026-04-08T09:13:55Z

I am not sure if the azure pipelines generator actually are tested or currently work, so I would leave them aside.

traversaro · 2026-04-08T09:20:26Z

Thanks a lot for the contribution @han-xudong , can you check the CI failures?

fyi @wep21

traversaro · 2026-04-08T09:23:33Z

vinca/generate_gha.py

+    if publish_mode == "platform-finalize" and prev_batch_keys:
+        add_publish_job(azure_template, "win-64", vm_imagename, prev_batch_keys)
+


Can you explain why this is necessary?

Copilot

Pull request overview

Adds an opt-in “platform-finalize” publish mode to generated CI workflows so that per-batch jobs build (and upload artifacts) without publishing, and a final per-platform job/stage publishes only after the full platform build succeeds.

Changes:

Add --publish-mode {immediate,platform-finalize} to both GHA and Azure pipeline generators.
In platform-finalize mode, batch jobs set VINCA_SKIP_UPLOAD=1 and upload artifacts; a final publish job/stage downloads artifacts and uploads packages.
Adjust Unix artifact upload path generation to use ~/conda-bld/<target>.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
`vinca/generate_gha.py`	Adds publish-mode CLI + deferred publish flow via per-batch artifacts and a final publish job.
`vinca/generate_azure.py`	Adds publish-mode CLI + deferred publish flow via pipeline artifacts and a final publish stage.
`vinca/test_generate_publish_workflows.py`	Adds tests validating the generated YAML structure for platform-finalize mode and artifact paths.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-08T09:28:17Z

vinca/generate_azure.py

+        append_azure_publish_stage(
+            azure_template,
+            target,
+            stage_names,
+            azure_template.get("pool"),


In platform-finalize mode, the publish stage depends on stage_names, but stage_names is populated for every computed stage even when that stage ends up omitted from azure_stages (because it has zero jobs). That can produce a dependsOn reference to a stage that is not present in the emitted YAML, causing the pipeline to fail to load. Build the dependency list from the stages actually appended (e.g., derive names from azure_stages) before calling append_azure_publish_stage.

Copilot · 2026-04-08T09:28:17Z

vinca/generate_azure.py

+        append_azure_publish_stage(
+            azure_template,
+            target,
+            stage_names,
+            azure_template.get("pool"),


Same issue as Linux: stage_names can include stages that were never appended to azure_stages, so the publish stage may dependsOn non-existent stages. Consider constructing the dependsOn list from the emitted azure_stages only.

Copilot · 2026-04-08T09:28:18Z

vinca/generate_azure.py

+        append_azure_publish_stage(
+            azure_template,
+            "win-64",
+            stage_names,


Same issue as Linux/OSX: stage_names includes stage names even for stages that were dropped (no jobs), so the publish stage may end up depending on stages that don't exist in the final YAML. Derive the publish stage dependencies from azure_stages instead.

Suggested change

stage_names,

[stage["stage"] for stage in azure_stages],

Copilot · 2026-04-08T09:28:18Z

vinca/generate_azure.py

+        f"""curl -fsSL https://pixi.sh/install.sh | bash
+export PATH="$HOME/.pixi/bin:$PATH"
+shopt -s globstar nullglob
+files=("$(Pipeline.Workspace)"/artifacts/**/*.conda "$(Pipeline.Workspace)"/artifacts/**/*.tar.bz2)
+if (( ${{#files[@]}} == 0 )); then
+    echo "No built packages found for {target}"
+    exit 1
+fi
+pixi run upload "${{files[@]}}" --force"""


The Unix publish stage installs pixi via curl ... | bash, which is a supply-chain risk and makes builds less reproducible. Prefer a pinned, checksum-verified installer (or another deterministic install mechanism) to avoid executing remote content directly.

Suggested change

f"""curl -fsSL https://pixi.sh/install.sh | bash

export PATH="$HOME/.pixi/bin:$PATH"

shopt -s globstar nullglob

files=("$(Pipeline.Workspace)"/artifacts/**/*.conda "$(Pipeline.Workspace)"/artifacts/**/*.tar.bz2)

if (( ${{#files[@]}} == 0 )); then

echo "No built packages found for {target}"

exit 1

fi

pixi run upload "${{files[@]}}" --force"""

f"""shopt -s globstar nullglob

files=("$(Pipeline.Workspace)"/artifacts/**/*.conda "$(Pipeline.Workspace)"/artifacts/**/*.tar.bz2)

if (( ${{#files[@]}} == 0 )); then

echo "No built packages found for {target}"

exit 1

fi

for file in "${{files[@]}}"; do

anaconda -t "$ANACONDA_API_TOKEN" upload "$file" --force

done"""

Copilot · 2026-04-08T09:28:18Z

vinca/generate_gha.py

+        {
+            "name": "Setup pixi",
+            "uses": "prefix-dev/setup-pixi@v0.9.4",
+            "with": {


The publish job sets up pixi without pinning a pixi-version, while the Windows build jobs pin pixi-version: v0.63.2. To keep publish behavior reproducible and aligned with the build jobs, consider pinning the same pixi version here as well.

Suggested change

"with": {

"with": {

"pixi-version": "v0.63.2",

traversaro · 2026-04-08T09:32:49Z

@han-xudong I asked a GitHub Copilot PR just to help me, but feel free to ignore comments that do not have sense, for example as they just refer to code copied from the rest of the codebase.

han-xudong · 2026-04-08T09:44:33Z

@han-xudong I asked a GitHub Copilot PR just to help me, but feel free to ignore comments that do not have sense, for example as they just refer to code copied from the rest of the codebase.

@traversaro Thank you for your quick reply! I am checking the code and will submit a new commit as soon as possible. Copilot's suggestions are helpful~

han-xudong · 2026-04-08T10:07:12Z

@traversaro Thanks for the review.

The reason for this mode is to avoid partial per-platform publication during long RoboStack builds. Builds can already run in batches, but if uploads happen batch-by-batch, a platform can temporarily expose only part of its dependency graph while the rest is still building. For ROS packages with dense interdependencies, that can create an inconsistent intermediate state.

platform-finalize only changes that publication point: the build still runs in batches, but upload is delayed until all batches for the same platform have completed successfully. The current behavior stays the default; this is opt-in.

I also pushed a follow-up update for the Copilot comments:

the Azure final publish stage now depends only on stages that are actually emitted
the Azure Unix final publish step now uses anaconda-client directly instead of bootstrapping Pixi via curl | bash

I added regression coverage for both and re-ran:

pixi run pytest vinca/test_generate_publish_workflows.py -q
pixi run fmt-check

Add platform-finalize publish mode

9f92914

traversaro reviewed Apr 8, 2026

View reviewed changes

traversaro requested a review from Copilot April 8, 2026 09:23

Copilot started reviewing on behalf of traversaro April 8, 2026 09:24 View session

Copilot AI reviewed Apr 8, 2026

View reviewed changes

Refine platform-finalize publish workflows

aeee495

		if publish_mode == "platform-finalize" and prev_batch_keys:
		add_publish_job(azure_template, "win-64", vm_imagename, prev_batch_keys)

Uh oh!

Conversation

han-xudong commented Apr 8, 2026

Uh oh!

han-xudong commented Apr 8, 2026

Uh oh!

traversaro commented Apr 8, 2026

Uh oh!

traversaro commented Apr 8, 2026

Uh oh!

traversaro Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

Uh oh!

traversaro commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

han-xudong commented Apr 8, 2026

Uh oh!

han-xudong commented Apr 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

traversaro commented Apr 8, 2026 •

edited

Loading