remove reliance on rapidsai/ci-conda, use a versions.yaml by jameslamb · Pull Request #834 · rapidsai/docker

jameslamb · 2025-12-18T20:40:54Z

Contributes to #832

Removes reliance on https://hub.docker.com/r/rapidsai/ci-conda in this repo. It was only being used in 2 places, both of which could be run in simpler environments:

in a compute-matrix job that just needs bash, jq and yq --> regular old ubuntu-latest GitHub Actions VM is fine
in a part of a multi-stage build that just needs bash, git, pip, and a Python interpreter --> python:{version} container is fine

Other changes based on PR reviewers:

puts versions to be managed by renovate in a versions.yaml (borrowing the approach from rapidsai/ci-imgs)
updates shell code in Dockerfiles to match our preferred style (including the array format for package lists from use bash arrays for package lists, other misc. maintenance ci-imgs#343)
sets all empty arg defaults to notset (ref: add CUDA 12.5 images #689 (comment))
- weakly prefer notset to unset because unset is a valid shell command
alphabetizes build args in Dockerfiles

Notes for Reviewers

This is only the beginning

To be clear, this project still heavily relies on rapidsai/miniforge-cuda (the base image for rapidsai/ci-conda. So there's more work to do for #832.

But this at least means we no longer need to care about anything in these lines breaking the things in this repo 😁 :

https://github.com/rapidsai/ci-imgs/blob/55b24734b7d594e768599771dcae2e718af52f9a/ci-conda.Dockerfile#L183-L313

jameslamb · 2025-12-18T22:22:14Z

.github/workflows/build-test-publish-images.yml

  compute-matrix:
    runs-on: ubuntu-latest
-    container:
-      image: rapidsai/ci-conda:latest


This is just some string manipulation in bash + jq + yq... I don't think we need to pay the cost of pulling a container image, and I think we can pretty safely just accept whatever jq / yq version GitHub is shipping in its images (ref: https://github.com/actions/runner-images/tree/main/images/ubuntu).

jameslamb · 2025-12-18T22:22:55Z

.gitignore

+*.tar.gz
+*.tgz
+*.whl
+*.zip


Saw .ruff_cache not .gitignore'd and that led me to updating this file which led me to adding a few more things just for extra safety.

jameslamb · 2025-12-18T22:24:26Z

Dockerfile

+    --prefer-binary        \
+    --upgrade              \
+      'conda-merge==0.3.*' \
+      'rapids-dependency-file-generator==1.20.*'


Pinning these to the version ranges we're currently getting just for a bit of extra safety (since I'm touching the line anyway).

jameslamb · 2025-12-19T17:11:46Z

I think the test jobs are failing because we're pulling in 25.12 conda packages but using the 26.02 development branches of the repos.

I think it's probably this 25.12.00b tag being after 26.02.00a:

git checkout main
git pull upstream main
git describe --tags --abbrev=0
# v25.12.00b

Deleting it should fix this, but I asked RAPIDS Ops offline to confirm before I do that.

jameslamb · 2025-12-19T18:33:32Z

good news: removing that 25.12.00b tag worked 🎉

bad news: the builds are failing because some libraries were missed in rapidsai/build-planning#233

#13 16.23        │  │  ├─ cuxfilter 26.02.00a5 would require
#13 16.23        │  │  │  └─ python_abi =3.12 *_cp312, which requires
#13 16.23        │  │  │     └─ python =3.12 *_cpython, which conflicts with any installable versions previously reported;
#13 16.23        │  │  └─ cuxfilter 26.02.00a5 would require
#13 16.23        │  │     └─ python_abi =3.13 *_cp313 but there are no viable options
#13 16.23        │  │        ├─ python_abi 3.13 would require
#13 16.23        │  │        │  └─ python =3.13 *_cpython, which conflicts with any installable versions previously reported;
#13 16.23        │  │        └─ python_abi 3.13 would require
#13 16.23        │  │           └─ python =3.13 *_cp313, which conflicts with any installable versions previously reported;
#13 16.23        │  └─ dask-cuda =26.2 *, which requires
#13 16.23        │     └─ numba-cuda >=0.22.1,<0.23.0 * but there are no viable options
#13 16.23        │        ├─ numba-cuda 0.22.1 conflicts with any installable versions previously reported;
#13 16.23        │        ├─ numba-cuda 0.22.1 would require
#13 16.23        │        │  └─ python_abi =3.11 *_cp311 but there are no viable options
#13 16.23        │        │     ├─ python_abi 3.11, which cannot be installed (as previously explained);
#13 16.23        │        │     └─ python_abi 3.11 conflicts with any installable versions previously reported;
#13 16.23        │        ├─ numba-cuda 0.22.1 would require
#13 16.23        │        │  └─ python_abi =3.12 *_cp312, which cannot be installed (as previously explained);
#13 16.23        │        └─ numba-cuda 0.22.1 would require
#13 16.23        │           └─ python_abi =3.13 *_cp313, which cannot be installed (as previously explained);
#13 16.23        └─ rapids 26.02.00a10 would require
#13 16.23           └─ cuda-version >=13,<14.0a0 *, which conflicts with any installable versions previously reported.

(build link)

I'll go update them and come back to this.

jameslamb · 2025-12-23T15:31:49Z

The tagging issue and cuxfilter issue have been resolved.

Next, CI's failing because cugraph hasn't had successful nightlies in a while and the older ones aren't usable alongside newer ones of other RAPIDS packages.

ImportError: /opt/conda/lib/python3.12/site-packages/cugraph/structure/../../../../libcugraph.so: undefined symbol: _ZN3rmm13device_bufferC1EPKvmNS_16cuda_stream_viewENS_6detail23cccl_async_resource_refIN4cuda2mr3__418basic_resource_refILNS8_10_AllocTypeE1EJNS8_17device_accessibleEEEEEE

(build link)

~~I think rapidsai/cugraph#5379 is trying to fix that.~~ yes, fixed

bdice

A couple cleanup suggestions.

bdice · 2025-12-31T22:55:22Z

Dockerfile

 ARG RAPIDS_BRANCH="main"

-SHELL ["/bin/bash", "-euo", "pipefail", "-c"]
+ARG YQ_VER=4.49.2


Could we make this use a versions.yaml like in ci-imgs? https://github.com/rapidsai/ci-imgs/blob/main/versions.yaml

Sure, I can expand this PR to include that.

I've pushed commits doing this. Spot-checked a few and I think it's working.

ref: https://github.com/rapidsai/docker/actions/runs/20798063149/job/59736606018?pr=834

bdice · 2025-12-31T22:56:35Z

Dockerfile

 RUN <<EOF
 apt-get update
-apt-get install -y --no-install-recommends rsync
+apt-get install             \


Generally we keep the flags on the first line (it's not that long) and we also avoid indenting the \ characters. Example: https://github.com/rapidsai/ci-imgs/blob/55b24734b7d594e768599771dcae2e718af52f9a/ci-conda.Dockerfile#L208-L215

Let's follow that style here.

Sure, no problem. This style of indenting the \ over is something I've grown used to looking at lots of RAPIDS devcontainers code recently (like this), I kind of like it.

But don't feel strongly at all, I'll change this to match what we do in other Dockerfiles, no problem.

I've updated this in recent commits, and also pulled in the put-packages-in-arrays stuff from rapidsai/ci-imgs#343 / rapidsai/ci-imgs#342 (comment)

…conda

…tize args

jameslamb · 2026-01-07T22:36:01Z

I think this is ready for another review.

jameslamb · 2026-01-08T17:43:42Z

/merge

jameslamb · 2026-01-08T17:44:15Z

rapidsai/miniforge-cuda next 😈

remove reliance on rapidsai/ci-conda

d0756cb

jameslamb added improvement non-breaking labels Dec 18, 2025

jameslamb added 2 commits December 18, 2025 14:43

just use echo, gha-tools is unnecessary

71b9610

install yq and jq

3c02404

jameslamb commented Dec 18, 2025

View reviewed changes

jameslamb added 3 commits December 19, 2025 09:02

install newer yq, required for notebooks.sh

a378910

fine hadolint, fine

455c452

notebooks image needs build args too

68114cc

jameslamb mentioned this pull request Dec 19, 2025

Migrate RAPIDS to numba-cuda>=0.22.1,<0.23.0 rapidsai/build-planning#233

Closed

6 tasks

jameslamb added 2 commits December 19, 2025 15:26

skip 'rapids' metapackage, just to test

117df11

switch back to rapids metapackage

dcb9ac0

jameslamb mentioned this pull request Dec 23, 2025

remove reliance on rapidsai/miniforge-cuda (and therefore ci-imgs), upgrade cuvs-bench-cpu to miniforge 25.9.1-0 #836

Merged

jameslamb changed the title ~~WIP: remove reliance on rapidsai/ci-conda~~ remove reliance on rapidsai/ci-conda Dec 31, 2025

jameslamb marked this pull request as ready for review December 31, 2025 21:40

jameslamb requested a review from a team as a code owner December 31, 2025 21:40

jameslamb requested a review from msarahan December 31, 2025 21:40

bdice reviewed Dec 31, 2025

View reviewed changes

jameslamb and others added 7 commits January 7, 2026 15:23

use a versions.yaml

56494e7

Merge branch 'main' into no-ci-conda

be6d19d

Merge branch 'main' of github.com:rapidsai/docker into no-ci-conda

8729973

Merge branch 'no-ci-conda' of github.com:jameslamb/docker into no-ci-…

6a07d40

…conda

copyrights

af498f9

fix workflow

252316f

prefer 'notset' to 'unset', remove unnecessary MINIFORGE_VER, alphabe…

b4d6998

…tize args

match new renovate version

d5797dd

jameslamb changed the title ~~remove reliance on rapidsai/ci-conda~~ remove reliance on rapidsai/ci-conda, use a versions.yaml Jan 7, 2026

bash array, not command evaluation

a8cc963

jameslamb requested a review from bdice January 7, 2026 22:35

bdice approved these changes Jan 8, 2026

View reviewed changes

rapids-bot bot merged commit 8533b67 into rapidsai:main Jan 8, 2026
64 checks passed

jameslamb deleted the no-ci-conda branch January 8, 2026 17:44

jameslamb mentioned this pull request Jan 8, 2026

[FEA] Decouple this project from rapidsai/ci-imgs #832

Closed

5 tasks

jameslamb mentioned this pull request Jan 23, 2026

Automate SBOM generation for container images #805

Open

Conversation

jameslamb commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Notes for Reviewers

This is only the beginning

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jameslamb commented Dec 19, 2025

Uh oh!

jameslamb commented Dec 19, 2025

Uh oh!

jameslamb commented Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bdice left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jameslamb commented Jan 7, 2026

Uh oh!

jameslamb commented Jan 8, 2026

Uh oh!

Uh oh!

jameslamb commented Jan 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jameslamb commented Dec 18, 2025 •

edited

Loading

jameslamb commented Dec 23, 2025 •

edited

Loading