Skip to content

Add :python option to uv_init/2 for free-threaded Python support#45

Merged
jonatanklosko merged 2 commits intolivebook-dev:mainfrom
nyo16:add-python-option
Feb 16, 2026
Merged

Add :python option to uv_init/2 for free-threaded Python support#45
jonatanklosko merged 2 commits intolivebook-dev:mainfrom
nyo16:add-python-option

Conversation

@nyo16
Copy link
Contributor

@nyo16 nyo16 commented Feb 14, 2026

Adds a new :python option to Pythonx.uv_init/2 that gets passed as --python to uv sync, allowing users to request a specific Python variant — most notably free-threaded builds like "3.14t".

Motivation

Free-threaded Python (experimental in 3.13t, officially supported in 3.14t) removes the GIL, enabling true parallelism when calling Pythonx.eval/2 from multiple Elixir processes concurrently. As noted in
#23 and #39, this has been a requested feature. Now that Python 3.14t is officially supported and uv can install free-threaded builds via --python, this can be offered as an opt-in option without any
breaking changes.

No C++ NIF changes are needed — PyEval_RestoreThread/PyEval_SaveThread still exist in free-threaded Python as lightweight no-ops, and the wildcard lib/libpython3.*{.dylib,.so} already matches
libpython3.14t.{dylib,so}.

Changes

Follows the same pattern as :native_tls (#41):

  • lib/pythonx.ex — Adds :python to opts validation and documentation, threads it through to Uv.fetch/3 and Uv.init/3
  • lib/pythonx/uv.ex — Accepts :python in fetch/3 and init/3, passes --python <version> to uv sync when set, and includes the python value in the project_dir cache key MD5 so that different
    variants (e.g., 3.14 vs 3.14t) get separate cache directories

Usage

# Standard Python (unchanged)
Pythonx.uv_init("""
[project]
name = "project"
version = "0.0.0"
requires-python = "==3.14.*"
""")

# Free-threaded Python (opt-in)
Pythonx.uv_init("""
[project]
name = "project"
version = "0.0.0"
requires-python = "==3.14.*"
""", python: "3.14t")

Validation

Tested on macOS aarch64. uv downloaded cpython-3.14.0rc1+freethreaded-macos-aarch64-none and confirmed:

Python 3.14.0rc1 free-threading build | GIL disabled: 1

Parallelism benchmark

8 concurrent tasks running CPU-bound pure Python work:

Pythonx.uv_init(
  """
  [project]
  name = "test-free-threaded"
  version = "0.0.0"
  requires-python = "==3.14.*"
  """,
  python: "3.14t"
)

cpu_work = """
total = 0
for i in range(2_000_000):
    total += i
total
"""

# Sequential baseline
t0 = System.monotonic_time(:millisecond)

for _ <- 1..8 do
  Pythonx.eval(cpu_work, %{})
end

sequential_ms = System.monotonic_time(:millisecond) - t0
IO.puts("Sequential (8 tasks):  #{sequential_ms} ms")

# Concurrent
t0 = System.monotonic_time(:millisecond)

tasks =
  for _ <- 1..8 do
    Task.async(fn -> Pythonx.eval(cpu_work, %{}) end)
  end

Task.await_many(tasks, 60_000)
concurrent_ms = System.monotonic_time(:millisecond) - t0
IO.puts("Concurrent (8 tasks):  #{concurrent_ms} ms")

speedup = sequential_ms / max(concurrent_ms, 1)
IO.puts("Speedup:               #{Float.round(speedup, 2)}x")

Results:

┌────────────┬────────┐
│    Mode    │  Time  │
├────────────┼────────┤
│ Sequential │ 531 ms │
├────────────┼────────┤
│ Concurrent │ 124 ms │
├────────────┼────────┤
│ Speedup    │ 4.28x  │
└────────────┴────────┘

True parallelism confirmed — with standard Python this would be ~1.0x since the GIL serializes pure Python work.

Cache separation

Different :python values produce different cache directories:
- No option: c4mnqgs63q7e66whs4cuoeftui
- "3.14t": wkfkw7orrgqor6yjyu37kglmgu

Backwards compatibility

All 74 existing tests pass unchanged (8 doctests + 66 tests, 0 failures).

Caveats

- Not all Python packages support free-threading yet (tracker)
- Users must ensure their Python code is thread-safe when using free-threaded Python with concurrent Elixir callers
- ~5-10% single-thread overhead in free-threaded mode vs standard Python 3.14

Allow specifying the Python version passed to uv via --python, enabling
users to request free-threaded Python builds (e.g., "3.14t"). The python
option is included in the project cache key so different variants get
separate cache directories.
Copy link
Contributor

@josevalim josevalim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks simple and lightweight enough to me!

Copy link
Member

@jonatanklosko jonatanklosko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we would be able to specify it via pyproject.toml, but from what I checked that's not possible. That said, I think the option doesn't hurt and we can revisit as things change.

@nyo16
Copy link
Contributor Author

nyo16 commented Feb 16, 2026

@jonatanklosko yes I agree. I was trying to find the least resistance path while python ecosystem resolving it :)

@jonatanklosko jonatanklosko merged commit 36cf67e into livebook-dev:main Feb 16, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments