using PyPlot hangs/kills julia 1.8.5 or 1.9.0 on MacOS

[deszoeke]

using PyPlot hangs or kills julia 1.8.5 or 1.9.0 on MacOS

Synopsis

using PyPlot hangs julia. No progress for >10 hours. No error message. User ^C exits julia with Killed: 9.

With repeated testing, behavior has evolved to crash immediately with Killed: 9, where it had hung.

Demonstration

Carefully recursively force-removed the conda directory $HOME/.julia/conda, which seems to be shared between julia 1.8 and 1.9.

Start julia, making sure dynamic dependencies are consistent with the conda python installation.

LD_LIBRARY_PATH="$HOME/.julia/conda/3/lib" /Applications/Julia-1.8.app/Contents/Resources/julia/bin/julia

or

LD_LIBRARY_PATH="$HOME/.julia/conda/3/lib" /Applications/Julia-1.9.app/Contents/Resources/julia/bin/julia

Make sure julia is using the conda python distribution and build PyCall.

ENV["PYTHON"]=""
]build PyCall
    Building Conda ─→ `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/e32a90da027ca45d84678b826fffd3110bb3fc90/build.log`
    Building PyCall → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/62f417f6ad727987c755549e9cd88c46578da562/build.log`
...

Quit and restart julia, then run

using PyPlot

Since we started by wiping ~/.julia/conda, matplotlib is not installed. Conda is automatically deployed to install it:

[ Info: Installing matplotlib via the Conda matplotlib package...
[ Info: Running `conda install -y matplotlib` in root environment
...
Executing transaction: done

It hangs there.

Kill julia with ^C -> Killed: 9.

Try again now that matplotlib is installed:

LD_LIBRARY_PATH="$HOME/.julia/conda/3/lib" /Applications/Julia-1.8.app/Contents/Resources/julia/bin/julia
ENV["PYTHON"]=""
using PyPlot

Result: It just hangs immediately. So, I think probably conda installs matplotlib (maybe OK?), but julia is unable to use it.

I have to kill julia with ^C -> Killed: 9.

Trying again.

LD_LIBRARY_PATH="$HOME/.julia/conda/3/lib" /Applications/Julia-1.8.app/Contents/Resources/julia/bin/julia
ENV["PYTHON"]=""
using Conda  # returns
using PyCall  # returns
using PyPlot  # crashes
Killed: 9

Crashes and exits.

Even wiping .julia/conda again, it just crashes now with Killed: 9

Background

PyPlot had previously been working for me.
The first sign of trouble was an MKL dylib conflict error message and crash when running using PyPlot.
Unfortunately after many ^Cs I can't find that error message.

I updated julia's conda after that. Then I got the behavior reported above.

[stevengj]

Works for me… but I haven't updated my conda packages recently, maybe there is a problem with the latest anaconda release? You could try pinning numpy or matplotlib to an older version…

[deszoeke]

I recently installed mamba (outside julia), and it appears julia PyCall is using it:

using PyCall
os = pyimport("os")
os.environ
 ... , 'CONDA_PYTHON_EXE': '$HOME/mambaforge/bin/python'

I had installed anaconda on my system, but the conda environment became unsolvable, so I switched to mamba. I didn't expect this to affect julia. I had thought the conda in julia was a separate installation (I use ENV["PYTHON"]="" before building PyCall), but it appears mine got connected to mamba.

[kevinpolisano]

On Mac OS 14.0, using PyPlot leads to:

julia> using PyPlot
[1]    13395 killed     /Applications/Julia-1.9.app/Contents/Resources/julia/bin/julia

Saving session...completed.

[Opération terminée]

julia> versioninfo()
Julia Version 1.9.4
Commit 8e5136fa297 (2023-11-14 08:46 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (x86_64-apple-darwin22.4.0)
  CPU: 16 × Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
  Threads: 1 on 16 virtual cores

[stevengj]

Works fine for me on MacOS 14 and Julia 1.9.4 … maybe your Python installation is broken?

If you are using Python via Conda.jl (which is the default on MacOS), you could try removing conda (rm -rf ~/.julia/conda) and rebuilding PyCall and PyPlot.