[Docs] Add a new page to help for first time contributors

[v01dXYZ]

Issue: No issue

Brief Summary

My first contribution was a little bit harder than expected. So I wrote down some pieces of advice for new contributors.

It is a WIP and you're welcome to edit it!

[hughperkins]

I don't agree with building LLVM oneself. It is very heavy and difficult. I never have to do this.

Please can you look at how our CI runs the build, and identify what is different between your current build process and the CI build process?

[hughperkins]

(meaning how Quadrants runs the build I mean)

[v01dXYZ]

Sorry to disagree on this but I don't think it is really hard to build LLVM as it is mainly cmake <option_to_use_ninja_or_make> -S llvm -B build -D<option_to_enable_targets or shared libs> && cmake --build build -jN. It takes some time then but it works well.

Why is it fine to document it? Sometimes, there could be some bugs in LLVM and it is great to have a version that could output the intermediate passes. or a debug version (that one is the really hard one as LLVM debug symbols are HUGE).

The CI build process uses download_llvm.py.

[hughperkins]

Sorry to disagree on this but I don't think it is really hard to build LLVM as it is mainly cmake <option_to_use_ninja_or_make> -S llvm -B build -D<option_to_enable_targets or shared libs> && cmake --build build -jN. It takes some time then but it works well.

Yes, it takes a long time to build.

Why is it fine to document it? Sometimes, there could be some bugs in LLVM and it is great to have a version that could output the intermediate passes. or a debug version (that one is the really hard one as LLVM debug symbols are HUGE).

It's documented by the CI at https://github.com/Genesis-Embodied-AI/quadrants-sdk-builds/blob/main/.github/workflows/llvm-ci.yml I feel.

I'm fine with linking to this in some "advanced, here be dragons" section. But not as a 'this is the standard build process' doc.

The CI build process uses download_llvm.py.

Yes. It should just download and work. I'd like a diagnosis into why this doesn't work for you first please, before considering any documentation beyond an 'advanced, here be dragons' type section, please.

[v01dXYZ]

:D I didn't say it doesn't work. I went straight to that as I wanted to understand why Instruction Selection failed.

But yes, it is for advanced users that want to find out why LLVM panics or if some passes could be better implemented.

Thinking again you're right, let's cut it off.

[hughperkins]

Oh, I see, you are covering multiple approaches:

• OS install of LLVM
• download_llvm.py
• build oneself

I actually missed that you were covering two other approaches, on initial skimming, to be honest.

Note that the build process as far as llvm and clang is a little bit confused currently, and could probably be cleaned up a little. We actually use three somewhat orthogonal things:

• some C/C++ compiler, e.g. 'msvc': for building C++ files
• clang/clang++ compiler: used to build the LLVM runtime bitcode
• LLVM library: linked with, at build time

The C/C++ compiler is pretty much always using a platform-native compiler, e.g. gcc, or msvc, currently. Which seems reasonable I feel. This is defined here

quadrants/.github/workflows/scripts/ti_build/entry.py

Line 43 in a346d2d

setup_clang(as_compiler=False)

The clang/clang++ compiler currently uses an OS-installed clang/clang++ on all platforms, I believe though I was pushing to use the downloaded clang/clang++ for a while. I saw advantages/disadvantages as:

good points of using packaged/downloaded clang/clang++:

• same version of clang/clang++ everywhere

downsides of using packaged/downloaded clang/clang++:

• inflates the size of the llvm library download (and the llvm library build)
• hard to get it to work with permissions on Mac

Lastly we have the LLVM libraries themselves, which are always downloaded, by default. They are linked with, when building Quadrants. This is the only bit which must be a specific version, and that is tightly coupled with the code, AFAIK.

Anyway... I'm not averse to having a section on building clang and/or llvm yourself, if you feel this is useful, but should clearly be marked as 'advanced here be dragons' (or similar) I feel.

It would probably be worth documenting clearly the difference between the three things above, somehow, somewhere?

[v01dXYZ]

My only intent is to debug the LLVM code that is called by the library. But what you said is worth mentioning about what is compiled.

To reformulate, quadrants being a compiler for running kernels, there are at least three pieces:

• quadrants cpp code: compiled with OS compiler
• LLVM used by quadrants: provided by download_llvm.py
• runtime bitcode (code that will be loaded in GPU memory to setup the kernel): compiled with clang as the compiler for that needs to target CUDA/AMDGPU and gcc/mvcc does not support them.

[hughperkins]

Are you sure this is needed? I never set this myself.

I simply do:

./build.py --shell
python setup.py develop

[v01dXYZ]

What does --shell? Does it activate a venv with all the deps?

[hughperkins]

it sets a bunch of env vars

there's also a version for AI. I don't pay close attention to what AI does, but it seems to be doing somethign like:

./build.py -w env.sh
source env.sh
python setup.py develop

[hughperkins]

Lets keep just the second sentence, to keep as concise as posible "There are three pieces that need to be compiled or are themselves part
of a compiler project.". Actually we can shorten further: "Quadrants comprises three parts:".

But actually, what about the python code that is part of Quadrants? One can argue it is included in the 'host runtime' part. But on the other hand, below it says that the 'host runtime' is compiled, usng a C/C++ compiler. Thoughts?

[hughperkins]

(I am wary of saying that Quadrants 'comprises 3 parts' to be honest. There are also .pyi stub files from pybind11, for example. Thoughts?)

[hughperkins]

Just to check, you are still addressing the comments right? Not expecting anything from me on this currently I think?

[hughperkins]

do you mean, when we run ./build.py --shell?

[v01dXYZ]

it is appended by setup.py before invoking cmake.

To me, it doesn't seem related to build.py --shell.

[hughperkins]

I've never run cmake directly to be honest. I've always done ./build.py --shell. I have no objection to making the build more and more standard, but I would like some explanation please of:

• why are you not simply using ./build.py --shell and python setup.py develop?
• how does calling cmake direclty relate to the above?

[v01dXYZ]

1. I didn't know build.py existed and I used at first pip as it is the way. But now I use build.py.
2. I don't call cmake directly as I have then to copy manually the shared object. But using the cmake options is required to disable some unwanted backends. It's not necessary though.

[v01dXYZ]

There is an env var that is read by setup.py to add cmake CLI args.

[hughperkins]

ther'es an env var that is read by build.py too I believe.

[v01dXYZ]

Yes, it is called CMakeArgsManager.

[hughperkins]

They're used by a lot more than final lowering. They're used throughout the entire c++ side, that receives the python side AST, via pybind11 api calls, transforms them to CHIIR, optimizes them, then gradually lowers them to LLVM IR.

[v01dXYZ]

I'll take a look at that to be more accurate.

[hughperkins]

This hasnt been addressed yet right?

[v01dXYZ]

Not yet.

[v01dXYZ]

sorry, but I don't understand what you said. You talk about a Chi-IR which is an intermediate representation. Could you talk a bit more about this, b/c there is no mention of this in the source code?

Once it is lowered into LLVM IR, do you have custom passes or do you rely only on the ones provided by LLVM?

[hughperkins]

custom passes.

Please consider asking Opus 4.6 to walk you trhough how the passes work, what passes there are, etc.

Personally I like Cursor CLI interface https://cursor.com/docs/cli/overview

Most people prefer Claude Code for reasons unclear to me.

[v01dXYZ]

Sorry to state it as plainly but I don't have the money to use Opus 4.6.

I grep-ed the code base to look at LLVM pass and I hardly found ones. I'm talking about LLVM passes not the ones related to higher IRs.

[hughperkins]

Ah, yes, I believe our custom passes are for our custom IR, right.

[v01dXYZ]

yes, indeed, it's in transforms.

[hughperkins]

Just to check, have you addressed all comments, and would like a fresh review?

Alternatively, once you believe you have addressed all comments, and would like a fresh review, please add a comment stating that.

[v01dXYZ]

I was trying to understand the PR about f32 atomics support with Vulkan/Metal, so I left this PR aside a little bit. I plan to take a look before the end of the day.

[hughperkins]

@claude review

[claude]

🔴 The contributing guide instructs new contributors to run ./tests/run_tests.py, but tests/run_tests.py is not marked executable (mode -rw-r--r--) and has no shebang line (it begins with import argparse). Following this verbatim from a fresh checkout will fail with "Permission denied". Either change the doc to python ./tests/run_tests.py (matching every CI invocation in the repo) or add a #!/usr/bin/env python3 shebang and chmod +x the script.

Extended reasoning...

The bug

The new contributing doc at docs/source/user_guide/contributing.md (line 5) tells contributors:

You have to run the test suite using ./tests/run_tests.py which sets up the right test environment for pytest.

The literal command ./tests/run_tests.py requires the script to be (a) marked executable and (b) have a shebang line so the kernel knows what interpreter to invoke. Neither is true for this repo.

Step-by-step proof

1. Fresh clone of the repo.
2. ls -l tests/run_tests.py shows mode -rw-r--r-- — not executable.
3. head -1 tests/run_tests.py shows import argparse — no shebang.
4. Run the doc's command verbatim: ./tests/run_tests.py.
5. Shell responds: bash: ./tests/run_tests.py: Permission denied.

Even if a user runs chmod +x tests/run_tests.py locally to work around step 5, the next attempt would fail with exec format error (or invoke the script under /bin/sh) because there is no shebang directing the kernel to use Python.

Why existing code doesn't prevent it

This is the very first instruction in the new contributing guide, aimed precisely at first-time contributors who have no other context. There is no surrounding text suggesting the user prepend python. Every other invocation in the repository — .github/workflows/scripts_new/linux/4_test.sh, .github/workflows/scripts_new/manylinux_wheel/5_test.sh, .github/workflows/scripts_new/macosx/4_test.sh, .github/workflows/scripts_new/win/3_test.ps1, .github/workflows/test_gpu.yml, and the docstrings in tests/test_utils.py — uses python tests/run_tests.py. So "run it as a script" was never the intended invocation.

Impact

Low technical impact (a seasoned developer figures it out in seconds), but disproportionately user-visible: this is the first instruction in a guide whose entire purpose is to lower the barrier for first-time contributors. A contributor copy-pasting the very first command and getting "Permission denied" is a poor first impression for a doc whose stated motivation in the PR description is "My first contribution was a little bit harder than expected".

Fix

Two equally valid options:

• Update the doc to match how CI invokes it: python ./tests/run_tests.py (or python tests/run_tests.py). Smallest change, consistent with all existing usage in the repo.
• Add #!/usr/bin/env python3 as the first line of tests/run_tests.py and chmod +x it (and commit the executable bit). Then the doc's command works as-is.

The first option is preferred since it is purely a docs change and matches established convention in the repo.

[claude]

🟡 Line 92 of contributing.md ends with a trailing space (after "compiler."), which will fail the trailing-whitespace pre-commit hook configured in .pre-commit-config.yaml (line 26). Ironically, line 7 of this very doc instructs new contributors to run pre-commit run -a before pushing — so the first thing a new contributor does after following the guide will be to auto-modify this file. Trim the trailing space at line 92.

Extended reasoning...

The issue

Line 92 of docs/source/user_guide/contributing.md reads:

1. `quadrants` host runtime: Made with a mix of Python and C++. The C++ core is compiled using the OS default C/C++ compiler.·

(where · denotes a trailing space). Grep with pattern $ confirms line 92 is the only line in the file that matches.

Why it matters here specifically

This isn't just a generic style nit — it's directly self-defeating in the context of this PR. Line 7 of the same file says:

format/linter: Before pushing any commits, ensure you set up pre-commit and run it using pre-commit run -a

The repo's .pre-commit-config.yaml (lines 23–27) enables the pre-commit-hooks v5.0.0 trailing-whitespace hook with no Markdown exclusion (the only exclude in that config targets tests/python/test_exception.py and external/). So a new contributor following this guide will, on their very first pre-commit run -a, have the contributing guide itself fail and get auto-rewritten — a poor first impression for the audience this guide is meant to help.

Step-by-step proof

1. A new contributor reads contributing.md line 7, installs pre-commit, and runs pre-commit run -a as instructed.
2. The trailing-whitespace hook from pre-commit-hooks v5.0.0 scans every tracked file (no Markdown is excluded).
3. It encounters line 92 of docs/source/user_guide/contributing.md and detects the trailing space.
4. The hook fails with a non-zero exit code and rewrites the file to strip the trailing space.
5. The contributor now sees docs/source/user_guide/contributing.md show up as "modified" in git status — a file they never touched, dirtied by following the guide's own instructions.

Fix

Delete the single trailing space at the end of line 92. One-character change.

[claude]

🟡 The example after "if you don't need to have any GPU support" only sets -DQD_WITH_CUDA=OFF -DQD_WITH_AMDGPU=OFF, but cmake/QuadrantsCore.cmake defines four GPU backends and QD_WITH_METAL defaults to ON (it's the only GPU backend enabled by default on macOS). On macOS, following this example still builds the Metal backend, contradicting the lead-in. Either add -DQD_WITH_METAL=OFF -DQD_WITH_VULKAN=OFF for completeness, or rephrase to something like "to disable the CUDA and AMDGPU backends".

Extended reasoning...

What the example claims vs. what it does

Lines 71-75 of docs/source/user_guide/contributing.md introduce the example with:

if you don't need to have any GPU support, you could use:

export QUADRANTS_CMAKE_ARGS="-DQD_WITH_CUDA=OFF -DQD_WITH_AMDGPU=OFF"

The claim is broad ("any GPU support"), but the command only touches two of the four GPU-backend options.

What the build system actually defines

cmake/QuadrantsCore.cmake lines 3-7 declare four GPU backend options:

option(QD_WITH_METAL  "Build with the Metal backend"  ON)   # default ON
option(QD_WITH_CUDA   "Build with the CUDA backend"   ON)   # default ON
option(QD_WITH_AMDGPU "Build with the AMDGPU backend" OFF)  # default OFF
option(QD_WITH_VULKAN "Build with the Vulkan backend" OFF)  # default OFF

Platform-conditional overrides at lines 32-50 then force CUDA/AMDGPU off on macOS and force METAL off when not on OS X. The net effect of the defaults per platform is:

Platform	METAL	CUDA	AMDGPU	VULKAN
Linux	OFF (forced)	ON	OFF	OFF
macOS	ON	OFF (forced)	OFF (forced)	OFF
Windows	OFF (forced)	ON	OFF (forced)	OFF

Step-by-step proof on macOS

1. User reads "if you don't need to have any GPU support" and runs export QUADRANTS_CMAKE_ARGS="-DQD_WITH_CUDA=OFF -DQD_WITH_AMDGPU=OFF".
2. CMake processes the option declarations: METAL=ON, CUDA=ON, AMDGPU=OFF, VULKAN=OFF.
3. The user-supplied -D flags set CUDA=OFF and AMDGPU=OFF (both are already effectively OFF on macOS via the lines 32-39 overrides anyway).
4. METAL is untouched and stays at its default ON. The OS X branch at lines 41-43 only turns METAL off on non-OS X — on macOS it leaves it ON.
5. Lines 94-95 then add -DQD_WITH_METAL to CMAKE_CXX_FLAGS, and line 205 pulls in the Metal backend sources. So the Metal backend is still compiled, contradicting the doc's "any GPU support" claim.

Why existing code does not save the user

There is no code path that infers user intent from CUDA+AMDGPU=OFF and disables METAL/VULKAN; each option is independent. So the only way to get a truly GPU-free build on macOS is to pass -DQD_WITH_METAL=OFF explicitly.

Impact and fix

Impact is low — wasted build time / unexpected backend on macOS, no runtime crash. Fix is a one-line doc edit; either:

• expand the example to -DQD_WITH_CUDA=OFF -DQD_WITH_AMDGPU=OFF -DQD_WITH_METAL=OFF -DQD_WITH_VULKAN=OFF (VULKAN is harmless since it defaults OFF, but including it makes the example self-documenting), or
• rephrase the lead-in to "to disable the CUDA and AMDGPU backends".

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[hughperkins]

Thank you! 🙌