Disabled unit Tests for CPU_FP32 and GPU_FP32 - #10
Merged
Conversation
suryasidd
self-requested a review
March 9, 2020 20:24
suryasidd
approved these changes
Mar 9, 2020
sfatimar
pushed a commit
that referenced
this pull request
Aug 28, 2023
### Description
Release OrtEnv before main function returns. Before this change, OrtEnv
is deleted when C/C++ runtime destructs all global variables in ONNX
Runtime's core framework.
The callstack is like this:
```
* frame #0: 0x00007fffee39f5a6 libonnxruntime.so.1.16.0`onnxruntime::Environment::~Environment(this=0x00007fffee39fbf2) at environment.h:20:7
frame #1: 0x00007fffee39f614 libonnxruntime.so.1.16.0`std::default_delete<onnxruntime::Environment>::operator()(this=0x00007ffff4c30e50, __ptr=0x0000000005404b00) const at unique_ptr.h:85:2
frame #2: 0x00007fffee39edca libonnxruntime.so.1.16.0`std::unique_ptr<onnxruntime::Environment, std::default_delete<onnxruntime::Environment>>::~unique_ptr(this=0x5404b00) at unique_ptr.h:361:17
frame #3: 0x00007fffee39e2ab libonnxruntime.so.1.16.0`OrtEnv::~OrtEnv(this=0x00007ffff4c30e50) at ort_env.cc:43:1
frame #4: 0x00007fffee39fa96 libonnxruntime.so.1.16.0`std::default_delete<OrtEnv>::operator()(this=0x00007fffefff8f78, __ptr=0x00007ffff4c30e50) const at unique_ptr.h:85:2
frame #5: 0x00007fffee39f394 libonnxruntime.so.1.16.0`std::unique_ptr<OrtEnv, std::default_delete<OrtEnv>>::~unique_ptr(this=0x7ffff4c30e50) at unique_ptr.h:361:17
frame #6: 0x00007ffff78574b5 libc.so.6`__run_exit_handlers + 261
frame #7: 0x00007ffff7857630 libc.so.6`exit + 32
frame #8: 0x00007ffff783feb7 libc.so.6`__libc_start_call_main + 135
frame #9: 0x00007ffff783ff60 libc.so.6`__libc_start_main@@GLIBC_2.34 + 128
frame #10: 0x0000000000abbdee node`_start + 46
```
After this change, OrtEnv will be deleted before the main function
returns and nodejs is still alive.
sfatimar
pushed a commit
that referenced
this pull request
Aug 8, 2024
… transient connection exceptions. (microsoft#21612) ### Description Improve docker commands to make docker image layer caching works. It can make docker building faster and more stable. So far, A100 pool's system disk is too small to use docker cache. We won't use pipeline cache for docker image and remove some legacy code. ### Motivation and Context There are often an exception of ``` 64.58 + curl https://nodejs.org/dist/v18.17.1/node-v18.17.1-linux-x64.tar.gz -sSL --retry 5 --retry-delay 30 --create-dirs -o /tmp/src/node-v18.17.1-linux-x64.tar.gz --fail 286.4 curl: (92) HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2) ``` Because Onnxruntime pipeline have been sending too many requests to download Nodejs in docker building. Which is the major reason of pipeline failing now In fact, docker image layer caching never works. We can always see the scrips are still running ``` #9 [3/5] RUN cd /tmp/scripts && /tmp/scripts/install_centos.sh && /tmp/scripts/install_deps.sh && rm -rf /tmp/scripts #9 0.234 /bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) #9 0.235 /bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) #9 0.235 /tmp/scripts/install_centos.sh: line 1: !/bin/bash: No such file or directory #9 0.235 ++ '[' '!' -f /etc/yum.repos.d/microsoft-prod.repo ']' #9 0.236 +++ tr -dc 0-9. #9 0.236 +++ cut -d . -f1 #9 0.238 ++ os_major_version=8 .... #9 60.41 + curl https://nodejs.org/dist/v18.17.1/node-v18.17.1-linux-x64.tar.gz -sSL --retry 5 --retry-delay 30 --create-dirs -o /tmp/src/node-v18.17.1-linux-x64.tar.gz --fail #9 60.59 + return 0 ... ``` This PR is improving the docker command to make image layer caching work. Thus, CI won't send so many redundant request of downloading NodeJS. ``` #9 [2/5] ADD scripts /tmp/scripts #9 CACHED #10 [3/5] RUN cd /tmp/scripts && /tmp/scripts/install_centos.sh && /tmp/scripts/install_deps.sh && rm -rf /tmp/scripts #10 CACHED #11 [4/5] RUN adduser --uid 1000 onnxruntimedev #11 CACHED #12 [5/5] WORKDIR /home/onnxruntimedev #12 CACHED ``` ###Reference https://docs.docker.com/build/drivers/ --------- Co-authored-by: Yi Zhang <your@email.com>
ai-fw-intg
pushed a commit
that referenced
this pull request
Mar 23, 2026
…icrosoft#27091) ### Description This PR fixes test errors encountered during the build and compilation of onnxruntime-cann. error1： Unrecognized because HiSilicon CPU info was not added, leading to CTest errors. ``` onnxruntime cpuid_info warning: Unknown CPU vendor. cpuinfo_vendor value: 15 10: [----------] Global test environment tear-down 10: [==========] 15 tests from 1 test suite ran. (772 ms total) 10: [ PASSED ] 15 tests. 10/10 Test #10: onnxruntime_ep_graph_test ............... Passed 0.98 sec 90% tests passed, 1 tests failed out of 10 Total Test time (real) = 674.68 sec The following tests FAILED: 1 - onnxruntime_test_all (Failed) Errors while running CTest ``` error2: Some Python tests are failing here due to a previously submitted PR (microsoft#25867). In that PR, we introduced a new parameter enable_cann_subgraph to control subgraph partitioning for unsupported operators, with a preference for executing the entire graph as a whole. However, this change causes certain test cases to fail when specific operator versions in the models are not supported, leading to execution errors.
jatinwadhwa921
pushed a commit
that referenced
this pull request
Jul 8, 2026
…path (microsoft#29525) ### Description `com.microsoft.GroupQueryAttention`'s optional `attention_bias` input (input #10) is implemented by the CPU EP (microsoft#23944) and the WebGPU EP (microsoft#25285, microsoft#26769), but the CUDA EP rejects it at runtime. This PR wires it through. No new kernel is needed: the unfused GQA fallback added for microsoft#28195 calls `LaunchUnfusedAttention`, whose kernel already implements an additive bias with dim-0/dim-1 broadcast, per-batch `seqlens_k`, causal/sliding-window masking and softcap — the op just passed `attn_bias=nullptr`. The change is dispatch plumbing: - **`group_query_attention.cc`** — remove the blanket rejection; validate the bias element type; set `broadcast_attn_bias_dim_0/1` from the bias shape (the fields already exist on `AttentionParameters`); add `!has_attention_bias` to the XQA / cuDNN SDPA / flash / flash-fast-decode / MEA eligibility so bias-carrying nodes dispatch to the unfused fallback; set `data.attention_bias`. - **`attention_data.h`** — add the `attention_bias` pointer to `GroupQueryAttentionData`. - **`group_query_attention_impl.cu`** — pass the real pointer and broadcast flags in `UnfusedGqaAttention` (previously hardcoded `nullptr`/`false`). Why each fused path stays disqualified with a bias: - flash / flash fast-decode: `flash_api.h` has no bias parameter (same exclusion as MHA and the ONNX `Attention`-op CUDA kernel). - XQA: no bias parameter. - cuDNN SDPA: GQA's cuDNN path is bottom-right causal, which cuDNN documents as incompatible with a bias (`multihead_attention.cc` has the same restriction). - cutlass MEA: the wrapper computes the bias row stride from `kv_sequence_length`, which GQA sets to the KV-cache capacity (`seqlen_present_kv_cache`) rather than `total_sequence_length`, so rows would be misaligned under past/present buffer sharing. Left for a follow-up (needs an explicit `attn_bias_strideM` in `MemoryEfficientAttentionParams`). Kept `NOT_IMPLEMENTED` (explicit, clear errors instead of the previous blanket rejection): bias × quantized KV cache (unfused requires `T == U`), bias × smooth-softmax/head_sink. ### Tests New `TestGQAAttentionBias` in `test_gqa.py`: prompt and past/decode parity across packed/unpacked QKV, shared/separate KV buffer, rotary, odd head sizes (40/80), and a subsequent multi-token prompt. The bias is **non-zero random** so a kernel that silently ignores the input fails parity (the harness previously modeled a zeros bias). Also fixes the test graph builder declaring the bias input's last dim as the KV-cache capacity instead of `total_sequence_length` (the shape the op validates). Full `test_gqa.py` suite: 476 tests pass, no regressions (SM89, CUDA 12.8). ### Motivation and Context Fixes microsoft#29506. Transformers.js-exported speech models carry non-causal attention patterns as `attention_bias` and currently cannot run on the CUDA EP at all, while running fine on WebGPU and CPU: - [`onnx-community/Voxtral-Mini-4B-Realtime-2602-ONNX`](https://huggingface.co/onnx-community/Voxtral-Mini-4B-Realtime-2602-ONNX) (streaming ASR) - [`onnx-community/cohere-transcribe-03-2026-ONNX`](https://huggingface.co/onnx-community/cohere-transcribe-03-2026-ONNX) End-to-end validation with this patch on an RTX 4070 SUPER: the full Voxtral-Mini-4B-Realtime streaming pipeline (q4f16) transcribes correctly at **RTF 0.23** (31 s clip, 25.6 tok/s sustained decode, 6.4 GB VRAM) — on this workload the unfused-attention path outperforms the same model on the WebGPU EP (RTF 0.26). (While validating, an unrelated pre-existing issue surfaced: `GroupQueryAttentionFusion` breaks graphs whose GQA nodes carry >9 inputs — filed as microsoft#29524.) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
hdharpure9922
pushed a commit
that referenced
this pull request
Jul 28, 2026
…rosoft#29880) ### Description `import onnxruntime` segfaults during `dlopen` of `onnxruntime_pybind11_state.so` on Linux. This removes the global initializer in `onnxruntime_pybind_state.cc` that eagerly calls `Env::Default()`, and resolves the platform `Env` on first use at its two call sites instead. ### Motivation and Context The module has a namespace-scope dynamic initializer: ```cpp static Env& platform_env = Env::Default(); ``` Since POSIX telemetry landed (microsoft#27379), `Env::Default()` constructs `PosixEnv`, whose `PosixTelemetry` member initializes the 1DS SDK in its constructor. That path reads `defaultRuntimeConfig`, a namespace-scope `static ILogConfiguration` defined in the 1DS SDK's `RuntimeConfig_Default.hpp`, which lives in a **different translation unit of the same shared library**. Dynamic initialization order across translation units is unspecified, and the pybind TU's initializer runs first. `Variant::merge_map` therefore iterates a still zero-initialized `std::map`: `_M_node_count == 0`, but `_M_header._M_left` is `nullptr` rather than self-pointing, so `begin() != end()` and the loop dereferences null. Textbook static initialization order fiasco. Backtrace (Release build relinked without `--strip-all` to recover symbols): ``` #0 std::map<..., Variant>::lower_bound stl_map.h:1307 #1 std::map<..., Variant>::operator[] stl_map.h:509 #2 Variant::merge_map VariantType.hpp:508 #3 RuntimeConfig_Default::RuntimeConfig_Default RuntimeConfig_Default.hpp:97 #4 LogManagerImpl::LogManagerImpl LogManagerImpl.cpp:183 #5 LogManagerFactory::Create LogManagerFactory.cpp:36 #6 LogManagerFactory::lease #7 LogManagerFactory::Get LogManagerFactory.hpp:71 #8 LogManagerProvider::Get LogManagerProvider.cpp:16 #9 onnxruntime::PosixTelemetry::Initialize() #10 onnxruntime::PosixTelemetry::PosixTelemetry() #11 onnxruntime::(anonymous namespace)::PosixEnv::PosixEnv() #12 onnxruntime::Env::Default() #13 _GLOBAL__sub_I_onnxruntime_pybind_state.cc #14 call_init elf/dl-init.c:74 ... #21 _dl_open elf/dl-open.c:905 ``` The crash is independent of `LD_LIBRARY_PATH` and `CUDA_VISIBLE_DEVICES` — it happens before any ORT runtime code runs. `libonnxruntime.so` is unaffected because it contains no global initializer that reaches `Env::Default()`, which is why C/C++ and onnxruntime-genai consumers do not see it. Both remaining uses of `platform_env` are inside pybind lambdas that run long after load, so calling `Env::Default()` there is safe. This also removes the now-stale `TODO: we may delay-init this variable` and a pre-existing `#pragma warning(push)` that should have been `pop`. Note for follow-up: `Env::Default()` is now unsafe to call from any dynamic initializer. This was the only such call site in the tree, but hardening `PosixTelemetry` to defer SDK initialization out of its constructor would remove the hazard entirely. ### Tests Verified on a Linux CUDA 13 Release build (`onnxruntime_USE_TELEMETRY` enabled): - `dlopen` of `onnxruntime_pybind11_state.so` succeeds (previously SIGSEGV). - `import onnxruntime` reports the version and `['CUDAExecutionProvider', 'CPUExecutionProvider']`. - `onnxruntime.enable_telemetry_events()` / `disable_telemetry_events()` — the two call sites changed here — work. - CPU and CUDA inference sessions produce correct results. - Reproduced and verified with `LD_LIBRARY_PATH` unset and `CUDA_VISIBLE_DEVICES` empty.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixed tests for CPUFP32 and GPUFP32