Enable Mac pipeline by raymondxyang · Pull Request #9 · microsoft/onnxruntime

raymondxyang · 2018-11-20T07:01:42Z

Enable Mac build pipeline in CI.

See successful build log here: https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=27&view=logs

Create protobuf for wire contracts

### Description Release OrtEnv before main function returns. Before this change, OrtEnv is deleted when C/C++ runtime destructs all global variables in ONNX Runtime's core framework. The callstack is like this: ``` * frame #0: 0x00007fffee39f5a6 libonnxruntime.so.1.16.0`onnxruntime::Environment::~Environment(this=0x00007fffee39fbf2) at environment.h:20:7 frame #1: 0x00007fffee39f614 libonnxruntime.so.1.16.0`std::default_delete<onnxruntime::Environment>::operator()(this=0x00007ffff4c30e50, __ptr=0x0000000005404b00) const at unique_ptr.h:85:2 frame #2: 0x00007fffee39edca libonnxruntime.so.1.16.0`std::unique_ptr<onnxruntime::Environment, std::default_delete<onnxruntime::Environment>>::~unique_ptr(this=0x5404b00) at unique_ptr.h:361:17 frame #3: 0x00007fffee39e2ab libonnxruntime.so.1.16.0`OrtEnv::~OrtEnv(this=0x00007ffff4c30e50) at ort_env.cc:43:1 frame #4: 0x00007fffee39fa96 libonnxruntime.so.1.16.0`std::default_delete<OrtEnv>::operator()(this=0x00007fffefff8f78, __ptr=0x00007ffff4c30e50) const at unique_ptr.h:85:2 frame #5: 0x00007fffee39f394 libonnxruntime.so.1.16.0`std::unique_ptr<OrtEnv, std::default_delete<OrtEnv>>::~unique_ptr(this=0x7ffff4c30e50) at unique_ptr.h:361:17 frame #6: 0x00007ffff78574b5 libc.so.6`__run_exit_handlers + 261 frame #7: 0x00007ffff7857630 libc.so.6`exit + 32 frame #8: 0x00007ffff783feb7 libc.so.6`__libc_start_call_main + 135 frame #9: 0x00007ffff783ff60 libc.so.6`__libc_start_main@@GLIBC_2.34 + 128 frame #10: 0x0000000000abbdee node`_start + 46 ``` After this change, OrtEnv will be deleted before the main function returns and nodejs is still alive.

### Description Release OrtEnv before main function returns. Before this change, OrtEnv is deleted when C/C++ runtime destructs all global variables in ONNX Runtime's core framework. The callstack is like this: ``` * frame #0: 0x00007fffee39f5a6 libonnxruntime.so.1.16.0`onnxruntime::Environment::~Environment(this=0x00007fffee39fbf2) at environment.h:20:7 frame microsoft#1: 0x00007fffee39f614 libonnxruntime.so.1.16.0`std::default_delete<onnxruntime::Environment>::operator()(this=0x00007ffff4c30e50, __ptr=0x0000000005404b00) const at unique_ptr.h:85:2 frame microsoft#2: 0x00007fffee39edca libonnxruntime.so.1.16.0`std::unique_ptr<onnxruntime::Environment, std::default_delete<onnxruntime::Environment>>::~unique_ptr(this=0x5404b00) at unique_ptr.h:361:17 frame microsoft#3: 0x00007fffee39e2ab libonnxruntime.so.1.16.0`OrtEnv::~OrtEnv(this=0x00007ffff4c30e50) at ort_env.cc:43:1 frame microsoft#4: 0x00007fffee39fa96 libonnxruntime.so.1.16.0`std::default_delete<OrtEnv>::operator()(this=0x00007fffefff8f78, __ptr=0x00007ffff4c30e50) const at unique_ptr.h:85:2 frame microsoft#5: 0x00007fffee39f394 libonnxruntime.so.1.16.0`std::unique_ptr<OrtEnv, std::default_delete<OrtEnv>>::~unique_ptr(this=0x7ffff4c30e50) at unique_ptr.h:361:17 frame microsoft#6: 0x00007ffff78574b5 libc.so.6`__run_exit_handlers + 261 frame microsoft#7: 0x00007ffff7857630 libc.so.6`exit + 32 frame microsoft#8: 0x00007ffff783feb7 libc.so.6`__libc_start_call_main + 135 frame microsoft#9: 0x00007ffff783ff60 libc.so.6`__libc_start_main@@GLIBC_2.34 + 128 frame microsoft#10: 0x0000000000abbdee node`_start + 46 ``` After this change, OrtEnv will be deleted before the main function returns and nodejs is still alive.

Refine qlinearconv and added missed quantize ops

### Description Release OrtEnv before main function returns. Before this change, OrtEnv is deleted when C/C++ runtime destructs all global variables in ONNX Runtime's core framework. The callstack is like this: ``` * frame #0: 0x00007fffee39f5a6 libonnxruntime.so.1.16.0`onnxruntime::Environment::~Environment(this=0x00007fffee39fbf2) at environment.h:20:7 frame microsoft#1: 0x00007fffee39f614 libonnxruntime.so.1.16.0`std::default_delete<onnxruntime::Environment>::operator()(this=0x00007ffff4c30e50, __ptr=0x0000000005404b00) const at unique_ptr.h:85:2 frame microsoft#2: 0x00007fffee39edca libonnxruntime.so.1.16.0`std::unique_ptr<onnxruntime::Environment, std::default_delete<onnxruntime::Environment>>::~unique_ptr(this=0x5404b00) at unique_ptr.h:361:17 frame microsoft#3: 0x00007fffee39e2ab libonnxruntime.so.1.16.0`OrtEnv::~OrtEnv(this=0x00007ffff4c30e50) at ort_env.cc:43:1 frame microsoft#4: 0x00007fffee39fa96 libonnxruntime.so.1.16.0`std::default_delete<OrtEnv>::operator()(this=0x00007fffefff8f78, __ptr=0x00007ffff4c30e50) const at unique_ptr.h:85:2 frame microsoft#5: 0x00007fffee39f394 libonnxruntime.so.1.16.0`std::unique_ptr<OrtEnv, std::default_delete<OrtEnv>>::~unique_ptr(this=0x7ffff4c30e50) at unique_ptr.h:361:17 frame microsoft#6: 0x00007ffff78574b5 libc.so.6`__run_exit_handlers + 261 frame microsoft#7: 0x00007ffff7857630 libc.so.6`exit + 32 frame microsoft#8: 0x00007ffff783feb7 libc.so.6`__libc_start_call_main + 135 frame microsoft#9: 0x00007ffff783ff60 libc.so.6`__libc_start_main@@GLIBC_2.34 + 128 frame microsoft#10: 0x0000000000abbdee node`_start + 46 ``` After this change, OrtEnv will be deleted before the main function returns and nodejs is still alive.

… transient connection exceptions. (#21612) ### Description Improve docker commands to make docker image layer caching works. It can make docker building faster and more stable. So far, A100 pool's system disk is too small to use docker cache. We won't use pipeline cache for docker image and remove some legacy code. ### Motivation and Context There are often an exception of ``` 64.58 + curl https://nodejs.org/dist/v18.17.1/node-v18.17.1-linux-x64.tar.gz -sSL --retry 5 --retry-delay 30 --create-dirs -o /tmp/src/node-v18.17.1-linux-x64.tar.gz --fail 286.4 curl: (92) HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2) ``` Because Onnxruntime pipeline have been sending too many requests to download Nodejs in docker building. Which is the major reason of pipeline failing now In fact, docker image layer caching never works. We can always see the scrips are still running ``` #9 [3/5] RUN cd /tmp/scripts && /tmp/scripts/install_centos.sh && /tmp/scripts/install_deps.sh && rm -rf /tmp/scripts #9 0.234 /bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) #9 0.235 /bin/sh: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8) #9 0.235 /tmp/scripts/install_centos.sh: line 1: !/bin/bash: No such file or directory #9 0.235 ++ '[' '!' -f /etc/yum.repos.d/microsoft-prod.repo ']' #9 0.236 +++ tr -dc 0-9. #9 0.236 +++ cut -d . -f1 #9 0.238 ++ os_major_version=8 .... #9 60.41 + curl https://nodejs.org/dist/v18.17.1/node-v18.17.1-linux-x64.tar.gz -sSL --retry 5 --retry-delay 30 --create-dirs -o /tmp/src/node-v18.17.1-linux-x64.tar.gz --fail #9 60.59 + return 0 ... ``` This PR is improving the docker command to make image layer caching work. Thus, CI won't send so many redundant request of downloading NodeJS. ``` #9 [2/5] ADD scripts /tmp/scripts #9 CACHED #10 [3/5] RUN cd /tmp/scripts && /tmp/scripts/install_centos.sh && /tmp/scripts/install_deps.sh && rm -rf /tmp/scripts #10 CACHED #11 [4/5] RUN adduser --uid 1000 onnxruntimedev #11 CACHED #12 [5/5] WORKDIR /home/onnxruntimedev #12 CACHED ``` ###Reference https://docs.docker.com/build/drivers/ --------- Co-authored-by: Yi Zhang <your@email.com>

Clean up old APIs and options

…ntation - P2 #7: Tighten CPU fp32 test tolerance (no TF32 on CPU) - P2 #9: Add TODO documenting GQA kv_seq==q_seq limitation blocking TensorScatter external KV cache pattern - P2 #10: Document nonpad_kv_seqlen_data as device pointer on CUDA Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

) ### Description `import onnxruntime` segfaults during `dlopen` of `onnxruntime_pybind11_state.so` on Linux. This removes the global initializer in `onnxruntime_pybind_state.cc` that eagerly calls `Env::Default()`, and resolves the platform `Env` on first use at its two call sites instead. ### Motivation and Context The module has a namespace-scope dynamic initializer: ```cpp static Env& platform_env = Env::Default(); ``` Since POSIX telemetry landed (#27379), `Env::Default()` constructs `PosixEnv`, whose `PosixTelemetry` member initializes the 1DS SDK in its constructor. That path reads `defaultRuntimeConfig`, a namespace-scope `static ILogConfiguration` defined in the 1DS SDK's `RuntimeConfig_Default.hpp`, which lives in a **different translation unit of the same shared library**. Dynamic initialization order across translation units is unspecified, and the pybind TU's initializer runs first. `Variant::merge_map` therefore iterates a still zero-initialized `std::map`: `_M_node_count == 0`, but `_M_header._M_left` is `nullptr` rather than self-pointing, so `begin() != end()` and the loop dereferences null. Textbook static initialization order fiasco. Backtrace (Release build relinked without `--strip-all` to recover symbols): ``` #0 std::map<..., Variant>::lower_bound stl_map.h:1307 #1 std::map<..., Variant>::operator[] stl_map.h:509 #2 Variant::merge_map VariantType.hpp:508 #3 RuntimeConfig_Default::RuntimeConfig_Default RuntimeConfig_Default.hpp:97 #4 LogManagerImpl::LogManagerImpl LogManagerImpl.cpp:183 #5 LogManagerFactory::Create LogManagerFactory.cpp:36 #6 LogManagerFactory::lease #7 LogManagerFactory::Get LogManagerFactory.hpp:71 #8 LogManagerProvider::Get LogManagerProvider.cpp:16 #9 onnxruntime::PosixTelemetry::Initialize() #10 onnxruntime::PosixTelemetry::PosixTelemetry() #11 onnxruntime::(anonymous namespace)::PosixEnv::PosixEnv() #12 onnxruntime::Env::Default() #13 _GLOBAL__sub_I_onnxruntime_pybind_state.cc #14 call_init elf/dl-init.c:74 ... #21 _dl_open elf/dl-open.c:905 ``` The crash is independent of `LD_LIBRARY_PATH` and `CUDA_VISIBLE_DEVICES` — it happens before any ORT runtime code runs. `libonnxruntime.so` is unaffected because it contains no global initializer that reaches `Env::Default()`, which is why C/C++ and onnxruntime-genai consumers do not see it. Both remaining uses of `platform_env` are inside pybind lambdas that run long after load, so calling `Env::Default()` there is safe. This also removes the now-stale `TODO: we may delay-init this variable` and a pre-existing `#pragma warning(push)` that should have been `pop`. Note for follow-up: `Env::Default()` is now unsafe to call from any dynamic initializer. This was the only such call site in the tree, but hardening `PosixTelemetry` to defer SDK initialization out of its constructor would remove the hazard entirely. ### Tests Verified on a Linux CUDA 13 Release build (`onnxruntime_USE_TELEMETRY` enabled): - `dlopen` of `onnxruntime_pybind11_state.so` succeeds (previously SIGSEGV). - `import onnxruntime` reports the version and `['CUDAExecutionProvider', 'CPUExecutionProvider']`. - `onnxruntime.enable_telemetry_events()` / `disable_telemetry_events()` — the two call sites changed here — work. - CPU and CUDA inference sessions produce correct results. - Reproduced and verified with `LD_LIBRARY_PATH` unset and `CUDA_VISIBLE_DEVICES` empty.

Enable Mac pipeline

53ce3e7

raymondxyang requested review from fs-eire and pranavsharma November 20, 2018 07:01

pranavsharma approved these changes Nov 20, 2018

View reviewed changes

pranavsharma merged commit 0f96064 into master Nov 20, 2018

raymondxyang deleted the mac-ci branch November 20, 2018 07:42

yufenglee mentioned this pull request May 7, 2019

MKLDNN-DO Subgraph Optimization #966

Closed

tmccrmck pushed a commit to tmccrmck/onnxruntime that referenced this pull request Aug 28, 2019

Merge pull request microsoft#9 from NonStatic2014/bohu/protobuf

e18b5f7

Create protobuf for wire contracts

yueyihua mentioned this pull request Jan 9, 2020

Runtime eror before my code return #2804

Closed

rashedkoutayni mentioned this pull request Apr 1, 2020

Segmentation Fault when running on Raspberry Pi 3B+ #3296

Closed

Liujingxiu23 mentioned this pull request Jun 12, 2020

How to get the current provider used in C++ inference stage? #4140

Closed

Molaire mentioned this pull request Jan 16, 2021

Failing to build from Dockerfile.source #6368

Closed

matthill mentioned this pull request Apr 15, 2021

Memory leaks and valgrind errors when running with TensorRT #7286

Closed

chilo-ms mentioned this pull request Apr 22, 2021

TRT EP memory leak fix #7415

Merged

slackzh mentioned this pull request May 18, 2021

android app crash when use nnapi #7744

Closed

pvavercak mentioned this pull request May 19, 2021

TensorRT execution provider SEGFAULT #7757

Closed

sunhmy mentioned this pull request Aug 2, 2021

Failed to use self created stream for a new cuda session #8578

Closed

jun-lv-17 mentioned this pull request Sep 2, 2021

Crash with bus error on Android (arm-v7a) when creating InferenceSession with SSD model #4103

Closed

leejinxi mentioned this pull request Jan 17, 2022

Segmentation Fault when calling Provider Host CPU #9874

Closed

AliceSum mentioned this pull request Mar 27, 2022

[Lot of people have this issue] The model can not "really" do inference; Loading issue is solved. #10990

Closed

msftlincoln mentioned this pull request Jul 19, 2022

cpplint & Eager mode: refactor and add comments to empty_* functions, general lint cleanup in ort_aten #12238

Merged

adjhawar mentioned this pull request Nov 18, 2022

Help in running onnxruntime with SNPE as execution provider #13693

Closed

mszhanyi mentioned this pull request Jan 16, 2023

upgrade protobuf to 3.20.2 and onnx to 1.13 #14279

Merged

ns-wxin mentioned this pull request Jan 31, 2023

How to release a session properly? #11529

Closed

guschmue pushed a commit that referenced this pull request May 17, 2023

Support Gather and don't reshape scalar to [1] (#9)

7776f6d

lfch mentioned this pull request Oct 13, 2023

GPU Usage 100% #17942

Closed

justgo-now mentioned this pull request Dec 11, 2023

segmentation fault when do inference sometimes with onnxruntime-1.15.1 also can produce by onnxruntime-1.16.3 #18776

Closed

horror-proton mentioned this pull request Feb 21, 2024

A bug occurs when the program terminates #15174

Closed

hubin858130 mentioned this pull request Mar 4, 2024

[Mobile] #19753

Closed

tianleiwu mentioned this pull request Mar 20, 2024

Recognize NaN operands in Min and Max ops #19984

Merged

chenfeiyue-cfy pushed a commit to chenfeiyue-cfy/onnxruntime that referenced this pull request Apr 15, 2024

Merge pull request microsoft#9 from chenfeiyue-cfy/vsinpu

c9bf8d5

Refine qlinearconv and added missed quantize ops

sansrem mentioned this pull request Jun 11, 2024

ONNXRuntime 1.18 crashing with TensorRT EP when dealing with big inputs #21001

Closed

dxur mentioned this pull request Jul 8, 2024

[Build] Heap overflow caused by the onnx runtime #21283

Open

sai-fpp mentioned this pull request Sep 6, 2024

onnxruntime c++ logging crash #22013

Closed

frenetj mentioned this pull request Oct 15, 2024

Crash running inference on a specific ONNX model #22448

Closed

FaithyQi mentioned this pull request Oct 22, 2024

[Mobile] null pointer dereference #22538

Closed

Consulting4J mentioned this pull request Feb 20, 2025

[Mobile] [urgent] iOS application crash at CreateEnv (pointer being freed was not allocated) #23759

Closed

pl121lp mentioned this pull request Feb 26, 2025

Selecting XNNPACK as execution provider for Android following the documentation example results in program termination #23826

Closed

yuanyao-nv mentioned this pull request Mar 9, 2025

What's the right way to construct custom ops with the same name but different output types? #23891

Closed

gedoensmax pushed a commit to gedoensmax/onnxruntime that referenced this pull request Apr 22, 2025

Merge pull request microsoft#9 from ankan-ban/options-clean-up

7dfa562

Clean up old APIs and options

hui-li-xf mentioned this pull request Jun 9, 2025

[Mobile] null pointer dereference: SIGSEGV #24991

Closed

rainysiu mentioned this pull request Jul 5, 2025

[Build] /install-utils.js Error: Failed to download build list. HTTP status code = 302 #25293

Closed

laurenspriem mentioned this pull request Nov 28, 2025

[Mobile] Crash on OnePlus15 devices #26678

Closed

aarifzafar1 mentioned this pull request Dec 9, 2025

[Mobile] #26760

Closed

pawanake2333 mentioned this pull request Jan 6, 2026

[Mobile] app crashed in matmul on snapdragon 8gen5, with cpu provider #26921

Closed

titaiwangms mentioned this pull request Feb 28, 2026

Support nonpad_kv_seqlen and external kv cache in-place update in Attention-24 (CUDA) #27486

Closed

tianleiwu mentioned this pull request May 9, 2026

Add FLOAT8E8M0 data type support in ONNX Runtime #28381

Merged

justinchuby mentioned this pull request Jun 3, 2026

QMoE: prepack int4/int8 expert weights in PrePack hook (symmetric with MatMulNBits) #28749

Merged

KilLogic98 mentioned this pull request Jun 15, 2026

C API memory leak Asan #29049

Open

jiangzhuo mentioned this pull request Jul 3, 2026

GroupQueryAttentionFusion produces an invalid graph when a GQA node has more than 9 inputs (e.g. attention_bias) — "sum of input arg count is not equal to size of input defs" #29524

Open

Copilot AI mentioned this pull request Jul 5, 2026

Skip GroupQueryAttentionFusion when GQA node has inputs beyond sin_cache #29552

Draft

hariharans29 mentioned this pull request Jul 16, 2026

[MLAS][KleidiAI] Add MLAS Arm64 half GEMM and convolution support #28786

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable Mac pipeline - #9

Enable Mac pipeline#9
pranavsharma merged 1 commit into
masterfrom
mac-ci

raymondxyang commented Nov 20, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

raymondxyang commented Nov 20, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants