Describe the bug
Running vdk dag data job and the job fails the output summary is very hard to read
Here is an example https://gist.github.com/antoniivanov/783577b7e1dc237495ef18a9e9ebde47
The surround block can be disabled using VDK_LOG_EXCEPTION_FORMATTER=plain
https://gist.github.com/antoniivanov/9d54761ef0783d6c65853f930ec9df22
THey still can use improvement
Steps To Reproduce
Easiest way you can reuse rag-dag-pipeline dag job
sset
export DAGS_JOB_EXECUTOR_TYPE=local
to run it locally
Expected behavior
The error logs are well-formatted and easier to read the error.
Version (please complete the following information):
Versatile Data Kit (VDK)
Version: 0.3.1177416838
Build details: RELEASE_VERSION=0.3.1177416838, BUILD_DATE=Thu Feb 15 07:57:26 UTC 2024, BUILD_MACHINE_INFO=Linux runner-trzkizxz-project-28359933-concurrent-0rvcnn 5.4.235-144.344.amzn2.x86_64 #1 SMP Sun Mar 12 12:50:22 UTC 2023 x86_64 GNU/Linux, GITLAB_CI_JOB_ID=6177494231, GIT_COMMIT_SHA=a4c5a23b69ed2bd135b7f14658abd18c143e8759, GIT_BRANCH=main
Python version: 3.10.3 64bit (/Users/aivanov/.pyenv/versions/3.10.3/envs/vdk-oss-310/bin/python3.10)
Installed plugins:
vdk-logging-format (from package vdk-logging-format, version 0.1.948436673)
vdk-control-service-properties (from package vdk-plugin-control-cli, version 0.1.948436673)
vdk-execution-skip (from package vdk-plugin-control-cli, version 0.1.948436673)
vdk-plugin-control-cli (from package vdk-plugin-control-cli, version 0.1.948436673)
vdk-ingest-file (from package vdk-ingest-file, version 0.1.948436673)
vdk-server (from package vdk-server, version 0.1.1174992336)
vdk-ingest-http (from package vdk-ingest-http, version 0.2.1156222304)
vdk-postgres (from package vdk-postgres, version 0.0.1179665895)
vdk-sqlite (from package vdk-sqlite, version 0.1.1156222304)
dag (from package vdk-dag, version 0.1.1171967383)
vdk-structlog (from package vdk-structlog, version 0.1.1179037110)
vdk-poc-anonymize-plugin (from package vdk-poc-anonymize, version 0.1)
Describe the bug
Running vdk dag data job and the job fails the output summary is very hard to read
Here is an example https://gist.github.com/antoniivanov/783577b7e1dc237495ef18a9e9ebde47
The surround block can be disabled using
VDK_LOG_EXCEPTION_FORMATTER=plainhttps://gist.github.com/antoniivanov/9d54761ef0783d6c65853f930ec9df22
THey still can use improvement
Steps To Reproduce
Easiest way you can reuse rag-dag-pipeline dag job
sset
to run it locally
Expected behavior
The error logs are well-formatted and easier to read the error.
Version (please complete the following information):