← 返回首页
vdk-structlog: put vdk init logs config behind flag by DeltaMichael · Pull Request #3107 · vmware/versatile-data-kit · GitHub
Skip to content

Navigation Menu

Toggle navigation
Sign in
Appearance settings
Search or jump to...

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted
Cancel Submit feedback

Saved searches

Use saved searches to filter your results more quickly

Cancel Create saved search
Appearance settings
Resetting focus

vdk-structlog: put vdk init logs config behind flag#3107

Merged
DeltaMichael merged 1 commit into
mainfrom
person/mdilyan/structlog-vdk-init
Feb 16, 2024
Merged

vdk-structlog: put vdk init logs config behind flag#3107
DeltaMichael merged 1 commit into
mainfrom
person/mdilyan/structlog-vdk-init

Conversation

DeltaMichael commented Feb 13, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Why?

Configuring the logs in the vdk_initialize hook causes commands like vdk server and vdk info to have the same verbose logging format as data job logs.

What?

  • Put the log configuration for vdk_initialize behind a config option and disable it by default for local runs and enable it by default for cloud runs. Configuring the logging format in vdk_initialize might still be useful for cloud runs and we don't care about other vdk commands output in cloud environments

  • Move config classes and functions into separate file

How was this tested

(venv) 11:01:14 ~ $ export VDK_STRUCTLOG_FORMAT_INIT_LOGS=True (venv) 11:01:19 ~ $ vdk create 2024-02-15 11:01:21,922 [VDK] [INFO ] vdk.internal.builtin_plugins.b builtin_hook_impl.py :98 vdk_initialize - Setting: OP_ID: 14b52d54-af7d-4eab-a0a5-05b50d534aed-1707987681, ATTEMPT_ID: 14b52d54-af7d-4eab-a0a5-05b50d534aed-1707987681-aa061, EXECUTION_ID: 14b52d54-af7d-4eab-a0a5-05b50d534aed-1707987681 2024-02-15 11:01:21,923 [VDK] [INFO ] vdk.internal.cli_entry cli_entry.py :135 vdk_main - Start CLI vdk with args ['create'] Job Name: (venv) 11:01:52 ~ $ export VDK_STRUCTLOG_FORMAT_INIT_LOGS=False (venv) 11:01:58 ~ $ vdk create Job Name:
(venv) 11:03:25 ~ $ export VDK_STRUCTLOG_FORMAT_INIT_LOGS=True (venv) 11:03:27 ~ $ vdk version 2024-02-15 11:03:30,267 [VDK] [INFO ] vdk.internal.builtin_plugins.b builtin_hook_impl.py :98 vdk_initialize - Setting: OP_ID: 907e2fef-40c7-45c1-a492-f1cda0717748-1707987810, ATTEMPT_ID: 907e2fef-40c7-45c1-a492-f1cda0717748-1707987810-3568d, EXECUTION_ID: 907e2fef-40c7-45c1-a492-f1cda0717748-1707987810 2024-02-15 11:03:30,267 [VDK] [INFO ] vdk.internal.cli_entry cli_entry.py :135 vdk_main - Start CLI vdk with args ['version'] 2024-02-15 11:03:30,292 [VDK] [INFO ] vdk.internal.builtin_plugins.v version.py :85 version - Versatile Data Kit (VDK) Version: 0.3.1177416838 Build details: RELEASE_VERSION=0.3.1177416838, BUILD_DATE=Thu Feb 15 07:57:26 UTC 2024, BUILD_MACHINE_INFO=Linux runner-trzkizxz-project-28359933-concurrent-0rvcnn 5.4.235-144.344.amzn2.x86_64 #1 SMP Sun Mar 12 12:50:22 UTC 2023 x86_64 GNU/Linux, GITLAB_CI_JOB_ID=6177494231, GIT_COMMIT_SHA=a4c5a23b69ed2bd135b7f14658abd18c143e8759, GIT_BRANCH=main Python version: 3.11.7 64bit (/Users/mdilyan/Projects/versatile-data-kit/projects/vdk-plugins/vdk-structlog/venv/bin/python) Installed plugins: vdk-ingest-http (from package vdk-ingest-http, version 0.2.1156222304) vdk-sqlite (from package vdk-sqlite, version 0.1.1156222304) vdk-server (from package vdk-server, version 0.1.1174992336) vdk-structlog (from package vdk-structlog, version 0.1.0) vdk-control-service-properties (from package vdk-plugin-control-cli, version 0.1.948436673) vdk-execution-skip (from package vdk-plugin-control-cli, version 0.1.948436673) vdk-plugin-control-cli (from package vdk-plugin-control-cli, version 0.1.948436673) vdk-ingest-file (from package vdk-ingest-file, version 0.1.948436673) -------------------------------------------------------------------------------- (venv) 11:04:04 ~ $ export VDK_STRUCTLOG_FORMAT_INIT_LOGS=False (venv) 11:04:09 ~ $ vdk version Versatile Data Kit (VDK) Version: 0.3.1177416838 Build details: RELEASE_VERSION=0.3.1177416838, BUILD_DATE=Thu Feb 15 07:57:26 UTC 2024, BUILD_MACHINE_INFO=Linux runner-trzkizxz-project-28359933-concurrent-0rvcnn 5.4.235-144.344.amzn2.x86_64 #1 SMP Sun Mar 12 12:50:22 UTC 2023 x86_64 GNU/Linux, GITLAB_CI_JOB_ID=6177494231, GIT_COMMIT_SHA=a4c5a23b69ed2bd135b7f14658abd18c143e8759, GIT_BRANCH=main Python version: 3.11.7 64bit (/Users/mdilyan/Projects/versatile-data-kit/projects/vdk-plugins/vdk-structlog/venv/bin/python) Installed plugins: vdk-ingest-http (from package vdk-ingest-http, version 0.2.1156222304) vdk-sqlite (from package vdk-sqlite, version 0.1.1156222304) vdk-server (from package vdk-server, version 0.1.1174992336) vdk-structlog (from package vdk-structlog, version 0.1.0) vdk-control-service-properties (from package vdk-plugin-control-cli, version 0.1.948436673) vdk-execution-skip (from package vdk-plugin-control-cli, version 0.1.948436673) vdk-plugin-control-cli (from package vdk-plugin-control-cli, version 0.1.948436673) vdk-ingest-file (from package vdk-ingest-file, version 0.1.948436673) --------------------------------------------------------------------------------

CI/CD

What kind of change is this?

Bugfix

DeltaMichael linked an issue Feb 13, 2024 that may be closed by this pull request
non-run CLI commands log output without context #3105
Closed
DeltaMichael force-pushed the person/mdilyan/structlog-vdk-init branch from f7dcec6 to 3940f40 Compare February 13, 2024 12:28
antoniivanov reviewed Feb 13, 2024
View reviewed changes
Comment thread projects/vdk-plugins/vdk-structlog/README.md Show resolved Hide resolved
DeltaMichael force-pushed the person/mdilyan/structlog-vdk-init branch 5 times, most recently from aaf50db to 11063df Compare February 14, 2024 14:34

antoniivanov commented Feb 15, 2024

Copy link
Copy Markdown
Contributor

Please expand on the

How was this tested Manually

What were the commands and their output and how it was verified. I always discourage saying "manually" without explanation in testing done.

It's good idea to verify it works well with a few commands . E.g vdk list, vdk create, vdk depoy, vdk version

DeltaMichael force-pushed the person/mdilyan/structlog-vdk-init branch 4 times, most recently from a614c44 to 7f18a7b Compare February 15, 2024 13:12
vdk-structlog: put vdk init logs config behind flag …
0b33c47
Why? Configuring the logs in the vdk_initialize hook causes commands like vdk server and vdk info to have the same verbose logging format as data job logs. What? - Put the log configuration for vdk_initialize behind a config option and disable it by default for local runs and enable it by default for cloud runs. Configuring the logging format in vdk_initialize might still be useful for cloud runs and we don't care about other vdk commands output in cloud environments - Move config classes and functions into separate file How was this tested Manually CI/CD What kind of change is this? Feature/non-breaking Signed-off-by: Dilyan Marinov <mdilyan@vmware.com>
DeltaMichael force-pushed the person/mdilyan/structlog-vdk-init branch from 7f18a7b to 0b33c47 Compare February 15, 2024 15:10
DeltaMichael merged commit 5feed5b into main Feb 16, 2024
DeltaMichael deleted the person/mdilyan/structlog-vdk-init branch February 16, 2024 08:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers
antoniivanov antoniivanov approved these changes
duyguHsnHsn duyguHsnHsn approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cla-not-required

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

non-run CLI commands log output without context

4 participants

Footer

© 2026 GitHub, Inc.