← 返回首页
Add "misra_help" query docs generator by data-douser · Pull Request #1114 · github/codeql-coding-standards · GitHub
Skip to content

Navigation Menu

Toggle navigation
Sign in
Appearance settings
Search or jump to...

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Resetting focus

Add "misra_help" query docs generator#1114

Draft
data-douser wants to merge 6 commits into
mainfrom
dd/misra-qhelp/1
Draft

Add "misra_help" query docs generator#1114
data-douser wants to merge 6 commits into
mainfrom
dd/misra-qhelp/1

Conversation

Copy link
Copy Markdown
Contributor

Introduces scripts/generate_rules/misra_help/, a two-stage pipeline for (mostly) idempotent generation of per-query .md help files. It uses MISRA rule text as input and creates (or updates) documentation for codeql-coding-standards queries in C and C++.

Initial supported standards:

  • MISRA C 2012 / 2023
  • MISRA C++ 2023

Stage 1 — deterministic, docling-based extraction and rendering, with a JSON sidecar for downstream consumption.

Stage 2 — a headless Python driver for the Copilot SDK that rewrites each help file from the JSON sidecar against a fixed Markdown schema, normalized to American English.

See scripts/generate_rules/misra_help/README.md for usage, architecture, and operational notes.

Description

Adds a new internal tooling package under scripts/generate_rules/misra_help/ that automates generation of per-query Markdown help files for MISRA C/C++ queries. No query files, query metadata, rule packages, shared libraries, tests, .expected files, or release artifacts are modified by this PR — it is purely additive tooling (7 new files, ~2.1k lines, all under scripts/generate_rules/misra_help/).

The pipeline is split so that the deterministic extraction stage can be re-run cheaply and audited independently of the LLM-driven rewrite stage. The JSON sidecar is the contract between the two stages, which keeps Stage 2 reproducible against a pinned input.

Change request type

  • Release or process automation (GitHub workflows, internal scripts)
  • Internal documentation
  • External documentation
  • Query files (.ql, .qll, .qls or unit tests)
  • External scripts (analysis report or other code shipped as part of a release)

Rules with added or modified queries

  • No rules added
  • Queries have been added for the following rules:
    • rule number here
  • Queries have been modified for the following rules:
    • rule number here

Release change checklist

A change note (development_handbook.md#change-notes) is required for any pull request which modifies:

  • The structure or layout of the release artifacts.
  • The evaluation performance (memory, execution time) of an existing query.
  • The results of an existing query in any circumstance.

If you are only adding new rule queries, a change note is not required.

Author: Is a change note required?

  • Yes
  • No — tooling-only change under scripts/generate_rules/misra_help/; no release artifacts, query results, or query performance are affected.

🚨🚨🚨
Reviewer: Confirm that format of shared queries (not the .qll file, the
.ql file that imports it) is valid by running them within VS Code.

  • N/A — no .ql/.qll files modified.

Reviewer: Confirm that either a change note is not required or the change note is required and has been added.

  • Confirmed

Query development review checklist

For PRs that add new queries or modify existing queries, the following checklist should be completed by both the author and reviewer:

N/A for this PR — no queries are added or modified. This section is left as-is for the reviewer to confirm.

Author

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

Reviewer

  • Have all the relevant rule package description files been checked in?
  • Have you verified that the metadata properties of each new query is set appropriately?
  • Do all the unit tests contain both "COMPLIANT" and "NON_COMPLIANT" cases?
  • Are the alert messages properly formatted and consistent with the style guide?
  • Have you run the queries on OpenPilot and verified that the performance and results are acceptable?
    As a rule of thumb, predicates specific to the query should take no more than 1 minute, and for simple queries be under 10 seconds. If this is not the case, this should be highlighted and agreed in the code review process.
  • Does the query have an appropriate level of in-query comments/documentation?
  • Have you considered/identified possible edge cases?
  • Does the query not reinvent features in the standard library?
  • Can the query be simplified further (not golfed!)

Introduces scripts/generate_rules/misra_help/, a two-stage pipeline for (mostly) idempotent generation of per-query .md help files. Uses MISRA rules as input and creates (or updates, as needed) documentation for codeql-coding-standards queries for C and C++. Focuses on immediate support for: - MISRA C 2012/2023 - MISRA C++ 2023. Stage 1: deterministic docling-based extraction and rendering, with a JSON sidecar for downstream consumption. Stage 2: a headless Python driver for the Copilot SDK that rewrites each help file from the JSON sidecar against a fixed Markdown schema and American English spelling. Adds docs via -> "scripts/generate_rules/misra_help/README.md"
data-douser self-assigned this Apr 21, 2026
Comment thread scripts/generate_rules/misra_help/populate_help.py Fixed Show fixed Hide fixed
Comment thread scripts/generate_rules/misra_help/populate_help.py Show resolved Hide resolved
#
# If none of those resolve to exactly one file, we abort with a clear message.
PDF_ENV_VARS = {
"MISRA-C-2023": "MISRA_C_PDF",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality Hide comment

So just a clarification here. We should not differentiate C-2023 and C-2012 at all.

Every rule we have is both a part of MISRA C 2012, and a part of MISRA C 2023, there isn't an actual distinction. MISRA C 2012 with all amendments included = MISRA C 2023

Comment thread scripts/generate_rules/misra_help/populate_help.py Show resolved Hide resolved
Comment thread scripts/generate_rules/misra_help/populate_help.py Show resolved Hide resolved
Comment thread scripts/generate_rules/misra_help/populate_help.py Show resolved Hide resolved
14 hidden conversations Load more…
Comment thread scripts/generate_rules/misra_help/rewrite_help.py Show resolved Hide resolved
Comment thread scripts/generate_rules/misra_help/rewrite_help.py Show resolved Hide resolved
Comment thread scripts/generate_rules/misra_help/rewrite_help.py Show resolved Hide resolved
Comment thread scripts/generate_rules/misra_help/rewrite_help.py Show resolved Hide resolved
Comment thread scripts/generate_rules/misra_help/rewrite_help.py Outdated Show resolved Hide resolved
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Footer

© 2026 GitHub, Inc.