CI: accept --runID in uploadMetadata, emit run_id on Pub/Sub#4169
CI: accept --runID in uploadMetadata, emit run_id on Pub/Sub#4169openroad-ci wants to merge 1 commit intoThe-OpenROAD-Project:masterfrom
Conversation
Adds a per-run UUID (provided by the Jenkins shared library) to every Pub/Sub message so downstream consumers can group all (platform, design, variant) records belonging to a single CI invocation. Signed-off-by: Miguel Pedro <miguel.pedro@precisioninno.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a new --runID command-line argument to the metadata upload utility and includes it in the Pub/Sub payload. Feedback suggests moving the argument definition to the general arguments section for better organization and ensuring the run_id is also added to the Firestore payload to maintain data consistency across backends.
| parser.add_argument( | ||
| "--pubsubCred", type=str, help="Service account credentials file for Pub/Sub" | ||
| ) | ||
| parser.add_argument("--runID", type=str, help="Unique identifier for this CI run") |
There was a problem hiding this comment.
The --runID argument is added within the Pub/Sub specific arguments block. However, the motivation describes it as a general identifier for the CI run (similar to buildID or pipelineID). Consider moving it to the general arguments section (near line 29). Furthermore, while this PR updates the Pub/Sub payload, it appears to miss adding run_id to the Firestore upload_data payload (around line 63). To ensure data consistency across both backends while Firestore is still supported, please include run_id in the Firestore document as well.
2 participants
Summary
--runIDCLI argument toflow/util/uploadMetadata.pyrun_idas a top-level field in every Pub/Sub message published by the scriptMotivation
Each invocation of
orfsUploadMetadata(in thejenkins-cishared library) publishes one Pub/Sub message per(platform, design, variant)tuple. There is currently no shared identifier tying those messages together as a single CI run — existing fields (build_id,pipeline_id,commit_sha) are either sequential integers or not globally unique.The companion
jenkins-cichange generates a UUID per invocation and passes it here via--runID. Downstream consumers (e.g. the PalantirMessageConsumerWorker) can then group records byrun_id.Rollout
This PR is backwards-compatible:
args.runIDisNone, payload gets"run_id": null. Works.This PR should merge before the matching
jenkins-ciPR to avoid a broken window where the updated Groovy passes--runIDto a Python script that doesn't yet know about it.