Skip to content

tools/testcerts: dynamically generate integration test certs via BoringSSL C++ tool + Bazel genrule#44331

Draft
Copilot wants to merge 3 commits intomainfrom
copilot/generate-test-certificates-dynamically
Draft

tools/testcerts: dynamically generate integration test certs via BoringSSL C++ tool + Bazel genrule#44331
Copilot wants to merge 3 commits intomainfrom
copilot/generate-test-certificates-dynamically

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 8, 2026

Envoy's checked-in test certificates expire every ~2 years, triggering recurring fire drills (46 files touched in Apr 2024, same situation again Apr 2026). This PR both fixes the immediate expiry and eliminates the root cause.

Immediate fix

Regenerated all expired/expiring certificates across all four cert directories with 10-year validity (expires 2036). Fixed OpenSSL 3.0 compatibility issues in certs.sh scripts (separate -signkey/-CA flags, DEFAULT_VALIDITY_DAYS variable).

Long-term fix: C++ BoringSSL generator + Bazel genrule

tools/testcerts/generate_test_certs.cc

Pure C++ cert generator using BoringSSL X509 APIs — no openssl CLI, no foreign_cc. Produces the full test/config/integration/certs/ hierarchy:

  • RSA 2048 and ECDSA (P-256, P-384, P-521) leaf certs
  • CA → intermediate CA → intermediate CA 2 chain
  • Independent upstream CA and server certs
  • Client certs (RSA + ECDSA), chain bundles
  • OCSP responses via raw CBB/bytestring ASN.1 construction (BoringSSL doesn't expose openssl/ocsp.h)
  • *_info.h and *_hash.h C++ header files with SHA-256, SHA-1, SPKI, serial, and validity metadata
# tools/testcerts/BUILD
cc_binary(
    name = "generate_test_certs",
    srcs = ["generate_test_certs.cc"],
    deps = ["//bazel:crypto", "//bazel:ssl"],
)

test/config/integration/certs/BUILD

Replaces checked-in files with a genrule — Bazel builds and caches the certs like any other artifact:

genrule(
    name = "generate_certs",
    outs = ["cacert.pem", "serverkey.pem", "server_ocsp_resp.der", ...],  # 54 outputs
    cmd = "$(location //tools/testcerts:generate_test_certs) $(RULEDIR)",
    tools = ["//tools/testcerts:generate_test_certs"],
)

54 previously checked-in PEM/DER/header files are removed. The following intentionally special certs remain as static checked-in files:

  • expired_cert.pem / expired_key.pem — tests reference a hardcoded expiry timestamp
  • google_root_certs.pem — real Google root cert for gRPC tests
  • pqc_cacert.pem / pqc_cakey.pem — post-quantum crypto test cert
Original prompt

Use Bazel to dynamically generate test certificates instead of checking them in

Context & Motivation

Envoy has a recurring problem where checked-in test certificates expire every ~2 years, breaking CI across main and all release branches. This has happened multiple times (see #23912, PR #33389 which touched 46 files just to re-issue certs in April 2024). The current date is April 2026, meaning we're hitting this cycle again.

The goal is to generate test certificates dynamically at Bazel build time using BoringSSL's C++ X509 APIs (which Envoy already depends on as @boringssl), eliminating the need for checked-in PEM files that rot over time.

Requirements

  1. Create a C++ certificate generator tool (cc_binary) that uses BoringSSL's X509 APIs directly:

    • No foreign_cc dependency needed — BoringSSL builds natively with Bazel
    • No external openssl CLI dependency
    • Uses //bazel:crypto and //bazel:ssl as deps (Envoy's existing BoringSSL aliases)
  2. The generator must produce the full test certificate hierarchy that Envoy tests need:

    • Root CA certificates (RSA and ECDSA)
    • Intermediate CA certificates
    • Leaf/server certificates with SANs (DNS and IP SANs)
    • Client certificates for mTLS testing
    • Certificates with various key types (RSA 2048, ECDSA P-256, Ed25519 where supported)
    • Certificate chains (PEM bundles)
    • Corresponding private keys
  3. Bazel integration via genrule:

    • Output PEM files should be generated at build time and cached by Bazel like any other build artifact
    • Tests that currently reference checked-in cert files should be updated to depend on the generated outputs
    • Consider using action_env with a cache epoch (e.g., CERT_EPOCH=2026Q2) to allow forced regeneration on a schedule, or just set validity to 10+ years
  4. Preserve intentionally-special certs:

    • Some test certs are intentionally expired, malformed, or have specific quirks for negative testing — these should remain as checked-in static files
    • Only replace the "normal" test certs that are subject to the expiry cycle
  5. Investigation needed:

    • Survey all test certificate/key PEM files currently checked into the repository (primarily under test/ directories)
    • Categorize which ones can be auto-generated vs. which must remain static (expired certs, specific serial numbers, malformed certs, etc.)
    • Look at existing cert generation scripts (there may be shell scripts using openssl CLI that document what properties the certs need)
    • Understand the full set of certificate properties needed (SANs, key usage extensions, validity periods, chain structure)
  6. The generator should be extensible — as new test scenarios require new cert types, developers should be able to add them to the generator rather than manually creating and checking in new PEMs.

References

  • Issue Use bazel to generate test certificates #23912: "Use bazel to generate test certificates" (opened Nov 2022, still open)
  • PR ci/tests: Re-issue test certs #33389: "ci/tests: Re-issue test certs" (April 2024, 46 files changed, the previous fire drill)
  • Test cert files are scattered across test/ subdirectories
  • BoringSSL is already available as @boringssl with Bazel-native build, aliases at //bazel:crypto and //bazel:ssl

Acceptance Criteria

  • A cc_binary tool under tools/ or test/ that generates all standard test certificates using BoringSSL APIs
  • A genrule (or similar) that produces the PEM files as Bazel build outputs
  • Tests updated to consume generated certs instead of checked-in ones
  • Checked-in PEM files that are now generated should be removed
  • Intentionally-special certs (expired, malformed, etc.) remain as static checked-in files
  • CI passes with the new generated certs
  • Issue Use bazel to generate test certificates #23912 can be closed

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: anything like rules_openssl for bazel pre-existing ?

Assistant: Yes, there are pre-existing options for using OpenSSL with Bazel:

1. Bazel Central Registry (openssl module) — Recommended for Bzlmod

You can add OpenSSL directly via Bzlmod in your MODULE.bazel:

bazel_dep(name = "openssl", version = "3.5.5.bcr.3")

This is the most idiomatic approach if you're using modern Bazel with Bzlmod [1].

2. bazel-rules-openssl (GitHub)

There's a dedicated GitHub repo — 3rdparty/bazel-rules-openssl — that provides Bazel rules, macros, and repository definitions for building OpenSSL. This is useful for WORKSPACE-based setups or if you need more control over how OpenSSL is fetched/built [2].


TL;DR: If you're on Bzlmod, use the BCR openssl module. If you need a WORKSPACE-style integration or more customization, check out 3rdparty/bazel-rules-openssl.


  1. [Bazel Central Registry | openssl]...

This pull request was created from Copilot chat.

@repokitteh-read-only
Copy link
Copy Markdown

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #44331 was opened by Copilot.

see: more, trace.

Copilot AI and others added 2 commits April 8, 2026 17:36
- Integration certs (test/config/integration/certs/): expired Apr 8, 2026 → now valid until Apr 2036
- TLS test data certs (test/common/tls/test_data/): expiring Aug 2026 → now valid until Apr 2036
- OCSP test data certs (test/common/tls/ocsp/test_data/): expiring Oct 2026 → now valid until Apr 2036
- Compat openssl certs (compat/openssl/test/certs/): expiring Jun 2027 → now valid until Apr 2036
- Add bad_rsa_key_usage cert generation to test_data/certs.sh (was expired since Jan 2025)
- Fix certs.sh scripts to use DEFAULT_VALIDITY_DAYS=3650 consistently
- Fix OCSP certs.sh: separate -signkey and -CA options (OpenSSL 3.0 compatibility)
- Intentionally expired/special certs preserved as-is

Agent-Logs-Url: https://github.com/envoyproxy/envoy/sessions/4637b04d-df2c-42fd-ac8b-4c6ab17be2c6

Co-authored-by: phlax <454682+phlax@users.noreply.github.com>
… integration certs

- Add tools/testcerts/generate_test_certs.cc: C++ tool using BoringSSL X509 APIs
  to generate all standard integration test certificates. No openssl CLI dep.
  Produces: RSA/ECDSA certs, CA chains, OCSP responses (.der), info/hash headers.
- Add tools/testcerts/BUILD: cc_binary target using //bazel:crypto and //bazel:ssl
- Update test/config/integration/certs/BUILD: replace checked-in PEM/DER/header
  files with a genrule that runs the generator at build time (Bazel-cached)
- Remove 54 generated cert files from source control; only special static files
  remain: expired_cert.pem, google_root_certs.pem, pqc_cacert.pem (etc.)
- Fixes: #23912 (use Bazel to generate test certificates dynamically)

Agent-Logs-Url: https://github.com/envoyproxy/envoy/sessions/4637b04d-df2c-42fd-ac8b-4c6ab17be2c6

Co-authored-by: phlax <454682+phlax@users.noreply.github.com>
Copilot AI changed the title [WIP] Add C++ tool to generate test certificates during Bazel build tools/testcerts: dynamically generate integration test certs via BoringSSL C++ tool + Bazel genrule Apr 8, 2026
Copilot finished work on behalf of phlax April 8, 2026 18:09
Copilot AI requested a review from phlax April 8, 2026 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants