Add failOpen to CES Guardrail llmPromptSecurity#17634
Conversation
|
Googlers: For automatic test runs see go/terraform-auto-test-runs. @roaks3, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look. You can help make sure that review is quick by doing a self-review and by running impacted tests locally. |
|
@DanKotowski fyi. thanks |
|
Hi there, I'm the Modular magician. I've detected the following information about your changes for commit b0c46c3: Diff reportYour PR generated the following diffs in downstream repositories:
Test reportAnalytics
Affected Service Packages
Step 1: Replaying Mode Action takenFound 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests
View the replaying VCR build log Step 2: Recording Mode
🟢 All tests passed! View the recording VCR build log or the debug logs folder for detailed results. |
|
Hi there, I'm the Modular magician. I've detected the following information about your changes for commit 734e91c: Diff reportYour PR generated the following diffs in downstream repositories:
Test reportAnalytics
Affected Service Packages
Step 1: Replaying Mode 🟢 All tests passed in Replaying mode! No Recording was needed. View the replaying VCR build log |
roaks3
left a comment
There was a problem hiding this comment.
LGTM, but wondering if the tests/docs are appropriate
| @@ -0,0 +1,33 @@ | |||
| resource "google_ces_app" "ces_app_for_guardrail" { | |||
There was a problem hiding this comment.
Just checking, is this worth of its own entire test and example in the docs? I can't tell if this is a major use case, or a simple toggle of which there are many.
There was a problem hiding this comment.
I added this as a separate example because of the API level constraint and to document a distinct configuration path. llm_prompt_security block has two mutually exclusive configuration paths in the backend Root level fail_open and custom_policy.fail_open. If custom_policy is configured, the backend API ignores the root-level fail_open completely and does not return it in GET responses.
The existing example (ces_guardrail_generative_answer_llm_prompt_security) focuses on the custom_policy setup where users configure their own Gemini models and prompts. The new example demonstrates default root level fail_open settings which as added as part of the PR.
@roaks3 please let me know if this looks good or you have additional feedback.
| } | ||
| enabled = true | ||
| llm_prompt_security { | ||
| fail_open = true |
There was a problem hiding this comment.
Similar to my other question: could this be simply added to an existing test, or is it representing an isolated use case where we want to test updating the field on its own.
There was a problem hiding this comment.
This is kept as separate test case to verify the root-level fail_open lifecycle without causing state issues or diff loops.
If we tried to merge this into existing GenerativeAnswerLlmPromptSecurity test (which uses custom_policy), the API would ignore the root-level fail_open and omit it from the response.
Also, Optional booleans in proto3 APIs are omitted from the response when set to false (default value). To test this robustly, the test uses an inverted lifecycle flow (Create without the block -> Update to add the block with fail_open = true). This lets us verify both false (by baseline omission) and true (by explicit update) without hitting diff loops.
@roaks3 hope this is fine.
roaks3
left a comment
There was a problem hiding this comment.
LGTM, thanks for the responses
56dcd42
Add failOpen field to llmPromptSecurity in google_ces_guardrail and google_ces_app_version
This PR adds the missing
fail_openattribute to thellm_prompt_securityblock in thegoogle_ces_guardrailandgoogle_ces_app_versionresources, achieving full API parity for this security feature.Documentation
Rationale
The
failOpenfield at the root of thellmPromptSecurityobject was missing from the Terraform provider schemas. This field determines whether the guardrail fails open (allowing user queries to pass through if LLM classification fails) or closed. Without it, users were unable to configure this critical fallback behavior when using the default system security settings.Technical Details
failOpen(Boolean) tollmPromptSecurityproperties inGuardrail.yaml.failOpen(Boolean, read-onlyoutput: true) under the nestedguardrails.llmPromptSecurityproperties inAppVersion.yamlto ensure complete schema parity across resources.ces_guardrail_llm_prompt_security_fail_open.tf.tmplto demonstrate usage withdefault_settingsandfail_open = true.TestAccCESGuardrail_cesGuardrailLlmPromptSecurityFailOpenExample_updatetoces_guardrail_test.go.fail_open = true) to robustly handle proto3 default-value (false) omission in API responses, avoiding Terraform "empty nested block" diff loops.Verification Results
googleandgoogle-betaproviders usingPRODUCT=ces.TestAccCESGuardrail_cesGuardrailLlmPromptSecurityFailOpenExample_update).