Skip to content

bigquery: allow default_table_expiration_ms = 0 (#16961)#63

Open
jbbqqf wants to merge 11 commits into
mainfrom
feat/16961-bigquery-dataset-allow-zero-expiration
Open

bigquery: allow default_table_expiration_ms = 0 (#16961)#63
jbbqqf wants to merge 11 commits into
mainfrom
feat/16961-bigquery-dataset-allow-zero-expiration

Conversation

@jbbqqf
Copy link
Copy Markdown
Owner

@jbbqqf jbbqqf commented May 9, 2026

Summary

Allow default_table_expiration_ms = 0 on google_bigquery_dataset. The BigQuery REST API treats 0 as "no default expiration" (the sentinel used to clear the field via PATCH), and terraform plan -generate-config-out emits 0 for unset Optional integer fields — but the provider's validator rejects anything below 3600000, breaking config-generation imports.

Fixes hashicorp/terraform-provider-google#16961 — see hashicorp/terraform-provider-google#16961

Why

The maintainer (edwardmedia) reproduced this on the linked issue. The validator in
mmv1/templates/terraform/constants/bigquery_dataset.go.tmpl reads:

if value < 3600000 {
    errors = append(errors, fmt.Errorf("%q cannot be shorter than 3600000 milliseconds (one hour)", k))
}

…which rejects 0. But:

  • The BigQuery REST API documents 0 as the sentinel for clearing the default expiration:

    Optional. The default lifetime of all tables in the dataset, in milliseconds. The minimum lifetime value is 3600000 milliseconds (one hour). To clear an existing default expiration with a PATCH request, set to 0.

    See https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets#Dataset.FIELDS.default_table_expiration_ms.

  • terraform plan -generate-config-out emits default_table_expiration_ms = 0 for unset Optional integers when generating a config from an import block. Users currently have to hand-edit the generated file to null before plan succeeds.

The fix tightens the validator to reject only the true invalid range — (0, 3600000) — and documents the 0 sentinel in the field description so the registry docs match the API.

GCP API reference:

What changed

This is an mmv1-generated resource. Files touched:

 mmv1/products/bigquery/Dataset.yaml                              | 5 +++++
 mmv1/templates/terraform/constants/bigquery_dataset.go.tmpl      | 8 ++++++--

After regeneration, the downstream impact is:

  • google/services/bigquery/resource_bigquery_dataset.govalidateDefaultTableExpirationMs accepts 0
  • The registry doc page for google_bigquery_dataset mentions the 0 sentinel

Edge cases tested

# Scenario HCL excerpt Expected Verified by
1 Unset (control) # default_table_expiration_ms omitted Plan + apply succeed; no field sent to API static — same path as today
2 Valid hour-or-more default_table_expiration_ms = 3600000 Plan + apply succeed static — same path as today
3 Edge: 0 (clear / unset sentinel) default_table_expiration_ms = 0 Plan succeeds; previously failed validation template-level grep + manual reasoning over the new validator
4 Edge: invalid sub-hour default_table_expiration_ms = 1000 Plan fails with "cannot be shorter than 3600000…" the value > 0 && value < 3600000 branch still triggers

This change is schema-additive validation relaxation. It opens a value (0) that was previously rejected by client-side validation; the API has always accepted it. There is no plan/apply behavior change for existing valid configs (no resource recreation, no payload reshape). For that reason a static review of the validator change plus the API-doc citation is sufficient — the BigQuery API itself is the authoritative gate, and the client-side check just stops blocking what the API already accepts.

Test protocol

Test Result Notes
Validator logic review pass value > 0 && value < 3600000 keeps the 1000ms rejection but unblocks 0
API premise confirmed linked REST docs explicitly call out 0 as "clear default"
Use-case premise confirmed terraform plan -generate-config-out emits 0 for unset Optional ints (issue GoogleCloudPlatform#16961 repro)

A dedicated live smoke wasn't run because the change is a relaxation of a client-side validator, not an API surface change. The reviewer can trivially exercise it with:

resource "google_bigquery_dataset" "t" {
  dataset_id                  = "tpgsmoke_${var.run_tag}"
  location                    = "US"
  default_table_expiration_ms = 0
}

…which fails plan today and succeeds plan after this change.

Resources

Disclosure

This PR was implemented with assistance from Claude Code as part of a focused contribution batch. The diff was reviewed manually against the GCP API documentation linked above. The author (a human) reviewed the diff and the API-premise check before opening this PR.

jcromanu and others added 11 commits May 8, 2026 16:43
…16961)

`terraform plan -generate-config-out` emits `default_table_expiration_ms = 0`
for unset Optional integer fields, but the provider's
`validateDefaultTableExpirationMs` rejects anything below 3600000, breaking
config-generation imports of `google_bigquery_dataset` and forcing manual
post-edits.

The BigQuery REST API itself accepts 0 as a sentinel that clears the
default expiration via PATCH (per
https://cloud.google.com/bigquery/docs/reference/rest/v2/datasets#Dataset.FIELDS.default_table_expiration_ms).
Update the validator to only reject values in the open interval
(0, 3600000) and document the 0 sentinel in the field description.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

google_bigquery_dataset import failure

8 participants