go.chromium.org/luci@v0.0.0-20240309015107-7cdc2e660f33/tools/cmd/bqschemaupdater/README.md (about)

     1  bqschemaupdater is a tool for adding and updating BigQuery table schema.
     2  
     3  # Usage
     4  
     5  schemas should be written in .proto format.
     6  
     7  Bqschemaupdater uses protoc found in $PATH.  Please make the latest
     8  [protobuf library](https://github.com/google/protobuf) available in $PATH.
     9  
    10  More information can be had with:
    11  
    12  ```
    13  bqschemaupdater --help
    14  ```
    15  
    16  ## Supported Uses
    17  
    18  The operations supported by this tool include:
    19  
    20  * Creating a new table
    21  * Adding NULLABLE or REPEATED columns to an existing table
    22  * Making REQUIRED fields NULLABLE in an existing table
    23  
    24  # Standard Practices
    25  
    26  Table IDs and Dataset IDs should be underscored delimited, e.g. `test_results`.
    27  
    28  ## Schema Definitions
    29  
    30  Columns in BigQuery tables cannot be modified or deleted once they have been
    31  created. However, it is easy to add columns, so err on the side of not adding a
    32  column if you are unsure.
    33  
    34  BigQuery provides useful types such as RECORD and TIMESTAMP. It is a good idea
    35  to take advantage of these types.
    36  
    37  Events usually have an associated timestamp field recording the time the event
    38  took place.
    39  
    40  BigQuery discourages JOINs and encourages denormalizing data.
    41  
    42  Flattening repeated fields can be costly. If you know that a repeated field
    43  will only ever contain 0 or 1 value, consider not making that field repeated.
    44  
    45  # BigQuery Limits
    46  
    47  Please see [BigQuery
    48  docs](https://cloud.google.com/bigquery/quotas#datasettableupdates) for the most
    49  updated limits for creating and modifying tables. It is not expected that we
    50  will exceed these limits through bqschemaupdater usage. If you are planning
    51  a project which might, please contact the Monitoring Team.