StudyCerts
StudyCertsProfessional Cloud Data Engineer
professional-cloud-data-engineer
Advanced
Gauge your current knowledge
Advanced
Gauge your current knowledge
Designing the project, dataset, and table architecture to ensure proper data governance
...
professional-cloud-data-engineer
Gauge your current knowledge
Gauge your current knowledge
BigQuery security starts with Google Cloud's Identity and Access Management (IAM) system. You assign roles at the dataset level, and those permissions automatically apply to all tables inside it, which makes managing access for many users easier. The principle is to give people the least privilege they need to do their job.
Data classification involves tagging your data based on its sensitivity. You create a system of policy tags (like "Confidential" or "Public") and attach them to specific columns in your tables. Services like Sensitive Data Protection can automatically find and label sensitive information, such as credit card numbers. These tags then work with IAM to control who can see which columns.
Column-level security uses those policy tags to protect data directly in a column. You can set up data masking so that users without special permission see "****" instead of the real values. For even stronger protection, you can use column-level encryption, where the data is encrypted, and a user needs both table access and a specific encryption key to read it.
Row-level security controls which rows of data a user can see. You can create an authorized view with a filter (like WHERE department = 'sales') and only grant access to that view. BigQuery also has native row-level security policies that can filter rows based on who is asking, which is useful for multi-tenant setups where different customers should only see their own data.
Authorized views are a key tool for sharing data safely. You write a query that selects the data you want to share, save it as a view, and then authorize that view to read the source tables. Users query the view but never touch the original tables, and you can build complex security logic right into the view's query.
Additional security measures include default encryption for data at rest and in transit, with options to use your own keys via Cloud KMS. VPC Service Controls create a security perimeter to stop data from being moved outside approved areas. You can also set permissions on individual tables for more precise control than dataset-level permissions allow.
Designing your tables involves choosing the right type for how the data will be used and managed. You select between managed tables (fully stored in BigQuery), external tables (data stays in Cloud Storage), and materialized views (pre-computed summaries). Your choice depends on performance needs, cost, and how often the data changes.
To manage costs and improve query performance, you use partitioning and clustering. Partitioning splits a large table into smaller pieces based on a date or integer column, so queries only scan relevant data. Clustering sorts the data within those partitions based on other columns, which makes certain queries much faster. Together, they help govern both cost and speed.
You must also define data retention and expiration policies to meet compliance rules. You can set tables or partitions to automatically delete after a certain time. For protection against mistakes, BigQuery Snapshots let you save a point-in-time copy of a table beyond the standard recovery window. For strict legal holds, you can use Cloud Storage Bucket Lock to make data immutable.
To comply with regulations about where data can be stored, you use resource location policies. These are organization-level rules that restrict where new resources like datasets can be created, ensuring data stays in approved geographic regions. For the highest level of key control, you can use Cloud External Key Manager (EKM) to store encryption keys outside of Google Cloud or use customer-managed encryption keys (CMEK) in a Cloud HSM.
Data governance on Google Cloud is built on a structured resource hierarchy: Organization, Folders, Projects, and then Resources (like datasets). You design this hierarchy to separate data by environment (like Development vs. Production), sensitivity, or business unit. This creates natural boundaries for applying security rules and isolating data.
You design a multi-tiered hierarchy by creating folders under your organization for major categories. Under these folders, you create separate projects for different purposes, like a project for confidential data and another for data ingestion tools. This separation lets you apply IAM policies and organization policies at the right level, making it easier to enforce rules like least-privilege access for each team.
To enforce strict boundaries and prevent data from being copied out improperly, you use VPC Service Controls. This creates a security perimeter around specific projects containing sensitive data. It controls how data moves in and out of managed services like BigQuery, adding a layer of protection even if someone's IAM permissions are incorrectly set.
You control where data lives using organization policy constraints. For example, you can set a rule (gcp.resourceLocations) that only allows resources to be created in certain countries or regions, meeting data residency laws. Inside your projects, you use tools like Data Catalog and Sensitive Data Protection to discover and classify sensitive data, then apply policy tags in BigQuery to manage who can see it.
Finally, you operationalize governance with automation and monitoring. You define organization policies as code for consistency. Security Command Center monitors for policy violations and threats, while Cloud Logging and audit logs keep a complete record of who did what. This provides evidence for compliance audits and helps with security investigations.