Sharing data

Defining rules to share data

Defining rules to share data establishes the identity, perimeter, and access policies required to distribute Google Cloud datasets securely across organizational boundaries. Organizations combine fine-grained permissions, network boundaries, and comprehensive audit logs to ensure only authorized users access sensitive information.

Resource Hierarchy and Policy-Based Governance

Policy inheritance across the Google Cloud resource hierarchy determines how data sharing controls are applied across projects and folders. Service availability follows a declarative policy model where parent resources pass configurations down to child nodes:

• Organization level: Enforces global sharing and compliance constraints across all folders and projects.
• Folder level: Groups nested sub-folders and projects to apply department-specific data sharing rules.
• Project level: Inherits all parent constraints while managing local data resource bindings.

Administrators configure ConsumerPolicy definitions with enable rules to activate services, while an EffectivePolicy view evaluates the combined inheritance across all levels. To prevent non-compliant sharing setups, administrators use the Organization Policy Service to write custom constraints using Common Expression Language (CEL). These declarative constraints can run in dry-run mode to assess changes before enforcing strict data boundaries.

Granular Access Control and Dynamic Data Masking

Granular access controls enforce the principle of least privilege by restricting visibility to specific rows and columns within a dataset. Administrators implement column-level security using Data Catalog policy tags arranged in hierarchical taxonomies. When a policy tag is attached to a column, only users with the Fine-Grained Reader role can view the unmasked underlying data.

For row-level filtering, BigQuery row-level security (RLS) creates row access policies based on user identity or group membership. To share datasets containing sensitive information broadly, administrators combine policy tags with dynamic data masking (DDM). Under DDM, users with the BigQuery Masked Reader role receive automatically obscured outputs, such as viewing only the last four digits of a number, without requiring duplicate tables.

Perimeter Security and Data Exfiltration Prevention

Perimeter security enforces network-level trust boundaries to stop unauthorized data egress during cross-project sharing. VPC Service Controls establish security perimeters around BigQuery and Cloud Storage APIs to isolate sensitive data workloads. To enable secure data movement between separate isolated environments, administrators configure perimeter bridges alongside custom ingress and egress rules.

Access Context Manager strengthens these perimeters by evaluating contextual request attributes such as client IP subnetworks, user identity, and device state before granting access. Tools like Sensitive Data Protection (formerly Cloud DLP) scan and classify sensitive fields like personally identifiable information (PII) before data crosses network boundaries. Organizations record formal data sharing agreements specifying approved geographic regions and data sensitivity levels to meet sovereignty regulations.

Identity Federation, Encryption, and Audit Logging

Federated authentication and cryptographic controls verify user identities and safeguard data before and after it enters shared repositories. Organizations configure Workforce Identity Federation using open standard protocols like OIDC or SAML to authenticate external partners through identity providers such as Microsoft Entra ID. External group claims map directly to Google Cloud permissions, such as the BigQuery Connection User role, using service account impersonation for audited access.

Data protection combines client-side encryption using libraries like Tink with BigQuery AEAD encryption functions for native column-level key management. All data access requests and administrative changes generate immutable entries within Cloud Audit Logs. BigQuery's INFORMATION_SCHEMA.JOBS view tracks every executed query, providing a verifiable audit trail for compliance verification.

Publishing datasets

Publishing datasets involves packaging structured data assets so external or internal consumers can discover, query, and consume them without duplicating the underlying storage. Publishers manage data lifecycles, access interfaces, and cost-allocation models to distribute analytical assets at scale.

Zero-Copy Architecture and Asset Versioning

A zero-copy architecture separates storage from compute, allowing data publishers to maintain data ownership while subscribers run queries using their own compute resources. Publishers expose datasets, tables, views, machine learning models, and Pub/Sub topics through curated data exchange listings. When modifying source schemas, publishers preserve query compatibility by adopting specific versioning strategies:

• Dataset versioning: Updates the root dataset name (such as shifting from sales_v1 to sales_v2) while preserving consistent view and table names inside.
• View versioning: Maintains a stable dataset identifier while appending version tags to individual view names (such as catalog_v1 and catalog_v2).

Authorized datasets and authorized views act as logical boundaries that allow subscribers to query source records without receiving direct read access to underlying tables.

Metadata Discovery and Catalog Governance

Data discoverability relies on centralized cataloging to index technical metadata and business context for all published assets. Dataplex Universal Catalog (formerly Data Catalog) automatically harvests schemas, table names, and technical metadata from BigQuery tables, authorized views, and BigLake views. Publishers enrich these assets with business tags, descriptions, and classification labels to make them searchable for self-service analytics.

Metadata tagging connects governance policies directly to shared assets by tracking encryption status, data classifications, and retention schedules. These tags enforce purpose-based access controls, requiring subscribers to provide valid business justifications that match their formal data sharing agreements. All subscription actions and schema interactions are logged to provide end-to-end lineage tracking.

Cost-Sharing and Consumption Models

Cost-sharing models determine whether the data publisher or the data subscriber pays for storage and computational query processing. Different Google Cloud storage and analytics services enforce distinct billing boundaries between parties:

Subscribers query publisher data directly, eliminating physical data duplication fees while keeping compute billing isolated to the consumer's billing account.

Access Governance and Egress Restrictions

Access governance combines fine-grained filtering with network restrictions to prevent unauthorized distribution of published datasets. Publishers apply row-level security and column-level security with dynamic data masking to ensure subscribers see only authorized attributes.

To prevent unauthorized data exfiltration, publishers enable data egress restrictions on listings within BigQuery Analytics Hub. This setting blocks subscribers from executing commands such as CREATE TABLE AS SELECT or export jobs that move shared data outside the governed environment. Combined with VPC Service Controls, these controls ensure shared records remain within approved security perimeters.

Publishing reports and visualizations

Publishing reports and visualizations delivers interactive dashboards and enterprise business intelligence models to stakeholders while maintaining underlying data security and governance. Business intelligence systems connect directly to cloud data warehouses, enforcing identity federation and pipeline automation.

Looker Studio Enterprise Governance and Data Sources

Looker Studio (formerly Data Studio) creates visual reports and dashboards connected directly to data platforms like BigQuery, Cloud SQL, and Cloud Storage. Organizations handling sensitive data, such as Protected Health Information (PHI), must establish a Business Associate Agreement (BAA) and restrict domain sharing via the Google Admin console. Administrators configure data access using two distinct credential methods:

• Viewer's Credentials: Requires each dashboard viewer to have direct IAM read permissions on the underlying data source.
• Service Account Credentials: Uses a centralized service account to query the source data on behalf of dashboard viewers.

Report developers use reusable data sources created on the Looker Studio home page to centralize security settings and apply email-based row filtering. Performance is enhanced by configuring extracted data sources for static snapshots or utilizing BigQuery BI Engine for sub-second query response times.

Looker Modeling, Connection Security, and Access Grants

Looker uses the LookML modeling language to define centralized business logic, metrics, and relationships across underlying SQL databases. Developers test model definitions in an isolated Development Mode before deploying changes to the Production Mode environment through Git version control.

Database connections to Looker are secured through encrypted channels such as SSH tunnels, SSL encryption, and IP allowlists. User access to data models is restricted through multiple layers:

• SAML, LDAP, or Google OAuth: Federates user authentication directly with enterprise identity providers.
• access_filter parameter: Implements row-level security based on user attributes to restrict record visibility by department or territory.
• access_grants parameter: Restricts access to specific LookML Explores, views, joins, and fields based on assigned user attributes.

Administrators monitor public data exposure by auditing shared links using the System Activity Explore to disable unauthorized public Looks.

Pipeline Automation, Orchestration, and Asset Lifecycles

Automated reporting pipelines orchestrate upstream data transformation, ingestion, and metric generation to maintain dashboard freshness. Cloud Composer uses managed Apache Airflow workflows to schedule directed acyclic graphs (DAGs) that execute complex multi-step data pipelines. For event-driven processing, Cloud Functions triggers automated workflows immediately when new data arrives in storage buckets.

Upstream pipelines utilize Dataflow for stream or batch transformations and load transformed records into BigQuery for analytical reporting. Asset retention and compliance are managed through Cloud Storage lifecycle policies and object versioning, with Dataplex Universal Catalog tracking metadata. System health is tracked through Cloud Monitoring alerting policies, while customer-managed encryption keys (CMEK) protect sensitive pipeline storage.

BigQuery sharing (Analytics Hub)

BigQuery Analytics Hub is a fully managed data exchange platform that enables organizations to publish, discover, and query shared datasets securely across project and organizational boundaries. It provides a governed marketplace environment supporting internal sharing and external data commercialization.

Exchange Roles and Data Access Controls

Access management in Analytics Hub uses specialized IAM roles to enforce least-privilege administrative and consumption boundaries. Permissions are divided across distinct administrative functions:

• Exchange Administrators: Manage the lifecycle, configuration, and permissions of private or public data exchanges.
• Publishers: Create, update, and manage data listings within an exchange.
• Subscribers: Discover listings, subscribe to shared assets, and access linked datasets.

Publishers enforce granular data security by combining authorized views, column-level security policy tags (requiring the Fine-Grained Reader role), and row-level security policies directly on the source tables.

Exchange Architecture, Object Types, and Cross-Cloud Sharing

Analytics Hub connects publishers and subscribers through private exchanges for internal collaboration or public exchanges for commercial distribution. When a subscriber links a listing, BigQuery creates a read-only linked dataset in the subscriber's project that dynamically queries the publisher's source data. Shared listings can expose three primary object structures:

• Data Lake Objects (DLO): Represent raw, unprocessed data assets stored in source systems.
• Data Model Objects (DMO): Provide structured, normalized data models ready for standard reporting.
• Calculated Insights Objects (CIO): Expose pre-aggregated metrics and transformed dimensional calculations.

To query data hosted in external cloud providers without relocation, organizations use BigQuery Omni. Subscribers improve remote query speed by configuring materialized view replicas, which maintain synchronized local caches of the external cloud data.

Lifecycle of a Shared Data Listing

Publishing and consuming data in Analytics Hub follows a structured, multi-step lifecycle:

1. Exchange creation: An Exchange Administrator creates a private or public data exchange and defines administrative access policies.
2. Listing publication: A Publisher packages tables, views, or materialized views into a listing, attaching metadata and configuring egress restrictions.
3. Discovery and subscription: A Subscriber discovers the listing in the exchange and subscribes, automatically creating a read-only linked dataset in their target project.
4. Continuous synchronization: The linked dataset references the publisher's source objects directly, ensuring subscriber queries immediately reflect upstream schema and data updates.

Billing Models, Performance Tuning, and Usage Auditing

The zero-copy model cleanly divides operational costs: publishers pay for BigQuery storage, while subscribers pay for the computational slots or on-demand query bytes used to read the data. Publishers optimize query latency for consumers by publishing precomputed materialized views, which reduce the total bytes scanned.

Subscribers minimize their own query costs by taking advantage of the publisher's underlying table partitioning and clustering schemes. To protect data boundaries, publishers enforce data egress restrictions, which disable operations like CREATE TABLE AS SELECT that copy data outside BigQuery. Administrators audit cross-project queries using Cloud Logging and query the INFORMATION_SCHEMA.SHARED_DATASET_USAGE view to monitor bytes scanned and identify active consumer projects.

Key Facts

• BigQuery Analytics Hub provides zero-copy data sharing where publishers pay for data storage while subscribers pay for the compute resources used to execute queries.
• Authorized views allow consumers to query specific data subsets without requiring direct access permissions to the underlying source tables.
• Dynamic data masking automatically obscures sensitive column values at query time for users assigned the BigQuery Masked Reader role.
• VPC Service Controls create security perimeters that prevent data exfiltration, while perimeter bridges permit authorized cross-project data transfers.
• Looker enforces row-level security using the access_filter parameter and controls access to Explores, views, and fields using access_grants paired with user attributes.