Sharing Data with BigQuery and Looker

Defining rules to share data is the practice of letting authorized collaborators query data without copying it into uncontrolled environments while recording every access. BigQuery sharing (Analytics Hub) (opens in a new tab) and BigQuery data clean rooms provide the controlled framework for cross-organization collaboration, while Cloud Storage and Looker Action Hub extend sharing to objects and dashboards. Cloud Audit Logs, Cloud Logging, and Cloud Monitoring provide the audit trail that proves who accessed data and whether sharing rules were followed.

1.1Choosing a sharing method that keeps control with the provider

A secure sharing method defines where data lives, who can query it, and what is allowed to leave the boundary. BigQuery sharing lets a provider publish datasets once and lets subscribers query them across organizations without moving copies, while BigQuery data clean rooms add a privacy-enforced environment where multiple parties can join and analyze sensitive data under strict rules. Reusable data source sharing (opens in a new tab) through Looker Action Hub sends results directly from dashboards into built-in destinations, so dashboard data flows from Looker to the destination without manual export.

Choose BigQuery sharing when consumers need governed read access to curated datasets, choose data clean rooms when two parties must combine sensitive data without exposing raw rows, and choose Cloud Storage IAM when the unit of sharing is files that need distinct upload, view, and management rights.

1.2Enforcing least privilege with IAM and fine-grained controls

Least-privilege access means every identity receives only the minimum permission needed for its role, such as no-access, read-only, or write. Identity and Access Management (IAM) (opens in a new tab) enforces this by mapping predefined or custom roles to a user, group, or service account, refining grants with IAM Conditions and resource tags, and adding row-level security and column-level security in BigQuery to restrict rows and columns inside a dataset. Cloud Storage applies the same principle at the bucket level, where a policy can separate bucket management from object viewing.

The finance-marketing pattern shows separation in practice. IT staff receive Storage Legacy Bucket Owner on the bucket, which allows bucket management such as deleting objects and changing the IAM policy plus listing bucket contents, but not viewing or downloading object data. The marketing analyst receives Storage Object User on the same bucket, which allows uploading, viewing, updating, and deleting objects. Temporary elevation is handled by Privileged Access Manager with just-in-time elevation and audit logging, while principal access boundary (PAB) policies cap what a principal can reach and Identity-Aware Proxy (IAP) with context-aware access restricts access by IP address and device attributes.

1.3Containing movement with perimeters and encryption

A sharing rule must also control exfiltration, which is data leaving the trusted boundary through APIs, networks, or copies. VPC Service Controls creates a service perimeter around AI, ML, and data resources, and ingress and egress rules on that perimeter control which data flows enter and leave. Firewall rules then restrict inbound and outbound traffic under a deny-all-by-default policy that explicitly allows only traffic meeting defined criteria.

Encryption and sharing guardrails protect data at rest, in transit, and from public exposure. Customer-managed encryption keys (CMEKs) in Cloud KMS give the customer control of keys while CMEK-integrated services including Vertex AI Agent Platform, Cloud Storage, Cloud SQL, Spanner, and BigQuery encrypt data at rest. HTTPS protects API calls, with HTTPS load balancers used to enforce it, while organization controls such as domain restricted sharing, uniform bucket-level access, public access prevention, and restrictions on public access to BigQuery datasets and Cloud Storage buckets keep sharing inside authorized domains and IAM-only paths.

1.4Building governance with catalogs, lineage, and audit logs

A data governance layer monitors data flows, transformations, and access across teams and preserves logs for audit. Knowledge Catalog (opens in a new tab) provides the organization-wide platform to centrally discover, manage, monitor, and govern data and AI artifacts and to grant access to trusted users. Knowledge Catalog responsibilities include managing column-level lineage from BigQuery to trace source tables to shared outputs, managing data quality rules (opens in a new tab), data profiles, and automatic validation before sharing, managing discovery across data marts plus feature and model metadata, creating a business glossary with aspects and aspect types, unifying governance across BigLake and open formats such as Iceberg and Delta, and building a data mesh with decentralized ownership to domain teams.

Operational audit trails depend on complementary log sources that together record API activity, pipeline behavior, and network movement. Cloud Audit Logs collects audit trails of API calls, Cloud Logging captures real-time pipeline logs with live tailing, and Cloud Monitoring collects metrics and alerting to detect anomalies and unauthorized attempts. VPC Flow Logs, load balancer logging, retained audit records, and on-demand audit log access extend that trail to network and compliance review. Do not log personally identifiable information (PII) or confidential data in experiments or log servers, because logs themselves become a sharing surface. Federated sharing adds a five-level hierarchy of PROJECT_ID.CATALOG_ID.SHARE_NAME.SCHEMA_NAME.TABLE_NAME, where the Catalog maps to the Lakehouse connection and the Share is the data product unit, and Organization Policy enforces centralized constraints across organization, folder, and project levels.

Publishing datasets is the publication of BigQuery assets inside and across organizations without creating unmanaged copies. BigQuery Analytics Hub (opens in a new tab) is the publication layer where a publisher creates a data exchange as a container, publishes listings that describe specific assets, and consumers discover and subscribe. Because administration stays in the publisher project, access can be granted, changed, or revoked centrally while control of source data remains with the owner.

2.1Data exchanges, listings, and linked datasets

A data exchange and its listings are the foundation for Analytics Hub sharing. The exchange sets the sharing boundary and ownership, while each listing points to one shared asset and defines how a subscriber can use it. A publisher builds sharing in a fixed order because a listing depends on its parent exchange and subscribers depend on the listing for governed access.

1. Create exchange: Create the exchange in the publisher project to own and administer sharing policy.
2. Create listing: Add a listing to the exchange that references the source dataset or topic.
3. Publish and manage: Publish the listing for internal or external discovery, then manage membership, access, and lifecycle.

Subscribing creates a read-only linked dataset in the subscriber project that is a live pointer to the shared dataset, so queries always see current publisher data without a copy. When egress is restricted on a listing, copy, clone, export, snapshot, CREATE TABLE AS SELECT, CREATE VIEW AS SELECT, writing to a destination table, and BigQuery Data Transfer Service on the linked dataset become unavailable.

2.2Sharing streaming data and collaborating in clean rooms

A Pub/Sub (opens in a new tab) topic listing shares real-time streaming data through Analytics Hub instead of a static table snapshot. Data flows from the publisher-owned Pub/Sub topic to each authorized subscriber, so subscribers receive new events as they are published. The publisher retains ownership of the topic and its listing, while subscribers attach and consume within their own projects. Choose Pub/Sub topic sharing when consumers need continuous event streams rather than periodic queries.

A data clean room is an Analytics Hub environment for multi-organizational collaboration on sensitive data. The clean room allows participating organizations to bring governed datasets together and run joint analysis under shared controls, rather than exchanging raw copies outside any boundary. Control stays with each contributor, who administers what data enters the clean room and what analysis is permitted. Choose data clean rooms when the requirement is collaborative insight on restricted data where direct dataset sharing would expose too much.

2.3Protecting shared assets and monetizing listings

VPC Service Controls (VPC SC) adds a network and identity perimeter around shared Analytics Hub assets to protect against data exfiltration. The perimeter encloses trusted publisher and subscriber projects on the inside, and traffic that attempts to copy or move data to resources on the outside is denied even when IAM permissions would otherwise allow access. The shared asset depends on both its listing permission and the perimeter rule, so access requires passing both checks.

Google Cloud Marketplace (opens in a new tab) is the commercial channel for Analytics Hub assets. A publisher commercializes an existing exchange listing by offering it as a paid product, so discovery, entitlement, and billing flow through Marketplace to the external buyer. The trust model does not change: the publisher still owns the source and administers the listing, while the buyer receives only the access granted by the listing. Choose Marketplace commercialization when the goal is data monetization beyond internal or partner sharing, including direct event publishing flows (opens in a new tab) for event-driven products.

2.4Governing access with row-level and column-level controls

Robust access governance combines coarse project permissions with fine-grained filters so each consumer sees only entitled rows, columns, and results. In BigQuery, the publisher keeps the source dataset in its own project and shares it in place, while IAM roles, authorized resources, row-level policies, and column-level policies enforce the boundary between publisher storage and subscriber queries. Because BigQuery separates compute from storage, subscriber queries run with subscriber compute against publisher storage.

Authorized views, authorized datasets, and authorized routines including user-defined functions (UDFs), table functions, and stored procedures let subscribers query derived results without source table access. Sensitive Data Protection helps find columns to tag by scanning for PII, financial, and health data.

2.5Making listings discoverable and assigning costs

BigQuery sharing publishes data in place without replication, and catalogs make that data searchable through listings and governed metadata. A private data exchange is visible only to granted users or groups, while a public data exchange lets all authenticated Google Cloud users discover and subscribe. A complete listing carries description, sample queries, documentation links, subscriber instructions, contact email, and provider details, plus governed metadata such as glossary terms, lineage, and data quality results.

There is no additional charge for managing exchanges or listings. Cost-sharing follows resource type, with storage staying with the publisher and consumption billed to the subscriber:

BigQuery public datasets (opens in a new tab) extend the same model at internet scale. Public datasets are hosted by Google under projects such as bigquery-public-data, with Google paying for storage and each querier paying only for queries it runs, with the first 1 TB per month free. Subscribers find them in the console Explorer pane, through BigQuery sharing, through the bq command-line tool, or through the REST API, and must query with a fully qualified name such as bigquery-public-data.bbc_news.fulltext.

Publishing reports and visualizations is the delivery of governed metrics and dashboards to business users without exposing raw uncontrolled data. Looker is a business intelligence platform where data teams describe tables, joins, and business logic in LookML, and Looker generates SQL from that LookML when a user opens a Look, Explore, or dashboard. This creates a fixed dependency chain: the Explore depends on model and view files, the generated SQL depends on the Explore, and results flow back from the database to the visualization.

3.1Deploying governed Looker models and Explores

LookML models, views, and Explores form the governed layer that controls what users can query and how joins behave. A model file names the database connection and declares which Explores are available, while a view file defines the dimensions, measures, and field logic for a table or derived table. An Explore combines one or more views into a queryable starting point, so traffic flows from the user-selected Explore to the joined views to the database as generated SQL.

Looker development with Git version control separates personal work from shared content through isolated branches and ordered promotion. A developer works in Development Mode on an isolated Git branch, and only after commit and deployment does that content become visible in Production Mode for all users.

1. Set up and test Git connection: Connect the LookML project to its remote Git repository.
2. Configure version control settings: Set branch and repository options for the project.
3. Configure webhook deploy secret: Secure automated deployment signals from the repository.
4. Branch, commit, and deploy: Create a branch in Development Mode, commit changes, and deploy to Production Mode.

Teams can generate a basic LookML model automatically, create a project manually, or build with the visual modeling canvas. Looker Blocks are prebuilt LookML models maintained by the domain team that knows the underlying data model.

3.2Securing Looker connections and controlling access

A secure Looker database connection defines the trust boundary between Looker and the warehouse, so the connection must be secured before any model is published. Secure access starts with encrypted connections in transit, a tunnel server for any Secure Shell (SSH) tunnel, and an allow list that permits only Google-specific IP addresses for external access. The database grant follows least privilege, and Google is not given write or administrative access to customer databases. SQL Runner issues SQL directly against that connection to debug queries and build derived tables, with data flowing from SQL Runner to the database and back as raw results.

Looker access control combines identity authentication with data-level filtering enforced in the model. The access_filter parameter works together with user attributes to apply row-level, column-level, or field-level security, so two users querying the same Explore receive different rows based on attribute values. Operational hardening includes limiting administrator and SQL Runner privileges, requiring two-factor authentication or SAML single sign-on (SSO), restricting login_special_email to a maximum of two users when SSO is used, preventing sharing with excluded third-party services, and auditing users, groups, API keys, public links, and sharing settings at least quarterly.

3.3Automating dataset refresh and report delivery pipelines

Automating reporting pipelines means using versioned, scheduled orchestration to refresh datasets and publish reports (opens in a new tab) without manual steps. On Google Cloud, pipelines run as part of an enterprise platform where Cloud Build builds and promotes templates, Managed Service for Apache Airflow and Cloud Dataflow orchestrate movement and transformation, and Cloud Scheduler or Cloud Run functions trigger refreshes. Refreshed data flows from source systems through the orchestrated pipeline to BigQuery tables, Cloud Storage buckets, or Drive folders where visualizations read them.

A production refresh follows an ordered promotion path from commit to execution:

1. Git commit triggers build: A Git commit triggers a Cloud Build pipeline that validates code.
2. Cloud Build stages artifacts: Cloud Build copies the template and configuration files to Cloud Storage.
3. Continuous delivery promotes artifacts: Artifacts move to preproduction for end-to-end tests, and production updates only after tests pass.
4. Orchestrator launches execution: Managed Airflow runs batch workflows, Cloud Scheduler schedules batch jobs, or an event invokes a function that launches a Dataflow job.
5. Data lands for reporting: Transformed data flows from the Dataflow job to the reporting dataset in BigQuery or to Cloud Storage.

Batch pipelines support direct redeployment as a new Dataflow job, while streaming pipelines may require parallel Dataflow jobs during cutover to avoid disruption. For stored reports, Cloud Storage lifecycle controls manage retention, regional buckets replicate synchronously across zones while dual-region and multi-region buckets replicate asynchronously, and buckets used for report delivery must not retain a retention policy that would block updates. If Uniform Bucket Level Access with Domain Restricted Sharing blocks required principals, the workaround is to request Google default handling or grant access through an internal group.

3.4Connecting and governing Data Studio enterprise reports

Configuring Data Studio for enterprise reporting means connecting governed sources to reusable dashboards and controlling access, performance, and distribution. Data Studio is a self-service business intelligence platform for visualizations, dashboards, and reports. Data Studio connects directly to BigQuery tables, views, and custom queries, Apigee analytics, and logs exported to BigQuery. Data Studio Pro adds IAM permissions management, team workspaces, a mobile app, and support.

Connecting Data Studio to BigQuery means authorizing Data Studio and pointing it at a project-scoped dataset, table, view, or custom query. The connector workflow runs in order: create a report, select the BigQuery connector to create or reuse a data source, configure access for a table, view, or custom SQL with separate projects for data access and billing when needed, then add to canvas. Projects determine where query costs are billed when reports exceed free limits, and access requires a project with a billing account.

Managing access depends on the credential mode for the data source. To query BigQuery, an identity needs BigQuery Data Viewer to read data and BigQuery Job User to run jobs in the billing project.

Governing deployments defines network and audit boundaries. If the organization uses VPC Service Controls, the Data Studio service account must have egress and ingress rules that allow bigquery.googleapis.com and bigquerydatatransfer.googleapis.com. Every Data Studio query to BigQuery can be audited through INFORMATION_SCHEMA.JOBS using labels with requestor set to looker_studio plus report and data source IDs for the last 7 days.

3.5Sharing and optimizing Data Studio reports for distribution

Sharing and optimizing Data Studio reports controls whether recipients can view or edit and how fast dashboards load. Sharing opens the report, selects Share, enters email or Google Group addresses, chooses view or edit access, and sends the invitation. Prebuilt templates for Apigee, reCAPTCHA, and Backup and DR provide customizable charts that can be copied from view-only access. Published reports support email sharing, CSV export, PDF and PNG download, and scheduled email delivery, with data freshness settings controlling refresh cadence.

BigQuery BI Engine improves Data Studio performance while reducing compute costs, and teams reserve BI Engine capacity for dashboards that need faster response. Exploring query results from the BigQuery console creates dashboards backed by temporary tables stored for up to 24 hours, and charts built this way can visualize a maximum of 5,000 rows. In full Data Studio reports backed by BigQuery datasets, up to 2 million rows can be viewed and up to 750,000 rows can be exported from a chart. The connector also provides Record Count and automatic Google Analytics metrics, plus GEOGRAPHY polygon support with Google Maps. Data Studio is free for viewing and creating, while BigQuery access is billed with no charge for the first 1 TiB per month, consistent with Cloud Architecture guidance (opens in a new tab) for cost-aware report setup (opens in a new tab).

BigQuery (opens in a new tab) data sharing organizes collaboration into exchanges that hold listings, and listings that provision linked datasets for consumers. A data exchange is the organizational sharing boundary, a listing is the structured published entry that exposes a source dataset or table, and a linked dataset is the read-only consumer-side reference that queries the source. The provider project continues to host storage and governance while consumer projects attach through their own query execution, billing, and slot quotas.

4.1Designing exchanges, listings, and linked datasets for scale

Sharing requirements determine whether an exchange must be private to specific projects, users, and groups or public for broader discovery. BigQuery supports cross-project sharing where different projects can query datasets hosted in other projects. Separation of query execution lets autonomous teams work within their own projects by separating slot quotas and query billing from the projects that host datasets. Storage resources are allocated as data is consumed and deallocated as data is removed, so an exchange does not require pre-provisioned storage per consumer.

Location policy forms an additional boundary. The Organization Policy Service can restrict locations where datasets can be created, which constrains where exchanges and source datasets may exist. BigQuery determines the location to run jobs based on datasets referenced in the request, and a query accessing data in more than one location can be treated as a global query. Colocation preserves performance: Cloud SQL and Spanner federated queries require the source instance to be in the same location as the BigQuery dataset, Bigtable external tables require the same location with multi-region not recommended, and Managed Service for Apache Spark and Vertex AI Workbench should be colocated with the dataset. Data analytics planning (opens in a new tab) should include a geographic management plan for regional resources.

Regional availability is controlled by dataset location and automatic replication. Single-region storage replicates across zones in the same region and protects against zone issues, while multi-region storage replicates across regions and protects against zone or region unavailability. If multi-region storage is not available, BigQuery Data Transfer Service can copy a dataset to a different region, or tables can be backed up with exports scheduled through Cloud Scheduler or Managed Airflow using Avro with SNAPPY or JSON with GZIP, stored in Nearline or Coldline Storage with a lifecycle policy for deletion.

4.2Enforcing identity, perimeters, and tenant isolation on shared assets

Enforcing security on shared assets means combining project separation, granular IAM, network perimeters, and fine-grained controls. The shared data project holds sanitized information copied from the internal project, and the internal project owns the service account that updates the shared data. External access is managed through groups with viewer permission, while VPC Service Controls, authorized views, and row-level and column-level security enforce governance.

IAM in BigQuery controls who can publish, administer, and consume shared datasets through organization, folder, project, and resource hierarchy. Datasets are child resources of projects, and tables inherit permissions from their dataset. IAM provides basic roles such as Owner, Editor, and Viewer, predefined roles such as BigQuery Data Editor and Analytics Hub Admin, and custom roles for specialized needs. Stronger governance adds IAM Conditions, PAB policies, and Privileged Access Manager for just-in-time elevation. Service accounts are limited to a soft quota of 100 per project, and BigQuery concurrency defaults to 100 queries per issuing project.

VPC Service Controls creates a boundary that prevents accidental exposure outside internal projects. The recommended design uses an internal data perimeter that enforces BigQuery access and an externally shared data perimeter that hosts shareable datasets, with data flowing from internal projects across a perimeter bridge into external share projects only after the bridge allows egress. VPC Service Controls is limited to 100 perimeters including bridges per organization and 10,000 projects within perimeters organization-wide. When a project-per-tenant design would exceed that scale, a dataset-per-tenant design is preferred.

Authorized views and subset tables give each tenant access to only a subset of a large fact table:

Row-level security filters rows by user conditions, column-level access control uses policy tags such as 3_Confidential, 2_Private, and 1_Sensitive, and column masking obscures values while keeping rows readable. Encryption uses default at-rest and in-transit protection, with optional CMEK in Cloud KMS with Cloud HSM and AEAD encryption for tenant-sensitive fields. Analysis rules can be set only on views, not tables, and query templates built on table-valued functions share predefined queries without sharing underlying tables.

4.3Managing billing, performance, and auditability for subscribers

BigQuery sharing is a zero-copy exchange where a publisher shares a dataset in place and a subscriber queries it through a live pointer. The publisher pays for storing the shared BigQuery dataset, while the subscriber pays for queries it runs against the shared data. For streaming, the publisher pays for data written plus egress, while the subscriber pays for data read plus egress. There is no additional charge for managing exchanges or listings. Subscriber query cost uses on-demand pricing for bytes scanned or capacity-based pricing through reservations, with concurrency governed in the subscriber project and fair redistribution between new and running queries.

Query performance on a linked dataset is the performance of a query that reads publisher storage but executes in the subscriber project. Subscribers can view the query plan in the console and request execution details with INFORMATION_SCHEMA.JOBS or the jobs.get API to locate expensive stages. For BigLake tables over Cloud Storage, columnar formats such as Parquet or ORC perform better than JSON or CSV, Hive partitioning enables pruning, and file count and size must be balanced. If BigLake queries cannot meet requirements, converting to BigQuery storage and redefining authorized views generally does not disrupt consumers because the linked dataset still points to the same logical view, enabling materialized views and BI Engine.

Publisher visibility comes from provider usage metrics that track jobs, consumption by subscriber project and organization, and total rows and bytes processed. Detailed auditing uses telemetry query and view tools (opens in a new tab) with Cloud Logging, BigQuery audit logs, and INFORMATION_SCHEMA views to analyze who ran which job and how much data each job scanned. Usage for shared Pub/Sub topics appears in Metrics and Cloud Monitoring, including publish count, requests, throughput, top subscriptions, and ingested bytes, but not in INFORMATION_SCHEMA.SHARED_DATASET_USAGE. Billing for Marketplace listings is tracked through Marketplace orders, not provider usage metrics. Recovery is bounded: historical snapshots can be queried for up to seven days and deleted tables restored within two days using time travel, while agent analytics (opens in a new tab) and migration guidance (opens in a new tab) support ongoing optimization.

Key Facts

• BigQuery sharing publishes through data exchanges that contain listings, and subscribing creates a read-only linked dataset that queries publisher storage without copying data.
• Storage Legacy Bucket Owner can manage a Cloud Storage bucket and list its contents but cannot view object data, while Storage Object User can upload, view, update, and delete objects.
• An authorized view in BigQuery automatically inherits column-level policy tags from its base table, while a separate subset table must have each policy reapplied.
• The publisher of a shared BigQuery dataset pays for data storage while the subscriber pays for queries run against the shared data in the subscriber project.
• VPC Service Controls perimeters with ingress and egress rules plus Cloud Audit Logs, Cloud Logging, and Cloud Monitoring together control exfiltration and prove compliance for shared data.