Prepare and test your skills

Prepare and test your skills

An integrated data ecosystem is a connected set of pipelines, analytics services, and shared datasets that moves data from where it is created to where decisions are made. In Google Cloud, operational databases and business applications feed unified pipelines, BigQuery (opens in a new tab) provides large-scale analysis, and sharing services distribute trusted results to many teams. This connection shortens the path from raw information to action and helps different teams work from the same evidence.
Unified data pipelines are automated paths that capture changes in source systems and deliver them to analytics destinations without manual copying. Datastream (opens in a new tab) is a serverless change data capture (CDC) and replication service that synchronizes data reliably and with minimal latency. Datastream supports database sources such as MySQL, Oracle, PostgreSQL including AlloyDB for PostgreSQL, SQL Server, MongoDB, and Spanner, plus application sources such as Salesforce, Microsoft Dataverse, and ServiceNow. Because Datastream is serverless, there are no resources to provision and the service scales up and down automatically as data volumes shift.
Data flows from sources to analytics destinations through an ordered sequence:
A left-to-right process flow showing operational databases and apps captured by Datastream, landed in BigQuery and Cloud Storage, then distributed via Dataflow, with Knowledge Catalog lineage underneath.
Centralized metadata and lineage management with Knowledge Catalog (formerly Dataplex Universal Catalog) lets teams view Datastream resources in the context of all data sources across Google Cloud.
Real-time analytics is the ability to analyze new operational changes within seconds or minutes so teams can act while the information is still current. Datastream enables near real-time insights in BigQuery by replicating operational changes with low latency and with minimal impact to source performance. BigQuery then applies massive processing power to large volumes of fresh data for reports, dashboards, and machine learning. This combination supports analytics, database replication, cloud migration and hybrid-cloud synchronization, and event-driven architectures.
Faster access to current data increases efficiency because teams spend less time waiting for extracts and managing infrastructure. Teams can focus on deriving up-to-date insights and responding to high-priority issues instead of performance tuning or resource provisioning. Accurate and reliable replication with transparent status reporting also helps maintain confidence when data and schemas change.
Shared datasets are curated data assets that multiple teams can discover, request, and reuse instead of maintaining separate copies. Analytics Hub, built on BigQuery, supports sharing through exchanges and listings within or outside an organization, including sharing real-time streaming data with Pub/Sub topics. Knowledge Catalog organizes sharing further through data products, which are logical, curated collections of data assets packaged to be discoverable, trusted, and accessible for a specific business problem.
A data product named Ecommerce Business Data, for example, could package BigQuery tables such as order_details and user_traffic with a view such as user_demographics that excludes sensitive fields. Each data product links assets, people, and rules: an asset is a pointer to a physical resource such as a BigQuery dataset, table, or view; an access group maps friendly roles such as Reader or Analyst to Google Groups or service accounts; a data product owner manages quality, access, and documentation; a data product consumer uses the product to generate insights; and a contract defines expectations such as refresh schedule and quality standards.
| Sharing option | What is shared | Privacy and control |
|---|---|---|
| Analytics Hub exchanges and listings | Curated tables, views, and streaming Pub/Sub topics for use in BigQuery | Sharing within or outside the organization, with VPC Service Controls available to help prevent data exfiltration and with commercialization through Google Cloud Marketplace |
| Knowledge Catalog data products | Logical grouping of one or more assets with descriptions, documentation, and contracts | Self-service evaluation with access groups and default or asset-specific Identity and Access Management (IAM) roles |
| Data clean rooms | Joint analysis across multiple parties built on BigQuery scalability | Security-enhanced environment to share, join, and analyze data assets without moving or revealing the underlying data |
Exam tip: Choose data clean rooms when parties need joint analysis of sensitive data without revealing the underlying data, while Analytics Hub listings and data products are for distributing discoverable datasets with managed access.
Trusted visibility means every user can see where data came from, how it changed, and who depends on it. Data lineage in Knowledge Catalog provides a visual map that tracks the entire lifecycle of data, showing its origin, its destinations, and all transformations along the way. Supported Google Cloud services such as BigQuery and Dataflow automatically report lineage events when data is moved or transformed, and the map can be viewed in the Google Cloud console or retrieved through the Data Lineage API.
This visibility directly supports faster and safer decisions across functions. Trust and verification improve because teams can confirm numbers come from a trusted source. Troubleshooting improves by tracing an error in a final report back through every step to its root cause. Change management improves by identifying every downstream report or model that relies on a column or table before changing it. Compliance improves by showing how sensitive customer or financial information is used across the organization.
AI and machine learning in Google Cloud turn previously untapped unstructured data into business value by extracting meaning that people and traditional databases cannot easily search or analyze. Structured data already lives in rows and tables, while unstructured data lives in documents, images, audio, video, and free-form text. Cloud services collect and store both types at scale, and AI services convert the unstructured content into structured signals that can drive automation, prediction, and new insights.
Unstructured data holds most of an organization's real-world information, but it is hard to use without AI. Examples include scanned contracts, invoices, support emails, product photos, call-center recordings, and chat messages. Traditional queries cannot count, sort, or summarize this content because it has no fixed schema. Google Cloud unlocks value by combining scalable storage and analytics, such as BigQuery, with AI that reads text, images, and speech and outputs usable data for analysis and action.
Vertex AI is a fully managed, unified AI development platform for building and using both predictive machine learning and generative AI. The platform provides pre-built models for common tasks so teams can start quickly without building models from scratch. Teams can also perform custom model training with their own data to tailor solutions to specific needs. Built-in machine learning operations (MLOps) capabilities support the full lifecycle, where models move from training to deployment to monitoring and management in one consistent workflow with familiar tools and APIs.
The platform approach gives organizations three related advantages:
Together, these capabilities let the same data flow from raw files to trained models to live predictions without rebuilding the toolchain for each step.
AI extraction services convert unstructured inputs into text and structured outputs that analytics and applications can use. Optical Character Recognition (OCR) extracts text from images and documents, so a scanned invoice becomes searchable fields. Speech-to-Text converts spoken language into written text, so a customer call becomes a transcript that can be searched, summarized, and analyzed. Vertex AI Translation translates text between multiple languages, so the same content can be used across regions and teams.
A four-step process flow showing unstructured content like documents, images, and audio flowing through AI extraction services (OCR, Speech-to-Text, Generative AI) into BigQuery analytics and machine learning, and finally into business value through automation, prediction, and insights.
Generative AI on Vertex AI builds on this foundation by creating new content and answers from the extracted meaning. An application sends documents, transcripts, or images to a generative model, and the model returns summaries, classifications, answers, or drafted content. Vertex AI Workbench supports this work by providing a managed JupyterLab notebook environment where analysts and developers explore data, test models, and build machine learning workflows.
| Service | Input it reads | Output it creates |
|---|---|---|
| Optical Character Recognition (OCR) | Images and documents, including scans and photos | Machine-readable text extracted from visual content |
| Speech-to-Text | Audio with spoken language, such as calls and recordings | Written transcripts that can be searched and analyzed |
| Generative AI | Text, documents, and other combined context | New summaries, classifications, answers, and generated content |
Once text flows from the source file or audio stream to the AI service, the resulting output can flow onward to storage, analytics, and automation systems.
Exam tip: Choose OCR when the input is an image or scanned document, and choose Speech-to-Text when the input is spoken audio; both produce text that generative AI and analytics can then use.
Extracted meaning creates business value when it feeds automation and prediction. Automation happens when document text flows directly into a business process, such as approving an invoice, routing a claim, or updating a record without manual retyping. Prediction happens when historical examples train a model that then scores new cases, such as forecasting demand, flagging risk, or recommending a next action. New insights happen when the structured outputs land in analytics platforms where teams can query, visualize, and share them.
Three Google Cloud paths connect AI outputs to decisions. Online Prediction deploys a trained model so applications can send new data and receive a prediction in return. BigQuery ML lets analysts build and train machine learning models directly with SQL queries inside BigQuery, so teams that know analytics can create predictions without moving data to a separate system. Gemini assistance and business intelligence tools in BigQuery and Looker then help users explore results, so raw documents and audio become dashboards, alerts, and automated actions that improve efficiency and customer experience.
Cloud-native storage and processing create business value by letting an organization collect structured and unstructured data at any scale and turn that raw information into answers without managing servers. BigQuery (opens in a new tab) is the central analysis platform, Cloud Storage holds raw files and bulk data for loading or direct querying, and Dataflow builds pipelines that process batch and streaming data in real time. Together they remove the capacity limits, resource conflicts, and maintenance work that restrict on-premises databases.
BigQuery is a fully managed, AI-ready data platform for managing and analyzing data with built-in machine learning, search, geospatial analysis, and business intelligence. Analysts use familiar SQL and Python with zero infrastructure management, so they focus on questions rather than provisioning servers. BigQuery provides a uniform way to work with both structured and unstructured data, supports open table formats like Apache Iceberg, Delta, and Apache Hudi, and its distributed engine can query terabytes in seconds and petabytes in minutes.
BigQuery architecture separates into two independent parts: a storage layer that ingests, stores, and optimizes data and a compute layer that provides analytics capabilities. The two layers communicate over Google's petabit-scale network, so each layer can dynamically allocate resources without impacting the performance or availability of the other. Legacy databases usually share one resource pool for reads, writes, analysis, and management tasks like assigning permissions, which creates conflicts and slows queries while data is written or read.
This separation also lets storage and compute improvements deploy independently without downtime or loss of performance. Because the BigQuery engineering team handles updates and maintenance, customers do not provision or manually scale resources. The result is a fully managed serverless warehouse where operational work shifts from database administration to delivering analysis.
BigQuery storage stores data in a columnar format optimized for analytical queries and presents data as tables, rows, and columns with full ACID transaction semantics. BigQuery storage is automatically replicated across multiple locations to provide high availability. Data can arrive continuously for immediate analysis or in bulk from files, and data can also stay where it lives and still be queried.
BigQuery supports three ingestion patterns that differ by how data moves from its source to analysis:
| Ingestion pattern | How data moves | Sources supported |
|---|---|---|
| Streaming ingestion | Data flows continuously from producers through the Storage Write API into BigQuery tables | Real-time event and application data |
| Batch loading | Data flows in bulk from files into BigQuery tables | Local files or Cloud Storage objects |
| Federated query | Data stays in place while the BigQuery compute layer queries it through external tables | Cloud Storage, Bigtable, Spanner, or Google Sheets in Google Drive |
Batch loading from local files or Cloud Storage accepts multiple file and database formats:
Automated ingestion is handled by BigQuery Data Transfer Service, which moves data on a schedule without custom loading code. This combination lets structured records and previously untapped unstructured files follow the same path from Cloud Storage or live streams into governed tables for analysis.
Exam tip: Use loaded tables when BigQuery should store and optimize the data, and use external tables with federated queries when BigQuery should analyze data where it lives in Cloud Storage, Bigtable, Spanner, or Google Sheets.
BigQuery analytics is the capability to run descriptive and prescriptive analysis, including business intelligence, ad hoc analysis, geospatial analytics, and machine learning, on data stored in BigQuery or on external data. Users query with ANSI-standard SQL with joins, nested fields, aggregations, and spatial functions, or with a Pandas-compatible Python API through BigQuery DataFrames. Results can be shared as views, explored in BI Engine with Looker, Data Studio, Google Sheets, Tableau, and Power BI, and extended with BigQuery ML for predictive models and BigQuery Studio notebooks with version control.
Processing before analysis is handled by Dataflow, which develops batch and streaming data processing pipelines, and by Pub/Sub, which ingests event streams from anywhere at any scale. In a typical flow, event data flows from producers to Pub/Sub, then through Dataflow pipelines for cleaning and transformation, then into BigQuery tables, where the BigQuery compute layer queries it. Built-in governance powered by Knowledge Catalog helps teams discover data with semantic search, validate it with data lineage and quality controls, and share data and AI assets with access control.
Cloud-native services remove operational barriers by replacing fixed capacity and manual maintenance with serverless scaling and centralized control. Teams access BigQuery through the Google Cloud console, command-line tool, client libraries for Python, Java, JavaScript, and Go, REST and RPC APIs, and ODBC and JDBC drivers, so existing tools continue to work. Identity and Access Management (IAM) secures data and compute resources with the same access model used across Google Cloud, while centralized administration manages jobs that load, export, query, or copy data.
On-premises value is limited because shared pools strain under concurrent reads, writes, and permission changes, and staff must provision hardware for peak demand. BigQuery avoids that tradeoff because storage growth and query demand scale independently, streaming keeps fresh data available for analysis, and governance features keep diverse datasets discoverable and trusted. Organizations can therefore analyze more data types, more quickly, without adding database management work.
Use data clean rooms when multiple parties need joint analysis of sensitive data without moving or revealing the underlying data. Analytics Hub listings and Knowledge Catalog data products are for distributing discoverable datasets with managed access, while clean rooms provide a security-enhanced environment built on BigQuery scalability for shared analysis.
Use Optical Character Recognition (OCR) when the input is an image or scanned document such as a scanned invoice, and use Speech-to-Text when the input is spoken audio such as a customer call. Both produce machine-readable text that generative AI and analytics can then search, summarize, and analyze.
Use loaded tables when BigQuery should store and optimize the data, and use external tables with federated queries when BigQuery should analyze data where it lives. Federated queries let the BigQuery compute layer query data in place in Cloud Storage, Bigtable, Spanner, or Google Sheets in Google Drive.
A healthcare organization wants to eliminate manual data entry and unlock business value from its previously untapped unstructured "dark data." Specifically, the organization has the following requirements:
Which combination of Google Cloud services should the organization implement to meet these requirements?
Cloud Translation, Text-to-Speech, and Vertex AI TensorBoard
Video Intelligence API, Dialogflow CX, and Model Monitoring
AutoML Vision classification, Transcoder API, and Recommendations
Document AI, Speech-to-Text, and Vertex AI
Last updated