Exporting Logs to External SIEM Systems

Aggregated log sinks are centralized routing rules in Google Cloud Logging that forward copies of log entries to a designated destination for security analysis. A sink combines a scope, a destination, an inclusion filter, and optional exclusion filters. The Log Router evaluates every incoming entry against those rules, which lets one pipeline carry high-fidelity audit, network, and threat telemetry from many projects to a central Cloud Storage bucket or external Security Information and Event Management (SIEM) system.

1.1Sink Scope and Destination Architecture

Sink scope defines which logs a sink can see, and the destination defines where matching entries are delivered for centralized detection and auditing. A project-level sink routes logs generated in that project, while a sink created at folder or organization scope aggregates matching entries from all child projects and folders into one destination. Data flows from Cloud Logging to the sink destination, and then from the destination to the SIEM feed, so the sink depends on the destination bucket or SIEM endpoint existing before routing starts.

For Google Cloud, the destination is a Cloud Storage bucket identified by a URI such as gs://gcp-ids-logs, and the sink's writer identity must be granted permission to write to that bucket. For Google Distributed Cloud air-gapped, the destination is configured with a SIEMOrgForwarder (opens in a new tab) custom resource that names the external SIEM instance, such as Splunk or Elastic SIEM. The SIEMOrgForwarder must live in the same project namespace as the Kubernetes Secret that holds its token or password, and a separate SIEMOrgForwarder resource is required for each log type when both audit and operational logs are exported.

1.2Inclusion Filter Design for Audit and Network Telemetry

Inclusion filters are advanced Logging query expressions that select exactly which entries a sink forwards, most often by combining resource type, log name, and payload fields. The filter runs against fields such as resource.type, which identifies the monitored resource, logName, which identifies the log stream, and jsonPayload fields, which carry connection, instance, and rule details. Because filters use AND, OR, and field matching, one sink can carry broad Compute telemetry while another sink carries only intrusion or DNS threat telemetry to a different bucket.

Cloud Intrusion Detection System (Cloud IDS) (opens in a new tab) telemetry, Compute Engine audit and flow telemetry, and DNS threat telemetry all follow the resource-plus-log-name pattern above. App Engine (opens in a new tab) and application telemetry follows the same pattern but requires resource.type="gae_app", with narrower filters for request logs or for stdout and stderr application logs. Compute (opens in a new tab) and network-focused filters can add payload conditions such as jsonPayload.connection.dest_ip:*, jsonPayload.connection.src_ip:*, or jsonPayload.rule_details.action=("ALLOW" OR "BLOCK") to keep only entries with connection detail or a specific enforcement action. Choose a broad resource-type filter when centralized visibility is needed, and add log-name and payload conditions when only high-fidelity security events should pay the cost of export and SIEM ingestion. DNS Threat Detector (opens in a new tab) filters use the DnsThreatDetector resource type and encoded log name shown in the table.

1.3Exclusion Rules, Writer Identity, and Batched Delivery

Exclusion rules are optional filters on the same sink that discard noisy entries after inclusion matching, so only security-relevant events reach the destination. The effective export set is the inclusion match minus the exclusion match, which preserves audit completeness while reducing volume, cost, and parser load in the SIEM. Identity and connectivity bound the whole pipeline: the Cloud Logging service account writer identity, which looks like service-123456789012@gcp-sa-logging.iam.gserviceaccount.com, needs Storage Object Creator on the destination bucket, while the SIEM reader needs Storage Object Viewer, or Storage Object Admin when the feed is allowed to delete transferred files.

A reliable export pipeline is built in an explicit order:

1. Establish scope and connectivity: Confirm organization permissions for the sink and, for air-gapped export, confirm the uplink connection from Google Distributed Cloud to the customer network with the Infrastructure Operator.
2. Create the destination and secret: Create the Cloud Storage bucket first, and for Splunk or Elastic create the Secret holding the token or password in the target project namespace.
3. Define name and destination: Create the sink in Logging greater than Log Router with a meaningful name and the bucket URI, including the trailing slash in feed configuration such as gs://dns-threat-detector-logs/.
4. Define inclusion and exclusion filters: Add the resource, log-name, and payload expression first, then add exclusion filters to remove low-value entries.
5. Grant writer access and verify flow: Grant the sink writer identity its write role, then verify that hourly batches begin arriving before connecting the SIEM feed.

Delivery is batched and retried rather than instantaneous. New sinks to Cloud Storage save routed entries in hourly batches, and the first entries can take 2 to 3 hours to appear. In air-gapped export, the logging stack retries the SIEM connection a default of five times, configurable by the Infrastructure Operator, and drops the audit batch if all retries fail.

Stream and ingest log telemetry into external SIEM platforms is the design of continuous pipelines that move security logs from Google Cloud and multicloud sources into a central analysis system. Telemetry flows from producers such as Cloud Intrusion Detection System (Cloud IDS) (opens in a new tab), Compute Engine, SentinelOne, Zscaler, and custom applications through either push-based ingestion or pull-based feeds into Google Security Operations (Google SecOps) (opens in a new tab), Splunk, Elastic SIEM, and other third-party SIEM solutions. The pipeline must preserve delivery reliability, enforce secure access, and normalize diverse formats into the Unified Data Model (UDM) for consistent detection and auditing.

2.1Push and Pull Ingestion Models for SIEM Delivery

Push and pull ingestion models define who initiates the transfer of telemetry into the SIEM and how quickly events become searchable. Push ingestion means the source or an intermediary sends events directly to the SIEM Ingestion API or a Webhook feed endpoint, which minimizes the time from event occurrence to detection and supports real-time threat response. Pull ingestion means Google SecOps polls customer storage using data feeds, which consolidates fragmented logs from multi-cloud, on-premises, and SaaS environments into one platform at scheduled or near real-time intervals. Both models normalize raw logs into UDM so a failed login from AWS and a failed login from Windows share the same schema for search and rules.

Push paths depend on domain and IP allowlisting for outbound API access, while V2 pull paths depend on STS workers reading the bucket and therefore require different allowlists and bucket permissions.

2.2Real-Time Push Feeds with Webhook and API Endpoints

Real-time push feeds deliver events directly to a Google SecOps feed without waiting for a polling cycle. The administrator creates the feed under SIEM Settings greater than Feeds greater than Add New Feed greater than Configure a single feed, selects Webhook, Third party API, Custom API, or Google Cloud Pub/Sub as the Source type, selects the Log type, and obtains a feed ID and customer ID. The source then sends HTTPS POST requests to a feed-specific URL in the form https://<CHRONICLE_REGION>-chronicle.googleapis.com/v1alpha/projects/<GOOGLE_PROJECT_NUMBER>/locations/<LOCATION>/instances/<CUSTOMER_ID>/feeds/<FEED_ID>:importPushLogs, where the region, project number, location, customer ID, and feed ID identify the destination instance and feed.

A Zscaler Nanolog Streaming Service deployment shows how push reliability is tuned at the source. Zscaler DLP and Zscaler CASB use Cloud NSS Feeds with NSS Type set for web traffic, SIEM Type set to Other, OAuth 2.0 Authentication set to Disabled, SIEM Rate left as Unlimited unless throttling is needed for licensing, and Max Batch Size set to a SIEM best practice such as 512 KB. The ingestion label on the feed, for example ZSCALER_DLP or ZSCALER_CASB, selects the parser that normalizes raw logs to UDM, while Asset Namespace separates overlapping asset names and Split delimiter defines how log lines are separated when no default framing exists. Because push originates from customer or Google infrastructure toward Google APIs, the firewall must periodically fetch and parse goog.json and cloud.json for third-party API paths and configure domain allowlisting for Ingestion API paths.

2.3Batch Pull Feeds from Cloud and Multicloud Storage

Batch pull feeds use Google SecOps feeds to fetch files staged in object storage. Supported V2 source types include Cloud Storage (V2), Amazon S3 (V2), Amazon SQS (V2), Azure Blob Storage (V2), and Cloud Storage Event Driven, all of which use Storage Transfer Service workers to pull data for near real-time ingestion. As of April 15, 2026 the v1 connector framework for Cloud Storage, Amazon S3, Amazon SQS, and Azure Blob Storage is deprecated, so new designs must use the V2 framework and allowlist the STS IP ranges published in ipranges.json.

Configuring a V2 pull feed follows an ordered dependency from bucket to feed to permissions:

1. Create the staging bucket: Create a globally unique Cloud Storage bucket in a location near the Google SecOps instance, select Standard storage class for frequently accessed logs, select Uniform access control, and configure encryption, versioning, or retention as needed.
2. Stage the logs: Write newline-delimited JSON or vendor-native logs to the bucket directly, via gsutil or client libraries, via SentinelOne (opens in a new tab) Cloud Funnel, or via a Storage Transfer Service job that copies from Amazon S3 to Cloud Storage.
3. Create the feed: In Google SecOps specify Source type as Google Cloud Storage V2, Log type matching the product, Storage bucket URL with a trailing slash such as gs://custom-security-analytics-logs/security-events/, Source deletion option, Maximum File Age defaulting to 180 days, Asset namespace, and Ingestion labels.
4. Grant the reader identity: Copy the unique Chronicle service account shown by Get Service Account and grant it access on the bucket, then verify feed Status is Active rather than Error or Paused.

Buffer capacity and lifecycle on pull feeds are controlled by deletion and age settings rather than by holding events in memory. Source deletion options are Never, Delete transferred files, and Delete transferred files and empty directories, where Never retains originals for testing and the delete options reclaim staging capacity only after successful transfer. Maximum File Age limits ingestion to files modified within the last N days, which bounds backfill volume after an outage. Amazon SQS (V2) adds queue-specific dependencies: Google SecOps must be explicitly granted permission to delete messages from the queue, and only one feed should consume a deprecated SQS queue because messages read by another application are not ingested by the current feed.

2.4Air-Gapped Forwarding to Splunk and Elastic Platforms

Air-gapped forwarding exports audit and operational logs from Google Distributed Cloud (GDC) (opens in a new tab) air-gapped to a customer-network SIEM over an uplink connection. The SIEMOrgForwarder custom resource acts as the forwarding configuration file that names the external SIEM instance, the log type, and the referenced credential, so administrators establish a streamlined pipeline without exposing the management plane directly to the SIEM. Before deploying the forwarder, the Organization IAM Admin must grant a SIEM Export Org creator, editor, or viewer role in the project namespace, and the Infrastructure Operator must confirm or establish uplink connectivity to the customer network.

Deployment enforces strict namespace and secret dependencies to protect SIEM credentials. First, the administrator creates an Opaque Secret in the project namespace holding a token for Splunk or a username and password for Elastic SIEM, using separate secrets and separate tokens for audit versus operational logs when isolation is required. Then the administrator deploys one SIEMOrgForwarder resource in the same namespace as the Secret for each log type, with audit and operational logs requiring separate resources and with splunkOutputs placed under the destinations field for release 1.17 and later. Delivery reliability is bounded by retry policy: the logging stack attempts to establish the SIEM connection and send audit logs up to five times by default, a count the Infrastructure Operator can reconfigure, and the stack drops the audit batch if all retries fail rather than blocking local logging indefinitely.

2.5Normalization, Monitoring, and Delivery Assurance

Normalization, monitoring, and delivery assurance make ingested telemetry searchable and prove that no feed is silently failing. Every feed applies a parser selected by Log type or Ingestion label that extracts fields from raw JSON, maps them to UDM, categorizes the event type, and enriches context such as network direction and resource type. When no prebuilt parser fits, the team requests a custom log type with 5 to 10 raw samples, builds a custom parser that maps source fields to UDM fields such as event timestamp, principal user, target IP, and security result action, tests it against samples, and then searches Raw Log Scan for metadata.log_type to verify arrival.

Operational visibility for feeds comes from feed metrics and from ingestion activity logs sent to Cloud Logging. Feed metrics expose throughput, errors, and Status transitions between Active, Error, and Paused, while ingestion activity logs under the chronicle-siem namespace allow filtering by log stream, ingestion mechanism, and feed ID or collector ID, including Storage Transfer Service logs for V2 pulls. Analyzing feed activity with Cloud Logging requires a Bring Your Own Project (BYOP) Google Cloud project and Logs Viewer or Private Logs Viewer roles, and it helps diagnose missing, delayed, or failing logs across both push and pull mechanisms. Together, UDM consistency, explicit deletion and age bounds, retry and batch limits, and continuous feed monitoring let security operations centralize detection while controlling cost, buffer growth, and audit completeness. Compute Engine (opens in a new tab) audit, flow, and firewall telemetry uses this same normalization path after Log Router staging.

A secure export pipeline moves telemetry from Cloud Logging through a log sink to a Cloud Storage bucket and then to an external SIEM system such as Google Security Operations (Google SecOps). The pipeline depends on three separate controls working together: a dedicated writer identity that can only write objects, a dedicated reader identity that can only read objects, and a hardened storage destination with encryption and access boundaries. In Google Distributed Cloud air-gapped environments, the same pattern uses a SIEMOrgForwarder (opens in a new tab) custom resource and an uplink connection to forward audit or operational logs to Splunk or Elastic SIEM.

3.1Least-Privilege Writer Identities for Log Sinks

A dedicated log sink writer identity is the Google-managed service account that Cloud Logging creates for each sink, typically in the form serviceAccount:service-123456789012@gcp-sa-logging.iam.gserviceaccount.com. Log entries flow from Cloud Logging to the sink and then from the sink to the destination bucket only when that writer identity has permission on the bucket. The least-privilege role for the writer identity on the destination bucket is Storage Object Creator (roles/storage.objectCreator), which allows creating objects without granting read, delete, or bucket administration rights.

Creating and managing the sink itself requires the Cloud Logging configuration role roles/logging.configWriter, while viewing and copying the writer identity is done through Log Router greater than View sink details. Granting broader roles such as Storage Admin (roles/storage.admin) on the bucket exceeds what the export path needs and widens blast radius if the logging identity is misused. For air-gapped export, least privilege also means requesting only the needed SIEM Export Org role — creator, editor, or viewer — for SIEMOrgForwarder resources in a specific project namespace.

The end-to-end write path follows an explicit order:

1. Filter: The sink selects entries with a Logging filter, such as resource.type="gae_app" or logName="projects/PROJECT_ID/logs/cloud-ids".
2. Write: Cloud Logging assumes the sink writer identity and writes hourly batches of sharded JSON files to the bucket, which can take 2 to 3 hours before first delivery.
3. Isolate: The Secret holding the Splunk token or Elastic password and the SIEMOrgForwarder that references it must live in the same project namespace, with separate secrets and separate forwarder resources for audit logs versus operational logs.

App Engine (opens in a new tab) export uses the gae_app filter in the first step, while Cloud Intrusion Detection System (Cloud IDS) (opens in a new tab) export uses the cloud-ids log-name filter.

3.2Reader Identities and Deletion-Based Access Boundaries

An external reader identity is the unique per-instance service account that Google SecOps uses to pull files from the staging bucket, for example chronicle-12345678@chronicle-gcp-prod.iam.gserviceaccount.com. Data flows from the bucket to the SecOps feed configured as source type Google Cloud Storage V2, and the feed depends on the reader identity having read access to the exact bucket URL with a trailing slash, such as gs://appengine-logs-export/. Each SecOps instance has a different service account, so an account copied from documentation or another instance will fail authentication.

The reader identity follows a strict privilege boundary tied to the feed deletion option:

Compute (opens in a new tab) audit, VPC flow, and firewall feeds use the same writer-to-bucket and bucket-to-reader separation.

3.3Bucket Encryption and Access Hardening

A storage destination is the Cloud Storage bucket that holds exported logs before SIEM ingestion, and its protection is set at creation time under access control and data protection options. The bucket uses Uniform access control and public access prevention, which can be locked by organization policy, so anonymous or broad access cannot be re-enabled below the policy. Optional protection tools include object versioning and retention policy, while location type, location, storage class, and labels determine residency, cost, and lifecycle behavior.

The bucket creation flow also includes a Data encryption selection that determines how object data is encrypted at rest. Because hierarchical namespace cannot be enabled on an existing bucket and dual-region buckets have separate options such as turbo replication and cross-bucket replication, encryption, namespace, and location choices must be fixed before the sink starts routing. Choosing Standard storage class for frequently accessed logs and setting Maximum File Age, with a default of 180 days on the SecOps feed, keeps hot telemetry available for detection without retaining every historical shard indefinitely.

3.4Perimeter-Controlled Telemetry Egress Paths

A perimeter-controlled egress path is a network and namespace boundary that limits how telemetry leaves Google Cloud toward an external SIEM. In air-gapped deployments, telemetry flows from the logging stack across an uplink connection to the customer network, and connectivity must be established with the Infrastructure Operator before any forwarder is deployed. The logging stack retries a failed SIEM connection a limited number of times, with a default of five configurable by the Infrastructure Operator, and then stops sending audit logs rather than queuing them without bound.

For bucket-staged export, the bucket acts as the controlled egress point between the producer side and the consumer side. The producer side is bounded by the sink filter, the writer identity, and Uniform access with public access prevention, while the consumer side is bounded by the SecOps feed URL, asset namespace, ingestion labels, and the reader identity. Keeping audit and operational forwarders separate, using separate tokens for each type, and requiring the Secret and SIEMOrgForwarder to share the same project namespace prevents one compromised credential or misconfigured forwarder from exposing both log classes across the perimeter.

Key Facts

• Aggregated log sinks at project, folder, or organization scope route matching Cloud Logging entries to a Cloud Storage bucket or SIEM, where folder and organization sinks aggregate logs from all child projects and folders.
• The effective export set of a sink is its inclusion filter match minus its exclusion filter match, using resource type, log name, and JSON payload fields to select high-fidelity audit, VPC flow, firewall, Cloud IDS, and DNS threat telemetry.
• New Cloud Storage sinks deliver hourly batches with a 2 to 3 hour delay before first delivery, and air-gapped SIEM forwarding retries failed connections five times by default before dropping the audit batch.
• The Cloud Logging sink writer identity requires Storage Object Creator on the destination bucket, while the Google SecOps reader requires Storage Object Viewer, elevated to Storage Object Admin only when the feed deletes transferred files.
• The SIEMOrgForwarder resource and its credential Secret must share the same project namespace, with separate resources and separate secrets for audit logs versus operational logs.