Authenticating Cloud API Calls with Service Accounts

Manage Authorization through IAM Roles and Scopes is the control of what a service account identity can access through two filters that must both allow the request. A service account (opens in a new tab) is the identity an application uses to call Google Cloud APIs, and its access is limited by Identity and Access Management (IAM) (opens in a new tab) roles plus the access scopes on the compute resource. IAM defines what the service account is allowed to do, while scopes can further limit what OAuth requests from that resource are allowed to do. Both configurations must allow access before the application can reach the resource.

1.1IAM Roles Define Service Account Permissions

IAM roles are the primary way to control what a service account can access across projects, folders, organizations, or single resources. A service account is a principal, so you can grant it a role on a project or on a specific resource such as a Cloud Storage bucket. When that service account is attached to a Compute Engine (opens in a new tab) virtual machine (VM) instance, any application running on that instance inherits the authorization from the role.

Follow the principle of least privilege by granting only the permissions the workload needs. Use predefined roles for common jobs, such as Storage Object Viewer (roles/storage.objectViewer) or Compute Admin (roles/compute.admin), and create custom roles when no predefined role matches. In production, do not grant the basic Owner, Editor, or Viewer roles, and create a dedicated single-purpose service account (opens in a new tab) for each application so a role change affects only that workload.

For example, a VM service account granted only three predefined roles has only the permissions in those roles:

• roles/compute.instanceAdmin.v1 for managing Compute Engine instances
• roles/storage.objectViewer for reading Cloud Storage objects
• roles/compute.networkAdmin for managing networks

1.2Access Scopes Limit OAuth Requests From a VM

Access scopes are the legacy method of authorization for a Compute Engine VM instance. An access scope defines the default OAuth 2.0 scopes used in requests made through the gcloud CLI and client libraries on that instance. Scopes apply on a per-VM basis for the life of the VM, can be set at creation or updated later, and have no effect unless the related API is enabled on the project.

Access scopes do not replace IAM and do not apply to calls made using gRPC. A new Compute Engine instance is automatically configured with a set of default scopes:

• Read-only access to Cloud Storage: https://www.googleapis.com/auth/devstorage.read_only
• Write access to write Compute Engine logs: https://www.googleapis.com/auth/logging.write
• Write access to publish metric data: https://www.googleapis.com/auth/monitoring.write
• Read-only access to Service Management for Google Cloud Endpoints: https://www.googleapis.com/auth/service.management.readonly
• Read or write access to Service Control for Google Cloud Endpoints: https://www.googleapis.com/auth/servicecontrol
• Write access to Cloud Trace: https://www.googleapis.com/auth/trace.append

Other common scopes include the full https://www.googleapis.com/auth/cloud-platform scope, full control of Compute Engine (https://www.googleapis.com/auth/compute), read-only Compute Engine (https://www.googleapis.com/auth/compute.readonly), and the logging write scope.

1.3How IAM Roles and Scopes Work Together

How IAM roles and scopes work together is a two-filter check where both controls must pass for OAuth requests from the gcloud CLI and client libraries. IAM restricts access based on the roles granted to the service account, while access scopes can further limit access to API methods without ever expanding it. The following comparison shows where each control is set and what it controls:

The best practice is to set the full cloud-platform scope (https://www.googleapis.com/auth/cloud-platform) on the VM and then control access with fine-grained IAM roles. For example, a VM with the cloud-platform scope plus roles/storage.objectViewer can only read objects, even though the scope is broad. In contrast, a VM with the narrow devstorage.read_only scope plus the broad roles/storage.objectAdmin role still cannot manage objects from the gcloud CLI or client libraries, because the read-only scope does not authorize write operations.

Evaluate Secure Credential Management Strategies is the choice between short-lived credentials obtained on demand and long-lived keys stored as files. Short-lived credentials (opens in a new tab) expire by default after 1 hour and are created by Google, while user-managed service account keys remain valid until deleted and must be protected by the holder. The secure pattern is to attach a service account to a workload or impersonate a service account to mint temporary tokens, and to use downloadable keys only when no alternative works.

2.1Long-Lived Keys Versus Short-Lived Credentials

Long-lived keys versus short-lived credentials differ in lifetime, handling, and risk when calling Google Cloud APIs. A service account key (opens in a new tab) is the private key of a user-managed RSA pair that signs a JSON Web Token (JWT) and exchanges it for an OAuth 2.0 access token, and anyone holding the key can authenticate as the service account without extra sign-in checks like 2-step verification. Short-lived credentials are OAuth 2.0 access tokens, OpenID Connect (OIDC) ID tokens, self-signed JWTs, or self-signed binary blobs created by the Service Account Credentials API from a Google-held key, so no private key file circulates.

Short-lived credentials create less risk because a leaked token stops working quickly, while a leaked key gives lasting access until it is rotated or deleted. Choose short-lived credentials for application calls to Google Cloud APIs, and choose a service account key only for systems that cannot use impersonation, attached service accounts, or workload identity federation.

2.2Google-Owned Keys Versus User-Managed Keys

Google-owned keys versus user-managed keys differ in who holds the private key and how it is used. Google-owned and managed key pairs are internal RSA pairs that Google uses to mint short-lived credentials for service accounts. The private key in a Google-owned pair is always held in escrow and can never be accessed directly, while the public key is published for signature verification at X.509, JSON Web Key (JWK), and raw endpoints. Google rotates actively used signing keys regularly with probabilistic ramp-up, and cached public keys should be kept for at most 24 hours but remain valid for at least 24 hours after retrieval.

User-managed key pairs are additional RSA pairs where Google stores only the public portion and the holder stores the private service account key. The Cloud Client Libraries can use a service account key file to obtain an access token automatically, or code can sign a JWT manually and exchange it for a token. Because possession of the private key equals the service account identity, keys must be stored in a hardware-based or software-based key store and never left in temporary directories, email, file shares, backups, source repositories, or program binaries.

2.3Service Account Impersonation

Service account impersonation (opens in a new tab) is the process where a source principal with permission creates short-lived credentials for a target service account without holding the target account key. The source principal, such as a user or a different service account with the Service Account Token Creator role, calls the Service Account Credentials API to mint an access token that carries the target service account identity to the destination API. Common flows include the gcloud CLI with the --impersonate-service-account flag, an external application authenticated by workload identity federation using a credential configuration file to impersonate a service account, and a service that mints temporary credentials for users.

Impersonation preserves accountability because Cloud Audit Logs record both the service account being impersonated and the identity that created the short-lived credential. Some services, including App Engine and Google Kubernetes Engine (GKE), do not log the creator identity, so dedicated service accounts per application and dedicated keys per machine help trace activity through the principalEmail and serviceAccountKeyName fields. Use impersonation for local development, external workloads, and temporary elevated access instead of downloading a key.

2.4Acquiring Tokens on Google Cloud Without Keys

Acquiring tokens on Google Cloud without keys uses the service account attached to the compute resource where code runs. When a Compute Engine virtual machine (VM) instance has an attached service account, workloads on that instance call Google Cloud APIs through the Cloud Client Libraries and ADC, which obtain short-lived credentials for the attached account from the platform. Data flows from the workload to the client library to the platform token service, and then the access token flows from the platform back to the workload for the API call.

Attachment is the trust boundary: for most services the service account must be attached when the resource is created, while Compute Engine allows attaching a service account to a VM at any time. This attached-identity path is the preferred authentication method for code running on Google Cloud because no private key is distributed and Google manages rotation. For domain-wide delegation without keys, the workload follows an ordered flow:

1. Authenticate first: Authenticate with an attached service account, Workload Identity Federation for GKE, or workload identity federation.
2. Construct JWT: Construct a JWT and set the sub claim to the user email for delegated access.
3. Sign JWT: Call the signJwt method of the Service Account Credentials API to sign the JWT.
4. Exchange for token: Pass the signed JWT to the OAuth 2.0 token resource to obtain an access token.

2.5Self-Impersonation Restrictions

Self-impersonation restrictions block a service account from using its own short-lived credential to generate a new credential for that same service account. The Service Account Credentials API prohibits using a short-lived credential to generate a new access token for the same account, and prohibits using a short-lived credential to sign a blob or JWT for calling the Identity and Access Management API, the Service Account Credentials API, or the OAuth 2.0 API. Google Cloud blocks these paths because they would let an attacker with a stolen token refresh it indefinitely.

A prohibited attempt returns FAILED_PRECONDITION: You can't create a token for the same service account that you used to authenticate the request. Recovery requires minting the new credential from different source credentials, such as end-user credentials or a different service account.

2.6Reducing Risk When Keys Cannot Be Avoided

Reducing risk when keys cannot be avoided relies on rotation, expiry, policy guardrails, and detection. Rotation replaces keys on a schedule so a leaked key is more likely invalid by the time it is abused, and an established rotation process enables fast response to suspected compromise. Expiry limits validity automatically, which suits temporary non-production development or third-party tools that require keys, but expiry should be avoided for production workloads, permanent non-production access such as continuous integration (CI), and rotation systems that already bound key age.

Organization policy adds guardrails around key sprawl and exposure. Constraints can limit which projects can create keys, and the Service Account Key Exposure Response constraint set to DISABLE_KEY causes Google Cloud to automatically disable detected leaked keys with disable_reason, extended_status, and extended_status_message metadata. Detection is not guaranteed, so keys should still be minimized, unused keys identified with metrics, externally sourced credential files validated with a type value of service_account, and uploaded X.509 certificates kept free of optional attributes that would become public.

Implement Application Default Credentials (ADC) is the use of a standard credential-discovery strategy so the same code runs locally and on Google Cloud. Application Default Credentials (ADC) (opens in a new tab) is a strategy used by Google authentication libraries to automatically find credentials based on the application environment. The authentication libraries make those credentials available to Cloud Client Libraries and Google API Client Libraries, which then use them to call Google Cloud services and APIs. When an application uses ADC, the same authentication code can run in a local development environment and in production without changing how the application authenticates.

3.1How ADC Discovers Credentials

How ADC discovers credentials is an ordered search in which the authentication libraries check three locations in sequence and use the first credentials they find. Data flows from the application code to the authentication libraries, and then from the authentication libraries to the client libraries that attach the credentials to API requests. The search order does not indicate which option is most secure or most recommended for production use.

1. GOOGLE_APPLICATION_CREDENTIALS environment variable: ADC checks whether the variable points to a credential JSON file.
2. Local ADC file from gcloud CLI: ADC checks the well-known file created by gcloud auth application-default login.
3. Attached service account from metadata server: ADC requests credentials for the service account attached to the Google Cloud resource where the code is running.

If credentials are found in the environment variable, ADC never checks the local file or the metadata server. If no local credentials exist, code running on Google Cloud falls through to the metadata server, which returns credentials for the attached service account.

3.2Credentials From the GOOGLE_APPLICATION_CREDENTIALS Environment Variable

Credentials from the GOOGLE_APPLICATION_CREDENTIALS environment variable come from the credential JSON file at the path stored in that variable. ADC reads the file at that path and supplies its identity to the client libraries, so the application authenticates as whatever identity the file represents. This mechanism is useful when the credentials must come from a specific file rather than from the default local login location.

The JSON file can be one of the following supported types:

• A credential configuration file for Workforce Identity Federation, which lets an external identity provider authenticate users to Google Cloud resources.
• A credential configuration file for Workload Identity Federation, which lets an external identity provider authenticate workloads to Google Cloud resources.
• A service account key, which creates a security risk and is not recommended because a compromised key can be used by an attacker without any additional information.

3.3Credentials From Local Login for Development

Credentials from local login for development are the JSON credentials created by the gcloud auth application-default login command on a workstation. The command stores user credentials, or impersonated service account credentials, in a well-known location on the workstation: $HOME/.config/gcloud/application_default_credentials.json on Linux and macOS, and %APPDATA%\gcloud\application_default_credentials.json on Windows. These ADC credentials are distinct from gcloud CLI credentials, which are the separate credentials the gcloud CLI itself uses to manage Google Cloud.

By default, access tokens generated from a local ADC file created with user credentials include the cloud-wide scope https://www.googleapis.com/auth/cloud-platform. Custom scopes can be requested with the --scopes flag, and access to services outside Google Cloud such as Google Drive requires extra setup. For OAuth authentication, create an OAuth client ID and pass it with the --client-id-file flag plus --scopes; for service account authentication, create a service account and pass its email with the --impersonate-service-account flag plus --scopes. Service account impersonation for a local ADC file requires the iam.serviceAccounts.getAccessToken permission, included in the Service Account Token Creator role (roles/iam.serviceAccountTokenCreator), and is supported only for the Go, Java, Node.js, and Python client libraries.

3.4Credentials From the Attached Service Account in Production

Credentials from the attached service account in production are short-lived credentials for the user-managed service account bound to the compute resource. The attached service account is the identity attached to the resource where production code runs, such as a Compute Engine virtual machine (VM) instance. When ADC finds no credentials in the environment variable or local file, traffic flows from the application to the metadata server, and the metadata server returns short-lived credentials for that attached service account. Using the attached service account (opens in a new tab) is the preferred method for production code running on Google Cloud because no long-lived key file is stored with the application.

Setting up production ADC depends on creating a least-privilege identity and binding it to the resource before the code needs it:

1. Create a user-managed service account: Create a dedicated service account for the workload instead of using the highly privileged default service account.
2. Grant least-privilege IAM roles: Grant only the predefined or custom roles the workload needs, and avoid Owner, Editor, or Viewer in production.
3. Attach the service account to the resource: Attach the service account to the resource that will run the code, and grant the deploying principal the Service Account User role when required.

Timing matters because most Google Cloud services require the service account to be attached when the resource is created, with no later replacement. Compute Engine is the exception because it lets an operator attach a service account to a VM instance at any time.

Key Facts

• Authorization for OAuth calls from a Compute Engine VM requires both the IAM role on the service account and the access scope on the VM to allow the request.
• Setting the full https://www.googleapis.com/auth/cloud-platform scope on a VM and controlling access with fine-grained IAM roles follows best practice.
• Short-lived credentials expire by default after 1 hour, while user-managed service account keys remain valid until deleted and each service account can have up to 10 keys.
• Application Default Credentials (ADC) searches in fixed order: GOOGLE_APPLICATION_CREDENTIALS file first, then the local file from gcloud auth application-default login, then the attached service account from the metadata server.