vLLM

Related topics

• GKE works with vLLM From the official documentation: “GKE supports these quantization options with vLLM:”
• Google Kubernetes Engine works with vLLM From the official documentation: “Deploy a vLLM container to your GKE cluster.”
• Cloud TPU works with vLLM From the official documentation: “Cloud TPU integrates with vLLM using the `tpu-inference` plugin, which supports JAX and PyTorch models.”
• Gemini Enterprise Agent Platform works with vLLM From the official documentation: “Gemini Enterprise Agent Platform uses an optimized and customized version of vLLM.”
• Google Cloud Storage works with vLLM From the official documentation: “The prebuilt vLLM container can download models from Hugging Face or from Cloud Storage.”
• host memory cache is part of vLLM From the official documentation: “Vertex Model Garden’s prebuilt vLLM additionally supports caching prefix kv in the VM host memory.”
• Model Garden works with vLLM From the official documentation: “The vLLM Cloud GPUs serving container is integrated into Model Garden the playground, one-click deployment, and Colab Enterprise notebook examples.”
• Prefix caching works with vLLM From the official documentation: “Try out Prefix Caching with vLLM”
• Speculative Decoding works with vLLM From the official documentation: “In the following section, you will try out speculative decoding with vLLM.”
• BigQuery works with Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• Cloud Logging works with Customer-managed encryption keys From the official documentation: “To use Cloud Logging with Customer-managed encryption keys (CMEK), you must complete the steps in the [Enable CMEK for an organization](/logging/docs/routing/managed-encryption#enable) page in the Cloud Logging documentation.”
• BigQuery works with Cloud Monitoring From the official documentation: “You can use Monitoring to export your metrics to BigQuery.”
• BigQuery is an alternative to Google Cloud Storage From the official documentation: “The training data can be either a CSV file in Cloud Storage or a table in BigQuery.”
• Cloud Logging works with GKE From the official documentation: “Enable Cloud Logging on GKE Clusters”
• Compute Engine is an alternative to GKE From the official documentation: “To use TPU7x, you can use Google Kubernetes Engine (GKE) or Compute Engine.”
• BigQuery works with Google Cloud Storage From the official documentation: “Load Cloud Storage data into BigQuery”
• Cloud Monitoring works with GKE From the official documentation: “Enable Cloud Monitoring on GKE Clusters”
• BigQuery works with Gemini Enterprise Agent Platform From the official documentation: “The BigQuery dataset stores all versions for the associated Gemini Enterprise Agent Platform dataset.”
• Cloud Run is an alternative to GKE From the official documentation: “The app uses Cloud SQL, Vertex AI, and either Google Kubernetes Engine (GKE) or Cloud Run.”
• Customer-managed encryption keys secures BigQuery From the official documentation: “You can either create CMEKs manually to protect your BigQuery resources or use Cloud KMS Autokey.”
• Customer-managed encryption keys secures Gemini Enterprise Agent Platform From the official documentation: “Enable customer-managed encryption keys (CMEK) for the Gemini Enterprise Agent Platform to gain more control over data encryption and key management.”
• BigQuery depends on Cloud Logging From the official documentation: “BigQuery utilizes Cloud Logging for some of your log data.”
• Cloud Logging works with Compute Engine From the official documentation: “Compute Engine sends you Cloud Logging events for scheduled, started, or completed maintenance.”
• Cloud Run works with Google Cloud Storage From the official documentation: “The Cloud Run function will receive syslog messages from FortiEDR via HTTP and write them to GCS.”
• Compute Engine works with Google Cloud Storage From the official documentation: “The usage export bucket is a Google Cloud Storage bucket where Compute Engine usage reports are stored.”
• Compute Engine is an alternative to Google Kubernetes Engine From the official documentation: “You can install MongoDB on a Compute Engine instance or you can deploy MongoDB on Google Kubernetes Engine.”
• GKE depends on Compute Engine From the official documentation: “For example, GKE deploys clusters on Compute Engine VMs.”
• Model Garden is part of Gemini Enterprise Agent Platform From the official documentation: “The following open models are offered as managed APIs on Gemini Enterprise Agent Platform Model Garden (MaaS):”
• BigQuery works with Cloud Run From the official documentation: “You can then use Cloud Run to transcode data present in the bucket and move it to BigQuery.”
• Cloud Logging works with Cloud Monitoring From the official documentation: “If a log entry matches the filter of the alerting policy, then Cloud Logging extracts labels and forwards the data to Cloud Monitoring.”
• Cloud Logging works with Cloud Run From the official documentation: “Cloud Logging captures, stores, and analyzes Cloud Run logs.”
• Cloud Logging is an alternative to Google Cloud Storage From the official documentation: “When you specify your own service account for builds, you must store your build logs in a Cloud Logging bucket, in a user-created Cloud Storage bucket, or in a Google Cloud-created and user-owned Cloud Storage bucket.”
• Cloud Logging works with Google Kubernetes Engine From the official documentation: “As an example, if you are using GKE, you can [use Cloud Logging to query](/logging/docs/view/building-queries) for GKE cluster audit logs.”
• Cloud Monitoring works with Cloud Run From the official documentation: “Cloud Monitoring records metrics regarding the execution of Cloud Run.”
• Cloud Monitoring works with Compute Engine From the official documentation: “Many Google services like Compute Engine send telemetry to Cloud Monitoring.”
• Cloud Run works with GKE From the official documentation: “Leverage Google Kubernetes Engine (GKE) and Cloud Run to deploy applications to the Google Cloud, optimizing costs, performance, and management overhead.”
• Cloud Run works with Google Kubernetes Engine From the official documentation: “Improved security: take advantage of automatic IAM JWT injection for more secure communication between GKE and Cloud Run.”
• Cloud TPU works with Compute Engine From the official documentation: “You can use TPUs through Compute Engine, Google Kubernetes Engine, and Vertex AI.”
• Cloud TPU works with Gemini Enterprise Agent Platform From the official documentation: “You can use TPUs through Compute Engine, Google Kubernetes Engine, and Gemini Enterprise Agent Platform.”
• Customer-managed encryption keys secures Cloud Logging From the official documentation: “Cloud Logging supports encrypting Log storage with CMEK keys.”
• Customer-managed encryption keys secures Compute Engine From the official documentation: “In IL5, CMEK is enabled by default for Compute Engine, Google Kubernetes Engine (GKE), Cloud Storage, and Cloud SQL.”
• Customer-managed encryption keys secures GKE From the official documentation: “CMEKs encrypt data at rest in Cloud SQL, Cloud Storage, and within GKE clusters.”
• Customer-managed encryption keys secures Google Kubernetes Engine From the official documentation: “In IL5, CMEK is enabled by default for Compute Engine, Google Kubernetes Engine (GKE), Cloud Storage, and Cloud SQL.”
• Google Kubernetes Engine depends on BigQuery From the official documentation: “This happens because Google Kubernetes Engine depends on BigQuery and Managed Service for Apache Airflow inherits this dependency from the Kubernetes Engine API.”
• Google Kubernetes Engine depends on Compute Engine From the official documentation: “Other products, such as [Google Kubernetes Engine](/kubernetes-engine) and [Dataflow](/dataflow), use Compute Engine resources and therefore, can be deployed in the same regions or zone.”
• BigQuery works with Compute Engine From the official documentation: “Compute Engine integrates with Google Cloud technologies such as Cloud Storage, Google Kubernetes Engine, and BigQuery, to extend beyond the basic computational capability to create more complex and sophisticated applications.”
• BigQuery works with GKE From the official documentation: “Subsequently, by integrating BigQuery with GKE, you can use BigQuery to efficiently handle massive datasets (like Google Analytics) and the model's natural language understanding capabilities to generate meaningful insights.”
• Cloud Logging is an alternative to Cloud Monitoring From the official documentation: “To see the status of your invalidation request or find a definitive list of recently submitted requests, you can use Cloud Logging or Cloud Monitoring.”
• Cloud Logging works with Cloud TPU From the official documentation: “Cloud TPU logs upcoming maintenance events in Cloud Logging by the individual VM instance, for example, `t1v-n-5bdca789-w-0`.”
• Cloud Logging works with Gemini Enterprise Agent Platform From the official documentation: “This page provides information about how to use Cloud Logging to view details about a Gemini Enterprise Agent Platform model.”
• Cloud Logging works with Google Cloud Storage From the official documentation: “See [Routing and storage overview](/logging/docs/routing/overview) to route logs from Logging to Cloud Storage, BigQuery, or Pub/Sub.”
• Cloud Monitoring works with Cloud TPU From the official documentation: “Cloud Monitoring automatically collects metrics from your TPUs and their host Compute Engine VMs.”
• Cloud Monitoring works with Gemini Enterprise Agent Platform From the official documentation: “Gemini Enterprise Agent Platform exports metrics to [Cloud Monitoring](/monitoring/docs).”
• Cloud Monitoring works with Google Cloud Storage From the official documentation: “You can configure Cloud Storage client libraries to to generate gRPC related metrics in Cloud Monitoring.”
• Cloud Monitoring works with Google Kubernetes Engine From the official documentation: “This page describes how to configure a Google Kubernetes Engine (GKE) cluster to send a curated set of cAdvisor/Kubelet metrics to Cloud Monitoring using Google Cloud Managed Service for Prometheus.”
• Cloud Run works with Compute Engine From the official documentation: “This lets Cloud Run clients use this backend service route to send requests directly to Compute Engine instances.”
• Cloud Run is an alternative to Compute Engine From the official documentation: “The decision of whether to use Cloud Run, GKE, or Compute Engine as your application host involves trade-offs between configuration flexibility and management effort.”
• Cloud Run is an alternative to Gemini Enterprise Agent Platform From the official documentation: “The application that's deployed on Cloud Run or on Gemini Enterprise routes the request to the SOC manager.”
• Cloud Run is an alternative to Google Kubernetes Engine From the official documentation: “Run agents locally or scale them globally using Runtime, Cloud Run, or Google Kubernetes Engine.”
• Cloud TPU works with Customer-managed encryption keys From the official documentation: “If you want to control your encryption keys, then you can use customer-managed encryption keys (CMEKs) in [Cloud KMS](/kms/docs) with CMEK-integrated services including Cloud TPU.”