#gke

After managing Kubernetes clusters across every major cloud provider for the past decade, I’ve developed a deep appreciation for what Google Kubernetes Engine (GKE) brings to the table. As the managed Kubernetes offering from the company that created Kubernetes itself, GKE represents the most mature and feature-rich container orchestration platform available today. Let me share the architectural…

ハイブリッドクラウド ゼロトラスト IaC 戦略 ハイブリッドクラウド ゼロトラスト IaC 戦略 よくある失敗を避け、投資効率を最大化する 1. 経営層への問いかけ：御社のハイブリッド戦略は「コスト増と事故リスク」になっていませんか？ ハイブリッドクラウド ゼロトラスト IaC 戦略を誤ると、その投資は即座にコスト増・事故リスクとして跳ね返ります。 クラウド移行で最も多いのは、従来の境界防御や手動運用をそのまま持ち込み、構造的な非効率とリスクを温存してしまうケースです。 弊社が多くの案件で目にしてきた「よくある誤った移行」の症例です。 症例 A：場当たり的なVPN依存: セキュリティ監査対応に20〜40時間／回の負荷が発生し、事故リスクが放置されています。従来の境界型防御に固執した結果です。 症例 B：レガシー依存の継続:…

Securing Generative AI with Apigee and Kubernetes No AI/Agents without APIs! Many users interact with generative AI daily without realizing the crucial role APIs play in delivering these experiences. As AI models become increasingly integrated into applications, securing and managing access to these models becomes paramount. Google recently announced a significant advancement: integrating the…

Hyperdisk ML can speed up the loading of AI/ML data. This tutorial explains how to use it to streamline and speed up the loading of AI/ML model weights on Google Kubernetes Engine (GKE). The main method for accessing Hyperdisk ML storage with GKE clusters is through the Compute Engine Persistent Disk CSI driver.

What is Hyperdisk ML?

You can scale up your applications with Hyperdisk ML, a high-performance storage solution. It is perfect for running AI/ML tasks that require access to a lot of data since it offers high aggregate throughput to several virtual machines at once.

Overview

It can speed up model weight loading by up to 11.9X when activated in read-only-many mode, as opposed to loading straight from a model registry. The Google Cloud Hyperdisk design, which enables scalability to 2,500 concurrent nodes at 1.2 TB/s, is responsible for this acceleration. This enables you to decrease pod over-provisioning and improve load times for your AI/ML inference workloads.

The following are the high-level procedures for creating and utilizing Hyperdisk ML:

Pre-cache or hydrate data in a disk image that is persistent: Fill Hyperdisk ML volumes with serving-ready data from an external data source (e.g., Gemma weights fetched from Cloud Storage). The disk image’s persistent disk needs to work with Google Cloud Hyperdisk.

Using an existing Google Cloud Hyperdisk, create a Hyperdisk ML volume: Make a Kubernetes volume that points to the data-loaded Hyperdisk ML volume. To make sure your data is accessible in every zone where your pods will operate, you can optionally establish multi-zone storage classes.

To use it volume, create a Kubernetes deployment: For your applications to use, refer to the Hyperdisk ML volume with rapid data loading.

Multi-zone Hyperdisk ML volumes

There is just one zone where hyperdisk ML disks are accessible. Alternatively, you may dynamically join many zonal disks with identical content under a single logical PersistentVolumeClaim and PersistentVolume by using the Hyperdisk ML multi-zone capability. The multi-zone feature’s referenced zonal disks have to be in the same area. For instance, the multi-zone disks (such as us-central1-a and us-central1-b) must be situated in the same area if your regional cluster is established in us-central1.

Running Pods across zones for increased accelerator availability and cost effectiveness with Spot VMs is a popular use case for AI/ML inference. Because it is zonal, GKE will automatically clone the disks across zones if your inference server runs several pods across zones to make sure your data follows your application.Image Credit To Google Cloud

The limitations of multi-zone Hyperdisk ML volumes are as follows:

There is no support for volume resizing or volume snapshots.

Only read-only mode is available for multi-zone Hyperdisk ML volumes.

GKE does not verify that the disk content is consistent across zones when utilizing pre-existing disks with a multi-zone Hyperdisk ML volume. Make sure your program considers the possibility of inconsistencies between zones if any of the disks have divergent material.

Requirements

The following Requirements must be met by your clusters in order to use it volumes in GKE:

Use Linux clusters with GKE 1.30.2-gke.1394000 or above installed. Make sure the release channel contains the GKE version or above that is necessary for this driver if you want to use one.

A driver for the Compute Engine Persistent Disk (CSI) must be installed. On new Autopilot and Standard clusters, the Compute Engine Persistent Disc driver is on by default and cannot be turned off or changed while Autopilot is in use. See Enabling the Compute Engine Persistent Disk CSI Driver on an Existing Cluster if you need to enable the Cluster’s Compute Engine Persistent Disk CSI driver.

You should use GKE version 1.29.2-gke.1217000 or later if you wish to adjust the readahead value.

You must use GKE version 1.30.2-gke.1394000 or later in order to utilize the multi-zone dynamically provisioned capability.

Only specific node types and zones allow hyperdisk ML.

Conclusion

This source offers a thorough tutorial on how to use Hyperdisk ML to speed up AI/ML data loading on Google Kubernetes Engine (GKE). It explains how to pre-cache data in a disk image, create a it volume that your workload in GKE can read, and create a deployment to use this volume. The article also discusses how to fix problems such a low it throughput quota and provides advice on how to adjust readahead numbers for best results.

Introducing Google’s Fleet-Argocd-Plugin, Simplifying Multi-Cluster Management for GKE Fleets

Give your teams self-service to empower them. Kubernetes with Argo CD and GKE fleets

It can be challenging to manage apps across several Kubernetes clusters, particularly when those clusters are spread across various environments or even cloud providers. Google Kubernetes Engine (GKE) fleets and Argo CD, a declarative, GitOps continuous delivery platform for Kubernetes, are combined in one potent and secure solution. Workload Identity and Connect Gateway further improve the solution.

This blog post explains how to use these offerings to build a strong, team-focused multi-cluster architecture. Google uses a prototype GKE fleet that has a control cluster to host Argo CD and application clusters for your workloads. It uses Connect Gateway and Workload Identity to improve security and expedite authentication, allowing Argo CD to safely administer clusters without having to deal with clumsy Kubernetes Services Accounts.

Additionally, it uses GKE Enterprise Teams to control resources and access, assisting in making sure that every team has the appropriate namespaces and permissions inside this safe environment.

Lastly, Google presents the fleet-argocd-plugin, a specially created Argo CD generator intended to make cluster management in this complex configuration easier. This plugin makes it simpler for platform administrators to manage resources and for application teams to concentrate on deployments by automatically importing your GKE Fleet cluster list into Argo CD and maintaining synchronized cluster information.

Follow along as Google Cloud:

Build a GKE fleet that includes control and application clusters.

Install Argo CD on the control cluster with Workload Identity and Connect Gateway set up.

Set up GKE Enterprise Teams to have more precise access control.

Install the fleet-argocd-plugin and use it to manage your multi-cluster, secure fleet with team awareness.

Using GKE Fleets, Argo CD, Connect Gateway, Workload Identity, and Teams, you will develop a strong and automated multi-cluster system by the end that is prepared to meet the various demands and security specifications of your company. Let’s get started!

Create a multi-cluster infrastructure using Argo CD and the GKE fleet

The procedure for configuring a prototype GKE fleet is simple:

In the selected Google Cloud Project, enable the necessary APIs. This project serves as the host project for the fleet.

Installing the gcloud SDK and logging in with gcloud auth are prerequisites.

Assign application clusters to your fleet host project and register them.

Assemble groups within your fleet. Assume you have a webserver namespace and a single frontend team.

a. You may manage which team has access to particular namespaces on particular clusters by using fleet teams and fleet namespace.

Argo CD should now be configured and deployed to the control cluster. As your application, create a new GKE cluster and set up Workload Identity.

To communicate with the Argo CD API server, install the Argo CD CLI. It must be version 2.8.0 or later. The CLI installation guide contains comprehensive installation instructions.

Install Argo CD on the cluster under control.

Argo CD generator customization

You have now installed Argo CD on the control cluster and your GKE fleet is operational. By saving their credentials (such as the address of the API server and login information) as Kubernetes Secrets inside the Argo CD namespace, application clusters are registered with the control cluster in Argo CD. It has a method to greatly simplify this process!

A customized Argo CD plugin generator called fleet-argocd-plugin simplifies cluster administration by:

Automatically configuring the cluster secret objects for every application cluster and loading your GKE fleet cluster list into Argo CD

Monitoring the state of your fleet on Google Cloud and ensuring that your Argo CD cluster list is consistently current and in sync

Let’s now see how to set up and construct the Argo CD generator.

Set up your control cluster with the fleet-argocd-plugin.

a. In this demonstration, the fleet-argocd-plugin is built and deployed using Cloud Build.

Provide the fleet-argocd-plugin with the appropriate fleet management permissions to ensure it functions as intended.

a. In your Argo CD control cluster, create an IAM service account and provide it the necessary rights. The configuration adheres to the GKE Workload Identity Federation’s official onboarding manual. b. You must also grant access to your artifacts repository’s pictures for the Google Compute Engine service account.

Launch the Argo CD control cluster’s fleet plugin!

Demo time

To ensure that the GKE fleet and Argo CD are working well together, let’s take a brief look. You ought to see that your application clusters’ secrets have been produced automatically.

Demo 1: Argo CD’s automated fleet management

Alright, let’s check this out! The guestbook sample app will be used. Google starts by deploying it to the frontend team’s clusters. After that, you should be able to see the guestbook app operating on your application clusters without having to manually handle any cluster secrets!

export TEAM_ID=frontend envsubst ‘$FLEET_PROJECT_NUMBER $TEAM_ID’ < applicationset-demo.yaml | kubectl apply -f – -n argocd

kubectl config set-context –current –namespace=argocd argocd app list -o name

Example Output:

argocd/app-cluster-1.us-central1.141594892609-webserver

argocd/app-cluster-2.us-central1.141594892609-webserver

Demo 2: Fleet-argocd-plugin makes fleet evolution simple

Let’s say you choose to expand the frontend staff by adding another cluster. The frontend team should be given a fresh GKE cluster. Next, see whether the new cluster has deployed your guestbook app.

gcloud container clusters create app-cluster-3 –enable-fleet –region=us-central1 gcloud container fleet memberships bindings create app-cluster-3-b \ –membership app-cluster-3 \ –scope frontend \ –location us-central1

argocd app list -o name

Example Output: a new app shows up!

argocd/app-cluster-1.us-central1.141594892609-webserver

argocd/app-cluster-2.us-central1.141594892609-webserver

argocd/app-cluster-3.us-central1.141594892609-webserver

Final reflections

We’ve demonstrated in this blog post how to build a reliable and automated multi-cluster platform by combining the capabilities of GKE fleets, Argo CD, Connect Gateway, Workload Identity, and GKE Enterprise Teams. You can improve security, expedite Kubernetes operations, and enable your teams to effectively manage and deploy apps throughout your fleet by utilizing these technologies.

Remember that GKE fleets and Argo CD offer a strong basis for creating a scalable, safe, and effective platform as you proceed with multi-cluster Kubernetes.