#hyperdisk

Do you want to pay less for storing GKE blocks? Storage Pool for Hyperdisks may assist Whether you’re managing GKE clusters, conventional virtual machines, or both, it’s critical to automate as many of your operational chores as you can in an economical way.

Pool Storage

Hyperdisk Storage Pool are a pre-purchased collection of capacity, throughput, and IOPS that you can then supply to your applications as required. Hyperdisk is a next-generation network connected block storage solution. Hyperdisk block storage disks allow you to optimize operations and costs by sharing capacity and performance across all the disks in a pool when you put them in storage pools. Hyperdisk Storage Pools may reduce your Total Cost of Ownership (TCO) associated with storage by up to 30–50%, and as of Google Kubernetes Engine (GKE) 1.29.2, they can be used on GKE!

Thin provisioning in Storage Pool makes this feasible by enabling you to use the capacity that is allocated inside the pool only when data is written, not when pool disks are provided. Rather of provisioning each disk for peak demand regardless of whether it ever experiences that load, capacity, IOPS, and throughput are bought at the pool level and used by the disks in the pool on an as-needed basis, enabling you to share resources as needed:

Why is Hyperdisk used?

Hyperdisk, the next generation of Google Cloud persistent block storage, is different from conventional persistent disks in that it permits control of throughput and IOPS in addition to capacity. Additionally, even after the disks are first configured, you may adjust their performance to match your specific application requirements, eliminating extra capacity and enabling cost savings.Image Credit Google Cloud

How about Storage Pool?

In contrast, storage pools allow you to share a thinly-provisioned capacity pool across many Hyperdisks in a single project that are all located in the same zone, or “Advanced Capacity” Storage Pool. Rather to using storage capacity that is provided, you buy the capacity up front and just use it for data that is written. Throughput and IOPS may be adjusted in a similar manner in a storage pool referred to as “Advanced Capacity & Advanced Performance.”

Combining Hyperdisk with Storage Pools reduces the total cost of ownership (TCO) for block storage by shifting management responsibilities from the disk level to the pool level, where all disks within the pool absorb changes. A Storage Pool is a zonal resource with a minimum capacity of 10TB and requires a hyperdisk of the same kind (throughput or balanced).

Hyperdisk

Storage Pool + Hyperdisk on GKE

Hyperdisk Balanced boot disks and Hyperdisk Balanced or Hyperdisk Throughput attached disks may now be created on GKE nodes within Storage Pool, as of GKE 1.29.2.

Let’s imagine you want to be able to adjust the performance to suit your workload for a demanding stateful application that is executing in us-central-a. You decide to use Hyperdisk Balanced for the workload’s block storage. You employ a Hyperdisk Balanced Advanced Capacity, Advanced Performance Storage Pools in place of trying to right-size each disk in your application. The capacity and performance are paid for beforehand.

Pool performance is used up when the disks in the storage pool notice an increase in IOPS or throughput, while pool capacity is only used up when your application writes data to the disks. Prior to creating the Hyperdisks inside the Storage Pool(s) must be created.

Google Cloud Hyperdisk

Use the following gcloud command to establish an Advanced Capacity, Advanced Performance StoragePools:gcloud compute storage-pools create pool-us-central1-a --provisioned-capacity=10tb --storage-pool-type=hyperdisk-balanced --zone=us-central1-a --project=my-project-id --capacity-provisioning-type=advanced --performance-provisioning-type=advanced --provisioned-iops=10000 --provisioned-throughput=1024

The Pantheon UI may also be used to construct Storage Pools.

You may also provide your node boot disks in the storage pool if your GKE nodes are utilizing Hyperdisk Balanced as their boot drives. This may be set up at cluster or node-pool construction, as well as during node-pool updates. You may use the Pantheon UI or the following gcloud command to provide your Hyperdisk Balanced node boot drives in your Storage Pool upon cluster setup. Keep in mind that your Storage Pool has to be established in the same zone as your cluster and that the machine type of the nodes needs to support Hyperdisk Balanced.

You must use the storage-pools StorageClass argument to define your Storage Pool in order to deploy the Hyperdisk Balanced disks that your stateful application uses in it. The Hyperdisk Balanced volume that your application will utilize is then provisioned using a Persistent Volume Claim (PVC) that uses the StorageClass.

The provisioned-throughput-on-create and provisioned-iops-on-create parameters are optional and may be specified by the StorageClass. The volume will default to 3000 IOPS and 140Mi throughput if provisioned-throughput-on-create and provisioned-iops-on-create are left empty. Any IOPS or Throughput from the StoragePool will only be used by IOPS and Throughput values that exceed these preset levels.

Google Hyperdisk

The allowed IOPS and throughput figures vary based on the size of the drive.

Only 40 MiB of throughput and 1000 IOPS will be used by volumes allocated with this StorageClass from the Storage Pools.

Next, create a PVC with a reference to the StorageClass storage-pools-sc.

The pooling-storage-sc When a Pod utilizing the PVC is formed, Storage Class’s Volume Binding Mode: Wait For First Consumer is used, delaying the binding and provisioning of a Persistent Volume.

Finally, utilize the aforementioned PVC to include these Hyperdisk Volumes into your Stateful application. It is necessary to schedule your application to a node pool that has computers capable of attaching Hyperdisk Balanced.

NodeSelectors are used in the Postgres deployment to make sure that pods are scheduled to nodes that allow connecting Hyperdisk Balanced, or C3 machine types.

You ought now be able to see that your storage pools has your Hyperdisk Balanced volume deployed.

Next actions

For your stateful applications, you may optimize storage cost reductions and efficiency by using a Storage Pools + Hyperdisk approach for GKE.

Google Compute Engine Cost

Compute Engine provides several options for cutting expenses, such as optimising your infrastructure and utilising sales. Google Cloud is sharing some useful advice to help you save Google Compute Engine cost in this two-part blog post. This guide has something for everyone, regardless of whether you work for a huge organisation trying to optimise its budget or a small business just starting started with cloud computing.

Examine your present budgetary plan

It would be helpful to have a map of your present circumstances and spending structure before you embark on a journey to optimise your Google Compute Engine cost. This will allow you to make well-informed decisions regarding your next course of action. That billing panel is the Google Cloud console. It provides you with a detailed breakdown of your spending, tracking each expense to a specific SKU. It can be used to examine the overall financial picture of your company and to determine how much a given product will cost to use for a given project.

You can find resources you are no longer paying for but no longer require by taking a closer look at your spending. Nothing is a better method to save money than simply not spending it, after all.

Examine the automated suggestions

On the page where your virtual machines are listed, have you noticed the lightbulbs next to some of your machines? These are Google Cloud’s automated suggestions for things you could do to cut costs. The following project management categories cost, security, performance, reliability, management, and sustainability are addressed by Recommendation Hub, a new technology. The recommendations system can make suggestions for actions that you might think about based on its understanding of your fleet structure. Helping you cut costs without sacrificing fleet performance is Google Cloud’s main objective.Image credit to Google Cloud

The machine can be scaled down according to its utilisation, or the type of machine can be changed (e.g., from n1 to e2). You get a summary of the recommended modification along with the expected cost savings when you click on one of the recommendations.  You have the option of applying the modification or not. Recall that the instance must be restarted in order for modifications to take effect.Image credit to Google Cloud

Check the types of discs you have

You must attach at least one persistent disc to each virtual machine in your fleet. Google Cloud offers a variety of disc formats with varying features and performance. The kinds that are offered are:

Hyperdisk

With a full range of data durability and administration features, Hyperdisk is a scalable, high-performance storage solution built for the most demanding mission-critical applications.

Hyperdisk Storage Pools 

Hyperdisk Storage Pools are pre-aggregated volumes, throughput, and IOPS that you can reserve in advance and allocate to your apps as required.

Persistent Disk 

Your virtual machines default storage option is called Persistent Disc. It may be regional or zonal. has four variations:

Standard

The desktop computer’s equivalent of an HDD disc. offers the least expensive storage with a slower I/O speed.

SSD

A speed-focused option with excellent I/O performance, albeit at a higher cost per gigabyte.

Balanced

The default setting for newly created compute instances; it strikes a compromise between “Standard” and “SSD.”

Extreme

Suitable for the hardest workloads. enables you to manage the disk’s IOPS in addition to its size.

Local SSD

An SSD that is physically attached to the host that powers your virtual machine is called a local SSD. incredibly quick but transient.

Since persistent disc storage is the most widely used type of storage, let’s concentrate on it. The Balanced disc, which offers a decent compromise between performance and cost, is the default disc type used when building a new virtual machine. Although this works well in a lot of situations, it might not be the ideal choice in every situation.

Fast I/O to disc is not needed, for instance, by stateless apps that are a component of auto-scaling deployments and keep all pertinent data in an external cache or database. These apps are excellent candidates for switching to Standard discs, which, depending on the region, can be up to three times less expensive per gigabyte than Balanced discs.

A list of the discs used in your project can be obtained using: the list of gcloud compute discs with the format “table(name, type, zone, sizeGb, users)”

You must clone the disc and make changes to the virtual machines that use it in order to start using the new disc in order to alter the disc type.

Free up any unused disc space

Moving on to storage, there are other factors besides disc type that influence price. You should also consider how much disc utilisation affects your budget.You will be paid for the full 100 GB of persistent disc space allocated for your project, whether you use 20%, 70%, or 100%. You may still want to monitor your boot discs closely even if your application does not use Persistent Discs for data storage.

If your stateless programme really needs a disc with many gigabytes of free space, think about reducing the size of the discs to match your actual needs. Because they enjoy round numbers, people frequently build 20 GB discs even when they only require 12 GB. Save money and act more like a machine.

Agree to make use of CUDs, or committed use discounts

Compute Engine is not the only product to which this advice is applicable. You can receive a significant discount if you can guarantee that you’ll use a specific number of virtual machines for three or more years, or at least a year! You can get substantially cheaper costs for local SSDs, GPUs, vCPUs, memory, sole-tenant nodes, and software licences by using a range of (CUDs). You are not even limited to allocating your vCPU and memory to a certain project, area, or machine series when using Flex CUDs.

Discounts for committed use are offered on a number of Google Cloud products. If you’re satisfied with Google Cloud and have no intention of switching providers anytime soon, you should seriously think about utilising CUDs whenever you can to save a lot of money. When it comes to computing, you can buy CUDs straight from the Google Cloud dashboard.

Hyperdisk Storage Pools, the first block storage solution from a hyperscale cloud provider to allow thin-provisioning, data reduction, and capacity pooling, will be generally available, as Google cloud revealed at Google Cloud Next 2024. By reducing your Total Cost of Ownership (TCO) by up to 30–50%, Hyperdisk Storage Pools enable you to streamline block storage management, modernise SAN-based operations, and increase efficiency. Storage Pools are available right now via the Google Cloud console. Let’s examine Hyperdisk Storage Pools’ operation and integration into your environment in this blog post.

Hyperdisk Storage Pools can store big amounts using Compute Engine. A Hyperdisk Storage Pool provides pre-purchased capacity, throughput, and IOPS to applications as needed. Create, manage, and use discs in pools with Hyperdisk Storage Pools for multiple workloads. Manage discs in bulk to save money and increase capacity and performance. By employing exactly the storage you need in Hyperdisk Storage Pools, you simplify capacity forecasts and decrease management by going from hundreds of discs to one pool.

Benefits of storage pools include:

Thin provisioning and data reduction enable Hyperdisk Storage Pools store data efficiently and achieve best-in-class TCO.

Hyperdisk Storage Pools take advantage of thin provisioning and data reduction to maximize resource utilization and minimize TCO.

Management overhead reduced Higher Flexibility – Hyperdisk Storage Pools allow workload owners to provide larger discs and only use what they need, eliminating capacity and performance forecasts and rescaling downtime.

Workloads use Hyperdisk volumes the same way with storage pools. No downtime or workload disruptions are needed.

Data on discs in a storage pool remain isolated as if they were not.

Use storage pools

Storage pools solve these issues:

Trouble predicting resource requirements when transitioning on-premise SAN workloads to Google Cloud.

Estimating application performance and capacity can take weeks and be error-prone, delaying a cloud migration or application rollout.

In Hyperdisk Storage Pools, you can overestimate the capacity needed to make discs and then only use the disc space you write data on.

Underuse of resources

Ensuring volume utilization is difficult and painful. Block storage is often underutilized because peak capacity and performance must be provided to avoid outages and slowdowns. However, few applications reach those levels.

Using Hyperdisk Storage Pools, you establish a pool to meet workload capacity needs. To keep utilization below 80%, the Hyperdisk Storage Pool automatically adds capacity.

Complex workload block storage management

Managing hundreds or thousands of disc volumes takes time and resources away from innovation.

When creating VM discs in a storage pool, you can choose a size bigger than expected workload. Workloads write to discs, reducing storage pool capacity. You only subtract the block storage you use from the storage pool capacity, not the disc creation amount. The disc size you choose limits capacity. Only change the disc size if this limit is reached.

If you build discs in the storage pool and your workload exceeds capacity planning over numerous discs, you can expand capacity. All discs in the storage pool can use the extra capacity.

Hyperdisk Storage Pools options

Features of storage pools include:

Allocating blocks as needed instead of all at once is capacity thin provisioning. This prevents low storage utilisation, where lots of disc space is assigned but not used.

Data reduction: Storage pools improve efficiency with various data reduction technologies. Data type strongly affects data reduction. Data compressed or encrypted before storage in a Hyperdisk Storage Pool won’t reduce.

To prevent failures caused by insufficient capacity, Hyperdisk Storage Pools automatically adds capacity when utilisation surpasses 80% of provisioned capacity.

Hyperdisk Storage Pool operation

A storage pool with the aggregate capacity and performance your workloads need is created, then discs are added. Connect the discs to your VMs. Create discs with a larger size than needed. This allows future growth without disc resizing.

Only the disc space your workloads require after data reduction is withdrawn from the Advanced capacity storage pool if the disc was generated there.

To increase disc capacity, increase the storage pool’s provided capacity. The discs’ disc space can then rise to the size you set when creating them. By generating discs in an Advanced capacity storage pool with a big initial size and allocating additional space, you consolidate disc storage administration and decrease costs.

Storage pool capacity is managed. Increase or decrease a storage pool’s provided capacity to change disc capacity. If the storage pool’s usage capacity or the aggregate capacity of all discs reaches 80% of its specified capacity, an advanced capacity storage pool automatically adds capacity. If auto-grow fails, you can manually add storage pool capacity up to 1 PiB.

If a storage pool achieves 100% utilization and has no free space, writes to all discs fail unless data or discs are deleted. Most commercial software treats full disc write errors like hardware failures.

To avoid out-of-space issues, actively maintain and monitor your storage pool. You should also know how your workload will react to a disc out of space issue in a storage pool.

Hyperdisk Storage Pool provisioning

You can configure a Hyperdisk Storage Pool with Standard or Advanced capacity.

Storage pools standard capacity

Standard capacity provisioning creates discs in the storage pool until all discs exceed the pool’s capacity. Standard capacity storage pool discs use capacity like non-storage pool discs.

Storage pools with high capacity

Advanced storage pools allow thin-provisioning and data reduction for capacity, allowing you to provision discs with more capacity than purchased. Advanced capacity storage pools use capacity based only on bytes written to discs after data reduction, so you can give end users and applications more capacity than you paid.

Advanced capacity lets you build discs in the storage pool that exceed its provisioned capacity by up to 500%. Data written determines storage pool capacity, not disc capacity. Advanced capacity storage pool discs use capacity differently from Standard capacity and non-storage pool discs.

You can fill discs in an Advanced capacity storage pool to their allotted size if the data written to all discs doesn’t exceed the capacity. The auto-grow feature adds capacity to the storage pool if utilization reaches 80% of provisioned capacity. If the storage pool is full, writes to all discs will fail until you delete data or discs to reduce its capacity. Since most software applications treat writes to full discs as hardware failures, it’s important to both:

Monitor your storage pool to minimise disc space shortages.

Know how your workload reacts if it does.

Remember that storage pools can’t see your file system. Deleted data is still in use until your OS marks it unused using DISCARD or TRIM. Most third-party OS images and all Google-provided OS images do this by default, but you should confirm this if you are not using one. Find out how to verify or configure this feature at Disable lazy initialization and enable DISCARD commands.

Hyperdisk Storage Pool Types

The discs you can generate in a Hyperdisk Storage Pool depend on its type.

Hyperdisk Throughput Storage Pool: You choose capacity and throughput while creating the storage pool. Created Hyperdisk Throughput discs in the storage pool use some allotted capacity and throughput.

Hyperdisk Balanced Storage Pool: You set capacity, throughput, and IOPS while constructing the storage pool. You employ some of the storage pool’s capacity and performance to make Hyperdisk Balanced discs with capacity and performance above baseline.

High-throughput storage pools

Storage Pools can manage Hyperdisk Throughput disc utilization.

Performance

Storage pooled discs perform the same as non-pooled discs.

Pricing

Capacity, throughput, and IOPS determine Hyperdisk Storage Pool pricing.

Standard capacity storage pools price capacity like discs. Standard capacity in a Hyperdisk Balanced Storage Pool costs the same as standalone capacity.

Thin provisioning and data reduction cost more, hence advanced capacity is more expensive. Despite this premium, thin-provisioning and data reduction can lower block storage costs by increasing efficiency and utilization.

Toda la información en: https://hardwaresfera.com/noticias/perifericos/hyperdisk-un-pequeno-ssd-externo-de-45-gramos-que-esta-en-kickstarter/

Actualmente los SSD están a un precio realmente asequible, entre otros, porque las memorias NAND Flash han bajado de precio. Ha pasado la fiebre de los smartphone y eso nos beneficia a todos. Esto permite que se estén desarrollando otras soluciones muy interesantes, como el HyperDisk. Y es que esta unidad SSD extremadamente compacta está …