#computeengine

Compute Engine’s compute-optimized machine family

High performance computing (HPC) workloads and compute-intensive tasks are best suited for instances of compute-optimized virtual machines (VMs). With their architecture that makes use of characteristics like non-uniform memory access (NUMA) for optimal, dependable, consistent performance, compute-optimized virtual machines (VMs) provide the best performance per core.

This family of machines includes the following machine series:

Two 4th-generation Intel Xeon Scalable processors, code-named Sapphire Rapids, with an all-core frequency of 3.0 GHz power H3 virtual machines. 88 virtual cores (vCPUs) and 352 GB of DDR5 memory are features of H3 virtual machines.

The third generation AMD EPYC Milan CPU, which has a maximum boost frequency of 3.5 GHz, powers C2D virtual machines. Flexible scaling between 2 and 112 virtual CPUs and 2 to 8 GB of RAM per vCPU are features of C2D virtual machines.

The 2nd-generation Intel Xeon Scalable processor (Cascade Lake), which has a sustained single-core maximum turbo frequency of 3.9 GHz, powers C2 virtual machines. C2 provides virtual machines (VMs) with 4–60 vCPUs and 4 GB of memory per vCPU.

H3 machine series

The 4th generation Intel Xeon Scalable processors (code-named Sapphire Rapids), DDR5 memory, and Titanium offload processors power the H3 machine series and H3 virtual machines.

For applications involving compute-intensive high performance computing (HPC) in Compute Engine, H3 virtual machines (VMs) provide the highest pricing performance. The single-threaded H3 virtual machines (VMs) are perfect for a wide range of modeling and simulation tasks, such as financial modeling, genomics, crash safety, computational fluid dynamics, and general scientific and engineering computing. Compact placement, which is ideal for closely connected applications that grow over several nodes, is supported by H3 virtual machines.

There is only one size for the H3 series, which includes a whole host server. You can change the amount of visible cores to reduce licensing fees, but the VM still costs the same. H3 virtual machines (VMs) have a default network bandwidth rate of up to 200 Gbps and are able to utilize the full host network capacity. However, there is a 1 Gbps limit on the VM to internet bandwidth.

H3 virtual machines are unable to allow simultaneous multithreading (SMT). To guarantee optimal performance constancy, there is also no overcommitment.

H3 virtual machines can be purchased on-demand or with committed use discounts (CUDs) for one or three years. Google Kubernetes Engine can be utilized with H3 virtual machines.

H3 VMs Limitations

The following are the limitations of the H3 machine series:

Only a certain machine type is offered by the H3 machine series. No custom machine shapes are available.

GPUs cannot be used with H3 virtual machines.

There is a 1 Gbps limit on outgoing data transfer.

The performance limits for Google Cloud Hyperdisk and Persistent Disk are 240 MBps throughput and 15,000 IOPS.

Machine images are not supported by H3 virtual machines.

The NVMe storage interface is the only one supported by H3 virtual machines.

Disks cannot be created from H3 VM images.

Read-only or multi-writer disk sharing is not supported by H3 virtual machines.

Different types of H3 machines

Machine typesvCPUs*Memory (GB)Max network egress bandwidth†h3-standard-8888352Up to 200 Gbps

With no simultaneous multithreading (SMT), a vCPU represents a whole core. † The default egress bandwidth is limited to the specified value. The destination IP address and several variables determine the actual egress bandwidth. Refer to the network bandwidth.

Supported disk types for H3

The following block storage types are compatible with H3 virtual machines:

Balanced Persistent Disk (pd-balanced)

Hyperdisk Balanced (hyperdisk-balanced)

Hyperdisk Throughput (hyperdisk-throughput)

Capacity and disk limitations

With a virtual machine, you can employ a combination of persistent disk and hyperdisk volumes, however there are some limitations:

Each virtual machine can have no more than 128 hyperdisk and persistent disk volumes combined.

All disk types’ combined maximum total disk capacity (in TiB) cannot be greater than:

Regarding computer types with fewer than 32 virtual CPUs:

257 TiB for all Hyperdisk or all Persistent Disk

257 TiB for a mixture of Hyperdisk and Persistent Disk

For computer types that include 32 or more virtual central processors:

512 TiB for all Hyperdisk

512 TiB for a mixture of Hyperdisk and Persistent Disk

257 TiB for all Persistent Disk

H3 storage limits are described in the following table: Maximum number of disks per VMMachine typesAll disk types All Hyperdisk typesHyperdisk BalancedHyperdisk ThroughputHyperdisk ExtremeLocal SSDh3-standard-88128648640Not supported

Network compatibility for H3 virtual machines

gVNIC network interfaces are needed for H3 virtual machines. For typical networking, H3 allows up to 200 Gbps of network capacity.

Make sure the operating system image you use is fully compatible with H3 before moving to H3 or setting up H3 virtual machines. Even though the guest OS displays the gve driver version as 1.0.0, fully supported images come with the most recent version of the gVNIC driver. The VM may not be able to reach the maximum network bandwidth for H3 VMs if it is running an operating system with limited support, which includes an outdated version of the gVNIC driver.

The most recent gVNIC driver can be manually installed if you use a custom OS image with the H3 machine series. For H3 virtual machines, the gVNIC driver version v1.3.0 or later is advised. To take advantage of more features and bug improvements, Google advises using the most recent version of the gVNIC driver.

Confidential virtual machine overview

Confidential VMs Google

One kind of Compute Engine virtual machine is a confidential virtual machine (VM). To help guarantee that your data and applications cannot be read or altered while in use, they employ hardware-based memory encryption.

Below are some advantages of confidential virtual machine instances:

Isolation: Only specialized hardware, unavailable to the hypervisor, generates and stores encryption keys.

Attestation: To ensure that important parts haven’t been tampered with, you can confirm the identity and condition of the virtual machine.

A Trusted Execution Environment is a term used to describe this kind of hardware isolation and attestation (TEE).

When you create a new virtual machine instance, you have the option to activate the Confidential VM service.

Confidential computing technology

Depending on the machine type and CPU platform you select, different Confidential Computing technologies can be employed while setting up a Confidential VM instance. Make sure the technology you select for Confidential Computing meets your budget and performance requirements.

AMD SEV

AMD Secure Encrypted Virtualization (SEV) on Confidential VM provides boot-time attestation using Google’s vTPM and hardware-based memory encryption via the AMD Secure Processor.

AMD SEV provides excellent performance for computationally intensive activities. Depending on the workload, the performance difference between a normal Compute Engine VM and a SEV Confidential VM can be negligible or nonexistent.

AMD SEV systems using the N2D machine type offer live migration, in contrast to other Confidential Computing technologies using Confidential VM.

AMD SEV-SNP

Adding hardware-based security to assist thwart malicious hypervisor-based attacks like data replay and memory remapping, AMD Secure Encrypted Virtualization-Secure Nested Paging (SEV-SNP) builds on SEV. Direct attestation results from the AMD Secure Processor are available upon request at any time.

Compared to SEV, AMD SEV-SNP requires greater resources due to its increased security measures. In instance, you may see higher network latency and decreased network bandwidth based on the demand.

TDX Intel

A hardware-based TEE is Intel Trust Domain Extensions (TDX). In order to manage and encrypt memory, TDX employs hardware extensions to establish an isolated trust domain (TD) inside of a virtual machine (VM).

By enhancing the defense of the TD against specific types of attacks that require physical access to the platform memory, such as active attacks of DRAM interfaces that involve splicing, aliasing, capturing, altering, relocating, and modifying memory contents, Intel TDX enhances the defense of the TD.

Confidential VM service

Confidential VM is used by the following Google Cloud services in addition to Compute Engine:

Secret All of your Google Kubernetes Engine nodes are required to use Confidential VM.

With a mutually agreed-upon workload, Confidential Space employs Confidential VM to allow parties to share sensitive data while maintaining ownership and confidentiality of that data.

Confidential VM-using Dataproc clusters are part of Dataproc Confidential Compute.

Features of Dataflow Confidential VM worker for dataflow confidential virtual machines.

Google Cloud is dedicated to keeping your data under your complete control and safe and secure. To start, use Confidential Computing to strengthen the Compute Engine virtual machines (VMs), which are the cornerstone of your compute architecture.

Using a hardware-based Trusted Execution Environment (TEE), Confidential Computing safeguards data throughout use and processing. TEEs are safe, segregated spaces that guard against illegal access to or alteration of data and applications while they’re in use.

Confidential Computing technologies and solutions have been early adopters and investments for us at Google. Google customers have been utilizing their expanded Confidential Computing products and new capabilities for over four years, utilizing them in creative ways to improve the security and confidentiality of their workloads. It was excited to announce the broad release of upgrades to the Google Cloud attestation service as well as numerous additional Confidential Computing choices.

Currently accessible to most people: Segmented virtual machine with AMD SEV on C3D platforms

It is pleased to inform you that Confidential VMs Google equipped with AMD Secure Encrypted Virtualization (AMD SEV) technology are now widely accessible on the general purpose C3D machine line. Using hardware-based memory encryption, Confidential VMs with AMD SEV technology help guarantee that your data and apps cannot be read or changed while in use. With Google’s Titanium hardware, the C3D machine series is built to provide optimal, dependable, and consistent performance, and is powered by the 4th generation AMD EPYC (Genoa) processor.

Prior to this, only the general-purpose N2D and C2D machine series offered Confidential VMs. The latest general purpose hardware with enhanced performance and data secrecy is now available to security-conscious customers with the expansion to the C3D machine line. Better performance comes from using the newest gear. Read more about the performance of the C3D machine series and confidential virtual machines here.

In any region and zone where C3D machines are available, confidential virtual machines featuring AMD SEV are accessible.

Now widely accessible: Intel TDX-powered confidential virtual machine on the C3 machine series

Confidential VMs Google equipped with Intel Trust Domain Extensions (Intel TDX) technology are now widely accessible on the general-purpose C3 machine series. Using hardware-based memory encryption, Confidential VMs with Intel TDX technology help guarantee that your data and apps cannot be read or changed while in use.

There are no code changes needed to enable confidential computing on a C3 virtual machine. You can use Intel Trust Authority’s remote attestation service or your own attestation provider to confirm that your hardened virtual machine (VM) is operating in a TEE. The 4th generation Intel Xeon Scalable CPUs (code-named Sapphire Rapids), DDR5 memory, and Google Titanium power the C3 machine line.

Intel AMX integrated CPU acceleration

By default, all C3 virtual machines (VMs), including Confidential VMs, have Intel Advanced Matrix Extensions (Intel AMX) enabled. To speed up workloads related to machine learning and artificial intelligence, Intel AMX is a novel expansion to the instruction set architecture (ISA). Two of the most popular processes in AI and ML are matrix multiplication and convolution, which may be carried out with the new instructions that AMX offers. You can execute AI/ML applications with an extra degree of protection by combining Intel AMX with Confidential VMs.

Asia-southeast1, US-central1, and Europe-west4 are the regions where Confidential VM with Intel TDX on the C3 machine series is accessible.

Confidential VM with AMD SEV-SNP on the N2D machine series is now widely accessible

Customers now have access to Confidential VMs with hardware-rooted attestation, data integrity, and data confidentiality thanks to the release of AMD Secure Encrypted Virtualization-Secure Nested Paging (AMD SEV-SNP) on the general purpose N2D machine series this past June. Prior to this, users could only access private VMs with AMD Secure Encrypted Virtualization (SEV), a private computing solution that guaranteed data confidentiality.

All Confidential VMs give users an extra line of defense and data protection against cloud administrators, operators, and insiders while also enabling them to retain control over their data in the public cloud and achieve cryptographic isolation in a multi-tenant environment. Confidential VMs with AMD SEV-SNP, on the other hand, come with further security measures that guard against harmful hypervisor-based assaults such memory remapping and data replay.

AMD SEV-SNP on the N2D machine series makes it simple and doesn’t require any code changes to create Confidential VMs. You also get the security advantages with less impact on performance.

Asia-southeast1, US-central1, Europe-west3, and Europe-west 4 are the regions where confidential virtual machines with AMD SEV-SNP on the N2D machine series are accessible.

Signed Intel TDX and AMD SEV-SNP UEFI binaries for Confidential Virtual Machines

With the addition of signed startup measures (UEFI binaries and initial state) to its Confidential VMs running AMD SEV-SNP and Intel TDX technologies, it is thrilled to announce a major security improvement. By signing these files, provided an additional degree of security against unauthorized changes or tampering with UEFI, the firmware that manages a computer’s startup procedure.

Gaining further transparency and confidence that the firmware operating on your Confidential VMs is authentic and uncompromised can be achieved by signing the UEFI and enabling you to validate the signatures. Your authenticated devices are operating in a secure and reliable environment if you can confirm the validity and integrity of the firmware.

Google intends to take other actions to create a system that is more verifiably reliable and secure.

AMD SEV Confidential VM is now supported by Google Cloud attestation

If your trust model permits it, you can use the Google Cloud attestation service in place of creating and executing an attestation verifier yourself. Use the Go-TPM tools to obtain an attestation quote from the vTPM of an AMD SEV Confidential VM instance, then transmit it to the Google Cloud Attestation service using the./go-tpm token command for verification.

You can verify whether or not the virtual machine (VM) can be trusted by comparing its details with your own policy once the Google Cloud Attestation has verified the attestation quote. Only AMD SEV is currently supported by Google’s attestation service.

Confidential VM costs

In addition to the Compute Engine price, there are additional expenses for Confidential VM. The cost of a Confidential VM instance is determined by several factors, including the type of Confidential Computing technology (such as AMD SEV, Intel TDX, or AMD SEV-SNP) and whether the instance is preemptible or on demand. The fees are flat rate per vCPU and per GB for Confidential VM.

See here for the price of the Confidential VM. See this page for Compute Engine’s price list.

Apache Airflow on Google Cloud

Are you considering utilizing Google Cloud to run Apache Airflow? This is a well-liked option for managing intricate sequences of operations, such Extract, Transform, and Load (ETL) or pipelines for data analytics. Airflow Apache is a sophisticated tool for scheduling and dependency graphing. It employs a Directed Acyclic Graph (DAG) to arrange and relate many tasks for your workflows, including scheduling the required activity to run at a specific time.

What are the various configuration options for Airflow Apache on Google Cloud? Making the incorrect decision could result in lower availability or more expenses. You could need to construct multiple environments, such as dev, staging, and prod, or the infrastructure could fail. It will examine three methods for using Airflow Apache on Google Cloud in this post and go over the benefits and drawbacks of each. It offer Terraform code, which is available on GitHub, for every method so you may give it a try.

It should be noted that this article’s Terraform has a directory structure. The format of the files under modules is the same as that of the Terraform default code. Consider the modules directory to be a type of library if you work as a developer. The real business code is stored in the main.tf file. Assume you are working on development: begin with main.tf and save the shared code in folders such as modules, libraries, etc.)

Apache Airflow best practices

Let’s examine three methods for utilizing Airflow Apache.

Compute Engine

Installing and using Airflow directly on a Compute Engine virtual machine instance is a popular method for using Airflow on Google Cloud. The benefits of this strategy are as follows:

It costs less than the others.

All you need to know about virtual machines.

Nevertheless, there are drawbacks as well:

The virtual computer needs to be maintained by you.

There is less of it available.

Although there can be significant drawbacks, Compute Engine can be used to quickly prove of concept Airflow adoption.

First, use the following terraform code to construct a Compute Engine instance (some code has been eliminated for brevity). Allow is a firewall configuration. Since Airflow Web uses port 8080 by default, it ought to be open. You are welcome to modify the other options.

main.tf

module “gcp_compute_engine” { source = “./modules/google_compute_engine” service_name = local.service_name

region = local.region zone = local.zone machine_type = “e2-standard-4” allow = { … 2 = { protocol = “tcp” ports = [“22”, “8080”] } } }

The code and files that take the variables it handed in before and actually build an instance for it was found in the google_compute_engine directory, which it reference as source in main.tf above. Take note of how it takes in the machine_type.

modules/google_compute_engine/google_compute_instance.tf

resource “google_compute_instance” “default” { name = var.service_name machine_type = var.machine_type zone = var.zone … }

Use Terraform to run the code you wrote above:

$ terraform apply

A Compute Engine instance will be created after a short wait. The next step is to install Airflow by connecting to the instance; Launch Airflow after installation.

You can now use your browser to access Airflow! You will need to take extra precautions with your firewall settings if you intend to run Airflow on Compute Engine. It should only be accessible to authorized users, even in the event that the password is compromised. It has only made the sample accessible with the barest minimum of firewall settings.

You ought to get a screen similar after logging in. Additionally, an example DAG from Airflow is displayed. Examine the contents of the screen.

GKE Autopilot

Using Google Kubernetes Engine (GKE), Google’s managed Kubernetes service, running Airflow Apache on Google Cloud is made extremely simple. Additionally, you have the option to operate in GKE Autopilot mode, which will automatically scale your cluster according to your demands and assist you avoid running out of compute resources. You can manage your own Kubernetes nodes without having to do so because GKE Autopilot is serverless.

GKE Autopilot provides scalability and high availability. Additionally, you may make use of the robust Kubernetes ecosystem. For instance, you can monitor workloads in addition to other business services in your cluster using the kubectl command, which allows for fine-grained control over workloads. However, if you’re not particularly knowledgeable with Kubernetes, utilizing this method may result in you spending a lot of time managing Kubernetes rather than concentrating on Airflow.

Cloud Composer

Using Cloud Composer, a fully managed data workflow orchestration service on Google Cloud, is the third option. As a managed service, Cloud Composer simplifies the Airflow installation process, relieving you of the burden of maintaining the Airflow infrastructure. But it offers fewer choices. One unusual scenario is that storage cannot be shared throughout DAGs. Because you don’t have as much control over CPU and memory utilization, you might also need to make sure you balance those usages.

Conclude

Three considerations must be made if you plan to use Airflow in production: availability, performance, and cost. Three distinct approaches of running Airflow Apache on Google Cloud have been covered in this post; each has advantages and disadvantages of its own.

Remember that these are the requirements at the very least for selecting an Airflow environment. It could be enough to write some Python code to generate a DAG if you’re using Airflow for a side project. But in order to execute Airflow in production, you’ll also need to set up the Executor (LocalExecutor, CeleryExecutor, KubernetesExecutor, etc.), Airflow Core (concurrency, parallelism, SQL Pool size, etc.), and other components as needed.

Google Vault Backup

Data backup is more crucial than ever. Due to the rise in ransomware attacks, more customers are upgrading their protection to protect their data. Thus, security and usability are prioritized. Regarding security, the frequency of ransomware and other cyberattacks puts a company’s finances and reputation at serious danger. Simultaneously, there is a strong need to decrease operational burden and boost agility by streamlining backup management.

Google is introducing three significant updates to the Google Cloud Backup and Disaster Recovery (DR) service, all of which are now in preview, to better meet your changing needs:

- Advertisement -

The new backup vault storage feature protects your backups from manipulation and unwanted deletion by providing immutable (preventing modification) and indelible (preventing deletion) backups.

A completely managed end-to-end solution for centralized backup management that facilitates direct integration into resource management flows and makes data protection simple.

Integration with the Compute Engine virtual machine formation process, enabling application owners to implement backup policies at the time of VM creation.

Protect your backups from unapproved deletion and alteration

Backups are frequently the last line of defense for data recovery in situations where production data is compromised or unreliable, as occurs after a major user error or cyberattack. It’s imperative to secure your backups from later alteration and deletion in addition to backing up your important workloads. Backup vault enables you to confidently achieve the protection your organization needs by offering secure storage for backups made by the Backup and DR service.

How to Backup Vault

Air-gapped backup isolation makes sense

Data from your backup vault is conceptually air-gapped from your self-managed Google Cloud project and kept in a Google-managed project. Direct assaults against the underlying backup vault resources are prevented since they are not visible to or accessible by users within your business. The only way to access backup vault data is via the APIs and user interface of Google Cloud Backup and DR services.

Control and adherence: mandatory storage

You can stipulate while building a backup vault that vaulted backups have to be kept under strict security against deletion and modification until the administrator-specified minimum enforced retention period has passed. You can meet backup immutability and indelibility goals with this layered protection. Security initiatives and regulatory compliance requirements are common drivers of these goals.

Dependable and adaptable recuperation

Fully self-contained, vaulted backups allow for recovery even in the event that the original resource is unavailable. Furthermore, backup vaults can be established in a project that is distinct from the source project to guarantee that backups are still available in the event that the source project or resource is destroyed. You may therefore set up your backup policy to offer robust resistance against the deletion of the source project. In the event of a cyberattack, this facilitates the prompt recovery of production applications to projects that already exist or are freshly established. Recovery into projects set up as isolated recovery environments (IREs) for pre-recovery testing and forensics is also supported.

- Advertisement -

In the upcoming months, the backup vault feature will become broadly accessible after being made available in preview form today. Oracle and SQL Server databases, as well as Compute Engine and VMware Engine virtual machines, are all protected.

Utilize centralized, fully managed backup administration

Consumers frequently request an infrastructure-free, self-serving, and basic backup solution made for cloud applications. Customers want a more flexible strategy that gives app developers the ability to backup their virtual machines (VMs) while maintaining governance and monitoring by the central backup team. This is to ensure that critical operational components do not impede the agility that businesses need.

With a fully managed solution that simplifies data protection and provides an integrated, developer-centric, self-service approach for app developers, Google cloud new centralized backup management experience offers simplicity.

Store your vital Compute Engine virtual machine data in backup vaults

The new fully managed experience, which offers first support for managing Compute Engine VM protection, simplifies backup setup into three simple steps: 1) building a backup vault (storage), 2) defining your backup plan (schedule), and 3) initiating your VMs’ protection. Because of this approach’s ease of use, complicated setups are not necessary, allowing you to concentrate on your main business activities rather than managing backups.

Providing direct connection to empower application owners

Thanks to an integrated experience during VM formation, platform administrators now have the ability to backup Compute Engine virtual machines (VMs). This feature streamlines workflows and lessens the administrative load on IT and central backup teams by enabling teams to take charge of their own backup strategy right from the VM creation process. It is possible to guarantee that your data protection standards are followed consistently right from the start by including backup tasks in the VM provisioning process. Admins have freedom and control because of their integration with Google Cloud Identity and Access Management.

Centralized oversight, reporting, and monitoring

By providing centralized management over backup policies and enabling application owners to handle their own backup operations, the backup service improves governance as well as supervision. This dual-layer strategy strikes the ideal mix between operational flexibility and centralized control, ensuring consistency and compliance throughout the business.

The system offers extensive monitoring and reporting features to further fortify data protection:

Backup and restore jobs that are scheduled

Monitor the progress of these jobs to make sure they are proceeding according to plan. Track advancement, failure, and success of jobs with a single dashboard.

Customizable reports

Produce thorough reports on protected resources, compliance, storage utilization, failed and skipped jobs, and more. Customize these reports to your own requirements to acquire insightful knowledge about your backup environment.

Notifications and alerts

Configure alerts and notifications to be informed when important backup events occur. You’ll get timely notifications to take the necessary action, whether it’s about a failed job or other significant updates.

Automation simplicity: Integrate at-scale protection with your current systems

Effective cloud resource management requires automation, and new product works with the VM management tools you already have. Easy integration into your automation workflows is made possible by the Backup vault, regardless of whether you’re using Terraform, APIs, or the gcloud CLI. You will be able to use the UI and the gcloud CLI to secure Compute Engine virtual machines during the preview. You can use APIs and Terraform to integrate backup operations as code on top of your current infrastructure and integrate them into your larger VM management strategy once the offering is made broadly available. With this feature, you can be sure that your backup procedures are optimized for your current infrastructure and that they are also effective.

Take instant snapshots to guard against errors and corruption on Compute Engine workloads.

Consider yourself an application administrator updating a program within a scheduled maintenance window of one hour, and all of a sudden the update fails. This is the time to start your rollback processes. Recovering from routine snapshots is the standard procedure for rolling back; the duration of this process varies based on the amount of data that needs to be restored and can take several minutes. Alternatively, you may try to restore from a costly-to-maintain pre-upgrade replica of the program.

What if you had a disk-size-independent, incredibly quick, and affordable recovery solution that would let you take use of the whole maintenance window and provide a predictable rollback time?

Google is pleased to present instant snapshots for Compute Engine, which offer high-frequency, nearly immediate point-in-time checks of a disk that can be quickly restored as necessary.

Recovery time objective (RTO) in the tens of seconds and recovery point objective (RPO) of seconds are provided via instant snapshots. The only hyperscaler that provides high-performance checkpointing that enables instant recovery is Google Cloud. While competing hyperscalers may require tens of minutes or even hours for the workload to recover to full performance, your Compute Engine workload operates at full disk performance as soon as an instant snapshot is restored.

Compute Engine instant snapshots allow you to recover in cases when the data needs to be rolled back to an earlier state but the fundamental infrastructure is still intact. Typical usage cases include of:

Facilitating quick recovery from file system corruption, application software errors, and user mistake.

Backup verification routines that take regular snapshots and instantly restore them in order to perform data consistency checks, like those for database workloads.

Creating restore points prior to an application update allows for a quick rollback in case the scheduled maintenance doesn’t go as expected.

Apart from the aforementioned, employing immediate snapshots can also:

Boost developer productivity: Unintentional mistakes and build failures lengthen the time needed to finish tasks in fast development cycles with lengthy build times or complex code. You can benefit from quick restores using instant snapshots instead of relying on off-site backups.

Check the status before retreating: Make sure the backups you make are in the appropriate state and useable. Before starting a long-term backup, use quick snapshots to checkpoint and clone drives to validate on secondary machines.

Boost the frequency of backups: You can take regular backups of business-critical, high-volume databases that can’t afford long backup windows by using immediate snapshots.

Important aspects of instant snapshots

Compared to conventional snapshots, Compute Engine Instant snapshots have several advantages.

Backups that are made in-place: Instant snapshots are made on regional or zonal drives, and they are stored in the same location as the media type and source disk. This indicates that an immediate snapshot taken from an SDD (or HDD) disk shall be kept on the same disk. Instant snapshots have a fixed creation fee and additional storage costs are billed based on usage.

Quick and incremental: Only modified, incremental data blocks since the last instant snapshot are stored in each instant snapshot, which are prepared in a matter of seconds. Compared to backup snapshots, you can take snapshots far more frequently thanks to its high-frequency snapshotting capability.

Quick disk restore: Every instant snapshot can be quickly restored in a matter of seconds to a new disk. These new disks take on the disk type of the snapshot because they are in the same zone.

Convertible to backup or archive: For long-term, geo-redundant storage, you can migrate instant snapshots to a separate point of presence.

The operation of instant snapshots

Image credit to Google Cloud

Instant snapshots can be made in a matter of seconds since they only store the data that has changed since the initial checkpoint.

Thus, in the example above, instant snapshots generates a checkpoint at the time of creation, but they don’t actively store data until the disk’s third block is overwritten with fresh information. Keep in mind that adding data like Google Cloud doing here by writing to blocks 6 and 7 doesn’t necessitate using more storage on the checkpoint.

This results in both efficient performance and extremely high storage efficiency: As previously indicated, producing and restoring from an instant snapshot can be completed in a matter of seconds and has no effect on the underlying disk’s performance.

You can easily and effectively safeguard your workloads by integrating instant snapshots into your routine change-management and maintenance processes, thanks to their storage, performance, and cost-effectiveness.

Comparing different sorts of snapshots

An overview of the various performance attributes of instant snapshots in comparison to backup standard snapshots is provided here.Image credit to Google Cloud

An illustration of a use case

Maya, an administrator of applications, must update programs stored on persistent disk drives. She is allotted one hour to complete this maintenance window. If the volumes have the new software at the end of the window, it will be considered a successful upgrade; if not, it will be considered a rollback, and they will be in the exact same state as they were right before the maintenance window began.

Examine Maya’s process for taking instantaneous versus frequent snapshots:Image credit to Google Cloud

Make use of it now

Instant snapshots provide high-frequency backup and nearly instantaneous on-device point-in-time backups for quick restorations. They facilitate quick restores in the event of unsuccessful upgrades or user mistake, shorten the time needed for off-site backup, and improve program upgrade windows.

Blockchain Analytics

Globally, blockchain businesses are setting the standard in sectors including identity verification, financial services, and gaming as the technology finds practical uses in these domains. Blockchain infrastructure development and management, however, can be difficult and time-consuming.

Herein lies the role of Google Cloud

A wide range of services and products from Google Cloud are available to assist blockchain companies in developing, implementing, and maintaining their apps in a timely, safe, and effective manner.

Blockchain technology issues

Numerous elements of google cloud life could be transformed by blockchain technology. But in spite of its potential, blockchain still has to overcome the following obstacles before it can be widely used.

Scalability

Blockchain networks may slow down and fail as user and transaction volumes climb. Because every network node must confirm every transaction, it can take a long time.

Achievement

Another significant obstacle facing blockchain technology is achieving quick and dependable network performance, especially as the volume of users and transactions on the network increases.

Safety

Due to their ability to alter the network and store important data, blockchain nodes are a target for hackers.

Information

The massive amount of data that blockchains contain can make it difficult to scale, maintain privacy, and retrieve data quickly.

Google Cloud provides a wide range of services and products that are specifically designed to meet the demands of blockchain companies in order to assist you in addressing these difficulties. Let’s examine a few of the important choices.

Use infrastructure that is highly scalable to manage rising transaction volumes

With the help of Compute Engine and Google Cloud’s strong network, entrepreneurs may easily host their blockchain nodes and develop decentralised applications (dApps). Managed instance groups (MIGs) with autoscaling are another feature that Compute Engine offers. These MIGs may smoothly adjust to varying demand by automatically spinning up extra virtual machines (VMs) as needed and scaling down when demand decreases. Google Cloud virtual machines (VMs) start up quickly, so your dApps stay responsive and effective.

web3 applications

Another well-liked Google Cloud tool that facilitates the deployment and management of containerised applications is Google Cloud managed Kubernetes service, Google Kubernetes Engine (GKE). For organisations creating and overseeing Web3 applications, GKE is an excellent option since it streamlines node management for blockchain suppliers such as Blockdaemon and gives developers access to a scalable, self-healing infrastructure and detailed monitoring.

Blockchain Node Engine, Google Cloud newest blockchain node hosting service, is a fully managed solution created especially for Web3 development. It relieves developers of the burden of managing the underlying infrastructure by enabling them to quickly and simply create specialised blockchain nodes. You don’t have to worry about SRE jobs when using Blockchain Node Engine because Google Cloud takes care of them. Blockchain Node Engine is another tool that startups can utilise to jumpstart their dApp initiatives. For example, in less than an hour, entrepreneurs can access Etherum block data, and the public preview of support for Polygon and Solana is already underway.

Utilise high-performance infrastructure to optimise dApps

Google Cloud network services use automation, cutting-edge AI, and programmability to scale, secure, and optimise your infrastructure for startups seeking quick and secure transaction processing. Google Cloud gives you fine-grained control over your networking architecture, including firewall rules and IP addresses, in addition to location flexibility.

High-throughput, low-latency interconnects between blockchain nodes and your dApp are automatically established over Google’s high-performance private network. Google Cloud top priorities are security and access control. To that end, Google cloud provide Identity and Access Management (IAM) and Key Management Service (KMS) to guarantee adherence to user access roles and to facilitate the safe submission and approval of transactions involving digital assets through the use of encryption keys and signatures.

Examine blockchain information

Analytics Blockchain

BigQuery makes it simple to retrieve historical blockchain data with Google Cloud Blockchain Analytics, which streamlines SQL query data analysis. Developers can save time and resources by not having to manage nodes or create specialised indexers. Deeper insights into user behaviour and business operations can be obtained by developers by combining blockchain data with internal data and utilising BigQuery’s query engine.

Furthermore, BigQuery public datasets now feature 11 more blockchains in preview, meeting the increasing need for Web3-based comprehensive data access. Avalanche, Arbitrum, Cronos, Ethereum (Görli), Fantom (Opera), Near, Optimism, Polkadot, Polygon Mainnet, Polygon Mumbai, and Tron are among the recently added blockchains. BigQuery gives developers, researchers, and companies the ability to fully utilise blockchain technology by offering an easily accessible and all-inclusive source of blockchain data.

In a similar vein, Vertex AI enables entrepreneurs to unleash the full potential of blockchain data by fusing real-time processing, scalability, and Google’s deep AI and machine learning know-how to produce insights that are unmatched. Vertex AI’s state-of-the-art generative powers enhance data analysis and open up new avenues for exploring the blockchain’s potential. Some of these avenues include creating hyper-realistic simulations, creating synthetic data, or even using real data to create dynamic NFTs that can be used to interact with customers.

Develop and expand your blockchain enterprise

The goal of the Google Cloud Web3 startup programme is to help and expedite Web3 initiatives and businesses. By taking part in the programme, participants can access a multitude of advantages that can help Web3 startups succeed.

With large credits for Google Cloud services, the programme offers substantial financial support, relieving you of the burden of infrastructure expenditures and freeing you up to concentrate on innovation. In addition, the programme provides companies with exceptional learning opportunities that help them stay ahead of the curve in the rapidly changing Web3 ecosystem. Teams that join can take advantage of cutting-edge Google Cloud technologies, including early access to new products, and sophisticated hands-on labs.

Web3 and Google Cloud Advancing Blockchain Technologies

At Google Cloud, they are dedicated to advancing blockchain innovation and providing assistance to the Web3 community. We are assisting hundreds of blockchain firms in growing and developing ground-breaking new solutions that will play a significant role in shaping the future of the internet through Google Cloud Web3 startup programme and by offering cloud tools and resources that developers need to build and scale dApps securely.

Polygon

The Web3 ecosystem will expand more quickly because to the collaboration between Google Cloud and Polygon Labs, which will make it simpler for developers to create and use Polygon protocols. Building, launching, and expanding Web3 products and dApps is made easier for developers using tools and infrastructure from Google Cloud, a strategic cloud provider for Polygon protocols.

Developers can overcome the difficulties of setting up, managing, and running their own blockchain nodes with the aid of Blockchain Node Engine. To power dApps fast and simply, Google Cloud Marketplace also provides one-click deployment of a Polygon PoS node. To further aid in decision-making, Google Cloud gives developers access to the Polygon blockchain dataset on BigQuery, enabling them to examine real-time, on-chain, and cross-chain data.

Nansen

The blockchain analytics platform Nansen can handle and analyse up to 1 petabyte of blockchain data per day, and BigQuery is essential to this process. The capacity to process data in real-time is crucial for Nansen to give its consumers current information and help them make wise decisions in the quickly changing cryptocurrency market.

Furthermore, Nansen’s AI predictive modelling and recommendations approach has been influenced by Google Cloud’s machine learning technologies, including BigQuery ML and Cloud Inference API. Nansen has been able to create complex algorithms that can find hidden patterns and trends in blockchain data with the help of Google Cloud, giving users insightful analysis and prognostic suggestions.

Matter Research/zkSync

Matter Labs’ Layer-2 ZkSync protocol scales Ethereum with cutting-edge zero-knowledge (ZK) technology. It has become one of the fastest-growing ZK rollups worldwide since entering the mainnet in March 2023. Since its founding, Matter Labs has used Google Cloud, utilising GKE and additional services to meet the rapid expansion and high demand of the company. They have therefore tailored their proofing method to work with Google Cloud.

Matter Labs’ new hyperchain offering a new scaling solution for Ethereum and Google Cloud strengthened their partnership in 2024 to support the company’s next stage of expansion

GDL Italia – Google Compute Engine overview

https://m.youtube.com/watch?v=ZJ6cLxGAeRs

We’ve been working for three Years on the virtualization technology and Compute Engine, So it’s not a copy of what you Can find in other places It’s really unique to Google So from the bottom up, You know what [ INAUDIBLE ] was Explaining that, because of our scale, we had to Build our product, We have to design our server, We have to build our Network…