#googlecloudnext

Join Onix as we bring the future of AI, data migration, and cloud modernization to life! Our experts will be front and center with dynamic speaking sessions, vibrant networking events, and unforgettable entertainment. Get ready to experience innovation like never before!

📍 April 9–11 | Meet us at Booth #3250 on the showroom floor!

🎯 April 8 – Kick off the week at Topgolf with an evening of fun, food, and community vibes. 🍻 April 9–10 – Visit our VIP Lounge for deep-dive discussions and real connections. 🪩 April 9, 7:30–10:30 PM – Don’t miss the Onix Afterparty: Bollywood Nights Live! Expect electrifying performances, dance-worthy music, and unforgettable energy!

🎤 Featured Speaking Sessions:

April 10, 9:00 AM – Cloud Talk: No data, no AI – How data fuels AI innovation

April 10, 11:45 AM – Breakout Session: Migrate to Google Workspace – Tips for a seamless transition

April 10, 1:30 PM – Lightning Talk at the Google Public Sector Booth

April 10, 2:00 PM – Roundtable: The Age of Agentic AI

April 10, 4:00 PM – Breakout Session: Unlock your data for a competitive edge – Insights from industry leaders

Adaptive audio

What is adaptive audio?

Adaptive Audio is a feature that automatically adjusts the level of noise cancellation and transparency mode based on your surroundings. It helps block out noise in loud places and lets in important sounds, like voices or alarms, when needed. This way, you can stay aware of your environment without having to change settings manually.

The Google Meet team put up a demo area with many laptops arranged side by side at this year’s Cloud Next. After bringing clients into the room, the team asked them what they would think would happen if all of the laptops entered the conference at once.

Following the global shift to video conferencing and, subsequently, hybrid work as a result of the pandemic, the team began working on adaptive audio. Due to supply chain difficulties at the time, obtaining new conference room hardware was difficult. Additionally, Huib notes that many businesses either lacked the funds for specialized meeting room technology or did not initially have enough video conferencing rooms.

Without having to cram themselves around a single laptop, teams needed to be able to set up ad hoc meeting areas. However, it’s far more difficult than it seems to allow everyone to connect from their own devices while keeping the “screams” quiet.

Consider the audio system in a movie theater. According to Meet Software Engineer Manager Henrik Lundin, “you have a number of speakers around you, and it’s a pleasant audio experience because they’re all connected to the same sound source, so they play out in an intended synchronicity.” Now, it would sound awful if multiple devices were playing the same music in the same room without synchronization. As if you were in a huge cathedral, you are receiving numerous copies of the same sounds. Similarly, when you talk in front of a group of microphones on several devices, they all record sound simultaneously even though they are not on the same time.

The echo issue comes next. You’ve undoubtedly observed that when you use video conferencing technologies, you occasionally hear an echo of your own voice.”The devices that conduct meetings have an echo canceller built in, so you don’t always get that,” Henrik says. It’s a signal processing method that attempts to determine which portion of the microphone signal is your speech and which portion is merely coming from the device’s speakers. When several laptops are in the same room playing the same audio and connecting to each other’s microphones, this becomes ten times more difficult.

The team had to spend a lot of time in the same room and figure out how to make their laptops recognize each other as being adjacent to each other in order to solve this audio challenge. Initially, they experimented with allowing attendees to join pre-established groups during the conference. “This was clearly prone to mistakes, but it allowed us to test the experience of synchronizing the microphones and speakers on all of the laptops,” Henrik adds.

They then experimented using ultrasound. The laptops may sense the presence of other computers nearby and start acting as a group by making high-frequency noises that are inaudible to the human ear. Users no longer had to choose the room they were in or manually configure their devices as a result. It was quite difficult, though, as Henrik explains, “because the ultrasound had to be accurate and dependable on any device if audio leaks from the room next door, it shouldn’t think you’re in the same room.” In order to improve accuracy, the researchers used a novel kind of ultrasound and adjusted the volume and frequency to maximize reach without being audible.

Adaptive audio immediately turns on when Meet recognizes that there are several laptops present, synchronizing the microphones and speakers on each laptop without shutting down any of them. Depending on who is speaking, it alternates between microphones to avoid echo and feedback. Before sending Adaptive audio to other participants, Meet also employs backend processing and a cloud denoiser to improve audio quality and eliminate background noise.

Adaptive audio is already used in numerous Google meetings every day, often without the participants’ knowledge. It is one of those technologies that relieves the user’s cognitive burden. Before attending a meeting, people don’t need to question whether they’re set up correctly, explains Ahmed Aly, Meet Interaction Design Lead. No matter how complex and incredible the tech is, from the end user’s perspective, it simply works anytime they open their laptop and attend a meeting.

In the future, the group is still investigating ways to facilitate connections, particularly in situations where meeting spaces or conferencing equipment are not available. Huib stated that “we hope it gives more flexibility and improves meeting equity and participation.” “You can be seen and heard clearly from wherever you are sitting because the camera and microphone are directly in front of you.”

PII data

Due to operational or regulatory constraints, Google Cloud clients who are interested in developing or updating their data lake architecture frequently have to keep a portion of their workloads and data on-premises.

You can now completely modernise your data lake with cloud-based technologies while creating hybrid data processing footprints that enable you to store and process on-prem data that you are unable to shift to the cloud, thanks to Dataproc on Google Distributed Cloud, which was unveiled in preview at Google Cloud Next ’24.

Using Google-provided hardware in your data centre, Dataproc on Google Distributed Cloud enables you to run Apache Spark processing workloads on-premises while preserving compatibility between your local and cloud-based technology.

For instance, in order to comply with regulatory obligations, a sizable European telecoms business is updating its data lake on Google Cloud while maintaining Personally Identifiable Information (PII) data on-premises on Google Distributed Cloud.

Google Cloud will demonstrate in this blog how to utilise Dataproc on Google Distributed Cloud to read PII data that is stored on-premises, compute aggregate metrics, and transfer the final dataset to the cloud’s data lake using Google Cloud Storage.

PII is present in this dataset. PII needs to be kept on-site in their own data centre in order to comply with regulations. The customer will store this data on-premises in object storage that is S3-compatible in order to meet this requirement. Now, though, the customer wants to use their larger data lake in Google Cloud to determine the optimal places to invest in new infrastructure by analysing signal strength by geography.

Full local execution of Spark jobs capable of performing an aggregation on signal quality is supported by Dataproc on Google Distributed Cloud, allowing integration with Google Cloud Data Analytics while adhering to compliance standards.

PII is present in this dataset. PII needs to be kept on-site in their own data centre in order to comply with regulations. The customer will store this data on-premises in object storage that is S3 compatible in order to meet this requirement. The customer now wants to analyse signal strength by location and determine the optimal places for new infrastructure expenditures using their larger data lake in Google Cloud.

Reading PII data with Google Distributed Cloud Dataproc requires various steps to assure data processing and privacy compliance.

To read PII data with Google Distributed Cloud Dataproc, just set up your Google Cloud environment.

Create a Google Cloud Project: If you don’t have one, create one in GCP.

Project billing: Enable billing.

In your Google Cloud project, enable the Dataproc API, Cloud Storage API, and any other relevant APIs.

Prepare PII

Securely store PII in Google Cloud Storage. Encrypt and restrict bucket and data access.

Classifying Data: Label data by sensitivity and compliance.

Create and configure Dataproc Cluster

Create a Dataproc cluster using the Google Cloud Console or gcloud command-line tool. Set the node count and type, and configure the cluster using software and libraries.

Security Configuration: Set IAM roles and permissions to restrict data access and processing to authorised users.

Develop Your Data Processing Job

Choose a Processing Framework: Consider Apache Spark or Hadoop.

Write the Data Processing Job: Create a script or app to process PII. This may involve reading GCS data, transforming it, and writing the output to GCS or another storage solution.

Job Submission to Dataproc Cluster

Submit your job to the cluster via the Google Cloud Console, gcloud command-line tool, or Dataproc API.

Check work status and records to guarantee completion.

Compliance and Data Security

Encrypt data at rest and in transit.

Use IAM policies to restrict data and resource access.

Compliance: Follow data protection laws including GDPR and CCPA.

Destruction of Dataproc Cluster

To save money, destroy the Dataproc cluster after data processing.

Best Practices

Always mask or anonymize PII data when processing.

Track PII data access and changes with extensive recording and monitoring.

Regularly audit data access and processing for compliance.

Data minimization: Process just the PII data you need.

Conclusion

PII processing with Google Distributed Cloud Dataproc requires careful design and execution to maintain data protection and compliance. Follow the methods and recommended practices above to use Dataproc for data processing while protecting sensitive data.

Dataproc

The managed, scalable Dataproc service supports Apache Hadoop, Spark, Flink, Presto, and over thirty open source tools and frameworks. For safe data science, ETL, and data lake modernization at scale that is integrated with Google Cloud at a significantly lower cost, use Dataproc.

ADVANTAGES

Bring your open source data processing up to date.

Your attention may be diverted from your infrastructure to your data and analytics using serverless deployment, logging, and monitoring. Cut the Apache Spark management TCO by as much as 54%. Create and hone models five times faster.

OSS for data science that is seamless and intelligent

Provide native connections with BigQuery, Dataplex, Vertex AI, and OSS notebooks like JupyterLab to let data scientists and analysts do data science tasks with ease.

Google Cloud integration with enterprise security

Features for security include OS Login, customer-managed encryption keys (CMEK), VPC Service Controls, and default at-rest encryption. Add a security setting to enable Hadoop Secure Mode using Kerberos.

Important characteristics

Completely automated and managed open-source big data applications

Your attention may be diverted from your infrastructure to your data and analytics using serverless deployment, logging, and monitoring. Cut the Apache Spark management TCO by as much as 54%. Integrate with Vertex AI Workbench to enable data scientists and engineers to construct and train models 5X faster than with standard notebooks. While Dataproc Metastore removes the need for you to manage your own Hive metastore or catalogue service, the Jobs API from Dataproc makes it simple to integrate large data processing into custom applications.

Use Kubernetes to containerise Apache Spark jobs

Create your Apache Spark jobs with Dataproc on Kubernetes so that you may utilise Dataproc to provide isolation and job portability while using Google Kubernetes Engine (GKE).

Google Cloud integration with enterprise security

By adding a Security Configuration, you can use Kerberos to enable Hadoop Secure Mode when you construct a Dataproc cluster. Additionally, customer-managed encryption keys (CMEK), OS Login, VPC Service Controls, and default at-rest encryption are some of the most often utilised Google Cloud-specific security features employed with Dataproc.

The best of Google Cloud combined with the finest of open source

More than 30 open source frameworks, including Apache Hadoop, Spark, Flink, and Presto, are supported by the managed, scalable Dataproc service. Simultaneously, Dataproc offers native integration with the whole Google Cloud database, analytics, and artificial intelligence ecosystem. Building data applications and linking Dataproc to BigQuery, Vertex AI, Spanner, Pub/Sub, or Data Fusion is a breeze for data scientists and developers.

The news keep flowing from San Francisco during the Google Cloud Next 2018 event: in case you've missed the first two days, you can view them on YouTube by clicking here for Day 1 Next Live Show and/or here for Day 2 Next Live Show. Following the great advances disclosing the Google AdWords rebrand into the new Google Ads framework and the Data Transfer Project lauched with Twitter, Facebook and Microsoft, the big announcement of the day is the new stand-alone version of Google Drive: a new subscription formula designed specifically for the business world that will allow the use of the famous cloud storage & sharing service - which, according to Google, is about to reach one billion users - to all the companies that are not interested in signing up for a G-Suite subscription - or cannot afford it. According to the Mountain View giant, the stand-alone version of Google Drive has always been a most wanted feature for a lot of potential customers: for this very reason, Google Drive will be provided as a dedicated ,stand-alone service, with all the features of online storage and sharing available to the version shipped with the G-Suite package: keep everything, share anything, as the product's catchphrase says.