#zeroetl

The general availability of Amazon Aurora PostgreSQL and Amazon DynamoDB zero-ETL integrations with Amazon Redshift.

The Amazon Aurora PostgreSQL-Compatible Edition zero-ETL integrations with Amazon Redshift are now generally available. By eliminating the need to create and maintain intricate data pipelines that carry out extract, transform, and load (ETL) activities, zero-ETL integration effortlessly makes transactional or operational data available in Amazon Redshift. It updates source data for you to use in Amazon Redshift for analytics and machine learning (ML) skills to extract timely insights and efficiently respond to important, time-sensitive events while automating source data replication to Amazon Redshift.

With these new zero-ETL integrations, you can conduct unified analytics on your data from various applications, eliminating the need to create and maintain separate data pipelines to write data from multiple relational and non-relational data sources into a single data warehouse.

Amazon Redshift is the target and a source is specified in order to construct a zero-ETL integration. The integration keeps an eye on the pipeline’s condition while replicating data from the source to the target data warehouse and making it easily accessible in Amazon Redshift.

Amazon Redshift integration of Aurora PostgreSQL zero-ETL

Near real-time analytics on petabytes of transactional data are made possible by the integration of Amazon Redshift and Amazon Aurora zero-ETL.

Why Aurora zero-ETL integration with Amazon Redshift?

Near real-time analytics and machine learning (ML) on petabytes of transactional data are made possible by the integration of Amazon Redshift with Amazon Aurora zero-ETL. Zero-ETL eliminates the need to create and maintain intricate data pipelines that carry out extract, transform, and load (ETL) activities by effortlessly making transactional data available in Amazon Redshift a few seconds after it was entered into Amazon Aurora.

Advantages

Access to data in almost real time

Run near real-time analytics and machine learning on petabytes of data by accessing transactional data from Aurora in Amazon Redshift in a matter of seconds.

Simple to use

Without having to create and maintain ETL pipelines to transfer transactional data to analytics platforms, you can quickly examine your transactional data in almost real time.

Smooth integration of data

To perform unified analytics across numerous apps and data sources, combine several tables from different Aurora database clusters and replicate your data to a single Amazon Redshift data warehouse.

Absence of infrastructure management

Using both Amazon Redshift Serverless and Amazon Aurora Serverless v2, you can do analytics on transactional data in almost real-time without managing any infrastructure.

Use cases

Operational analytics in near real time

To effectively respond to important, time-sensitive events, use Amazon Redshift analytics and machine learning capabilities to extract insights from transactional and other data in almost real-time. For use cases like fraud detection, data quality monitoring, content targeting, better gaming experiences, and customer behavior research, near real-time analytics can help you obtain more precise and timely insights.

Large-scale analytics

Petabytes of your transactional data pooled from several Aurora database clusters can be analyzed using Amazon Redshift’s capabilities thanks to the Aurora zero-ETL connector. You can benefit from Amazon Redshift’s extensive analytical features, including federated access to numerous data stores and data lakes, materialized views, built-in machine learning, and data sharing. With Amazon Redshift ML’s native integration into Amazon SageMaker, you can use simple SQL queries to generate billions of predictions.

lessen the operational load

It is frequently necessary to create, oversee, and run a sophisticated data pipeline ETL system in order to move data from a transactional database into a central data warehouse. You may easily transfer the schema, current data, and data modifications from your Aurora database to a new or existing Amazon Redshift cluster with a zero-ETL integration. Complex data pipeline management is no longer necessary with zero-ETL integration.

How to begin

You designate an Amazon Redshift data warehouse as the target and an Aurora DB cluster as the data source when creating your zero-ETL interface between Aurora and Redshift. Data from the source database is replicated into the target data warehouse through the integration. Within seconds, the data is accessible in Amazon Redshift, enabling data analysts to start utilizing the analytics and machine learning features of the platform.

Cost

Aurora zero-ETL integration with Amazon Redshift is free of charge via AWS. The change data produced by a zero-ETL integration is created and processed using pre-existing Aurora and Amazon Redshift resources, which you pay for. These resources could consist of:

By turning on change data capture, more I/O and storage are used.

For the first data export to seed your Amazon Redshift databases, the snapshot export costs

Extra Amazon Redshift storage for data replication

Extra Amazon Redshift computation for data replication processing

Cross-AZ data transfer fees for transferring data between sources and destinations.

Continuous data change processing via zero-ETL integration is provided at no extra cost. Please visit the Aurora price page for additional details.

Availability

The AWS regions for the US East (N. Virginia), US East (Ohio), US West (Oregon), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Europe (Frankfurt), Europe (Ireland), and Europe (Stockholm) now offer Aurora PostgreSQL zero-ETL integration with Amazon Redshift.

Zero ETL on Google cloud and Salesforce

The general availability of bidirectional data sharing between BigQuery and Salesforce Data Cloud excites us. Customers will be able to easily enhance their data use cases by safely merging data from several platforms, all without having to pay extra for complicated ETL (Extract, Transform, Load) pipelines and data infrastructure development.

More touchpoints and devices are available to provide instantaneous customer experiences, making prompt customer service more important than ever. However, as more data is generated, collected, and dispersed across SaaS apps and analytics platforms, it’s becoming harder

A partnership between Google Cloud and Salesforce was announced last year. According to the partnership, customers can easily combine data from both Salesforce Data Cloud and BigQuery, and can take advantage of the combined power of BigQuery and Vertex AI solutions to unlock and enrich new analytics and AI/ML scenarios.

They are making these features generally available today, allowing joint Google Cloud and Salesforce customers to safely access their data across various platforms and clouds. Consumers won’t need to set up or maintain infrastructure in order to access their Salesforce Data Cloud in BigQuery. They can also utilize their Google Cloud data to enhance Salesforce Customer 360 and other apps.

Customers of Salesforce and Google Cloud gain from these announcements in the following ways:

A single pane of glass and serverless, cross-platform data access requiring no ETL

Regulated and safe two-way access to their BigQuery and Salesforce Data Cloud data in almost real-time, without requiring the creation of data pipelines and infrastructure.

Enhance Salesforce Customer 360 and Salesforce Data Cloud with their Google Cloud data. Additionally, the capacity to enhance client data with minimal data movement by combining it with other pertinent public datasets.

utilizing unique Vertex AI and Cloud AI services for churn modeling, predictive analytics, and returning to customer campaigns via the integration of Vertex AI and Einstein Copilot Studio.

BigQuery Omni and Analytics Hub allow customers to view their data holistically across the Salesforce and Google platforms, spanning cloud boundaries. With the help of this integration, data scientists, marketing analysts, and other data and business users can now combine data from the Google and Salesforce platforms to conduct AI/ML pipelines, analyze data, and gain insights in a self-service manner without the assistance of infrastructure or data engineering teams.

Customers can concentrate on analytics and insights because this integration is completely managed and governed, and it spares them from a number of significant business difficulties that often arise when integrating important enterprise systems. These innovations uphold the data governance and access policies that administrators have established. Access to datasets is restricted to those that have been expressly shared, and only those with permission can exchange and examine the information. Relevant data is pre-filtered from Salesforce Data Cloud to BigQuery with minimal copying, lowering egress costs and data engineering overhead when data is spread across multiple clouds and platforms.

Simple and safe access from Google Cloud to Salesforce Data Cloud

Consumers want to be able to access and integrate their loyalty and point-of-sale data from Google analytics platforms with their marketing, commerce, and service data from Salesforce Data Cloud to gain actionable insights about their customer behavior, such as likelihood to buy, cross-sell/upsell recommendations, and the ability to run highly customized promotional campaigns. Additionally, they wish to use unique Google AI services to create machine learning models for training and predictions on top of combined Salesforce and Google Cloud data. This will enable use cases like price elasticity, market-mix modeling, churn modeling, customer funnel analysis, and A/B test experimentation.

Customers can now easily access their Salesforce Data Cloud data with the launch of unique BigQuery cross-cloud and data sharing features. They have access to all the pertinent data required to run powerful ad campaigns and conduct cross-platform analytics securely with other Google products. Administrators of Salesforce Data Cloud can quickly and easily share data with the appropriate BigQuery users or groups. Through the Analytics Hub UI, BigQuery users can effortlessly subscribe to shared datasets.

With this platform integration, information can be shared in multiple ways:

You can use a single cross-cloud join of your Salesforce Data Cloud and Google Cloud datasets for smaller datasets and ad hoc access, such as to identify the store with the highest sales last year, with little data movement or duplication.

You can use cross-cloud materialized views to access larger data sets that power your executive update, weekly business review, or marketing campaign dashboards. These views are automatically and incrementally updated, bringing in new data only on a periodic basis.

Add Google Cloud-stored data to Salesforce Customer 360

They also hear from customers, particularly retailers, who want to use the rich features of Salesforce Data Cloud to deliver personalized messaging, create a more comprehensive customer 360-degree profile, and access and combine behavioral data from their websites and mobile apps that was collected by Google Analytics with their own data. Breaking down data silos and providing customers with seamless real-time access to Google Analytics data within Salesforce Data Cloud to create more detailed customer profiles and personalized experiences is now easier than ever.

Customers of Salesforce Data Cloud can connect to their Google Cloud account with ease using point-and-click navigation, choose pertinent BigQuery datasets, and make them available as External Data Lake Objects, enabling real-time data access. After becoming Data Lake Objects, they function as native Data Cloud objects, enhancing Customer 360 data models and providing insights to support Analytics and Personalization for Customer 360 models. The operational overhead and latency associated with the conventional ETL copy approach are eliminated by this integration, which also removes the need to create and maintain ETL pipelines for data integration.Imagec redit to Google cloud

Tearing down the barriers separating Google and Salesforce data

With the help of this platform integration between Google Cloud and Salesforce Data Cloud, businesses can now break down data silos, obtain actionable insights, and provide outstanding customer service. Through the power of Google AI, unified access, and seamless data sharing, this partnership is revolutionizing how businesses use their data to achieve success.

Customers can directly access data stored in Salesforce Data Cloud and combine it with data in Google Cloud to further enrich it for business insights and activation through the unique cross-cloud functionality of BigQuery Omni and the data sharing capabilities of Analytics Hub. Customers no longer need to build custom ETL or move data in order to perform unmatched cross-cloud analytics or view their data across clouds.