Programmatic Ponderings

Java Development with Microsoft SQL Server: Calling Microsoft SQL Server Stored Procedures from Java Applications Using JDBC

View of the post’s Java project from JetBrains’ IntelliJ IDE

This is an updated version of a very popular blog post, originally published in August 2012.

Introduction

Enterprise software solutions often combine multiple technology platforms. Accessing an Oracle database via a Microsoft .NET application and vice-versa, accessing Microsoft SQL Server from a Java-based application is…

AWS IoT, LoRa, and LoRaWAN: Collecting and Analyzing IoT Data in Near Real-Time with AWS IoT, LoRa, and LoRaWAN

Gary Stafford · Collecting and Analyzing IoT Data in Near Real-Time with AWS IoT, LoRa, and LoRaWAN Introduction

In a recent post published on ITNEXT, LoRa and…

Presto Federated Queries: Getting Started with Presto Federated Queries using Ahana’s PrestoDB Sandbox on AWS

https://soundcloud.com/gary-stafford-214522524/introduction-getting-started-with-presto-federated-queries-using-ahanas-prestodb-sandbox-on-aws Audio introduction to the post Introduction

According to The Presto Foundation, Presto (aka PrestoDB), not to be confused with PrestoSQL, is an open-source, distributed, ANSI SQL compliant query engine. Presto is designed to run interactive ad-hoc…

LoRA and LoRaWAN for IoT: Getting Started with Long Range and LoRaWAN Specification for Low Power, Wide Area Networking

  Introduction

According to the LoRa Alliance, Low–Power, Wide-Area Networks (LPWAN) are projected to support a major portion of the billions of devices forecasted for the Internet of Things (IoT). LoRaWAN is designed from the bottom up to optimize LPWANs for battery lifetime, capacity, range, and cost. LoRa and LoRaWAN permit long-range connectivity for the Internet of Things (IoT) devices in…

https://programmaticponderings.files.wordpress.com/2020/03/blog.bd95ce07-abad-4924-8a09-3c051fc21d1b.mp3 Audio Version of Post

Writing a blog about your latest product’s features? Case studies on your latest’s customer integrations? Opinions and insights into your industry, or your customer’s industries? Well written blog posts can not only inform, but they also demonstrate you or your…

https://programmaticponderings.files.wordpress.com/2020/03/polly-saas-interactions-1.mp3 Audio Version of Post

You’re a software vendor selling a SaaS product hosted on the public cloud. As part of your solution, the SaaS product needs to interact with your customer’s cloud-based resources—their data, their applications, or perhaps their compute resources. As a vendor, you must architect your…

Streaming Data Analytics with Amazon Kinesis Data Firehose, Redshift, and QuickSight

In the following post, we will demonstrate the use of Amazon Kinesis Data Firehose, Amazon Redshift, and Amazon QuickSight to analyze streaming data. We will simulate time-series data, streaming from a set of IoT sensors to Kinesis Data Firehose. Kinesis will write the IoT data to an Amazon S3 Data Lake, where it will then be copied to Amazon Redshift in near real-time. In Amazon Redshift, we…

Architecting a Successful SaaS: Understanding Public Cloud SaaS Product Models

You’re a startup with an idea for a revolutionary new software product. You quickly build a beta version and deploy it to the cloud. After a successful social-marketing campaign and concerted sales effort, dozens of customers subscribe to your SaaS-based product. You’re ecstatic…until you realize you never architected your product for this level of success. You were so busy coding, raising…

Getting Started with Data Analysis on AWS using AWS Glue, Amazon Athena, and QuickSight: Part 2

Introduction

In part one, we learned how to ingest, transform, and enrich raw, semi-structured data, in multiple formats, using Amazon S3, AWS Glue, Amazon Athena, and AWS Lambda. We built an S3-based data lake and learned how AWS leverages open-source technologies, including Presto, Apache Hive, and Apache Parquet. In part two of this post, we will use the transformed and enriched data sources,…

Enabling Amazon Athena Queries Directly from JetBrains PyCharm

Amazon Athena

According to Amazon, Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Amazon Athena supports and works with a variety of popular data file formats, including CSV, JSON, Apache ORC, Apache Avro, and Apache Parquet.

The underlying technology behind Amazon Athena is Presto, the popular, open-source distributed SQL query engine…

Getting Started with Data Analysis on AWS using S3, Glue, Athena, and QuickSight

Introduction

According to Wikipedia, data analysis is “a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusion, and supporting decision-making.” In this two-part post, we will explore how to get started with data analysis on AWS, using the serverless capabilities of Amazon Athena, AWS Glue, Amazon QuickSight, Amazon…

Getting Started with Data Analytics using Jupyter Notebooks, PySpark, and Docker

(This post is an update of an earlier post, published November 2018, and includes updated and additional content.)

Introduction

There is little question, big data analytics, data science, artificial intelligence (AI), and machine learning(ML), a subcategory of AI, have all experienced a tremendous surge in popularity over the last few years. Behind the marketing hype, these technologies are…

Getting Started with Apache Zeppelin on Amazon EMR, using AWS Glue, RDS, and S3: Part 2

Introduction

In Part 1 of this two-part post, we created and configured the AWS resources required to demonstrate the use of Apache Zeppelin on Amazon Elastic MapReduce (EMR). Further, we configured Zeppelin integrations with AWS Glue Data Catalog, Amazon Relational Database Service (RDS) for PostgreSQL, and Amazon Simple Cloud Storage Service(S3) Data Lake. We also covered how to obtain the…

Getting Started with Apache Zeppelin on Amazon EMR, using AWS Glue, RDS, and S3: Part 1 – Setup

Introduction

There is little question big data analytics, data science, artificial intelligence (AI), and machine learning (ML), a subcategory of AI, have all experienced a tremendous surge in popularity over the last 3–5 years. Behind the hype cycles and marketing buzz, these technologies are having a significant influence on all aspects of our modern lives. Due to their popularity, commercial…

Amazon ECR Cross-Account Access for Containerized Applications on ECS

Recently, I was asked a question regarding sharing Docker images from one AWS Account’s Amazon Elastic Container Registry (ECR) with another AWS Account who was deploying to Amazon Elastic Container Service (ECS). The answer was relatively straightforward, use ECR Repository Policiesto allow cross-account access to pull images. However, the devil is always in the implementation details.…

Event-driven, Serverless Architectures with AWS Lambda, SQS, DynamoDB, and API Gateway

Introduction

In this post, we will explore modern application development using an event-driven, serverless architecture on AWS. To demonstrate this architecture, we will integrate several fully-managed services, all part of the AWS Serverless Computing platform. Serverless AWS offerings include Lambda, API Gateway, SQS, S3, and DynamoDB. The end result will be an application composed of small,…

Getting Started with PostgreSQL using Amazon RDS, CloudFormation, pgAdmin, and Python

Introduction

In the following post, we will explore how to get started with Amazon Relational Database Service (RDS) for PostgreSQL. We will use CloudFormation to build a master PostgreSQL database instance and a single read replica in a new VPC. We will use AWS Systems Manager Parameter Store to store our CloudFormation configuration values. Amazon RDS Event Notifications will send us mobile…