Working with Amazon Simple Notification Service
Hi! In this,the first of a 2 series of posts, I would like to talk about the Amazon Simple Notification Service web service we use here at XO. What it is, how it works and how to configure it.
Amazon Simple Notification Service (SNS) provides a convenient means for distributing data between applications. SNS is organized in user created âTopicsâ. Applications that generate the data to be distributed are called âPublishersâ and applications that will consume the data are called âSubscribersâ. The later can be HTTP/s endpoints, email, SMS, Mobile Apps or Amazon Simple Queue Service (SQS).
As a use case, letâs say we have an application that manages user accounts. Now letâs suppose that we have other applications that need to be be âawareâ of changes in user data. For example a newsletter mailing system, that would need to know when new users are created, their information updated or their accounts deleted. Or an analytics tool that creates reports on user statistics that manages itâs own database.
By using SNS, we can have these systems talk to each other in a loosely coupled fashion. The accounts managing application would be the publisher. Then the mailing and analytics systems would be subscribers to the SNS topic.
The publishing application can push user data changes to the topic in the form of messages by using one of the many SDK libraries (Ruby, .NET, Jscript, etcâŚ) . The Subscribers can, for example,  wrap their API around http REST endpoints that will parse the incoming messages from the topic.
Creating an SNS Topic and Subscriptions
We can create SNS topics and subscriptions through the AWS console.
https://console.aws.amazon.com/console/home
This is our home screen to access all services in the AWS constellation. Letâs go ahead and click on âSNS Push Notification Serviceâ
On the following screen, click on âCreate New Topicâ
Weâll call the topic âUser_Data_Updatesâ . Click on âCreate Topicâ.
arn:aws:sns:us-west-2:867544872691:User_Data_Updates
This key generated by AWS is the topicâs unique identifier, used by both publishers and subscribing applications.
Now, to add a subscription, click on âCreate subscriptionâ
SNS allows different types of subscriptions or âprotocolsâ, such as http, https, sns, email or SQS queues. We will choose HTTP and enter the endpointâs url. Finally, click on âsubscribeâ
The subscription will show up on the console with a status of âPending confirmationâ
Finalizing the subscription and message formats
In order for endpoints to begin receiving publisher messages, they need to âconfirmâ the Topic subscription to SNS.
Letâs talk a little about the message formats. SNS messages to subscribers come in 2 flavors:
-Subscription confirmation
Subscription confirmations are sent only once, when an endpoint is subscribed to the topic. This is what the message looks like:
POST / HTTP/1.1
x-amz-sns-message-type: SubscriptionConfirmation
x-amz-sns-message-id: 165545c9-2a5c-472c-8df2-7ff2be2b3b1b
x-amz-sns-topic-arn: arn:aws:sns:us-east-1:123456789012:MyTopic
x-amz-sns-subscription-arn: arn:aws:sns:us-east-1:123456789012:MyTopic:2bcfbf39-05c3-41de-beaa-fcfcc21c8f55
Content-Length: 1336
Content-Type: text/plain; charset=UTF-8
Host: example.com
Connection: Keep-Alive
User-Agent: Amazon Simple Notification Service Agent
{
 "Type" : âSubscriptionConfirmationâ,
 "MessageId" : â165545c9-2a5c-472c-8df2-7ff2be2b3b1bâ,
 "Token" : â2336412f37fb687f5d51e6e241d09c805a5a57b30d712f794cc5f6a988666d92768dd60a747ba6f3beb71854e285d6ad02428b09ceece29417f1f02d609c582afbacc99c583a916b9981dd2728f4ae6fdb82efd087cc3b7849e05798d2d2785c03b0879594eeac82c01f235d0e717736â,
 "TopicArn" : âarn:aws:sns:us-east-1:123456789012:MyTopicâ,
 "Message" : âYou have chosen to subscribe to the topic arn:aws:sns:us-east-1:123456789012:MyTopic.\nTo confirm the subscription, visit the SubscribeURL included in this message.â,
 "SubscribeURL" : âhttps://sns.us-east-1.amazonaws.com/?Action=ConfirmSubscription&TopicArn=arn:aws:sns:us-east-1:123456789012:MyTopic&Token=2336412f37fb687f5d51e6e241d09c805a5a57b30d712f794cc5f6a988666d92768dd60a747ba6f3beb71854e285d6ad02428b09ceece29417f1f02d609c582afbacc99c583a916b9981dd2728f4ae6fdb82efd087cc3b7849e05798d2d2785c03b0879594eeac82c01f235d0e717736â,
 "Timestamp" : â2012-04-26T20:45:04.751Zâ,
 "SignatureVersion" : â1â,
 "Signature" : âEXAMPLEpH+DcEwjAPg8O9mY8dReBSwksfg2S7WKQcikcNKWLQjwu6A4VbeS0QHVCkhRS7fUQvi2egU3N858fiTDN6bkkOxYDVrY0Ad8L10Hs3zH81mtnPk5uvvolIC1CXGu43obcgFxeL3khZl8IKvO61GWB6jI9b5+gLPoBc1Q=â,
 "SigningCertURL" : âhttps://sns.us-east-1.amazonaws.com/SimpleNotificationService-f3ecfb7224c7233fe7bb5f59f96de52f.pemâ
 }
Notice the following 2 headers:
x-amz-sns-message-type: SubscriptionConfirmation
x-amz-sns-topic-arn: arn:aws:sns:us-east-1:123456789012:MyTopic
The first header tells our application that SNS is awaiting a subscription confirmation. The second header has the topicâs ARN.
To confirm the subscription, the endpoint needs to send an HTTP GET request to the âSubscribeUrlâ included in the messageâs request payload.
Therefore, our endpoint needs to identify both the topic arn (to make sure it is the right topic) and the subscriptionConfirmation message.
Once the GET request is issued, we can verify that the subscription is confirmed, by seeing on the console that the topic assigned a Subscription ID to the endpoint.
Once the endpoint is âsubscribedâ, messages will start being sent to it. This time, they will be of type âNotificationâ and this is what they will look like:
POST / HTTP/1.1
x-amz-sns-message-type: Notification
x-amz-sns-message-id: da41e39f-ea4d-435a-b922-c6aae3915ebe
x-amz-sns-topic-arn: arn:aws:sns:us-east-1:123456789012:MyTopic
x-amz-sns-subscription-arn: arn:aws:sns:us-east-1:123456789012:MyTopic:2bcfbf39-05c3-41de-beaa-fcfcc21c8f55
Content-Length: 761
Content-Type: text/plain; charset=UTF-8
Host: ec2-50-17-44-49.compute-1.amazonaws.com
Connection: Keep-Alive
User-Agent: Amazon Simple Notification Service Agent
{
 "Type" : âNotificationâ,
 "MessageId" : âda41e39f-ea4d-435a-b922-c6aae3915ebeâ,
 "TopicArn" : âarn:aws:sns:us-east-1:123456789012:MyTopicâ,
 "Subject" : âtestâ,
 "Message" : âtest messageâ,
 "Timestamp" : â2012-04-25T21:49:25.719Zâ,
 "SignatureVersion" : â1â,
 "Signature" : âEXAMPLElDMXvB8r9R83tGoNn0ecwd5UjllzsvSvbItzfaMpN2nk5HVSw7XnOn/49IkxDKz8YrlH2qJXj2iZB0Zo2O71c4qQk1fMUDi3LGpij7RCW7AW9vYYsSqIKRnFS94ilu7NFhUzLiieYr4BKHpdTmdD6c0esKEYBpabxDSc=â,
 "SigningCertURL" : âhttps://sns.us-east-1.amazonaws.com/SimpleNotificationService-f3ecfb7224c7233fe7bb5f59f96de52f.pemâ,
 "UnsubscribeURL" : âhttps://sns.us-east-1.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:123456789012:MyTopic:2bcfbf39-05c3-41de-beaa-fcfcc21c8f55â
}
Notice that now the âmessage-typeâ header has a value of âNotificationâ.
The messageâs actual data is now present on the âMessageâ key on the payload.
We can test the endpoints by manually publishing to the topic. On the console, click on the âPublishâ button. Fill in the form, and click on âPublish Messageâ.
Then verify with whichever debugging / monitoring tool you are using, that the messages indeed are arriving.
With these simple steps, we can very quickly create distributed architectures that allow our systems to be loosely coupled. We can dynamically allocate resources to consume data in different ways, with ease, without the need to modify other participating systems.
On my next post I will get into the details of implementing http endpoints that subscribe to an SNS topic.
Thank you for reading and see you on my next post!
Alexander Copquin is a Senior Software Engineer working at XO Group on the Membership and Community team. He has over 20 years experience working in both the electronics and software arenas in 3 countries. He has developed and architected systems in several stacks and back-ends including .NET and Ruby on Rails. He also has a background in South American music, having a particular love for the harmony present in well-written code.