#sfscala

(via https://www.youtube.com/watch?v=LQg3ErGHSGw)

Scala-powered Data Pipelines By the Bay

Your app is a hit. Everyone's using your service. You've instrumented your website to collect real time usage data and you've set up a data processing pipeline to work with large amounts of data in real time, but how do you bridge the gap between web page and data pipeline? With websockets, Kafka and Akka Streams, of course!

This talk covers using Akka Streams and Akka HTTP to, in less than 100 lines of code, build a server that consumes streams of events via websocket and publishes them to a Kafka topic, after which you can use the data analysis framework of your choice to analyze them to your heart's content.

(via https://www.youtube.com/watch?v=imNYRPO74R8)

Scala-powered Data Pipelines By the Bay

Implementing a Distributed Hash Table with Scala and Akka

Tristan Penman will be talking about his experience implementing a distributed hash table, or DHT, based on the Chord protocol. In particular, the talk will cover some Scala and Akka best practices, and look at how these practices influence the design of a non-trivial actor system.

(via https://www.youtube.com/watch?v=NbWSKV1RnFg)

Data Fellas are speaking at data.bythebay.io!

Andy Petrella and Xavier Tordoir (Data Fellas) talk with Alexy Khrabrov about Spark Notebook, pivoting from physics to data science, where notebooks are going with Spark and Scala, and more.

(via https://www.youtube.com/watch?v=6J5oZcaduxE)

Spark Notebook is presented at Data By the Bay!

Data Science, a buzz word we've seen popping everywhere in 2015

Why? It turns out engineers explored the Big Data's value and the way to deal it, that is, digging the gold, Data Science, covering mathematics, statistics, machine learning, data preparation, software development and more

Data science came to the front because data is accumulating and exploiting the value is a key to competitivity. Data Science and Machine Learning in particular had traditionally been the smart and helpful tool mostly designed and developed in academia, the enterprise could only grasp at high premium

Now the game is changing drastically, methods have matured, libraries are available and more data scientists are entering the market

Still, there are many friction points in the development process of services exploiting data. It's true that Data Scientists are developers, but usually they are not software developers and even less devops which leads to a disrupted organization and a lack of efficiency

(via https://www.youtube.com/watch?v=VP-tJOXrf2o)

See also: Scala-powered Data Pipelines By the Bay

(via https://www.youtube.com/watch?v=VP-tJOXrf2o)

Scala is powering multiple data pipelines at Data By the Bay

(via https://www.youtube.com/watch?v=SXmKA1rNAgk)

It wasn't long ago that creating a highly specialized mini-language was a herculean undertaking. After dusting off the ol' "Dragon Book" we would implement a tokenizer and write our own BNF grammar. Then, we run these grammar rules through some arcane compiler-compiler tool (e.g. yacc) and iterate ad infinitum. But, now we have a truly scalable language at our fingertips. With the aid of predefined constructs and open source libraries, we can build our very own DSL, using this sophisticated language, requiring nothing more than the scalac compiler.

Yet, even for building a rather simple expression style DSL, which is at the very heart of many modern rule-based languages, it doesn't take long for the intrinsic complexity of language design to surface.  In this talk, we will dig deep in the functional programmer's tool chest, and uncover a little known gem known as the Kleisli.  We will explore how this deceptively simple piece of machinery in category theory can be used to build complex expression DSLs, with a completely modular library of operations and pluggable error handling.  The code examples will be entirely Scala, with dependencies on the scalaz library.  This presentation is geared towards those with an intermediate/advanced skill level in Scala and/or functional programming.

Ryan Delucchi has worked in the software industry for 12 years at various companies, including TiVo, Hotwire and Netflix. Furthermore, he has built a scripting engine for processing search results and mass transit route validation at startups Mobile Content Networks and Urban Mapping, respectively.

(via https://www.youtube.com/watch?v=VyYNotCzySA)

At x.ai we are building Amy and Andrew, AI assistants, who magically schedule meetings for you over email. Their brain is written in Scala, carefully employing functional programming techniques to improve correctness, testability and maintainability. This is the story of our decisions, how they turned out and what we overlooked. I’ll speak about relevant parts of our architecture and how we are using libraries like scalaz, shapeless and Monocle in production. I’ll give a gentle, beginner-targeted introduction to concepts such as Reader, Applicative, Monoid, Semigroup and Lenses and explain how we use them in Amy’s and Andrew’s brain.