It's Functional

(via https://www.youtube.com/watch?v=GXAgzxivze4)

Chris Moody is speaking at data.bythebay.io!

Chris Moody talks with Alexy about word2vec, the world of fashion and how deep learning can be used to identify trend vectors in it.

(via https://www.youtube.com/watch?v=eHcBeVnAiD4)

Chris is speaking at data.bythebay.io!

Q&A with Chris and Alexy

(via https://www.youtube.com/watch?v=3T32lweGxOA)

IBM is presenting at data.bythebay.io!

Berthold and Alexy talk abut Machine Learning on Spark, IBM's goals for Apache SystemML, and declarative approach.

(via https://www.youtube.com/watch?v=6VpiJK8Jydw)

IBM is presenting at data.bythebay.io!

Scalable machine learning is ubiquitous in virtually every industry ranging from insurance, manufacturing, finance, and health sciences. Expressing and and running machine learning algorithms for varying data characteristics and at scale is challenging. In this talk, we will discuss our experience in building Apache SystemML, peak at challenging optimization and implementation strategies in exploiting data-parallel platforms such as MapReduce and Spark, and provide performance and scalability insights.

(via https://www.youtube.com/watch?v=NbWSKV1RnFg)

Data Fellas are speaking at data.bythebay.io!

Andy Petrella and Xavier Tordoir (Data Fellas) talk with Alexy Khrabrov about Spark Notebook, pivoting from physics to data science, where notebooks are going with Spark and Scala, and more.

(via https://www.youtube.com/watch?v=6J5oZcaduxE)

Spark Notebook is presented at Data By the Bay!

Data Science, a buzz word we've seen popping everywhere in 2015

Why? It turns out engineers explored the Big Data's value and the way to deal it, that is, digging the gold, Data Science, covering mathematics, statistics, machine learning, data preparation, software development and more

Data science came to the front because data is accumulating and exploiting the value is a key to competitivity. Data Science and Machine Learning in particular had traditionally been the smart and helpful tool mostly designed and developed in academia, the enterprise could only grasp at high premium

Now the game is changing drastically, methods have matured, libraries are available and more data scientists are entering the market

Still, there are many friction points in the development process of services exploiting data. It's true that Data Scientists are developers, but usually they are not software developers and even less devops which leads to a disrupted organization and a lack of efficiency

(via https://www.youtube.com/watch?v=VP-tJOXrf2o)

See also: Scala-powered Data Pipelines By the Bay

(via https://www.youtube.com/watch?v=LQg3ErGHSGw)

Scala-powered Data Pipelines By the Bay

Your app is a hit. Everyone's using your service. You've instrumented your website to collect real time usage data and you've set up a data processing pipeline to work with large amounts of data in real time, but how do you bridge the gap between web page and data pipeline? With websockets, Kafka and Akka Streams, of course!

This talk covers using Akka Streams and Akka HTTP to, in less than 100 lines of code, build a server that consumes streams of events via websocket and publishes them to a Kafka topic, after which you can use the data analysis framework of your choice to analyze them to your heart's content.

(via https://www.youtube.com/watch?v=imNYRPO74R8)

Scala-powered Data Pipelines By the Bay

Implementing a Distributed Hash Table with Scala and Akka

Tristan Penman will be talking about his experience implementing a distributed hash table, or DHT, based on the Chord protocol. In particular, the talk will cover some Scala and Akka best practices, and look at how these practices influence the design of a non-trivial actor system.

(via https://www.youtube.com/watch?v=myFKoAXYiHI)

(via https://www.youtube.com/watch?v=AmusbnxOADM)

In this talk, we will discuss how we use Spark as part of a hybrid RDBMS architecture that includes Hadoop and HBase. The optimizer evaluates each query and sends OLTP traffic (including CRUD queries) to HBase and OLAP traffic to Spark. We will focus on the challenges of handling the tradeoffs inherent in an integrated architecture that simultaneously handles real-time and batch traffic. Lessons learned include: - Embedding Spark into a RDBMS - Running Spark on Yarn and isolating OLTP traffic from OLAP traffic - Accelerating the generation of Spark RDDs from HBase - Customizing the Spark UI The lessons learned can also be applied to other hybrid systems, such as Lambda architectures.

(via https://www.youtube.com/watch?v=zXluCSDfBPI)

Nikita Ivanov talks with Alexy Khrabrov before Nikita's talk at SF Text:

(via https://www.youtube.com/watch?v=pxkOZgkoMtg)

Speaker: Nikita Ivanov, Founder/CEO/CTO at GridGain/DataLingvo

In this talk I will talk about the technology behind DataLingvo - a Siri-like interface for the business data. Unlike many NLP-based startups DataLingvo is not based on ML but rather on a unique combination of semantic grammar modelling and human curation. Presentation will provide a deep dive into the reasons behind that technology choice as well as cover many tactical technical decisions. Presentation will include a live demo of DataLingvo system.

Nikita Ivanov is the guy behind DataLingvo, a cognitive analytics stealth startup that developed a Siri-like interface for the business data based on Human Curated Linguistics (HCL) technology. Nikita is also a founder and served as CEO/CTO for GridGain Systems, a company behind Apache Ignite, an industry leading in-memory data fabric.

Nikita is a serial entrepreneur with over 20 years of experience of starting companies and developing software including NLP, HPC and middleware platforms, contributing to the efforts of other startups and notable companies including Adaptec, Visa and BEA Systems. Nikita was one of the first pioneers using nascent Java technology for server side middleware development back in 1996.

(via https://www.youtube.com/watch?v=wA-PGz4taNQ)

SF Text (sftext.org) organizer Alexy Khrabrov Q&A with Ilya Kreymer, who talked about the Internet Archive, Wayback Machine, CommonCrawl, webrecorder, oldweb.today, history, formats and the best practices in archiving your web browsing experience, and accordion playing.

(via https://www.youtube.com/watch?v=wQXNk0I2INQ)

his talk will give a broad overview of the field of web archiving and its various forms, from large scale web crawling to high fidelity web recording.

We will cover how web data is crawled, stored and accessed by such existing services such as the Wayback Machine, CommonCrawl and the new Webrecorder project (https://webrecorder.io/), and what 'web archiving' even means. The talk will cover the ISO standard WARC format, web archive index APIs for CommonCrawl and Wayback Machine, and high-fidelity web archiving with Webrecorder, and present an overview of various open source tools and technologies available for working with web archives. Current and future challenges facing web archiving technology will also be discussed.

Speaker:

Ilya Kreymer has been worked in the field of web archiving for several years. He currently leads the development of the webrecorder in collaboration with Rhizome, a digital arts non-profit based in New York.

He has contributed to various open source web archiving and open access projects, including CommonCrawl project, Hypothesis annotation project and is also developing a new web archive replay and access toolset. Previously, he has worked at the Internet Archive focusing on web archive replay and access, and led the development of several new features and APIs for the Internet Archive Wayback Machine. Ilya is also a recipient of the Shuttleworth Foundation Flash Grant for Nov 2015.

(via https://www.youtube.com/watch?v=7fTyyI95Neo)

SF Scala organizer Alexy Khrabrov talks with the speaker Ryan Delucci about Ryan's adoption of Scala, engineering culture conducive of Functional Programming, and how to train new engineers for the best cultural fit.

(via https://www.youtube.com/watch?v=SXmKA1rNAgk)

It wasn't long ago that creating a highly specialized mini-language was a herculean undertaking. After dusting off the ol' "Dragon Book" we would implement a tokenizer and write our own BNF grammar. Then, we run these grammar rules through some arcane compiler-compiler tool (e.g. yacc) and iterate ad infinitum. But, now we have a truly scalable language at our fingertips. With the aid of predefined constructs and open source libraries, we can build our very own DSL, using this sophisticated language, requiring nothing more than the scalac compiler.

Yet, even for building a rather simple expression style DSL, which is at the very heart of many modern rule-based languages, it doesn't take long for the intrinsic complexity of language design to surface.  In this talk, we will dig deep in the functional programmer's tool chest, and uncover a little known gem known as the Kleisli.  We will explore how this deceptively simple piece of machinery in category theory can be used to build complex expression DSLs, with a completely modular library of operations and pluggable error handling.  The code examples will be entirely Scala, with dependencies on the scalaz library.  This presentation is geared towards those with an intermediate/advanced skill level in Scala and/or functional programming.

Ryan Delucchi has worked in the software industry for 12 years at various companies, including TiVo, Hotwire and Netflix. Furthermore, he has built a scripting engine for processing search results and mass transit route validation at startups Mobile Content Networks and Urban Mapping, respectively.