Storm Chasers: Hopper Hosts Boston Storm Users Group
On Wednesday, June 12, about thirty Boston-area developers met at Hopper’s hack/reduce space to discuss how they are using Twitter’s open-source real-time data processing framework Storm in their projects.
Phil Laflamme, Hopper’s back-end team lead, presented Hopper’s use case for Storm—after some good-natured digs at the Boston Bruins (as a Montrealer, he’s a Canadiens fan). Phil discussed the challenges the team encountered when originally constructing Hopper (Rebuilding an index of millions of crawled pages on every run? Agh!), and how Storm and Trident are well suited to solve them by allowing for incremental building, maintaining an intermediate state, and constructing priority queues. You can watch a video of Phil’s full presentation here.
Jamie Rothfeder, a developer at Crashlytics, discussed using Storm as a job scheduler and executor, and oversaw a lively debate over the pronunciation of the word “tuple,” which, despite polling the audience, remains unresolved!
Ilya Kaplun, a software engineer at Visible Measures and founder of the Boston Storm Users group, gave a talk on how and why to monitor Storm.
The Boston Storm Users group meets on the second Wednesday of every month, and pizza and soda are a guarantee. If you’re interested in giving a talk about your Boston company’s use of Storm, please contact Ilya here.
