Spark

Top Spark Summit Questions

Top 5 Questions Answered at Spark Summit

The MemSQL team enjoyed sponsoring and attending Spark Summit last week, where we spoke with hundreds of developers, data scientists, and architects all getting a better handle on modern data processing technologies like Spark and MemSQL. After a couple of days on the expo floor, I noticed several common questions. Below are some of the most frequent questions and answers exchanged in the MemSQL booth. 1. When should I use MemSQL? MemSQL shines in use cases requiring analytics on a changing...


Apache Spark Resources

Essential Resources for Apache Spark

There’s no doubt about it. Apache Spark is well on its way to becoming a ubiquitous technology. Over the past year, we’ve created resources to help our users understand the real-world use cases for Spark as well as showcase how our technologies compliment one another. Now, we’ve organized and consolidated those materials into this very post. Videos Pinterest Measures Real-Time User Engagement with Spark Demo of real-time data pipeline processing and analyzing re-pins...


MemCity

Modeling the City of the Future with Kafka and Spark

Today at Spark Summit in San Francisco, we are showcasing MemCity, a simulation that measures and maps the energy consumption across 1.4 million households in a futuristic, metropolitan city, approximately the size of Chicago. MemCity tracks, processes, and analyzes data from various energy devices that can be found in homes, measured by the minute in real-time. We define real-time as up to the last click, meaning all of the data being processed, up until the moment you hit enter on your query,...


Spark Summit West

Join MemSQL at Spark Summit

We’re excited to be at Spark Summit next week in our hometown of San Francisco. If you’re attending, stop by booth K6 for games and giveaways, and checkout our latest demo that showcases how organizations are using MemSQL and Spark for real-time analytics. Meet with us at Spark Summit Schedule an in-person meeting or demo at the event. Reserve a Time → MemSQL and Spark Highlights Over the past year, we’ve been working closely with our customers and the Spark community to build...


Enterprise Apache Spark

Harnessing the Enterprise Capabilities of Spark

As more developers and data scientists try Apache Spark, they ask questions about persistence, transactions and mutable data, and how to deploy statistical models in production. To address some of these questions, our CEO Eric Frenkiel recently wrote an article for Data Informed explaining key use cases integrating MemSQL and Spark together to drive concrete business value. The article explains how you can combine MemSQL and Spark for applications like stream processing, advanced analytics, and...


Geospatial Intelligence

MemSQL at Spark Summit East

We are happy to be in New York City this week for Spark Summit East. We will be sharing more about our new geospatial capabilities, as well as the work with Esri to showcase the power of MemSQL geospatial features in conjunction with Apache Spark. Last week we shared the preliminary release of MemSQL geospatial features introduced at the Esri Developer Summit in Palm Springs. You can read more about the live demonstration showcased at the summit here. The demonstration uses the “Taxistats”...


In-Memory and Apache Spark

Video: The State of In-Memory and Apache Spark

Strata+Hadoop World was full of activity for MemSQL. Our keynote explained why real-time is the next phase for big data. We showcased a live application with Pinterest where they combine Spark and MemSQL to ingest and analyze real-time data. And we gave away dozens of prizes to Strata+Hadoop attendees who proved their latency crushing skills in our Query Kong game. During the event, Mike Hendrickson of O’Reilly Media sat down with MemSQL CEO Eric Frenkiel to discuss: The state of in-memory...


Pinterest Apache Spark Demo

How Pinterest Measures Real-Time User Engagement with Spark

Setting the Stage for Spark With Spark on track to replace MapReduce, enterprises are flocking to the open source framework in effort to take advantage of its superior distributed data processing power. IT leads that manage infrastructure and data pipelines of high-traffic websites are running Spark–in particular, Spark Streaming which is ideal for structuring real-time data on the fly–to reliably capture and process event data, and write it in a format that can immediately be queried by...


Operationalize Spark

Operationalizing Spark with MemSQL

In Short: Combining the data processing prowess of Spark with a real-time database for transactions and analytics, where both are memory-optimized and distributed, leads to powerful new business use cases. MemSQL Spark Connector links at end of this post. Data Appetite and Evolution Our generation of, and appetite for, data continues unabated. This drives a critical need for tools to quickly process and transform data. Apache Spark, the new memory-optimized data processing framework, fills this...