spark

Spark Summit Boston

MemSQL at Spark Summit East 2017

Last week we announced the release of the MemSQL Spark 2 Connector with support for both Apache Spark 2.0 and 2.1. At Spark Summit Boston East 2017 next week we will showcase our new connector that operationalizes powerful advanced analytics. February 7-9 John B. Hynes Convention Center 900 Boylston Street, Boston, MA 02115 https://spark-summit.org/east-2017/ MemSQL CTO and Co-founder, Nikita Shamgunov and product manager, Steven Camiña will also deliver the following talks at the conference....


lambda architecture

Rethinking Lambda Architecture for Real-Time Analytics

Big data, as a concept and practice, has been around for quite some time now. Most companies have responded to the influx of data by adapting their data management strategy. However, managing data in real time still poses a challenge for many enterprises. Some have successfully incorporated streaming or processing tools that provide instant access to real-time data, but most traditional enterprises are still exploring options. Complicating the matter further, most enterprises need access to...


Technical Deep Dive into MemSQL Streamliner

MemSQL Streamliner, an open source tool available on GitHub, is an integrated solution for building real-time data pipelines using Apache Spark. With Streamliner, you can stream data from real-time data sources (e.g. Apache Kafka), perform data transformations within Apache Spark, and ultimately load data into MemSQL for persistence and application serving. Streamliner is great tool for developers and data scientists since little to no code is required – users can instantly build their...


Spark Streamliner

Build Real-Time Data Pipelines with MemSQL Streamliner

MemSQL Streamliner is now generally available! Streamliner is an integrated MemSQL and Apache Spark solution for streaming data from real-time data sources, such as sensors, IoT devices, transactions, application data and logs. The MemSQL database pairs perfectly with Apache Spark out-of-the-box. Apache Spark is a distributed, in-memory data processing framework that provides programmatic libraries for users to work with data across a broad set of use cases, including streaming, machine...


in-memory database survey

In-Memory Database Survey Reveals Top Use Case: Real-Time Analytics

To shed light on the state of the in-memory database market, we conducted a survey on the prevalent use cases for in-memory databases. Respondents included software architects, developers, enterprise executives and data scientists1. The results revealed a high demand for real-time capabilities, such as analytics and data capture, as well as a high level of interest in Spark Streaming. Real-Time Needs for In-Memory Databases It is no surprise that our survey results highlight real-time...


Top Spark Summit Questions

Top 5 Questions Answered at Spark Summit

The MemSQL team enjoyed sponsoring and attending Spark Summit last week, where we spoke with hundreds of developers, data scientists, and architects all getting a better handle on modern data processing technologies like Spark and MemSQL. After a couple of days on the expo floor, I noticed several common questions. Below are some of the most frequent questions and answers exchanged in the MemSQL booth. 1. When should I use MemSQL? MemSQL shines in use cases requiring analytics on a changing...


Apache Spark Resources

Essential Resources for Apache Spark

There’s no doubt about it. Apache Spark is well on its way to becoming a ubiquitous technology. Over the past year, we’ve created resources to help our users understand the real-world use cases for Spark as well as showcase how our technologies compliment one another. Now, we’ve organized and consolidated those materials into this very post. Videos Pinterest Measures Real-Time User Engagement with Spark Demo of real-time data pipeline processing and analyzing re-pins...


MemCity

Modeling the City of the Future with Kafka and Spark

Today at Spark Summit in San Francisco, we are showcasing MemCity, a simulation that measures and maps the energy consumption across 1.4 million households in a futuristic, metropolitan city, approximately the size of Chicago. MemCity tracks, processes, and analyzes data from various energy devices that can be found in homes, measured by the minute in real-time. We define real-time as up to the last click, meaning all of the data being processed, up until the moment you hit enter on your query,...


Spark Summit West

Join MemSQL at Spark Summit

We’re excited to be at Spark Summit next week in our hometown of San Francisco. If you’re attending, stop by booth K6 for games and giveaways, and checkout our latest demo that showcases how organizations are using MemSQL and Spark for real-time analytics. Meet with us at Spark Summit Schedule an in-person meeting or demo at the event. Reserve a Time → MemSQL and Spark Highlights Over the past year, we’ve been working closely with our customers and the Spark community to build...