Engineering

BPU Linux Performance

What is BPF and why is it taking over Linux Performance Analysis?

Performance analysis often gets bottlenecked by lack of visibility. At MemSQL, we architected our database to easily observe its inner workings. Observability allows our engineers to easily identify components that need to be faster. Faster components mean our database’s performance skyrockets. These tools also enable support engineers to react quickly and precisely to customer needs. In the spirit of using the best available tools to which we have access, the performance team is currently...


MemSQL Performance Benchmark

New Performance Benchmark for Live Dashboards and Fast Updates

Newest Upsert Benchmark showcases critical use case for internet billing with telcos, ISPs, and CDNs MemSQL achieves 7.9 million upserts per second, 6x faster than Cassandra Benchmark details and scripts now available on GitHub The business need for fast updates and live dashboards Businesses want insights from their data and they want it sooner rather than later. For fast-changing data, companies must rapidly glean insights in order to make the right decisions. Industry applications like IoT...


data ingest and concurrent analytics

Massive Data Ingest and Concurrent Analytics with MemSQL

The amount of data created in the past two years surpasses all of the data previously produced in human history. Even more shocking is that for all of that data produced, only 0.5% is being analyzed and used. In order to capitalize on data that exists today, businesses need the right tools to ingest and analyze data. At MemSQL, our mission is to do exactly that. We help enterprises operate in today’s real-time world by unlocking value from data instantaneously. The first step in achieving this...


dbbench database workloads

New Release of dbbench Streamlines Database Workload Testing

Our performance engineering team is committed to delivering high quality tools. Since we released dbbench 7 months ago, it has been widely adopted across our engineering and sales teams as the definitive tool for testing database workloads. Today we are announcing availability of a new version of dbbench, as well as a package of high level tools to enhance it. In this latest release, we enhanced both the flexibility and ease of use of the tool. We augmented capabilities of dbbench and added a...


Third Normal Form, Star Schema, and a
Performance Centric Data Strategy

Keeping it Straight Data value comes from sharing, so staying organized and providing common data access methods across different groups can bring big payoffs. Companies struggle daily to keep data formats consistent across applications, departments, people, divisions, and new software systems installed every year. Passing data between systems and applications is called ETL, which stands for Extract, Transform, and Load. It is the process everyone loves to hate. There is no glamour in...


RBAC security

MemSQL 5.1 Enhances Security for Real-Time Enterprises

Enterprises seek real-time data and analytics solutions to stay current in competitive, fast-evolving markets. Companies dealing in private information, such as healthcare organizations, financial institutions, and the public sector have historically been limited in their pursuit of real-time results, given stringent security requirements. Today, we announce the availability of MemSQL 5.1. This release adds Role-Based Access Control (RBAC) to the already powerful MemSQL 5, unlocking the gateway...


real-time monitoring

Monitoring A/B Experiments In Real Time

This post originally appeared on the Pinterest Engineering Blog by Bryant Xiao. As a data driven company, we rely heavily on A/B experiments to make decisions on new products and features. How efficiently we run these experiments strongly affects how fast we can iterate. By providing experimenters with real-time metrics, we increase our chance to successfully run experiments and move faster. We have daily workflows to compute hundreds of metrics for each experiment. While these daily metrics...


Should You Use a Rowstore or a Columnstore?

This is a repost of an article by Ankur Goyal, VP of Engineering, published on Medium ⇒ The terms rowstore and columnstore have become household names for database users. The general consensus is that rowstores are superior for online transaction processing (OLTP) workloads and columnstores are superior for online analytical processing (OLAP) workloads. This is close but not quite right — we’ll dig into why in this article and provide a more fundamental way to reason about when...


dbBench

dbbench: Bringing Active Benchmarking to Databases

In my last blog post, I investigated a Linux performance issue affecting a specific customer workload. In this post, I will introduce the tool I created to drive that investigation. Recently, a customer was running a test where data was loaded into MemSQL via LOAD DATA. The customer’s third-party benchmarking tool found that MemSQL took twice as long to load the same amount of data as a competing database; however, the numbers reported by this tool did not make sense. Local tests had shown...


Investigating Linux Performance

Investigating Linux Performance with Off-CPU Flame Graphs

The Setup As a performance engineer at MemSQL, one of my primary responsibilities is to ensure that customer Proof of Concepts (POCs) run smoothly. I was recently asked to assist with a big POC, where I was surprised to encounter an uncommon Linux performance issue. I was running a synthetic workload of 16 threads (one for each CPU core). Each one simultaneously executed a very simple query (select count(*) from t where i > 5) against a columnstore table. In theory, this ought to be a CPU...