10 Trends Driving a Renaissance in Database Technologies
These are the big themes fueling the growth of what is on track to become a $100 billion+ market
Today is a milestone: It’s been exactly one year since I launched the Cloud Database Report!
A big thanks to everyone who subscribes to, follows, and shares the Cloud Database Report for making our first year success. For the record, here’s my post from a year ago: “Introducing the Cloud Database Report.”
There hasn’t been a single dull moment in the past 365 days, and that won’t change. There’s just so much happening in the world of booming data.
I thought I would use this occasion to highlight a few big themes that we’ve talked about over the past 12 months, because they point forward to what’s next in modern data management.
By some estimates, the database market is on track to exceed $100 billion within a few years. Here are 10 major trends driving that growth and change.
There’s a Renaissance in database technologies enabled by the cloud. Today, there are more choices in database management systems than ever, with Carnegie Mellon University’s Database of Databases soon to surpass 800 different DBMS’s. See the Cloud Database Report 2022 for my full analysis.
Billions of dollars in funding are pouring into cloud-native startups. In 2021, the list included Databricks, Neo4j, Cockroach Labs, Yugabyte, Firebolt, Redis, SingleStore, PlanetScale, Yellowbrick, and ArangoDB. For details, see “Cockroach Labs, with New $5 Billion Valuation, Caps Big Year of Database Startup Funding.”
Traditional vendors are rapidly and urgently modernizing their database platforms and cloud services. They are adding AI, machine learning, multi-model support, and more. It’s proving difficult to displace them. See, “Oracle, Teradata, and Revenge of the Legacy Database Vendors.”
Data is growing in size and complexity. I sound like a broken record, but it bears repeating because everything else follows from the unrelenting growth of “big data,” as databases scale from terabytes to petabytes to exabytes. See, “There Are Zettabytes of Data in the World. So What?”
Database management is increasingly automated (thank God!). Managed cloud services, autonomous databases, and serverless capabilities take some of the intense pressure off developers, DBAs, and IT teams. This article explains it.
Multi-cloud is the new norm. One of the big breakthroughs in planning & deployment is that you can now run many databases (not all) in your choice of cloud environments—AWS, Google Cloud, Microsoft Azure, and others. That gives developers and IT teams unprecedented flexibility, enabling them to introduce new capabilities faster and optimized for business requirements.
Purpose-built databases drive performance for emerging applications. The plethora of databases for different data types (graph, time series, search, mobile, relational, object-relational, vector, document, etc.) means that businesses can create high-performing apps across industries and use cases. In just one example, Pinecone System’s new vector DB supports similarity search for things like fraud detection. Listen to my podcast interview with Pinecone CEO Edo Liberty.
Enterprise data warehouses give way to…Snowflake. The classic enterprise data warehouse in the data center hasn’t disappeared entirely, but they are giving way to data lakes, data clouds, and more flexible analytics tools and services. See my post, “Snowflake Hits 1 Billion Queries in 24 Hours.”
Data architectures must be designed for data distribution. Data movement can be tricky—essential for data sharing, but complex and expensive if not done right. Distributed databases and other new capabilities bring automation and intelligence to this long-standing challenge. See, “5 Ways to Accelerate Data Sharing Across Clouds.”
Database migrations are brutal but necessary. Vendors talk about fast-and-easy migrations, but slow-and-difficult is the reality for many organizations. Yet, there’s no escaping database migrations as more data is stored and managed in the cloud. So best practices are vital. See, “Cloud Vendors Confront ‘Highest Risk’ Projects: Database Migrations.”
Stay Tuned for More
That’s the big picture—or some of it. To be sure, there are other factors deserving attention, such as next-gen analytics, low-code development, AI/ML, universal databases, millisecond latency, and exabyte-scale databases.
As mentioned, there’s just so much happening in the world of booming data, so stay tuned for much more. And in case you missed it, the Cloud Database Report 2022, including our exclusive list of the Cloud Database Top 20 vendors, is available for free download.
Thanks everyone. Please share with a friend or colleague!