Previous Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous NextMARCH 2014

2014 State Of Database Tech

DOWNLOAD PDF

Conventional databases from Microsoft, Oracle, and IBM still dominate the enterprise. What will it take for NoSQL, DBaaS, and distributed systems to break through? >>

Nowhere in technology is the chasm between “state of the art” and “actual use” so wide as with databases. Tech journalists and industry pundits feed that gap,

expending barrels of virtual ink on innova-tions including Hadoop, in-memory systems, and Amazon’s Redshift. At publication time, a search for “Hadoop” on Google News yielded just shy of 15,000 results versus 13,400 for “Oracle database.”

Yet despite its online popularity, Hadoop is in production or pilot by only 13% of the 956 respondents to our 2014 State of Da-tabase Technology Survey, all of them in-volved with their organizations’ database strategies. Compare that with Microsoft SQL Server (75%) or Oracle (47%). Just 5% use MongoDB, 3% have bought SAP Hana, and

1% use Vertica — to name three databases getting big play in the press. Even FileMaker beats startup darlings Cassandra, Riak, and MariaDB.

And we’re not talking about small shops here: 48% of respondents hail from organiza-tions with more than 1,000 employees (23% have more than 10,000), and 37% have at least $100 million in annual revenue.

Today’s database landscape isn’t just static. It’s positively retro. Remember 2004? Face-book had just launched, the iPad wasn’t even a twinkle in Steve Jobs’s eye, and Gartner’s da-tabase market share report put IBM (34.1%), Oracle (33.7%), and Microsoft (20%) in the top spots. In our survey, Microsoft, Oracle, and IBM still hold the top spots; we do add MySQL, but that’s about it for innovation. The top six databases in use are all relational; you

have to go down to the 10th most widely used database in our survey to find NoSQL (MongoDB, at 5%).

And those relational databases from Mi-crosoft, Oracle, and IBM? They’re essentially just updated versions of the companies’ 2004 offerings.

Oracle gets the lion’s share of respondents’ database spending, with 46% of those using Oracle devoting more than half of their da-tabase budgets to it, followed by SQL Server (34% spend more than half ), Access (25%), and DB2 (24%). Yet despite stubbornly high pricing and concerns about using older tech-nology as data variety and volume expand, IBM, Microsoft, and Oracle needn’t worry about losing customers fast. Among respon-dents, 56% of SQL Server shops, 44% of both Oracle and MySQL users, and 29% of DB2

March 2014 2informationweek.com

Previous Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous Next

By Joe Masters Emison

DOWNLOAD PDF 2014 State Of Database TechConventional databases from Microsoft, Oracle, and IBM still dominate the enterprise. What will it take for NoSQL, DBaaS, and distributed systems to break through?

Previous Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous Next

Next Wave Of Business Tech

Engage with Oracle president Mark Hurd, Box founder Aaron Levie, UPMC CIO Dan Draw-baugh, GE Power CIO Jim Fowler, former Netflix cloud architect Adrian Cockcroft, and other lead-ers of the Digital Business move-ment at the InformationWeek Conference and Elite 100 Awards Ceremony, to be held in conjunc-tion with Interop in Las Vegas, March 31 to April 1. Click here for full agenda.

March 2014 3informationweek.com

shops plan to increase their use of those databases.

Where’s the disruption?

IT StrongholdCloud computing and shadow IT have

shaken up many enterprise tech fief-doms. Not so in databases. The difference is that cloud and shadow IT are driven by employees and developers getting work done without IT’s involvement. Da-tabases, however, are different for three big reasons. First, the constituency that most cares about databases, database ad-ministrators, is usually part of IT. Second, enterprise application developers don’t much care which database sits under the abstraction layer, at least not enough to go to battle. IT is charged with making code live, so again, the database decision remains with IT.

Finally, database hardware choice is insanely important. When you’re using one server, or a small set of colocated systems, to handle all of an application’s writes, the hardware’s speed and uptime are crucial. Very few respondents’ produc-tion databases even run on virtual ma-chines — forget about hybrid or public

cloud. And thus, the status quo sticks.So do you think you can hold pat for an-

other 10 years? Maybe. But it’s no mystery why venture

capitalists, cloud providers, and startups are bullish for NoSQL, and companies like ConAgra are ponying up for expensive in-memory technology for a reason. While conventional relational databases can, theoretically, serve any data store use, they’re not always the best choice for to-day’s global, varied, and mobilized work-loads. And that’s not just because the licensing and hardware demands of con-ventional relational databases are limiting and expensive. More often, alternatives make sense because modern applications have requirements that simply didn’t exist 10 years ago.

For example, Forbes estimates that for every minute Amazon.com is down, the company loses $66,240. We live in an al-ways-on world, where nightly scheduled maintenance windows are verboten. A high volume of writes, broad geographic distribution, and frequent upgrades are facts of life. New databases such as Mongo and Riak were built from the ground up to run on a distributed archi-

[2014 STATE OF DATABASE TECH]

Previous Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous Next

Previous Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous Next

Education And Networking

Learn how cloud computing, software-defined networking, virtualization, wireless, and other key technologies work together to drive business at Interop Las Vegas. It happens March 31 to April 4.

March 2014 4informationweek.com

tecture and address all of these factors. Setting up a distributed database to pro-

vide high availability is simple in concept. In practice, however, there are a lot of choices to be made. You need significantly different setups based on factors like the amount and type of data you’re writing, how consistent you need the data to be, and the volume of writes. (We explore these and other consid-erations here.)

There’s one core architectural difference between conventional RDBMSes and those created to handle distributed architectures. The latter group doesn’t require a single master node that handles all writes. They accept writes across multiple nodes. While there are multiple-master options for RDBM-Ses — Oracle RAC, MySQL Cluster, SQL Serv-er’s Peer-to-Peer Transactional Replication — all have limitations because their core architectures were designed with a single node in mind. Oracle RAC requires machines connected on a private network and shared disk. MySQL Cluster doesn’t perform well for large transactions or joins. And in Peer-to-Peer Transactional Replication, you can get conflicts if a row is updated at the same time on different servers.

Newer databases aren’t just structured

differently. They deviate from the licens-ing models IT has used with conventional RDBMSes. And that’s a good thing given that only MySQL and IBM DB2 surpass 50% of re-spondents saying they’re somewhat or very satisfied with their vendors’ licensing deals. Surprisingly, many respondents also still pur-chase CPU/core licenses for Microsoft SQL Server, Oracle, and IBM databases. Worst case,

these shops may be forced to compromise on architectural decisions simply to afford a par-ticular database.

From a hardware perspective, while bare metal is still the most popular way to run IBM DB2 and Oracle, a surprising 14% of respon-dents host Microsoft SQL Server on private clouds, and 30% run MySQL in hybrid or pub-lic clouds. These numbers will go up as dis-

[2014 STATE OF DATABASE TECH]

Previous Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous Next

Get This And All Our Reports

Our full report on enterprise mobile app dev is free with reg-istration. This report includes 37 pages of action-oriented analysis, packed with 30 charts.

What you’ll find:

> 7 biggest native vs. browser app dev challenges

> 13 most important IDE features

> 7 business drivers for mobilityPrevious Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous Next

March 2014 $99

Report ID: R7770314

Next

reports

2014 State of Database TechnologyIT’s stuck in neutral. Exciting techs like NoSQL, DBaaS, and

distributed architectures are in use by few of our 956respondents. Just 5% have Hadoop in production, and a

mere 1% use Amazon DynamoDB for critical functions.

What will it take to jump-start innovation?

By Joe Masters Emison

repor ts. informationweek.com

Which of these databases are used for your organization’s most critical functions?

Top 10 Databases

Microsoft SQL Server

Oracle Database

MySQL (Oracle or Community)

IBM DB2

Microsoft Access

PostgreSQL

SAP Sybase ASE

Teradata

MongoDB

SAP Hana

Data: InformationWeek 2014 State of Database Technology Survey of 956 business technology professionals, January 2014

57%

38%

13%

11%

11%

3%

3%

3%

1%

1%

March 2014 5informationweek.com

tributed (read: cloudified) architectures grow in popularity. Moreover, IT can get huge di-saster recovery/business continuity and geo-graphic distribution benefits from running databases across a private or hybrid cloud.

In past years we’ve highlighted respondents’ dissatisfaction with licensing, but that angst hasn’t been enough to push IT to try a new da-tabase model. Oracle, as usual, gets the most bad vibes, with 27% saying they’re dissatisfied with licensing. It’s no surprise that none of the 27 PostgreSQL users had complaints.

What’s Up With Hadoop?In contrast to the Big Three application da-

tabases, the state of the analytical database market has changed quite a bit in the past decade. There’s one primary driver: the shift away from complex and pricey enterprise data warehouses and toward analytical data-bases, where costs have dropped significantly. Consider Hewlett-Packard’s popular Vertica analytical database. HP offers a free commu-nity version that will run up to 1 TB of data on as many as three nodes. Amazon’s Redshift

lets IT run an analytical database-as-a-service for as little as $0.25 per hour. Such advances empower customers of older-era analytical databases and enterprise data warehouses to switch and save a lot of money. And the shift has created opportunity for a new breed of analytical database vendor.

These newcomers — the likes of Actian Ma-trix (formerly ParAccel), Pivotal GreenPlum, Teradata Aster, and Vertica — use a column-oriented design. That is, instead of data stored as rows, it’s stored as columns, where each column has the same data type. This enables fast summarizing over many rows and often returns a result more quickly than a conven-tional enterprise data warehouse with OLAP cubes. Loading data into a columnar database is likely a much faster process as well. So while there are still cases for star schemas, one of the newer columnar databases is likely a bet-ter choice than a data warehouse if you need to run general analytical queries.

The other thing that’s changed dramati-cally in the analytical space is ETL, in two areas. The first is a growing trend toward ELT instead of ETL — that is, “extract, load, transform” instead of “extract, transform, load.” Essentially, instead of using code to manipulate data before loading it into a da-

[2014 STATE OF DATABASE TECH]

Previous Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous Next

How satisfied are you with the licensing agreements for these databases?

20%5% 30%10% 35%

35%4% 20%6% 35%

36%5% 25%4% 30%

19%6% 26%13% 36%

17%7% 18%20% 38%

Very dissatisfied Somewhat dissatisfied Neutral Somewhat satisfied Very satisfied

MySQL (Oracle or Community)

IBM DB2

Microsoft Access

Microsoft SQL Server

Oracle Database

Data: InformationWeek 2014 State of Database Technology Survey of 956 business technology professionals (539 respondents using Microsoft SQL Server, 359 using Oracle Database, 124 using MySQL, 108 using IBM DB2, and 102 using Microsoft Access for their organizations’ most critical functions), January 2014

Satisfaction Varies

tabase, you load semistructured (or perhaps unstructured) data into a database and then use database commands to transform the data into a query-able form. Shilpa Lawande, VP of software engineering and customer support for Vertica, says this is what Zynga does for its analytical needs. There’s a great benefit to ELT from an ease-of-operation standpoint as well; it should be easier to maintain and store a series of database com-mands than it is proprietary code that ma-nipulates arbitrary files.

The second big change related to ETL is Hadoop. While only 5% of our survey respon-dents use Hadoop in production, it’s one of the most widely discussed software packages today. Hadoop makes it (relatively) easy to write massively scalable software that pro-cesses big input files. If you need to extract parts of an exabyte of data and load that in-formation into your analytical database, it’s likely that your source data is just too big for ELT. Or perhaps you have a constant fire hose of data streaming in, and you need to pull out particular chunks for analysis. These are ideal cases for Hadoop, where you can write processing code and effortlessly scale it to as many nodes as you wish.

Hadoop is wonderfully fault-tolerant, and it

[2014 STATE OF DATABASE TECH]

March 2014 6informationweek.com

Previous Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous Next

Respondents are on top of database security — just 18% don’t encrypt, and 80% say they have not been breached.

But there’s always room to improve. Here are a few points to consider.

Point No. 1: It’s usually far easier to com-promise an application than to crack the database directly. If the “outer shell” of se-curity for your application works, and you encrypt traffic between the application and the database, you’ve got most of the secu-rity battle won.

Point No. 2: The biggest database security risks come from insiders — developers or sys-tems administrators with the wrong access levels (think Edward Snowden) or the wrong execution permissions. Here are some factors to think about as you consider whether a da-tabase has appropriate security features:

>> The three A’s: authentication (verify the connecting user’s identity), authorization (verify that the user is allowed to do the ac-tion), and accounting (log what happens).

>> Verify granular, role-based authentica-

tion within the database platform itself. Don’t let developers implement security. Note that many newer databases had “if you have cre-dentials, you have all access” when first re-leased. Ensure that any databases you’re con-sidering have moved past that stage.

>> Demand encrypted communications channels, preferably with TLS.

>> Security and governance must extend across the whole application. Don’t consider database security a separate operation. Your databases should fit within your whole-stack security plan.

>> Watch permissions around code execu-tion as well as data access. Consider built-in calls (like mass deletes), stored procedures, and user-defined functions.

>> Penetration testing is critical. In addition to running intrusion-detection/intrusion-prevention systems, hire penetration experts to attempt to compromise your application stack. Do this on a regular basis, even if it’s not required by compliance mandate. This is money well spent. — Joe Masters Emison

Database Security: Progress, ProblemsDATA PROTECTION

March 2014 7informationweek.com

isn’t just for ETL. Among respondents us-ing Hadoop, the most common applica-tion is for running analytics. You may not need an analytical database at all if you have Hadoop to handle your input data and your analytical output is fairly simple to code from the data that’s coming in.

Database-As-A-ServiceIn theory, DBaaS offers compelling ben-

efits: automated scaling, backup, and re-covery; no need for database adminis-tration skills; and better business agility

because IT doesn’t need to set up or test database servers.

But in practice, it’s more complicated, and thus unlike SaaS and IaaS, DBaaS re-mains a niche market. Most databases still run on servers completely controlled by the same group that runs application servers. To understand why, let’s take a look at the two types of DBaaS offerings available.

The first option essentially comprises managed versions of standard database servers. The most-well-known of these is

[2014 STATE OF DATABASE TECH]

Previous Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous Next Does Your Organization Run Hadoop?

22%

5%

44%

21%

8%

Data: InformationWeek 2014 State of Database Technology Survey of 956 business technology professionals, January 2014

Yes, in productionYes, in pilot

No, but we’re considering it

Don’t know

No, and we’re not considering it

March 2014 8informationweek.com

[2014 STATE OF DATABASE TECH]

Previous Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous NextAmazon’s Relational Database Service. Cus-tomers trade some control to get some ben-efits, including automated security patches and backups, and easier distribution of nodes across datacenters. Other examples of this type of service include Heroku’s PostgreSQL host-ing and Google Cloud SQL. The problem: This type of DBaaS takes only half-measures toward the theoretical benefits of DBaaS. It doesn’t eliminate the need for database administration skills in-house, because functions like creating databases and designing tables are really im-possible to outsource. And to use one of these

services, you must have your application serv-ers running on the same cloud, which is often a nonstarter.

A second category of DBaaS gets you a lot closer to the full package of theoretical ben-efits. But it’s firmly a NoSQL, key-value-pair world. The most-well-known examples here are Amazon’s DynamoDB and Firebase. These systems abstract the database server such that your data is stored across many nodes, and all backups, restores, and scaling are done without IT being involved.

The first challenge here is whether you

can wedge your application’s data into one of these databases. It’s likely you’ll need to re-architect any application built for a rela-tional database. On the other hand, if you’re building a new application, or your current database just can’t get the job done (too slow, can’t handle the traffic), these may be great choices, at least for certain types of data.

And this brings us to the biggest obstacle to DBaaS today: IT tends to set up database servers to match application servers. If the ap-plication server is in a local datacenter, so is the database server. Concerns around latency and speed, and the desire for homogeneous architectures, partly drive this tendency. But it’s also because database servers are the ultimate “set it and forget it” device. The average server at my company, BuildFax, has been running for less than a month; our average database server has been running for more than a year. We set them up, they work, we don’t mess with them.

If it ain’t broke, why fix it?

The Next 10 YearsThe biggest questions about the state

of databases today are: When, if ever, will conventional relational databases lose their stranglehold on enterprises? Will dis-

Does your organization encrypt all databases that contain sensitive information?

28%

26%

20%

8%

18%

Database Encryption

Data: InformationWeek 2014 State of Database Technology Survey of 956 business technology professionals, January 2014

Yes; we use native database encryption capabilities

Yes; we use a third-party database encryption product

Yes, via a combination of native and third-party capabilities

We do not encrypt

We encrypt only some databases containing sensitive information

tributed databases become the norm for large organizations in two years? Five? Or will relational databases on bare metal still be the standard 10 years from now?

It’s difficult to oust an incumbent data-base. The stakes are high, and you’re likely to be second-guessed. If you ask a DBA who works with only two databases what the best choice is for a web-scale applica-tion … it’s going to be one of those two databases. Don’t allow comfort zone to trump progress. If you’re looking to pilot a distributed architecture, or just go cold turkey from Oracle, you need to budget time to do your own research. Enterprise use of new databases is likely broader than our survey reflects; generally, inno-vation is happening in specific greenfield projects, so talk to peers.

However, because the Big Three ven-dors will continue to innovate, they’ll keep sizeable market share. Almost all of our respondents have at least one DBA on staff, and this translates to entrench-ment. The benefits of a distributed sys-tem also seem questionable to many CIOs, given the performance of single-master architectures. Newer high-end

servers don’t go down often, and the majority of core enterprise applications can be run off a single master, especially considering how high IT can vertically scale today’s RDBMSes. It’s likely that only a small percentage of applications will require a new database — and IT will admit that only after repeated failures of the incumbent. That means the top five databases in the 2024 State of Databases Survey will likely have at least a few in common with 2014.

Joe Masters Emison oversees the award-winning Build-Fax cloud architecture and frequently speaks on cloud topics. Write to us at iwletters@ubm.com.

Previous Next

Previous Next

DownloadDownload

RegisterRegister

SubscribeSubscribe

Previous Next

Previous Next

March 2014 9

[2014 STATE OF DATABASE TECH]

informationweek.com