ARTICLE

Saama White Paper January 2010 www.Saama.com 

Choosing a BusinessIntelligence Appliance

As published in: 

The term business intelligence appliance is defined differently by manyfolks, however, a broadly accepted definition is that it is server hardwareand database software bundled to specifically meet data warehousingneeds.

Just like the name suggests, BI appliances are turning out to have somesimilarities with other familiar appliances, i.e., their kitchen counterparts.They come in many variations, so it takes some amount of groundworkand research to make sure that you choose the right appliance thatmeets your needs.

I was involved in two such bake-offs over the last year, and through theexercise, we worked out a pretty good process that helped us assessour options and make our choice. I will note some of the key highlightsin our approach.

First, the client was a large financial institution, and this initiative was ata departmental level. Our key objective was to consolidate four datamarts into a single infrastructure because we were getting manyrequests to combine information across these and eliminate datalatency between them. We were primarily facing challenges on the dataload side, which in some cases was greater than 30 hours for a run.Performance pain on the query side but wasn’t very significant. Also asa secondary benefit, we were attempting to consolidate databaselicenses and servers across these four environments, includingdevelopment and test boxes for each environment. Like many of thelarge enterprises, we had the server support outsourced to one of thelarge infrastructure support players and were being charged a hefty sumper server on a monthly basis.

Because the combined total data size was expected to be around 2TB,we were reluctant to even begin the process as we had heard about the5TB plus starting point for the appliance solutions to prove valuable.

“BI appliances

come in many

variations, so it 

takes some

amount of 

groundwork and 

research to make

sure that you

choose the right 

appliance that 

meets your 

needs.”

 

 

Page 2

Saama Article April 2010 www.Saama.com 

However, we really needed something to help the load situation anddecided to move ahead with this initiative.

We started with the usual “product evaluation” approach and broke itout into four key steps:

1. Long list – based on Internet research.2. Short list – based on discussions with analyst firms and

minimal interactions with the vendors.3. Proof of concept bake-off for the short list contenders.4. Final assessment and decision.

Step 1: Long ListFor the long list of candidates we got most of the players fromGartner’s Magic Quadrant. We found that we could broadlyclassify these into a few categories:

1. Hardware and software solution,2. Software solution or3. Hardware solution.

Step 2: Short ListWe trimmed down the initial list based on our size and performanceneeds. During this process we used input from Gartner and Forresterand had minimal interaction with the actual vendor sales reps. We didconsider customer references as part of the decision process.

So, that led us to two players, as our main contenders for our POCbake-off. Interestingly, it turned out to be a mix of a “proprietaryhardware plus software” player and a “commodity hardware plussoftware” player. Although one vendor appeared to be a startup player,they were given high marks by the analysts and, more importantly theyseemed to be working very closely with Sun and even had commonboard members, making their viability question a little less risky.

Step 3: POC Bake-OffWe put together a set clear and transparent of guidelines for thisprocess to ensure that we had an even playing field. Some of the thingswe laid out were:

•  We insisted that the POC be done on site.•  We would have one of our team members shadow the vendor

engineer during the entire process to understand and reportback on what it took.

•  We also time-bound this to be a one week on-site activity.

We knew that the on-site requirement would mean getting a bunch ofapprovals internally to allow for the vendor hardware to be set up in ourdata center, so we initiated that process during step one. By the time we

“We put 

together a set 

clear and 

transparent of 

guidelines for 

this process to

ensure that we

had an even

 playing field.”

 

 

Page 3

Saama Article April 2010 www.Saama.com 

had the final contenders, we had things in place from a legal/securityperspective to not hold.

For the POC task, we identified one load process that was takingapproximately 33 hours as a prime candidate. This process consisted ofa set of Informatica jobs and involved picking up data from flat files andmoving it to stage, to final schema and finally to a set of aggregatetables. Since we had existing investments in reporting and analyticalapplications, we had to ensure that the existing schema remainuntouched so as to avoid changes to these BI applications.

The load process consisted of all the three types of operations, i.e.,inserts, deletes and updates. We were going to measure the loadperformance of this entire task, and then we would have one of our BIenvironments point to the vendor appliance and benchmark runningsome of the long-running reports and queries.

Both the vendors shipped over their boxes to our data center, and forlogistic reasons, it turned out that we had the vendors come in and workon their tasks on staggered weeks. So we had vendor one on week 1and vendor 2 on week three. We would have preferred this to be thesame week, but that would have required some additional setup on ourside.

In both runs, we ran into some technical snags in moving the raw dataover to the appliance, but we used one of the big USB devices to moveit over. The runs were fairly smooth and the engineers were very good.They knew what they were doing and were able to carry out the taskswith minimal issues. The performance runs of the whole process in bothcases yielded mind-blowing results. The 33-hour process took less than40 minutes in both the cases. We also tested both using mixed loads,i.e., loading data and running reports simultaneously, and we did notsee much degradation. These systems have been architected to allowfor loads without impacting the end usage. Of course, this can also leadto some read consistency issues if the overall process is not designedproperly, but that’s a separate discussion.

Both of these appliances had proven themselves with goodperformance, with very little difference (less than four minutes) betweenthem. So, the decision process now switched from performance toprice/performance, price here being total cost of ownership over athree-year period, including accounting for the projected data growth.

This particular exercise we did in late 2008, and the economy was suchthat it was already a buyers market. Both vendors were willing to bendover backward to close the deal. So, even the price/performance wasbecoming a difficult metric to base the decision on.

“The performance

runs of the whole

 process in both

cases yielded 

mind-blowing

results. The 33-

hour process took 

less than 40

minutes in both

the cases.” 

 

 

Page 4

Saama Article April 2010 www.Saama.com 

We finally made our decision to go with a commodity hardware plussoftware solution rather than the proprietary hardware plus softwaresolution, our main reasoning behind being that the commodity hardwarebased system would stand to gain from the billions of dollars of R&Dbeing carried out by the hardware giants like Intel and AMD. This meantthat we would have to go with a relatively unknown player, but thestrong backing mitigated the risk.

The BI appliance industry is evolving rapidly, and new players are reallypushing hard to make their solutions affordable to even the sub-billion-dollar enterprises. I recommend that even enterprises having to dealwith just a TB of data (with expected data volume growth) shouldexplore the possibility of introducing a BI appliance into their ecosystem.I can almost assure you that you will be surprised by the performancegains from these systems on both the data load side as well as thequery side. Finally, I believe this market is going to evolve rapidly andthe commodity hardware plus software players will begin to dominatethe marketplace.

About the AuthorHaranath Gnana is senior principal at Saama Technologies Inc., a pure-play business intelligence solution provider. He has more than 15 yearsexperience in the area of information technology, specializing businessintelligence.

About Saama TechnologiesSaama Technologies, Inc. is a pure-play business intelligence solutionprovider that has revolutionized the way organizations make decisionsthrough business intelligence. Since 1997, the company has combinedits extreme BI technology expertise, unique intellectual property portfolioand strong relationships with the industry’s leading technology providersto deliver pure business intelligence to the world’s largest information-focused organizations. Saama’s customers are Fortune 500organizations within a wide range of industries, including life sciences,technology, financial services, and the public sector. Saama recentlyacquired data-integration software and NIEM-conformance pioneerSypherlink, which operates as an independent, wholly-ownedsubsidiary. For more information, visit www.saama.com.

“I can almost 

assure you that 

 you will be

surprised by the

 performance gains

 from these systems

on both the data

load side as well

as the query side.”