Building a Foundation For Big Data Storage and analyticS<FPO>

the digital data universe is doubling in size every two years, and idC is projecting the data we create and copy annually will reach 44 zettabytes by 2020. the growth is fueled by the continuing transfer of business online as well as by the surge of connected smart devices contrib-uting to the internet of things.1 the rise of structured and unstructured data presents an unprecedented opening for companies to parlay information into strategic insights. armed with better intelligence, companies can prac-tice more informed decision-making, resulting in tighter customer relationships, better conceived products, and streamlined operations.

Yet the primary characteristics of big data—commonly grouped as variety, velocity, and volume—are creating levels of complexity that threaten to overwhelm existing data warehouses and storage infrastructures, hindering organizations’ ability to capitalize on its benefits. according to gartner, 86% of companies can’t deliver the right information at the right time, and nearly three-quar-ters of current data warehouses will not scale to meet the new velocity and complexity requirements of big data.2

overwhelmed by the data deluge and faced with the shortcomings of existing data warehouse and storage infrastructures, organizations are continuing to explore new technologies that can help mitigate complexity and be more responsive to a data-driven enterprise.

“data is continuing to grow and to grow very substantially,” notes Wendy Harms, product manager for Hewlett Pack-ard’s ConvergedSystem 300 for Microsoft analytics Plat-form. “organizations want to be sure their it investments have the capability to scale to meet their needs over time.”

1 the digital universe of opportunities: rich data and the increasing Value of the internet of things, april 2014, http://idcdocserv.com/16782 gartner 2013 Bi Summit Stats, http://www.slideshare.net/cultureofperformance/gartner-idc-and-mckinsey-on-big-data

The big data opportunity has turned into a big challenge as organizations struggle to whip existing data warehouses and storage infrastructures into shape to convert mounting complexity into competitive business advantage.

2 nn Building a Foundation For Big data Storage and analYtiCS

aSSeSSing Big Data’S Pain PointS anD BuSineSS oBjectiveS

to capitalize on the changing landscape, companies are moving full speed ahead with new storage and analytics initiatives to ensure they derive optimal business value from big data. a new idg survey of it and business decision-makers found that 70% of respondents have already deployed, are in the process of deploying or implementing, or are planning to move forward on big data storage and analytics projects within the next 12 months.

there is no single motivator for these projects—instead, companies are hoping to address a variety of pain points with big data storage and analytics initiatives. Clearly, the growing number of users and data-generating apps and devices magnifies the issue, with half of respondents looking to big data storage and analytics initiatives as a means to more efficiently handle higher volumes. improved data management is also a goal. nearly half of survey respondents (47%) cited data quality improvements as a primary driver for big data storage and analytics projects while a third highlighted improved integra-tion of data and application silos as a top impetus.

toP Pain PointS Seeking to aDDreSS with Big Data Storage anD analyticS initiativeS

Handling increasing amounts of data, users, and applications in quick and efficient manner

improving data quality (accuracy, completeness, and consistency)

improving system response time

integrating and managing siloed data and applications

defining standards for information infrastructure & data management

toP BuSineSS oBjectiveS Driving Big Data Storage anD analyticS initiativeS

Creating executive dashboards for more informed, faster decision-making

Predicting and responding to customer needs in real-time

real-time tracking of financial data (cost, pricing, time to market)

Having the ability to quickly identify new business opportunities

SourCe: idg research Services, Big Data Storage and Analytics, 2015

For most survey respondents, speed of information to enable informed decision-making remains the overarching business objective. thirty-nine percent of respondents are creating executive dashboards for more informed, faster decision-making while 29% see opportunity in big data storage and analytics to better anticipate customer needs and track financial data in real-time. Slightly more than a quarter (26%) of partici-pants view big data projects as a means to quickly identify new business opportunities.

While improved decision-making is the highest ranked outcome for more than half of survey respondents regardless of size, larger companies gave greater weight to operational objectives. For example, companies with 1,000 or more employees were far more interested (than their small and mid-sized company counterparts) in reducing provisioning times and leveraging big data storage and analytics projects to more effectively scale up or down to accommodate business fluctuations.

For companies large and small, implementation remains a huge hurdle as they move forward with projects. to be most effective, big data storage and analytics architectures require a multifaceted data landscape encompassing centralized data warehouses (for 46% of respondents), in-database analytics (43%), high-performance computing (41%), and complex event processing (38%). larger companies were far more likely to add Hadoop to the mix—35% of those respondents compared to only 12% of smaller organi-zations. Hadoop—and all of the unstructured data that feeds into it—injects another layer of complexity into an already highly diverse environment.

other factors are stoking the complexity. Big data storage and analytics systems need to support multiple users simultaneously (a requirement cited by 80% of respondents) and be able to execute queries across both structured and unstructured data sets—a demand cited by nearly three quarters of respondents. at the same time, users don’t expect the complex nature of the environment to have a negative impact on performance or manageability. Seventy-one percent of respondents expect to be able to load large datasets quickly in order to keep pace with user demands, and nearly 80% of respon-dents want to be able to use standard and easily scalable tools for streamlined manageability.

due to the variety of moving parts and high range of expectations, organizations aren’t anticipating a quick deployment solution for big data storage and analytics projects. nearly 60% of respondents said they had a three- to 12-month window earmarked for implementation, with the average reasonable

50%

47%

34%

39%

29%

29%

26%

33%

31%

3 nn Building a Foundation For Big data Storage and analYtiCS

time frame estimated at seven months. limited it budgets, lack of on-site technical expertise in the new technology areas, and concerns about how to integrate the new environment with current data and storage infrastructures were considered potential obstacles to implementation.

convergeD SyStemS Drive Big Data efficiency—anD innovation

as companies work through these challenges, converged systems—which marry physical server hardware, software, networking, storage, and unified management and support into a single, out-of-box appliance—are emerging as a viable solution to meet the new big data requirements while reducing much of the complexity.

nearly half of respondents to the idg survey (47%) are either currently using or considering using converged systems as part of their it infrastructure over the next year. While just 10% are deploying converged systems specifically as part of big data projects, another two-thirds say they are at least somewhat likely to consider the technology when designing a big data storage and analytics infrastructure.

Converged systems, including those purpose-built for big data storage and analytics, are resonating with organizations for a variety of reasons. More than half of companies surveyed saw converged systems as a way to improve it staff efficiency and utilization of it resources—a finding that seems to correlate with the desire to get a handle on big data complexity and address lingering gaps in it domain expertise. Compa-nies with a longer term view of big data storage and analytics (those looking to do projects over the course of the next year or two) are drawn to the potential of converged systems for lowering total cost of owner-ship (tCo) and for improving business agility.

to address the requirements of big data storage and analytics, Microsoft and Hewlett-Packard have partnered on a converged system built specifically to handle data-intensive workloads.

Big Data Storage anD analyticS architectureS requiring SuPPort

Centralized data warehouses

in-database analytics

High-performance computing

Complex event processing

the HP ConvergedSystem 300 for Microsoft analytics Platform (also branded as Microsoft analytics Platform System) is a purpose-driven factory-built appliance based on the HP Converged infrastructure of servers, storage, and networking that seamlessly integrates traditional data warehousing with the power of massively parallel processing (MPP) and the improved performance of in-memory analytics. along with the unique HP-Support Pack utility to further reduce complexity, optimize performance and simplify support and upgrades, the platform comes preloaded with Microsoft Parallel data Warehouse (PdW) software. this includes Polybase, which supports simultaneous queries of both structured and unstructured data in Hadoop using the familiar t-SQl language. Customers also have the option to preload Hdinsight (Microsoft’s 100% apache Hadoop distribution) to the appliance, for additional big data capabilities.

integrating advanced data integration and robust Bi capabilities into a single appliance that is tested and tuned for data warehouse and analytics applications mitigates much of the complexity surrounding big data. For one thing, companies shrink their hardware footprint, reducing costs and minimizing administra-tion tasks associated with managing the platform. the ConvergedSystem 300 for Microsoft analytics Platform also eliminates the need for costly integration efforts, as it provides a single view of information across the enterprise and serves as a platform for building end-to-end data warehouse and analytics solutions that can be easily scaled to changing needs.

“it can be very time-consuming to configure a complex data warehouse system, properly balancing i/o, optimizing computing resources, and making sure you have the right components for networking and switching,” explains dan Kogan, senior product marketing manager for Microsoft’s analytics Platform System. “You can follow the recipe, but it is tough to pull off. the Converged-System 300 for Microsoft analytics Platform takes the guesswork out of the equation.”

While there’s no doubt that big data is opening the door to plenty of new opportunities, those oppor-tunities can only be captured by gaining access to the right information at the right time. to accom-modate that vision, organizations are tasked with re-architecting existing data warehouse and storage infrastructures into a next-generation platform that supports the needs of business. Converged systems purpose-built for data warehouse and big data analytics applications are proving to be a key enabler for helping organizations cut through complexity and get straight to the heart of data-driven innovation.

For more information, www.microsoft.com/en-us/server-cloud/products/analytics-platform-system/

SourCe: idg research Services, Big Data Storage and Analytics, 2015

46%

43%

41%

38%