campaign storage - spectra logic...campaign storage delivers the next evolution in large-scale file...

Campaign Storage delivers the next evolution in large-scale file systems that offers a novel approach to data management at scale. Campaign Storage’s Lustre Archive architecture integrates industry-leading technology to provide a seamless archive solution for large parallel file systems in the HPC community.

Fully scalable in capacity, object count, and throughput, Campaign Storage unlocks an affordable archive for large parallel file systems with features specifically designed to accommodate the most intense workflows. To create a modern solution for HPC users, an open steering council comprised of some of the top HPC sites worldwide, has contributed to Campaign Storage’s development priorities and strategies.

Direct archive access is provided, including enterprise disk staging, archive disk, enterprise and/or LTO tape, offsite ejected copies, remote site replication, as well as hybrid access to a variety of online cloud services for data replication.

While industry-standard tools are used, including integrated Lustre copy tools and optionally CEA’s Robinhood Policy Engine, the HSM system directly addresses well-known deficiencies in current archive systems. A subtree-indexed ZFS

Product Highlights and Features:

• Affordability

• Ease of management

• Sharding, striping, and packing of large and small files, speed scaling

• Multiple sources and multiple targets

• Efficient management of requests

• Instant renaming – renames become visible in the archive

• Nearly instantaneous subtree archive searches

• Well-designed verification subsystem

• Full control of disk vs. tape archive targets

• Ability to pre-stage to nearline disk layer

• Reliable data protection

Campaign Storage HSM for Lustre Archive

+1 800-833-1132 • +1 303-449-6400 • (Fax) +1 303-939-8844 • 6285 Lookout Road • Boulder, CO 80301 USA

Nearly unlimited capacity, scalability, & parallel performanceCampaign Storage offers scalability in capacity and performance for fast movement of data between the Campaign Storage system and premium high-performance storage tiers by simply scaling data movers and storage targets to accommodate the system needs. Advanced handling of small and large files through packing and sharding further accelerates system performance by optimizing parallelism of the backend.

Straightforward architecture & modular implementationCampaign Storage and Spectra provide onsite installation, including upgraded Lustre copy tools, the Campaign Storage HSM system with appropriate scale and appliance servers, modular BlackPearl systems and selected storage tiers, including a combination of disk, tape, and cloud gateway as well as multi-site replication systems.

Nearly instantaneous archive search Campaign Storage utilizes advanced subtree indexing on a ZFS metadata file system to provide nearly instantaneous search of all archive tiers.

Industry-standard componentsCampaign Storage uses native Lustre archive processes, data movers and the option of the Robinhood file scanner and policy engine for archive handling, leveraging existing workflows while enhancing both the

Key Advantages and Benefits:

file system is used as a metadata store, providing nearly instantaneous search of the archive. Advanced data management for both small and incredibly large files is also provided. By abstracting the metadata from the archive

itself, complete rename/move flexibility is also provided.

The Campaign Storage solution includes: updated copy tools for Lustre, automated changelog-driven indexes, metadata

store, Spectra® BlackPearl® Converged Storage System, and any combination of disk, tape, or cloud storage necessary for specific accessibility and long-term storage needs.


archive metadata store and the storage system itself. Access and control is via existing command line and scripting as well as advanced tools for archive search, verification, and monitoring.

Flexibility across multiple sources, targets, and file systemsCampaign Storage integrates into a number of different parallel file systems, beginning with Lustre. Campaign Storage supports multi-site replication,

multiple distributed data copies, and offsite data replication to other locations or cloud storage pools, all based on configurable policies.

Improved storage utilizationCampaign Storage helps organizations free up valuable space on a Lustre environment by offloading colder data to a significantly lower-cost tier of storage, while providing highly redundant data protection. By making the archive more accessible, larger data sets can be stored on archive and staged in the parallel file system only when needed.

Today the primary concern for storage software is to manage tiers. Campaign Storage does precisely that.

Peter BraamCEO of Campaign Storage LLC

Copyright ©2017 Spectra Logic Corporation. All rights reserved worldwide. Spectra and Spectra Logic are registered trademarks of Spectra Logic. All other trademarks and registered trademarks are the property of their respective owners. All library features and specifications listed are subject to change at any time without notice.

V1-31717


Spectra provides onsite installation including copy tools, appliance setup of the HSM system itself from 3 servers, BlackPearl systems, and appropriate storage.

The customer sets up automated archive jobs to be moved to Campaign Storage, including verification and release if selected.

Individual users or administrators can also initiate project-based jobs in support of specific compute jobs with archiving, post processing, or restaging of data prior to a computer job. Using data movers, data can also be pre-staged to warmer archive tiers to prepare for fast transfer if time constraints are an issue in a busy environment.

For large data sets comprised of large numbers of small files,

packing options combine files into byte-indexed blobs to maximize throughput on the backend and can be recovered as an entire job or individual jobs through byte-offset recovery. Similarly, extremely large files can be sharded for parallel archive and recovery.

When data within an archive needs to be re-categorized or moved, actions are performed exclusively on the metadata and, therefore, no extraneous data movement is required.

Files already archived can be searched simply and quickly on the ZFS metadata store for verification or restore jobs.

For any issues that arise, Spectra provides Tier 1 & Tier 2 24/7 phone support with a simple escalation path directly to Campaign Storage developers as needed.

The result is typically less loading on the parallel file system, leaving more available space to take advantage of its premium features, as well as easier access to the archive, less management time and a more streamlined system.

Who: A customer with a large compute cluster and a supporting Lustre parallel file system is low on space, their primary file system is aging, or they are running into organizational difficulties in searching, categorizing, or managing large sets of data.

What: The customer understands that the fastest and most economical way to upgrade their system and provide data protection is with a tape or disk archive solution.

How: Options include using the existing Lustre HSM binding, that includes a Robinhood file scanner, to initiate existing copy tools to move data on a policy or manual basis to another lower cost storage tier. To move to tape, the most affordable option available, an intermediary HSM system is needed to run the tape library.

Traditional Customer Example:

Pros:• Proven approach with

known tools

• Semi-automated access to extremely low cost storage tiers

Cons:• Slow and potentially difficult

search of the archive

• Limited and difficult scale-out for throughput and capacity

• Inflexible archive structure

• Limited monitoring, job restart and error handling

• High administrator time utilization

Customer Adds Campaign Storage HSM System:


campaign storage - spectra logic...campaign storage delivers the next evolution in large-scale file...

Documents