lattus rest api user’s guide - quantum...lattus rest api user’s guide xi preface document...
TRANSCRIPT
User’s Guide
Lattus REST API
6-67815-09 Rev B
ii Lattus REST API User’s Guide
Lattus REST API User’s Guide, 6-67815-09 Rev B, April 2015, Product of USA.
Quantum Corporation provides this publication “as is” without warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability or fitness for a particular purpose. Quantum Corporation may revise this publication from time to time without notice.
COPYRIGHT STATEMENT
© 2015 Quantum Corporation. All rights reserved.
Your right to copy this manual is limited by copyright law. Making copies or adaptations without prior written authorization of Quantum Corporation is prohibited by law and constitutes a punishable violation of the law.
TRADEMARK STATEMENT
Quantum, the Quantum Logo, Backup. Recovery. Archive. It's What We Do., Be Certain, Be Quantum Certain, DLT, the DLT Logo, DLTSage, DLTtape, the DLTtape Logo, DXi, DXi Accent, Dynamic Powerdown, FastSense, FlexLink, GoProtect, GoVault, iLayer, Lattus, MediaShield, Optyon, Pocket-sized., Well-armored., Preserving the World's Most Important Data. Yours., Q-Cloud, Quantum Certain, Quantum Certainty, Quantum vmPRO, Scalar, SDLT, SiteCare, SmartVerify, StorageCare, StorNext, Super DLTtape, SuperLoader, and Vision are either registered trademarks or trademarks of Quantum Corporation and its affiliates in the United States and/or other countries. All other trademarks are the property of their respective owners.
Products mentioned herein are for identification purposes only and may be registered trademarks or trademarks of their respective companies. All other brand names or trademarks are the property of their respective owners.
Quantum specifications are subject to change.
Contents
Preface xi
Chapter 1 Introduction 1
Chapter 2 S3 Integration 3
Starting with Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Query String Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Compatibility Calls. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5GET /?location. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5GET /?logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6GET /?versioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7GET /?acl . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Compatibility Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Service-API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Objects-API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Buckets-API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Differences Between Lattus S3 and Amazon S3 . . . . . . . . . . . 10Response Differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Other Behavioral Differences. . . . . . . . . . . . . . . . . . . . . . . . . . 14
Lattus REST API User’s Guide iii
Contents
Request Headers for GET and HEAD Object . . . . . . . . . . . . . . . . . . . . . 15Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Example Combined Header Request . . . . . . . . . . . . . . . . . . . . 20
Important Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Virtual Host Style . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Supported Bucketnames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Interpretation of Query Parameters . . . . . . . . . . . . . . . . . . . . 23Custom Headers You Want to Upload With the Object. . . . . . 24Headers You Would Like the Server to Ignore . . . . . . . . . . . . . 24x-amz-meta-* Headers Can be Uploaded with the Object. . . . 24Query String Authentication is Possible . . . . . . . . . . . . . . . . . 24Bucket Calls Can be Disabled . . . . . . . . . . . . . . . . . . . . . . . . . 25Anonymous GET Feature. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Permissions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Object Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26Bucket Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28Service Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Error Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32S3 Error Return Code Examples: . . . . . . . . . . . . . . . . . . . . . . . 35
Supported Bucket Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36PUT Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36GET Request (listing of objects). . . . . . . . . . . . . . . . . . . . . . . . 36HEAD Request. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Supported Multipart Bucket Operations . . . . . . . . . . . . . . . . . . . . . . . 40List Multipart Uploads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Supported Object Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42PUT Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42GET Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43HEAD Request. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44DELETE Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44PUT - Copy Request. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Supported Multipart Object Operations . . . . . . . . . . . . . . . . . . . . . . . 46Initiate Multipart Upload . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Upload Multipart Part . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48Complete Multipart Upload . . . . . . . . . . . . . . . . . . . . . . . . . . 49Abort Multipart Upload . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51List Parts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Multipart Upload (Copy) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
iv Lattus REST API User’s Guide
Contents
Supported Service Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54GET Request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Chapter 3 REST Definitions and Configuration 57REST Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
How To Enable REST in Lattus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Using Curl to Demonstrate the REST Functionality . . . . . . . . . . . . . . . 58
Other Interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
API Interface Documentation Assumptions . . . . . . . . . . . . . . . . . . . . . 59
Resource Identifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Transfer Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Behavior Common to All Requests. . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
What Has Changed Compared to the 3.2.X REST API . . . . . . . . . . . . . 61
Chapter 4 Design Considerations 63
Designing for Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63Connection Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63Parallelism. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Partial Reads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65Overwrites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Caching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Designing for Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Namespace Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66Connection Locality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
Chapter 5 Policy Operations 69
Create Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69http / text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 xml. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Get Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74http / text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
Lattus REST API User’s Guide v
Contents
xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
List Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77http / text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Delete Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Chapter 6 Namespaces 81
List Namespaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81http/text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
List Objects in a Namespace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83http/text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Create Namespace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86http/text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Update Namespace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91http/text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
Get Namespace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95http/text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
List Namespaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98http/text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Delete Namespace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
GET/HEAD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
List Namespace Capacity Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . 101json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
vi Lattus REST API User’s Guide
Contents
List Syncstores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Get Syncstore Capacity Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Chapter 7 Object Operations 113
Create or Overwrite Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
Transfer with Content-Length . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114http. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Transfer Without Content-Length . . . . . . . . . . . . . . . . . . . . . . . . . . . 117Request Headers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117If-Match . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118If-Modified-Since . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 If-None-Match . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119If-Unmodified-Since . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Transfer-Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Response Headers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Create Object with Generated Name. . . . . . . . . . . . . . . . . . . . . . . . . 121
Using Curl to Upload a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
Create an Object with Generated Name . . . . . . . . . . . . . . . . . . . . . . 123
Create Directory Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123http. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125Curl Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
Custom Metadata. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126http. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129Apply Updates to Existing Custom Metadata . . . . . . . . . . . . 130http. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
Lattus REST API User’s Guide vii
Contents
List Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133http / text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136Using Curl to List the Objects in a Namespace . . . . . . . . . . . 136
Move Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137Extra Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
Get Object Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138Complete Content. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138Request Headers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138If-Match . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138If-Modified-Since . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 If-None-Match . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139If-Unmodified-Since . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140Range . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140Response Headers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141Using Curl to Get an Object . . . . . . . . . . . . . . . . . . . . . . . . . 142
Get Object Metadata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142http / text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Delete Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145Using Curl to Delete an Object . . . . . . . . . . . . . . . . . . . . . . . 145
Adding A user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146http / text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
List Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150http / text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
GET User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152http / text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152json. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153xml . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
Delete A User. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
viii Lattus REST API User’s Guide
Contents
Chapter 8 Authentication 157
Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
How It Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
REST Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
API Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159Users. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159Paths and Flags. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159Q-Shell Interface for User Management . . . . . . . . . . . . . . . . 160Q-shell Interface for Assignment of Rights to a Specific Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
REST Authentication in Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166WWW-Authenticate header . . . . . . . . . . . . . . . . . . . . . . . . . 166Authorization header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166Curl Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Getting Started with Authentication . . . . . . . . . . . . . . . . . . . . . . . . . 168Out of the Box Authentication . . . . . . . . . . . . . . . . . . . . . . . 168Before you Start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168Allowing an Admin User to Create Namespaces and Polcies. 168Allowing Another User to Only Read Data from a Specific Namespace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169Inheriting Permissions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169Tips and Tricks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Chapter 9 Return Codes and Troubleshooting 173
Return Codes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
Chapter 10 Microsoft .net Bindings 177
AmplistorClientLib Class Documentation. . . . . . . . . . . . . . . . . . . . . . 177AmplistorClientLib.Types.AmpliObject Class Reference - Public Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177AmplistorClientLib.Types.AmpliPolicy Class Reference - Public Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
Lattus REST API User’s Guide ix
Contents
AmplistorClientLib Member Function Documentation. . . . . . . . . . . . 178Policy Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178Namespace Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179Object Operations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181Various Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185Getting the .net Bindings . . . . . . . . . . . . . . . . . . . . . . . . . . . 185Create a Policy and Namespace . . . . . . . . . . . . . . . . . . . . . . 185Create a Lattus Connection Object . . . . . . . . . . . . . . . . . . . . 185Upload an Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
Chapter 11 Working With REST 187Starting the q-shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187Setting up a Test Environment . . . . . . . . . . . . . . . . . . . . . . . 187Starting with Development . . . . . . . . . . . . . . . . . . . . . . . . . . 187Resetting the Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 188
Chapter 12 Working with Lattus S3 189
Starting the q-shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
Setting up an S3 Test Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 189
Enabling S3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190Adding a User for S3 Authentication . . . . . . . . . . . . . . . . . . 190Configure the Client Daemon to Listen on its S3 Interface . . 190Updating the HTTPS Proxy . . . . . . . . . . . . . . . . . . . . . . . . . . 190Defining the Policy for S3 Bucket Creation . . . . . . . . . . . . . . 191
Testing Your Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192Create a New Bucket. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193List Buckets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193Put a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193Get a File. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193List Files in a Bucket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193List All Objects in All Buckets . . . . . . . . . . . . . . . . . . . . . . . . 194Delete File From a Bucket . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Rebooting an S3 Test environment . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Tearing Down an S3 Test Environment . . . . . . . . . . . . . . . . . . . . . . . 195
x Lattus REST API User’s Guide
Contents
Getting Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195Output of Following qshell Command . . . . . . . . . . . . . . . . . 195Clientdaemon Log File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
Lattus REST API User’s Guide xi
Contents
xii Lattus REST API User’s Guide
Preface
Document Organization Following is a list of chapters in this guide:
• Chapter 1, Introduction
• Chapter 2, S3 Integration
• Chapter 3, REST Definitions and Configuration
• Chapter 4, Design Considerations
• Chapter 5, Policy Operations
• Chapter 6, Namespaces
• Chapter 7, Object Operations
• Chapter 8, Authentication
• Chapter 9, Return Codes and Troubleshooting
• Chapter 10, Microsoft .net Bindings
• Chapter 11, Working With REST
• Chapter 12, Working with Lattus S3
Lattus REST API User’s Guide xi
Preface
Notational Conventions This manual uses the following conventions:
The following formats indicate important information:
Note: Note emphasizes important information related to the main topic.
Caution: Caution indicates potential hazards to equipment or data.
WARNING: Warning indicates potential hazards to personal safety.
Convention Example
User input is shown in bold font. grep next-server /opt/qbase3/cfg/dhcpd/dhcpd.conf# dhcp-helper -s 10.1.0.204.
Computer output and command line examples are shown in monospace font.
GET /manage/namespace/VOL?meta=json HTTP/1.1Host: my.example.comDate: _date_
User input variables are enclosed in angle brackets.
#ip address <ip address>
For UNIX and Linux commands, the command prompt is implied.
hostname <name>is the same as#hostname <name>
File and directory names, menu commands, button names, and window names are shown in bold font.
“In the Add Namespace window, select the number of namespaces you want to add.”
Menu names separated by arrows indicate a sequence of menus to be navigated.
Dashboard > Administration > Storage Management > Storage Policies
xii Lattus REST API User’s Guide
Preface
• Right side of the system — Refers to the right side as you face the component being described.
• Left side of the system — Refers to the left side as you face the component being described.
• b — All binary numbers are succeeded by “b.”
• h — All hexadecimal numbers are succeeded by “h.”
• Error or attention conditions are represented in parenthesis that translate as follows:
(SK=S ASC=AA ASCQ=QQ)
where:
S — hexadecimal sense key value
AA — hexadecimal additional sense code
QQ — hexadecimal additional sense code qualifiers
Product Safety Statements
Quantum will not be held liable for damage arising from unauthorized use of the product. The user assumes all risk in this aspect.
This unit is engineered and manufactured to meet all safety and regulatory requirements. Be aware that improper use may result in bodily injury, damage to the equipment, or interference with other equipment.
WARNING: Before operating this product, read all instructions and warnings in this document and in the system, safety, and regulatory guide.
Lattus REST API User’s Guide xiii
Preface
xiv Lattus REST API User’s Guide
Preface
Contacts Quantum Home Page
Visit the Quantum home page at:
http://www.quantum.com
Getting More Information or Help
• Service and Support Website - Register products, license software, browse Quantum Learning courses, check backup software and operating system support, and locate manuals, FAQs, firmware downloads, product updates and more in one convenient location. Benefit today at:
http://www.quantum.com/ServiceandSupport/Index.aspx
• eSupport - Submit online service requests, update contact information, add attachments, and receive status updates via email. Online Service accounts are free from Quantum. That account can also be used to access Quantum’s Knowledge Base, a comprehensive repository of product support information. Sign up today at:
http://www.quantum.com/osr
For further assistance, or if training is desired, contact the Quantum Customer Support Center:
United States 1-800-284-5101 (toll free)+1-720-249-5700
EMEA +800-7826-8888 (toll free)+49-6131-3241-1164
APAC +800-7826-8887 (toll free)+603-7953-3010
For worldwide support:http://www.quantum.com/ServiceandSupport/Index.aspx
Lattus REST API User’s Guide xv
Preface
Worldwide End-User Product Warranty
For more information on the Quantum Worldwide End-User Standard Limited Product Warranty:
http://www.quantum.com/pdf/QuantumWarranty.pdf
xvi Lattus REST API User’s Guide
Chapter 1Introduction
Quantum Lattus™ Object Storage is disk-based storage that meets the extreme scalability, durability, and access requirements of large-scale Big Data archives.
Lattus object storage offers several different access methods to put data in and get data out of the Lattus storage, including StorNext access, Lattus A10 Access for NFS/CIFS, and HTTP/HTTPS REST access for customers who have ported their applications to use the Lattus REST APIs.
This document is designed for the REST API. You can adopt your applications to talk directly to the Lattus C10 controllers to be able to write objects natively to the Lattus system.
Lattus REST API User’s Guide 1
Chapter 1: Introduction
2 Lattus REST API User’s Guide
Chapter 2S3 Integration
Starting with Development
Quantum supports authentication and authorization on the bucket level (not on the object level).
In the following chapters, take into account the following:
• "bucketname" is usually the Lattus namespace name.
• The username is admin (having sufficient rights to perform the requested operation).
• This client daemon listens by default on port 7070 for S3 calls.
Authentication
The authentication information for S3 is hosted in the header field Authorization. This is of the form:
Authorization: AWSAccessKeyId:<Signature>
Lattus REST API User’s Guide 3
Chapter 2: S3 IntegrationAuthentication
For more information, see: http://docs.amazonwebservices.com/AmazonS3/latest/dev/RESTAuthentication.html
There is no anonymous user, every call to Lattus S3 needs to carry authentication information along with it.
Query String Authentication
You can authenticate certain types of requests by passing the required information as query-string parameters instead of using the Authorization HTTP header.
This is useful for enabling direct third-party browser access to your private Amazon S3 data without proxying the request. The idea is to construct a “pre-signed” request and encode it as a pre-signed URL that an end user’s browser can retrieve. Additionally, you can limit a pre-signed request by specifying an expiration time.
Instead of using special HTTP headers, the required authentication elements are specified as query string parameters:
Parameter Name Description Example Value
Expires The time when the signature expires, specified as the number of seconds since the epoch (00:00:00 UTC on January 1, 1970).A request received after this time (according to the server), will be rejected.
1141889120
4 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationCompatibility Calls
For more information, see: http://docs.amazonwebservices.com/AmazonS3/latest/dev/RESTAuthentication.html#RESTAuthenticationQueryStringAuth.
Compatibility Calls
To allow compatibility with some applications, the following calls have not been implemented but will return a dummy response. The dummy response eliminates the need for an exception handler in the calling routine.
GET /?location http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketGETlocation.html
s3cmd info s3://bucketname
AWSAccessKeyId Your AWS Access Key Id. The value is alphanumeric (A-Z, a-z and/or 0-9) and up to 256 characters.This specifies the AWS Secret Access Key used to sign the request, and (indirectly) the identity of the developer making the request.
AKIAIOSFODNN7-EXAMPLE
Signature The URL encoding of the Base64 encoding of the HMAC-SHA1 of StringToSign.
vjbyPxybdZaNmGa%2-ByT272YEAiv4%3D
Parameter Name Description Example Value
Lattus REST API User’s Guide 5
Chapter 2: S3 IntegrationCompatibility Calls
Request:
GET /?location HTTP/1.1Date: Mon, 26 Nov 2012 20:31:36 GMTAuthorization: AWS admin:W8QyXloRQVXXGeLkMzDE223iYMg=User-Agent: jclouds/1.5.0-beta.6 java/1.7.0_09Host: testfolder2.s3.amazonaws.com:7070Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2Connection: keep-alive""
Response:
<?xml version=\"1.0\" encoding=\"UTF-8\"?><LocationConstraintxmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\"/>
GET /?logging http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketGETlogging.html
s3cmd accesslog s3://bucketname
Request:
GET /bucketname?logging HTTP/1.1Host: 127.0.0.1:7070Accept: */*x-amz-acl:authenticate-readx-amz-date:Tue, 27 Nov 2012 08:43:34 GMTAuthorization: AWS admin:bAGHIFf98WMPu1FGpAmTAEHeaJM=""
Response:
<?xml version=\"1.0\" encoding=\"UTF-8\"?><BucketLoggingStatus
6 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationCompatibility Calls
xmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\"><LoggingEnabled><TargetBucket>mybucketlogs</TargetBucket><TargetPrefix>mybucket-access_log</TargetPrefix></LoggingEnabled></BucketLoggingStatus>
GET /?versioning http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketGETversioningStatus.html
[no s3cmd call exists]
Request:
GET /bucketname?versioning HTTP/1.1Host: 127.0.0.1:7070Accept: */*x-amz-acl:authenticate-readx-amz-date:Tue, 27 Nov 2012 08:43:34 GMTAuthorization: AWS admin:iC1TR8CvqFYxM+QOoCG9PY0r6cg=""
Response:
<?xml version=\"1.0\" encoding=\"UTF-8\"?><VersioningConfigurationxmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\"/>
GET /?acl http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketGETacl.html
s3cmd info s3://bucketname
Request:
GET /?acl HTTP/1.1Date: Mon, 26 Nov 2012 21:22:36 GMTAuthorization: AWS admin:05MWHGy46KX0gW6mL5DN1v6FfC8=User-Agent: jclouds/1.5.0-beta.6 java/1.7.0_09Host: testfolder5.s3.amazonaws.com:7070
Lattus REST API User’s Guide 7
Chapter 2: S3 IntegrationCompatibility Table
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2Connection: keep-alive""
Response:
<?xml version=\"1.0\" encoding=\"UTF-8\"?><AccessControlPolicyxmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\"/>
Compatibility Table
Service-API The following table shows which Service-API methods are supported and which are not supported.
Objects-API The following table shows which Objects-API methods are supported and which are not supported.
Supported Methods Non-Supported Methods
GET --
Supported Methods Non-Supported Methods
DELETE Object Delete Multiple Objects
GET Object (*) GET Object ACL
HEAD Object (*) GET Object torrent
PUT Object POST Object
PUT Object - Copy PUT Object ACL
8 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationCompatibility Table
(*) The GET and HEAD Object requests support “if-*” headers. For more information, refer to see the section Request Headers for GET and HEAD Objects.
Buckets-API The following table shows which Buckets-API methods are supported and which are not supported.
Initiate Multipart Upload
Upload Part
Upload Part - Copy
Complete Multipart Upload
Abort Multipart Upload
List Parts
Supported Methods Non-Supported Methods
Supported Methods Non-Supported Methods
DELETE Bucket DELETE Bucket lifecycle
GET Bucket (List Objects) DELETE Bucket policy
GET Bucket acl (dummy response)
DELETE Bucket website
GET Bucket location (dummy response)
GET Bucket lifecycle
GET Bucket logging (dummy response)
GET Bucket policy
GET Bucket versioning (dummy response)
GET Bucket notification
HEAD Bucket GET Bucket Object versions
List Multipart Uploads GET Bucket requestPayment
PUT Bucket GET Bucket website
Lattus REST API User’s Guide 9
Chapter 2: S3 IntegrationCompatibility Table
Differences Between Lattus S3 and Amazon S3
List Multipart Uploads
PUT Bucket acl
PUT Bucket lifecycle
PUT Bucket policy
PUT Bucket logging
PUT Bucket notification
PUT Bucket requestPayment
PUT Bucket versioning
PUT Bucket website
Supported Methods Non-Supported Methods
Situation Amazon AWS S3 Lattus S3
Number of characters for an object key name
Maximum up to 1024 characters
More than 1024 is allowed
Size of custom metadata perobject
Maximum of 2 KiB More than 2 KiB is allowed (see limitations)
Largest object that can be uploaded in a single PUT
5 GB More than 5 GB (which may result in errors)
Dummy headers HTTP 200 OK Returns errors
Creating an existing bucket HTTP 200 OK Returns an HTTP 409: conflict
Creating bucket with special characters ({, }, <, >, [, ] | ‘, ^, ”)
HTTP 400 Bad Request Returns an HTTP 501: Not Implemented
10 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationCompatibility Table
Object name with [, ], \, and ^ with s3cmd, unverified with other clients
HTTP 200 OK Returns an HTTP 501: Not Implemented
• S3 request with “Expires” parameter• S3 request with “TE” parameter
HTTP 200 OK Returns an HTTP 501: Not Implemented
Deleting non-existent objects HTTP 200 OK Returns errors
Calling GET Bucket (List Objects) with max-keys greater than 1000
Works without problems More than 1000 is not allowed, due to possible stack overflow
Performing a HEAD HTTP operation on a Bucket or key that does not exist
Returns an HTTP 404: page not found
Returns an HTTP 404: page not ound, together with the reason in the body:• NoSuchBucket: The specified
bucket does not exist• NoSuchKey: The specified
key does not exist
Performing a HEAD HTTPoperation on a bucket while the user does not have sufficient permissions to access them
Returns an HTTP 403: forbidden
Returns an HTTP 403: forbidden, together with the reason in the body:• AccessDenied: Access
Denied
Situation Amazon AWS S3 Lattus S3
Lattus REST API User’s Guide 11
Chapter 2: S3 IntegrationCompatibility Table
Cancelled multipart upload and then retry to upload another part.
Socket timeout HTTP 404: Not Found<?xml version="1.0"encoding="UTF-8"?><Error><Code>NoSuchUpload</Code><Message>The specified multipart upload does not exist. The upload ID might be invalid, or the multipart upload might have been aborted or completed.</Message> <RequestId></RequestId><Resource>testfile7</Resource></Error>
The timezone string 'CETrandomstring' is interpreted differently
This string is interpreted as 'CET' by Amazon S3
This string is interpreted as 'CETran' by Lattus, which is an unknown timezone
Request with unsupported time zone format
HTTP 403 Forbidden HTTP 500 Internal Server Error
Request without date header HTTP 403 Forbidden HTTP 500 Internal Server Error
Request with unsupported HTTP version
HTTP 505 HTTP Version Not Supported
HTTP 500 Internal Server Error
Request with bad URL encoding
HTTP 400 Invalid URI: EOF HTTP 501 Not Implemented
Request with invalid path HTTP 400 Invalid URI HTTP 500 Internal Server Error
Request with different “host” than the one configured in Lattus
HTTP 301 Moved Permanently HTTP 400 Bad Request
Incorrect md5sum value The Content-MD5 you specified did not match what we received
The Content-MD5 you specified was an invalid
HEAD bucket versioning request
HTTP 405 Method Not Allowed HTTP 501 Not Implemented
Situation Amazon AWS S3 Lattus S3
12 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationCompatibility Table
Response Differences
Request with Accept-Encoding: Identity
Transfer-Encoding is chunked Lattus ignores header and Transfer-Encoding is not chunked
Situation Amazon AWS S3 Lattus S3
Lattus Response Difference Request
• Extra period (”.”) mark • Missing element
“BucketName” in response
Delete non-empty bucket
Create too many buckets for the same S3 user
Content-Type in header has extra ”;charset=UTF-8”
HEAD bucket
Extra headers:• Content-Type• Pragma• Cache-Control• Expires• Date (omitted)
GET bucket versioning, dummy response
Missing elements:• CurrentNumberOfBuckets• AllowdNumberOfBuckets• HostId
Extra element:• Resource
Create too many buckets for the same S3 user
Missing elements:• BucketName• HostId
Put object in non-existent bucket
Lattus REST API User’s Guide 13
Chapter 2: S3 IntegrationCompatibility Table
Other Behavioral Differences
Following are other limitations of Lattus S3:
• Using one of the following response-header query parameters in the GET Object and Head request will not result in the correct response:
• response-content-type
• response-content-language
• response-expires
• response-cache-control
• response-content-disposition
• response-content-encoding
• Trying to PUT an object with the following headers may fail:
• x-amz-storage-class
• x-amz-server-side-encryption
• x-amz-website-redirect-location
• x-amz-grant-read
• x-amz-grant-write
• x-amz-grant-read-acp
• x-amz-grant-write-acp
• x-amz-grant-full-control
• The following common response headers are not implemented:
• x-amz-id-2
• x-amz-request-id
• The "x-amz-metadata-directive" header is not implemented.
• When trying to to do a server-side copy, the following x-amz-copy-source-if-* headers are not implemented:
Missing elements:• Key• HostId
GET /* where response is NoSuchKey
Lattus Response Difference Request
14 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationRequest Headers for GET and HEAD Object
• x-amz-copy-source-if-match
• x-amz-copy-source-if-none-match
• x-amz-copy-source-if-unmodified-since
• x-amz-copy-source-if-modified-since
• GET SERVICE requests are unable to handle several Query parameters when using a webdrive client. Be sure to specify a bucketname in the request.
Request Headers for GET and HEAD Object
The GET and HEAD Object requests support “if-*” headers. These include:
• If-Match: return the object if its entity tag (ETag) is the same as the one specified, or return HTTP 412 (precondition failed, XML response).
• If-Modified-Since: return the object if it has been modified since the specified time, or return HTTP 304 (object not modified).
• If-None-Match: return the object if its entity tag (ETag) is different from the one specified, or return HTTP 304 (object not modified).
• If-Unmodified-Since: return the object if it has not been modified since the specified time, or return HTTP 412 (precondition failed, XML response).
The time stamps used for If-Modified-Since and If-Unmodified-Since, must comply with the specifications of RFC2616.
The preferred time stamp is the RFC-822 (updated by RFC-1123): ‘Day, DD Mon YYYY HH:MM:SS GMT’.
The time stamp is case sensitive. See the following list:
• Day: Mon, Tue, Wed, Thu, Fri, Sat, Sun
• DD (*): two digits for the day of the month
• Mon: Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec
• YYYY: four digits for the year
Lattus REST API User’s Guide 15
Chapter 2: S3 IntegrationRequest Headers for GET and HEAD Object
• HH: two digits for the hour
• MM: two digits for the minutes
• SS: two digits for the seconds
• MT: time stamp must be represented in GMT
Note: If you don’t use two digits for the day of the month with an If-Modified-Since or If-Unmodified-Since request, you should receive respectively an HTTP 304 and HTTP 412 message. However, in Lattus you receive an HTTP 200 message but with warnings in the client daemon log.
The implementation of these headers are according the RFC2626. More information about these header optionscan be found in the RFC2626, section 14.24, 14.25, 14.26, and 14.27. (See http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html for more information.)
You can also combine the following requests:
• If-Match and If-Unmodified-Since
• If-None-Match and If-Modified-Since
If-Match If-Unmodified-Since Result
True True HTTP 200, object returned
True False XML Error 412 (precondition failed)
False True XML Error 412 (precondition failed)
False False XML Error 412 (precondition failed)
If-Match If-Unmodified-Since Result
True True HTTP 200, object returned
True False HTTP 200, object returned
False True HTTP 200, object returned
False False HTTP 304 (not modified)
16 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationRequest Headers for GET and HEAD Object
Examples Object created/modified on: Mon Jan 20 09:00:00 2014 etag: dbb126075a1d0b4ca64c3e2ae0159bc7
If-Match
• Matching ETag (HTTP 200):curl -X GET "http://monty.s3.amazonws.com:7070/python" -H "if-match:dbb126075a1d0b4ca64c3e2ae0159bc7"
HTTP/1.1 200 OKx-amz-id-2: V+INEFf6K34RtoqgQGfhDBiWo2oUauc8oh9Jpm2aA/XahYK8s3lt5s+pNGztkJh4x-amz-request-id: 8A0EA6052595BDB2Date: Fri, 14 Feb 2014 12:34:01 GMTLast-Modified: Mon, Jan 20 09:00:00 2014 GMTETag: "dbb126075a1d0b4ca64c3e2ae0159bc7"Accept-Ranges: bytesContent-Type: application/octet-streamContent-Length: 4Server: AmazonS3
• Different ETag (HTTP 412 Precondition Failed XML Response):curl -X GET "http://monty.s3.amazonws.com:7070/python" -H "if-match:dbb126075a1d0b4ca64c3e2ae0159xv3"
<Error><Code>PreconditionFailed</Code><Message>At least one of the pre-conditions you specified did not hold</Message><Condition>If-Match</Condition><RequestId>F58D72A68D9CF0C2</RequestId><HostId>xdvNIe0dyKfGOPu9lfwWcMbKKReL3oyO+vqRHtWI591rtqD7dZxHax02uOmr/HWM</HostId></Error>
Lattus REST API User’s Guide 17
Chapter 2: S3 IntegrationRequest Headers for GET and HEAD Object
If-None-Match
• Matching ETag:curl -X GET "http://monty.s3.amazonws.com:7070/python" -H "if-match:dbb126075a1d0b4ca64c3e2ae0159bc7"
–> returns HTTP Error 304
HTTP/1.1 304 Not Modifiedx-amz-id-2: T7kHsSgFbRy+DLcO2Y+jNkC/ZKVJN5KkKsAolS6Lmu9f0siX47sAdUEQsq/gzAr9x-amz-request-id: 58449835A39BB796Date: Fri, 14 Feb 2014 10:55:57 GMTLast-Modified: Mon, Jan 20 09:00:00 2014 GMTETag: "dbb126075a1d0b4ca64c3e2ae0159bc7"Server: AmazonS3
• Different ETag:curl -X GET "http://monty.s3.amazonws.com:7070/python" -H "if-match:dbb126075a1d0b4ca64c3e2ae0159xv3"
HTTP/1.1 200 OKx-amz-id-2: V+INEFf6K34RtoqgQGfhDBiWo2oUauc8oh9Jpm2aA/XahYK8s3lt5s+pNGztkJh4x-amz-request-id: 8A0EA6052595BF2CDate: Fri, 14 Feb 2014 12:34:01 GMTLast-Modified: Mon, Jan 20 09:00:00 2014 GMTETag: "dbb126075a1d0b4ca64c3e2ae0159bc7"Accept-Ranges: bytesContent-Type: application/octet-streamContent-Length: 4Server: AmazonS3
If-Modified-Since
• Object modified since given date:
18 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationRequest Headers for GET and HEAD Object
curl -X GET "http://monty.s3.amazonaws.com:7070/python" -H"if-modified-since: Tue Feb 18 08:38:47 2011"
HTTP/1.1 200 OKx-amz-id-2: V+INEFf6K34RtoqgQGfhDBiWo2oUauc8oh9Jpm2aA/XahYK8s3lt5s+pNGztkJh4x-amz-request-id: 8A0EA605259D345ADate: Fri, 14 Feb 2014 12:34:01 GMTLast-Modified: Mon, Jan 20 09:00:00 2014 GMTETag: "dbb126075a1d0b4ca64c3e2ae0159bc7"Accept-Ranges: bytesContent-Type: application/octet-streamContent-Length: 4Server: AmazonS3
• Object not modified since given date:curl -X GET "http://monty.s3.amazonaws.com:7070/python" -H"if-modified-since: Tue Feb 18 08:38:47 2015HTTP/1.1 304 Not Modifiedx-amz-id-2: T7kHsSgFbRy+DLcO2Y+jNkC/ZKVJN5KkKsAolS6Lmu9f0siX47sAdUEQsq/gzAr9x-amz-request-id: 58449835A39BB796Date: Fri, 14 Feb 2014 10:55:57 GMTLast-Modified: Mon, Jan 20 09:00:00 2014 GMTETag: "dbb126075a1d0b4ca64c3e2ae0159bc7"Server: AmazonS3
If-Unmodified-Since
• Object modified since given date:curl -X GET "http://monty.s3.amazonaws.com:7070/python" -H"if-unmodified-since: Tue Feb 18 09:40:53 2011"
<Error><Code>PreconditionFailed</Code>
Lattus REST API User’s Guide 19
Chapter 2: S3 IntegrationRequest Headers for GET and HEAD Object
<Message>At least one of the pre-conditions you specified did not hold</Message><Condition>If-Unmodified-Since</Condition><RequestId>F58D72A68D9CFDE1</RequestId><HostId>xdvNIe0dyKfGOPu9lfwWcMbKKReL3oyO+vqRHtWI591rtqD7dZxHax02uOmr/HWM</HostId></Error>
• Object not modified since given date:curl -X GET "http://monty.s3.amazonaws.com:7070/python" -H"if-unmodified-since: Tue Feb 18 09:38:47 2015"
HTTP/1.1 200 OKx-amz-id-2: V+INEFf6K34RtoqgQGfhDBiWo2oUauc8oh9Jpm2aA/XahYK8s3lt5s+pNGztkJh4x-amz-request-id: 8A0EA605259DCEF9Date: Fri, 14 Feb 2014 13:34:01 GMTLast-Modified: Mon, Jan 20 09:00:00 2014 GMTETag: "dbb126075a1d0b4ca64c3e2ae0159bc7"Accept-Ranges: bytesContent-Type: application/octet-streamContent-Length: 4Server: AmazonS3
Example Combined Header Request
• If-Match combined with If-Unmodified-Since:curl -X GET "http://monty.s3.amazonaws.com:7070/python" -H"if-unmodified-since: Tue Feb 18 09:40:53 2011" -H "if-match:dbb126075a1d0b4ca64c3e2ae0159bc6"
• If-None-Match combined with If-Modified-Since:curl -X GET "http://monty.s3.amazonaws.com:7070/python" -H"if-modified-since: Tue Feb 18 09:40:53 2011" -H "if-none-match:dbb126075a1d0b4ca64c3e2ae0159bc6"
20 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationImportant Information
Important Information
S3 uses a virtual host style. For more information and virtual hosting examples, see: http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/VirtualHosting.html#VirtualHostingExamples
Virtual Host Style If Your Host Header Ends in s3.amazonaws.com
As such, in the header bucket1.s3.amazonaws.com bucket1 is a valid bucketname.
If the host header has this format, the rest of the URL is the key to put in this bucket.
Example:PUT /myobjectname HTTP/1.1Host: mynewbucket.s3.amazonaws.comContent-Length: 0Content-Type: binary/octet-streamx-amz-acl: authenticated-readx-amz-date: Mon, 27 Aug 2012 13:41:44 GMTAuthorization: AWS admin:3gXQTb+Qx5/v2JmE18LlVyiuoc8=Expect: 100-continue""
If Your Host Header Ends in Something Else
When you want to send a host header with the following format:Host: testbucket.s3.mydomain.com:10101
You will need to add a configuration entry to the clientdeamon config file:
[s3]domain=s3.mydomain.com
Restart the clientdaemon for this change to be applied.
Requests using the abovementioned header will now result in a bucketname called testbucket
Lattus REST API User’s Guide 21
Chapter 2: S3 IntegrationImportant Information
Virtual Host Style Requests (Bucket Name as Part of the URL)
In this style of requests, the bucketname is part of the Host header field as first part of the URL.
Note: For all examples below, the header field is assumed to be:"Host: s3.amazonaws.com" or "Host: s3.cloudparrot.com" (if configured as such in the [s3] section in the domain field)
Examples:
PUT a bucket:
"PUT http://amplibucket.s3.amazonaws.com/ HTTP/1.1" puts the bucket with name amplibucket.
DELETE a bucket:
"DELETE http://amplibucket.s3.amazonaws.com/ HTTP/1.1" deletes the amplibucket.
PUT a file with the name "fileA" in the amplibucket:
"PUT http://amplibucket.s3.amazonaws.com/fileA HTTP/1.1" puts fileA in the amplibucket.
Path Style Requests
Note: We will assume path style request in all examples below and bucket name in the URL path.
In this style of requests; the host header has one of the following values:
• Either the exact string: "Host: s3.amazonaws.com" or "Host: s3.cloudparrot.com" (if configured as such in the [s3] section in the domain field).
• For bucket names that are not DNS aware: "Host: 192.169.12.148:7070".
• Is omitted completely.
PUT a bucket:
"PUT http://s3.amazonaws.com/amplibucket HTTP/1.1" puts the bucket with name amplibucket.
22 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationImportant Information
DELETE a bucket:
"DELETE http://s3.amazonaws.com/amplibucket HTTP/1.1" deletes the amplibucket.
PUT a file named "fileA" in the amplibucket:
"PUT http://s3.amazonaws.com/amplibucket/fileA HTTP/1.1" puts fileA in the amplibucket..
Supported Bucketnames
Use DNS compliant bucket names that comply to following rules:
• Bucket names must be at least 3 and no more than 63 characters long.
• Bucket name must be a series of one or more labels separated by a period (.), where each label:
• Must start with a lowercase letter or a number.
• Must end with a lowercase letter or a number.
• Can contain lowercase letters, numbers and dashes.
• Bucket names must not be formatted as an IP address (e.g., 192.168.5.4).
More information can be found here: http://docs.amazonwebservices.com/AmazonS3/latest/dev/BucketRestrictions.html
Interpretation of Query Parameters
At the server side we process the following 7 query parameters:
• With the GET bucket request:
• prefix
• delimiter
• marker
• max-keys
• With the GET object request:
• Expires
• AWSAccessKeyId
• Signature
Lattus REST API User’s Guide 23
Chapter 2: S3 IntegrationImportant Information
All other query parameters are not implemented and will throw an error to the client.
To avoid getting a client error you can do the following:
1 add to the client daemon configuration file:
[s3]
query_params_to_ignore=ignoreme
Custom Headers You Want to Upload With the Object
If you want to upload custom headers, configure the following:
[s3]
custom_headers_to_upload=uploadme
Now the header "uploadme", together with its value, will be stored with the object metadata.
Note: This is only supported for PUT object.
Headers You Would Like the Server to Ignore
If you want the server to ignore some headers, you can do the following:
[s3]headers_to_ignore=Origin
This way the server will not respond with an error if you send the Origin header. It will ignore it.
x-amz-meta-* Headers Can be Uploaded with the Object
For more information, see http://docs.amazonwebservices.com/AmazonS3/latest/dev/UsingMetadata.html.
Query String Authentication is Possible
We now can also do authentication when the GET request of an OBJECT specifies the 3 query parameters:
• Expires
24 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationImportant Information
• AWSAccessKeyId
• Signature
For more information, see: http://docs.amazonwebservices.com/AmazonS3/latest/dev/RESTAuthentication.html#RESTAuthenticationQueryStringAuth.
Bucket Calls Can be Disabled
The following three calls are enabled by default:
• enable_bucket_list
• enable_bucket_create
• enable_bucket_delete
You can disable them by setting the following parameters:[s3]enable_bucket_list = false # disables listing of buckets (does not disable listing of the content of buckets)enable_bucket_create = false # disables creation of bucketsenable_bucket_delete = false # disables deletion of buckets
If these calls are disabled, you will see the following error message: "action is disabled or not allowed on bucket"
You will receive the "Exc.MethodNotAllowed" exception.
Anonymous GET Feature
Anonymous GET Access
If an S3 bucket has READ access enabled for the special "everyone" user, then unauthenticated clients (including web browsers) can read objects within that bucket.
By default, the "everyone" user does not get any access to S3 buckets, but that access can be turned on with Q shell.
When anonymous GET is enabled, regular HTTP clients can fetch objects using URLs like http://s3.amplistore.host.com/bucketname/objectname.
Lattus REST API User’s Guide 25
Chapter 2: S3 IntegrationPermissions
Anonymous LIST Permission
If an S3 bucket has LIST access enabled for the "everyone" user, then unauthenticated clients can list the contents of that bucket by fetching a URL like http://s3.amplistore.host.com/bucketname.
The listing is an xml document in the format specified by S3.
Permissions
Permissions use the same paradigms as the AmpliStor REST Permission Schemes.
A bucket is conceptually the same as an AmpliStor Namespace. We map a bucket onto a namespace.
As an authenticated user you can create up to 100 buckets. The authenticated user creating the bucket is the "owner" of the bucket.
Note: Namespaces created using AmpliStor REST are not visible using S3 and vice versa
As an owner of a bucket you can do all below actions on your buckets.
As a non-owner, you need certain permissions to perform actions on these buckets (and you need to be authenticated).
Object Access PUT: update permissions if the object exists or create permissions if the object does not exist on the object path
Request:
"PUT /bucketname/a/b HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""Content-Length: 0"
26 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationPermissions
"Content-Type: binary/octet-stream""x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 13:41:44 GMT""Authorization: AWS admin:3gXQTb+Qx5/v2JmE18LlVyiuoc8=""Expect: 100-continue"""
Response:
HTTP/1.1 200 OK\r\nDAV: 1,3\Date: Mon, 27 Aug 2012 13:41:06 GMT\Server:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\Content-Type: text/plain;charset=UTF-8\r\nContent-Length: 0\Accept-Ranges:bytes\Last-Modified: Mon, 27 Aug 2012 13:41:06 GMT\ETag:\"3c46e4cdef1a6f620271991fbb84e7b6\"\
GET and HEAD: read permissions on the object path
Request:
"GET /bucketname/a/b HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 13:41:44 GMT""Authorization: AWS admin:wIVvNlW/VnYY+0aeKW6FSG9xuCc="""
Response:
HTTP/1.1 200 OK\r\nDAV: 1,3\Date: Mon, 27 Aug 2012 13:42:03 GMT\Server:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\Content-
Lattus REST API User’s Guide 27
Chapter 2: S3 IntegrationPermissions
Length: 0\r\nAccept-Ranges: bytes\Last-Modified: Mon, 27 Aug 2012 13:41:06GMT\ETag: \"f0a602cd8cd2d4e01c811daa4ba1b451\"\r\n
DELETE: delete permissions on the object path
Request:
"DELETE /bucketname/a/b/c/d/e HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 13:42:42 GMT""Authorization: AWS admin:b+zPIV1T/pCyic0vJ4WHFNh0sEQ="""
Response:
""DELETE response:HTTP/1.1 204 No Content\DAV: 1,3\Date: Mon, 27 Aug 201213:45:35 GMT\Server:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\Content-Type: text/plain\r\nContent-Length: 0\
Bucket Access PUT: you must be an authenticated user
Request:
"PUT /bucketname HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""Content-Length: 0"
28 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationPermissions
"x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 14:51:57 GMT""Authorization: AWS admin:XYzowiIv5TngegQsSmIRFPpk8mw=""Expect: 100-continue"""
Response:
HTTP/1.1 200 OK\DAV: 1,3\Date: Mon, 27 Aug 2012 14:51:19 GMT\Server:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\Content-Type: text/plain;charset=UTF-8\Content-Length: 0\
GET (listing) and HEAD: you must be the bucket owner or have list permissions on the bucket path: e.g: /namespace/bucketname
Request:
"GET /bucketname HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 13:46:14 GMT""Authorization: AWS admin:GQfdYkzk1UzDc8X3spAh1Wa1YzU="""
Response:
HTTP/1.1 200 OK\DAV: 1,3\Date: Mon, 27 Aug 2012 13:45:35 GMT\Server:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\r\nContent-Type: application/xml;charset=UTF-8\Content-Length: 5903\Pragma:no-cache\r\nCache-Control: no-cache\Expires: Thu, 01 Jan 1970 00:00:00 GMT\
Lattus REST API User’s Guide 29
Chapter 2: S3 IntegrationPermissions
body (in case file x and y are present as keys in bucket mynamespace):<?xml version=\"1.0\" encoding=\"UTF-8\"?><ListBucketResultxmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\"> <Name>bucketname</Name><Prefix></Prefix> <Marker></Marker> <MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated> <Contents> <Key>x</Key><LastModified>2012-09-04T11:15:12.000Z</LastModified><ETag>c10be79e50b74fff8c643b9760a33b3e</ETag> <Size>55</Size><StorageClass>STANDARD</StorageClass>\n <Owner>\n <ID></ID><DisplayName></DisplayName> </Owner> </Contents> <Contents><Key>y</Key> <LastModified>2012-09-04T11:15:07.000Z</LastModified><ETag>30cbeaa5e43c4443bc60d24aa6fd8d92</ETag> <Size>55</Size><StorageClass>STANDARD</StorageClass> <Owner> <ID></ID><DisplayName></DisplayName>\n </Owner> </Contents></ListBucketResult>
DELETE: you must be the owner of the bucket.
Request
"DELETE /bucketname HTTP/1.1""Host: s3.amazonaws.com""Accept-Encoding: identity""content-length: 0""Authorization: AWS admin:1KuDFb/uJzQI1+omTzJBU8i+3Ek=""x-amz-date: Tue, 04 Sep 2012 08:03:16 +0000"""
30 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationPermissions
Response
HTTP/1.1 204 No Content\r\nDAV: 1,3\r\nDate: Tue, 04 Sep 2012 08:03:16GMT\Server:Amplidata-AmpliStor/unknown-b55e743f085c9eb5bdea15501a8d60a9519e7ae4\Content-Length: 0\
Service Access GET: The following lists all the buckets owned by the authenticated sender of the request.
Request
"GET / HTTP/1.1""Host: s3.amazonaws.com""Accept-Encoding: identity""content-length: 0""Authorization: AWS admin:1KuDFb/uJzQI1+omTzJBU8i+3Ek=""x-amz-date: Tue, 04 Sep 2012 08:03:16 +0000"""
Response
HTTP/1.1 200 OKDAV: 1,3Date: Fri, 12 Oct 2012 11:48:28 GMTServer: Amplidata-AmpliStor/unknown-259aa153955d15a9f33a81e5992ae8c679c4e7eaContent-Type: application/xml;charset=UTF-8Content-Length: 525Pragma: no-cacheCache-Control: no-cacheExpires: Thu, 01 Jan 1970 00:00:00 GMTThe body for this request is in XML, and looks similar to the following example:<?xml version="1.0" encoding="UTF-8"?>
Lattus REST API User’s Guide 31
Chapter 2: S3 IntegrationError Codes
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>32</ID><DisplayName>admin</DisplayName></Owner><Buckets><Bucket><Name>mybucket</Name><CreationDate>2013-06-13T14:26:06.000Z</CreationDate></Bucket><Bucket><Name>myotherbucket</Name><CreationDate>2013-06-13T14:26:13.000Z</CreationDate></Bucket><Bucket><Name>joriske</Name><CreationDate>2013-06-13T11:56:02.000Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>
Error Codes
For more information about error responses in S3, see: http://docs.amazonwebservices.com/AmazonS3/latest/API/ErrorResponses.html
Error Code Description Http Status Code DSS Exception
InvalidBucketName The specified bucket is not valid.
400 Bad Request Exc.Invalid_namespace_name
32 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationError Codes
MetadataTooLarge Your metadata headers exceed the maximum allowed metadata size.
400 Bad Request
InvalidArgument Invalid Argument 400 Bad Request
TooManyBuckets You have attempted to create more buckets than allowed.
400 Bad Request
InvalidDigest The Content-MD5 you specified was invalid.
400 Bad Request
EntityTooLarge Your proposed upload exceeds the maximum allowed object size.
400 Bad Request
AccessDenied Access Denied 403 Forbidden None
SignatureDoesNotMatch The calculated request signature does not match the signature you provided. Check your AWS Secret Access Key and signing method.
403 Forbidden
InvalidAccessKeyId The AWS Access Key Id you provided does not exist in our records.
403 Forbidden
NoSuchBucket The specified bucket does not exist.
404 Not Found Exc.Namespace_not_found
Error Code Description Http Status Code DSS Exception
Lattus REST API User’s Guide 33
Chapter 2: S3 IntegrationError Codes
NoSuchKey The specified key does not exist.
404 Not Found Exc.Storage_object_not_found
NotSuchBucketPolicy The specified bucket does not have a bucket policy.
404 Not Found
MethodNotAllowed The specified method is not allowed against this resource.
405 Method Not Allowed
BucketAlreadyExists The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again.
409 Conflict Exc.Namespace_already _exists
BucketNotEmpty The bucket you tried to delete is not empty.
409 Conflict
MissingContentLength You must provide the Content-Length HTTP header.
411 Length Required
InternalError An internal error was encountered. Please try again.
500 Internal Server Error
NotImplemented A header you provided implies functionality that is not implemented.
501 Not Implemented
Error Code Description Http Status Code DSS Exception
34 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationError Codes
S3 Error Return Code Examples:
If the bucket does not exist, the following Error Response is generated:<?xml version=\"1.0\"encoding=\"UTF-8\"?><Error>\n<Code>NoSuchBucket</Code>\n<Message>The specified bucket does not exist.</Message>\n<Resource></Resource>\n<RequestId></RequestId>\n</Error>\nIf the key does not exist, the following error response is generated:<?xml version=\"1.0\"encoding=\"UTF-8\"?><Error>\n<Code>NoSuchKey</Code>\n<Message>The specified key does not exist.</Message>\n<Resource>fileX</Resource>\n<RequestId></RequestId>\n</Error>\n
Note: A requestId is not returned.
If a concurrent PUT request is detected:<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>ConcurrentPutDetected</Code><Message>"Concurrent put on this resource detected."</Message><RequestId></RequestId><Resource>/bucket/object</Resource>
</Error>
Concurrent PUT Detected
Concurrent PUT request detected for identical file. First PUT request wins
XML error Concurrent PUT on this resource detected
Error Code Description Http Status Code DSS Exception
Lattus REST API User’s Guide 35
Chapter 2: S3 IntegrationSupported Bucket Operations
Supported Bucket Operations
PUT Request http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketPUT.html
s3cmd mb s3://bucketname
Request:
"PUT /bucketname HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""Content-Length: 0""x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 14:51:57 GMT""Authorization: AWS admin:XYzowiIv5TngegQsSmIRFPpk8mw=""Expect: 100-continue"""
Response:
response: HTTP/1.1 200 OK\r\nDAV: 1,3\r\nDate: Mon, 27 Aug 2012 14:51:19GMT\r\nServer:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\r\nContent-Type: text/plain;charset=UTF-8\r\nContent-Length: 0\r\n
GET Request (listing of objects)
http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketGET.html
s3cmd ls s3://bucketname
36 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationSupported Bucket Operations
Example with full listing, no prefix:
Request:
"GET /bucketname HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 13:46:14 GMT""Authorization: AWS admin:GQfdYkzk1UzDc8X3spAh1Wa1YzU="""
Response:
HTTP/1.1 200 OK\r\nDAV: 1,3\r\nDate: Mon, 27 Aug 2012 13:45:35 GMT\r\nServer:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\r\nContent-Type: application/xml;charset=UTF-8\r\nContent-Length: 5903\r\nPragma:no-cache\r\nCache-Control: no-cache\r\nExpires: Thu, 01 Jan 1970 00:00:00 GMT\r\n
The body (in case file x and y are present as keys in bucket bucketname) looks like this:
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<ListBucketResultxmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\">\n <Name>bucketname</Name>\n<Prefix></Prefix>\n <Marker></Marker>\n <MaxKeys>1000</MaxKeys>\n<IsTruncated>false</IsTruncated>\n <Contents>\n <Key>x</Key>\n<LastModified>2012-09-04T11:15:12.000Z</LastModified>\n<ETag>c10be79e50b74fff8c643b9760a33b3e</ETag>\n <Size>55</Size>\n<StorageClass>STANDARD</StorageClass>\n <Owner>\n <ID></ID>\n<DisplayName></DisplayName>\n </Owner>\n </Contents>\n<Contents>\n <Key>y</Key>\n
Lattus REST API User’s Guide 37
Chapter 2: S3 IntegrationSupported Bucket Operations
<LastModified>2012-09-04T11:15:07.000Z</LastModified>\n<ETag>30cbeaa5e43c4443bc60d24aa6fd8d92</ETag>\n <Size>55</Size>\n<StorageClass>STANDARD</StorageClass>\n <Owner>\n <ID></ID>\n<DisplayName></DisplayName>\n </Owner>\n </Contents>\n</ListBucketResult>\n
Example with full listing with prefix:
Request:
"GET /bucketname?prefix=x HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 13:46:14 GMT""Authorization: AWS admin:GQfdYkzk1UzDc8X3spAh1Wa1YzU="""
Response:
HTTP/1.1 200 OK\r\nDAV: 1,3\r\nDate: Mon, 27 Aug 2012 13:45:35 GMT\r\nServer:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\r\nContent-Type: application/xml;charset=UTF-8\r\nContent-Length: 5903\r\nPragma:no-cache\r\nCache-Control: no-cache\r\nExpires: Thu, 01 Jan 1970 00:00:00 GMT\r\n
The body (in case file x and y are present as keys in bucket bucketname) looks like:
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<ListBucketResultxmlns=\"http://s3.amazonaws.com/doc/2006-03-01/\">\n <Name>bucketname</Name>\n<Prefix></Prefix>\n <Marker></Marker>\n <MaxKeys>1000</MaxKeys>\n
38 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationSupported Bucket Operations
<IsTruncated>false</IsTruncated>\n <Contents>\n <Key>x</Key>\n<LastModified>2012-09-04T11:15:12.000Z</LastModified>\n<ETag>c10be79e50b74fff8c643b9760a33b3e</ETag>\n <Size>55</Size>\n<StorageClass>STANDARD</StorageClass>\n <Owner>\n <ID></ID>\n<DisplayName></DisplayName>\n </Owner>\n </Contents>\n </ListBucketResult>\n
Note how the key with name 'y' is not present in the XML since it did not match the mentioned prefix):
HEAD Request http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketHEAD.html
No corresponding s3cmd call.
Request:
"HEAD /bucketname HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""x-amz-acl: authenticated-read""x-amz-date: Mon, 03 Sep 2012 14:54:17 GMT""Authorization: AWS admin:1oGc8tU8s4EWdcouB9esg6FE5so="""
Response:
HTTP/1.1 200 OK\r\nDAV: 1,3\r\nDate: Mon, 27 Aug 2012 13:45:35 GMT\r\nServer:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\r\nContent-Type: application/xml;charset=UTF-8\r\nContent-Length: 5903\r\nPragma:
Lattus REST API User’s Guide 39
Chapter 2: S3 IntegrationSupported Multipart Bucket Operations
no-cache\r\nCache-Control: no-cache\r\nExpires: Thu, 01 Jan 1970 00:00:00 GMT\r\n
DELETE Request
http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketDELETE.html
s3cmd rb s3://bucketname
Request:
"DELETE /bucketname HTTP/1.1""Host: s3.amazonaws.com""Accept-Encoding: identity""content-length: 0""Authorization: AWS admin:1KuDFb/uJzQI1+omTzJBU8i+3Ek=""x-amz-date: Tue, 04 Sep 2012 08:03:16 +0000"""
Response:
HTTP/1.1 204 No Content\r\nDAV: 1,3\r\nDate: Tue, 04 Sep 2012 08:03:16GMT\r\nServer:Amplidata-AmpliStor/unknown-b55e743f085c9eb5bdea15501a8d60a9519e7ae4\r\nContent-Length: 0\r\n
Supported Multipart Bucket Operations
List Multipart Uploads http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListMPUpload.html
40 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationSupported Multipart Bucket Operations
This operation shows all multipart uploads which are in progress. An in-progress multipart upload is a multipart upload that has been initiated, using the Initiate Multipart Upload request, but has not yet been completed or aborted.
Request:
GET /?uploads HTTP/1.1Host: 192.168.12.148:7070Date: Mon, 1 Nov 2013 20:34:56 GMTAuthorization: AWS admin:GQfdYkzk1UzDc8X3spAh1Wa1YzU=
Response:
HTTP/1.1 200 OKx-amz-id-2: Uuag1LuByRx9e6j5Onimru9pO4ZVKnJ2Qz7/C1NPcfTWAtRPfTaOFg==x-amz-request-id: 656c76696e6727732072657175657374Date: Mon, 1 Nov 2013 20:34:56 GMTContent-Length: 1330Connection: keep-aliveServer: Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b
The body in case there are in-progress multipart uploads:<?xml version="1.0" encoding="UTF-8"?><ListMultipartUploadsResult xmlns="http://192.168.12.148:7070"><Bucket>testbucket</Bucket><KeyMarker /><UploadIdMarker /><NextKeyMarker /><NextUploadIdMarker /><IsTruncated>false</IsTruncated><Upload><Key>TheHolyGrail.mp4</Key><UploadId>XMgbGlrZSBlbHZpbmcncyBub3QgaGF2aW5nIG11Y2ggbHVjaw</UploadId><Initiator>
Lattus REST API User’s Guide 41
Chapter 2: S3 IntegrationSupported Object Operations
<ID>arn:aws:iam::111122223333:user/user1-11111a31-17b5-4fb7-9df5-b111111f13de</ID>
<DisplayName>user1-11111a31-17b5-4fb7-9df5-b111111f13de</DisplayName>
</Initiator><Owner><ID>75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a</ID><DisplayName>OwnerDisplayName</DisplayName>
</Owner><StorageClass>STANDARD</StorageClass><Initiated>2010-11-10T20:48:33.000Z</Initiated>
</Upload></ListMultipartUploadsResult>
Supported Object Operations
PUT Request http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectPUT.html
s3cmd put /tmp/x s3://bucketname/x
Request:
"PUT /bucketname/a/b HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""Content-Length: 0""Content-Type: binary/octet-stream""x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 13:41:44 GMT""Authorization: AWS admin:3gXQTb+Qx5/v2JmE18LlVyiuoc8=""Expect: 100-continue"
42 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationSupported Object Operations
""
Response:
response: HTTP/1.1 200 OK\r\nDAV: 1,3\r\nDate: Mon, 27 Aug 2012 13:41:06GMT\r\nServer:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\r\nContent-Type: text/plain;charset=UTF-8\r\nContent-Length: 0\r\nAccept-Ranges:bytes\r\nLast-Modified: Mon, 27 Aug 2012 13:41:06 GMT\r\nETag:\"3c46e4cdef1a6f620271991fbb84e7b6\"\r\n
GET Request http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectGET.html
s3cmd get s3://bucketname/x /tmp/x.res
Request:
"GET /bucketname/a/b HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 13:41:44 GMT""Authorization: AWS admin:wIVvNlW/VnYY+0aeKW6FSG9xuCc="""
Response:
HTTP/1.1 200 OK\r\nDAV: 1,3\r\nDate: Mon, 27 Aug 2012 13:42:03 GMT\r\nServer:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\r\nContent-
Lattus REST API User’s Guide 43
Chapter 2: S3 IntegrationSupported Object Operations
Length: 0\r\nAccept-Ranges: bytes\r\nLast-Modified: Mon, 27 Aug 2012 13:41:06GMT\r\nETag: \"f0a602cd8cd2d4e01c811daa4ba1b451\"\r\n
HEAD Request http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectHEAD.html
No corresponding s3cmd call.
Request:
"HEAD /bucketname/a/b/c/d/e HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 13:42:42 GMT""Authorization: AWS admin:b+zPIV1T/pCyic0vJ4WHFNh0sEQ="""
Response:
HTTP/1.1 200 OK\r\nDAV: 1,3\r\nDate: Mon, 27 Aug 2012 13:42:03 GMT\r\nServer:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\r\nContent-Length: 1048576\r\nAccept-Ranges: bytes\r\nLast-Modified: Mon, 27 Aug 201213:42:03 GMT\r\nETag: \"fdf3a80e1ce052e7cd021563d98cb53b\"\r\n
DELETE Request http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTObjectDELETE.html
s3cmd del s3://bucketname/x
44 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationSupported Object Operations
Request:
"DELETE /bucketname/a/b/c/d/e HTTP/1.1""User-Agent: Mozilla/4.0 (Compatible; AMZS3; CVRF 1.0; Windows Server 2008 R2)""Host: 192.168.12.148:7070""Accept: */*""x-amz-acl: authenticated-read""x-amz-date: Mon, 27 Aug 2012 13:42:42 GMT""Authorization: AWS admin:b+zPIV1T/pCyic0vJ4WHFNh0sEQ="""
Response:
HTTP/1.1 204 No Content\r\nDAV: 1,3\r\nDate: Mon, 27 Aug 2012 13:45:35GMT\r\nServer:Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b\r\nContent-Type: text/plain\r\nContent-Length: 0\r\n
PUT - Copy Request http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html.
s3cmd cp s3://bucketname/xs3://bucketname/y
Request:
"PUT bucket/a HTTP/1.1"'content-length': '0''Authorization': 'AWS admin:8f5E54cGlLvt+NUrTZxmoEXAyQI=''x-amz-date': 'Thu, 13 Jun 2013 14:35:25 +0000''x-amz-copy-source': '/joriske/x''x-amz-metadata-directive': 'COPY'
Response:
HTTP/1.1 200 OK
Lattus REST API User’s Guide 45
Chapter 2: S3 IntegrationSupported Multipart Object Operations
Date: Thu, 13 Jun 2013 14:35:25 GMTDav: 1,3content-type: application/xmlcontent-length: 239server: Amplidata-AmpliStor/unknown-1fda372687b59bb934e108292c1d6873f519f37aThe body of this response is in XML:<?xml version="1.0" encoding="UTF-8"?><CopyObjectResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><LastModified>Thu, 13 Jun 2013 14:35:25 GMT</LastModified><ETag>"7c909b3e2820c8b47ed418753698a6da"</ETag></CopyObjectResult>
Supported Multipart Object Operations
Most of the S3 clients, (such as s3cmd or Cyberduck,) support multipart by default.
• Initiate Multipart Upload
• Upload Multipart Part
• Complete Multipart Upload
• Abort Multipart Upload
• List Parts
• Multipart Upload (Copy)
46 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationSupported Multipart Object Operations
Note: There are some default restrictions for the S3 multipart protocol:
• Maximum object size is 5 TB
• Minimum object size is 5 MB
• Minimum part size is 5 MB
• Maximum number of parts is 10,000 (default)
These settings can updated in the client daemon configuration file. The real maximum number of parts that can be used is 65,535.
Initiate Multipart Upload
http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadInitiate.html
Initiating a multipart upload does the initialization of the multipart upload and returns an upload ID and a storage object ID. The upload ID is used to associate all the parts in the specific multipart upload. A multipart upload request is stored with a key: mpoI_<bucketID>_<object name>_<upload id>
Request:
POST http://testbucket.mys3service.com/example-object.iso?uploads HTTP/1.1Host: testbucket.mys3service.comAccept-Encoding: identitycontent-length: 0content-type: application/x-iso9660-imageAuthorization: AWS thomas:HvU5l9u6ss3xHQxIPzUt9JzYA6c=x-amz-date: Fri, 13 Dec 2013 08:19:16 +0000content-encoding: UTF-8
Response:
HTTP/1.1 200 OKx-amz-id-2: JBP5qj+DsZcfrawymY/4UDTf2JKdYD0B/rmL+USGtVwg5AZKuu/icNAAiwqfEgX9
Lattus REST API User’s Guide 47
Chapter 2: S3 IntegrationSupported Multipart Object Operations
x-amz-request-id: 7F200D15AB4013C8Date: Fri, 13 Dec 2013 08:19:03 GMTTransfer-Encoding: chunkedServer: Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b
Body:
<?xml version="1.0" encoding="UTF-8"?><InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Bucket>testbucket.mys3service.com</Bucket><Key>c2c4dc85-38ac-4f89-aa29-16a19bbc3468</Key>
<UploadId>BbyOjajJe30TSjDP7Bq4Z.rmTu_pDwVOFBFi7tqHWwe5ZgUFMKGwZje2TQl4Zz_i4mBhjLD1vXxj8_c7rQC_BLwGQplJh_bcOftTq0b.U0o4_F8lyRkYJ8qBKFFmNwhS</UploadId></InitiateMultipartUploadResult>
Upload Multipart Part http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPart.html
A multipart part has a 1-on-1 mapping with a part, which can be considered a subobject. The part is first uploaded and created, and subsequently it is linked to the specific multipart part.
A possible previous "old" part will be overwritten and the "old" part is no longer accessible and will be deleted.
Request:
PUT/c2c4dc85-38ac-4f89-aa29-16a19bbc3468?partNumber=1&uploadId=BbyOjajJe30TSjDP7Bq4Z.rmTu_pDwVOFBFi7tqHWwe5ZgUFMKGwZje2TQl4Zz_i4mBhjLD1vXxj8_c7rQC_BLwGQplJh_bcOftTq0b.U0o4_F8lyRkYJ8qBKFFmNwhS HTTP/1.1Content-Length: 5242880Accept-Encoding: identity
48 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationSupported Multipart Object Operations
Accept: */*Content-MD5: p17Vdc2ncwx8FLQDUtoVVQ==Host: testbucket.mys3service.comx-amz-date: Fri, 13 Dec 2013 08:19:18 +0000Content-Type: application/octet-streamAuthorization: AWS thomas:HvU5l9u6ss3xHQxIPzUt9JzYA6c=
Response:
HTTP/1.1 200 OKx-amz-id-2: BD6QQJVqotgk5pzONQl8xQRLfY7XIUF24n0PZScTlnS3CP4fMSmwm5yf958KYDNpx-amz-request-id: F28B7459125E4943Date: Fri, 13 Dec 2013 08:19:05 GMTETag: "a75ed575cda7730c7c14b40352da1555"Content-Length: 0Server: Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b
Complete Multipart Upload
http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadComplete.html
Request:
POST /c2c4dc85-38ac-4f89-aa29-16a19bbc3468?uploadId=BbyOjajJe30TSjDP7Bq4Z.rmTu_\\pDwVOFBFi7tqHWwe5ZgUFMKGwZje2TQl4Zz_i4mBhjLD1vXxj8_c7rQC_BLwGQplJh_\\bcOftTq0b.U0o4_F8lyRkYJ8qBKFFmNwhS HTTP/1.1Content-Length: 243Accept-Encoding: identityAccept: */*Host: testbucket.mys3service.comx-amz-date: Fri, 13 Dec 2013 08:19:30 +0000Content-Type: application/octet-streamAuthorization: AWS thomas:HvU5l9u6ss3xHQxIPzUt9JzYA6c=
Lattus REST API User’s Guide 49
Chapter 2: S3 IntegrationSupported Multipart Object Operations
<CompleteMultipartUpload><Part><PartNumber>1</PartNumber><ETag>a75ed575cda7730c7c14b40352da1555</ETag></Part><Part><PartNumber>2</PartNumber><ETag>1f778a8d73d57b0e162667b635631173</ETag></Part></CompleteMultipartUpload>
Response:
HTTP/1.1 200 OKx-amz-id-2: vjq6W420mzRUGJcTPHSDTCDpYMbjba7RLXdhb0MSJzxaz5/cuLq6cHGFhAgPVfs/x-amz-request-id: CCC7CBBC674808C5Date: Fri, 13 Dec 2013 08:19:17 GMTContent-Type: application/xmlTransfer-Encoding: chunkedServer: Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b
Body:
<?xml version="1.0" encoding="UTF-8"?><CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Location>http://testbucket.mys3service.com/c2c4dc85-38ac-4f89-aa29-16a19bbc3468</Location><Bucket>testbucket.mys3service.com</Bucket><Key>c2c4dc85-38ac-4f89-aa29-16a19bbc3468</Key><ETag>"329454b9b991ccd9b34e0ba4ae0b5004-2"</ETag>
</CompleteMultipartUploadResult>
50 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationSupported Multipart Object Operations
Abort Multipart Upload http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadAbort.html
Request:
DELETE/example-object?uploadId=BbyOjajJe30TSjDP7Bq4Z.rmTu_pDwVOFBFi7tqHWwe5ZgUFMKGwZje2TQl4Zz_i4mBhjLD1vXxj8_c7rQC_BLwGQplJh_bcOftTq0b.U0o4_F8lyRkYJ8qBKFFmNwhS HTTP/1.1Host: testbucket.mys3service.comDate: Fri, 13 Dec 2013 08:19:22 GMTAuthorization: AWS thomas:HvU5l9u6ss3xHQxIPzUt9JzYA6c=
Response:
HTTP/1.1 204 OKx-amz-id-2: Weag1LuByRx9e6j5Onimru9pO4ZVKnJ2Qz7/C1NPcfTWAtRPfTaOFg==x-amz-request-id: 996c76696e6727732072657175657374Date: Fri, 13 Dec 2013 08:19:22 GMTContent-Length: 0Connection: keep-aliveServer: Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b
List Parts http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadListParts.html
List relevant multipart part entries.
Request:
GET/example-object?uploadId=BbyOjajJe30TSjDP7Bq4Z.rmTu_pDwVOFBFi7tqHWwe5ZgUFMKGwZje2TQ
Lattus REST API User’s Guide 51
Chapter 2: S3 IntegrationSupported Multipart Object Operations
l4Zz_i4mBhjLD1vXxj8_c7rQC_BLwGQplJh_bcOftTq0b.U0o4_F8lyRkYJ8qBKFFmNwhS&max-parts=2&part-number-marker=1 HTTP/1.1Host: testbucket.mys3service.comDate: Fri, 13 Dec 2013 08:19:32 GMTAuthorization: AWS thomas:HvU5l9u6ss3xHQxIPzUt9JzYA6c=
Response:
HTTP/1.1 200 OKx-amz-id-2: Uuag1LuByRx9e6j5Onimru9pO4ZVKnJ2Qz7/C1NPcfTWAtRPfTaOFg==x-amz-request-id: 656c76696e6727732072657175657374Date: Fri, 13 Dec 2013 08:19:19 GMTContent-Length: 985Connection: keep-aliveServer: Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b
Body:
<?xml version="1.0" encoding="UTF-8"?><ListPartsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Bucket>testbucket</Bucket><Key>example-object</Key>
<UploadId>BbyOjajJe30TSjDP7Bq4Z.rmTu_pDwVOFBFi7tqHWwe5ZgUFMKGwZje2TQl4Zz_i4mBhjLD1vXxj8_c7rQC_BLwGQplJh_bcOftTq0b.U0o4_F8lyRkYJ8qBKFFmNwhS</UploadId><Initiator><ID>arn:aws:iam::111122223333:user/some-user-11116a31-17b5-4fb7-9df5-b288870f11xx</ID><DisplayName>umat-user-11116a31-17b5-4fb7-9df5-b288870f11xx</DisplayName></Initiator><Owner><ID>75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a</ID><DisplayName>someName</DisplayName>
52 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationSupported Multipart Object Operations
</Owner><StorageClass>STANDARD</StorageClass><PartNumberMarker>1</PartNumberMarker><NextPartNumberMarker>3</NextPartNumberMarker><MaxParts>2</MaxParts><IsTruncated>true</IsTruncated><Part><PartNumber>2</PartNumber><LastModified>2010-11-10T20:48:34.000Z</LastModified><ETag>"7778aef83f66abc1fa1e8477f296d394"</ETag><Size>10485760</Size>
</Part><Part><PartNumber>3</PartNumber><LastModified>2010-11-10T20:48:33.000Z</LastModified><ETag>"aaaa18db4cc2f85cedef654fccc4a4x8"</ETag><Size>10485760</Size>
</Part></ListPartsResult>
Multipart Upload (Copy)
http://docs.aws.amazon.com/AmazonS3/latest/API/mpUploadUploadPartCopy.html
Request:
PUT/newobject?partNumber=2&uploadId=VCVsb2FkIElEIGZvciBlbZZpbmcncyBteS1tb3ZpZS5tMnRzIHVwbG9hZR HTTP/1.1Host: target-bucket.mys3service.comDate: Fri, 13 Dec 2013 08:25:19 GMTx-amz-copy-source: /source-bucket/sourceobjectx-amz-copy-source-range:bytes=500-6291456Authorization: AWS thomas:HvU5l9u6ss3xHQxIPzUt9JzYA6c=
Lattus REST API User’s Guide 53
Chapter 2: S3 IntegrationSupported Service Operations
Response:
HTTP/1.1 200 OKx-amz-id-2: Vvag1LuByRx9e6j5Onimru9pO4ZVKnJ2Qz7/C1NPcfTWAtRPfTaOFg==x-amz-request-id: 656c76696e6727732072657175657374Date: Fri, 13 Dec 2013 08:25:19 GMTServer: Amplidata-AmpliStor/unknown-c111f895b629566707818005fa6ff8f66346011b<CopyPartResult><LastModified>2008-01-29T08:22:00</LastModified><ETag>"9b2cf535f27731c974343645a3985328"</ETag></CopyPartResult>
Supported Service Operations
GET Request http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTServiceGET.html
s3cmd ls
Request:
"GET / HTTP/1.1""Host: s3.amazonaws.com""Accept-Encoding: identity""content-length: 0""Authorization: AWS admin:1KuDFb/uJzQI1+omTzJBU8i+3Ek=""x-amz-date: Tue, 04 Sep 2012 08:03:16 +0000"""
Response:
HTTP/1.1 200 OK\r\nDAV: 1,3\r\nDate: Fri, 12 Oct 2012 11:48:28 GMT\r\nServer:
54 Lattus REST API User’s Guide
Chapter 2: S3 IntegrationSupported Service Operations
Amplidata-AmpliStor/unknown-259aa153955d15a9f33a81e5992ae8c679c4e7ea\r\nContent-Type:application/xml;charset=UTF-8\r\nContent-Length: 525\r\nPragma:no-cache\r\nCache-Control: no-cache\r\nExpir es: Thu, 01 Jan 1970 00:00:00GMT\r\n
The body for this request is in XML:<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Owner><ID>32</ID><DisplayName>admin</DisplayName>
</Owner><Buckets>
<Bucket><Name>mybucket</Name><CreationDate>2013-06-13T14:26:06.000Z</
CreationDate></Bucket><Bucket>
<Name>myotherbucket</Name><CreationDate>2013-06-13T14:26:13.000Z</
CreationDate></Bucket>
<Bucket><Name>joriske</Name><CreationDate>2013-06-13T11:56:02.000Z</CreationDate></Bucket></Buckets></ListAllMyBucketsResult>
Lattus REST API User’s Guide 55
Chapter 2: S3 IntegrationSupported Service Operations
56 Lattus REST API User’s Guide
Chapter 3REST Definitions and
Configuration
In addition to the S3 implementation, you can also communicate with Lattus through a REST API. Details for using this REST API are provided in subsequent chapters.
REST Overview Any Lattus server can be configured to act as a REST Capable Storage Server. The more Lattus servers are available in the environment, the higher the potential overall throughput performance.
All Lattus servers can view and access the complete storage pool as a global namespace and put or receive data from all namespaces. An object stored on one controller is immediately visible and available for access from any other controller.
The Lattus server can be configured to listen on a dedicated port to receive incoming HTTP requests. The following REST commands are currently supported:
• Create policies
• Create, overwrite, retrieve, delete, and list objects
• Add, list, get, and delete users
• Retrieve policy metadata
• Create and delete namespaces
• Retrieve namespace metadata
Lattus REST API User’s Guide 57
Chapter 3: REST Definitions and ConfigurationHow To Enable REST in Lattus
• Store, retrieve, delete, list, set and retrieve object metadata
How To Enable REST in Lattus
In Lattus, REST is enabled by default on every controller and listening on port 8080.
If you are using the REST development virtual machine, a command line option is available to enable REST.
Using Curl to Demonstrate the REST Functionality
As a way to demonstrate the REST API, the program Curl is used in some of the following examples. Curl has the following options:
Note: Do not use wget. The most common versions of wget only provide HTTP 1.0 support, while HTTP 1.1 is required.
58 Lattus REST API User’s Guide
Chapter 3: REST Definitions and ConfigurationOther Interfaces
Other Interfaces
Besides the REST API interface, there is also a command line interface (CLI) and the Q-Shell interface, available on any controller. This guide describes mainly the REST API and the Q-Shell interfaces, but all functions described in this guide can also be executed from the CLI.
The CLI is available via /opt/qbase/bin/dss. Call the help function by adding --help to the command.
~# /opt/qbase3/bin/dss --permission-settings-get /namespace
{
"Flags":[],
"User-Permissions":
{
"everyone":["UPDATE","LIST","DELETE","CREATE","READ"]
}
}
API Interface Documentation Assumptions
For readability purposes, some parameters within the HTTP headers are replaced with a named parameter. The following named parameters are used:
• _date_: a date string following the HTTP 1.1 date specification, e.g. Fri, 03 Jun 2011 00:00:00 GMT.
• _length_: the length in bytes of the specified variable, e.g. for Content-length: length. This means that the length in bytes of the passed content must be filled in.
• _epoch_time_: time in seconds since Thu, 01 Jan 1970 00:00:00 GMT.
Lattus REST API User’s Guide 59
Chapter 3: REST Definitions and ConfigurationResource Identifiers
Resource Identifiers
The Lattus Web Service API uses Internationalized Resource Identifiers (IRI) (RFC 3987), to refer to the entities addressable in the storage system.
An IRI is similar to a Uniform Resource Identifier (URI) but without the limitation of containing only characters from the ASCII character set. As such, an IRI can contain Unicode characters. The RFC also documents how to encode this for systems handling only ASCII character set URIs.
IRI setup: /manage/policy is used for managing policies
Transfer Protocol
The transfer protocol for the Web Service is Hypertext Transfer Protocol v1.1, as documented in RFC 2616.
If the Lattus receives an HTTP/1.0 request, it will always return with an HTTP 505 Version Not Supported return code.
Behavior Common to All Requests
In the remainder of this document, the list of REST Web Service API requests and responses are described in detail.
Irrespective of the different requests, there is some base HTTP 1.1 behavior that an integrator can expect from the Lattus REST Web Service. Such behavior is described in this section.
60 Lattus REST API User’s Guide
Chapter 3: REST Definitions and ConfigurationWhat Has Changed Compared to the 3.2.X REST API
What Has Changed Compared to the 3.2.X REST API
• In its response headers, the client daemon will reply with Quantum-Lattus/3.6.0-<versionGUID>.
Lattus REST API User’s Guide 61
Chapter 3: REST Definitions and ConfigurationWhat Has Changed Compared to the 3.2.X REST API
62 Lattus REST API User’s Guide
Chapter 4Design Considerations
Integration of your application with Lattus requires a carefully architected integration in order to get the maximum out of Lattus's performance and scalability.
Take the following items into account prior to starting the actual development of the integration code.
Designing for Performance
Connection Management
Use a REST-library that can handle persisting connections. If your library can't handle this, you might encounter performance degradation since the connection will have to be re-established upon every REST operation.
Because of this, keep the following advice in mind, especially when using authentication:
• Do not use the Linux curl executable.
• Use the libcurl library instead.
Do not direct all connections from your application towards a single controller. If you have multiple controllers at your disposal, spread and
Lattus REST API User’s Guide 63
Chapter 4: Design ConsiderationsDesigning for Performance
balance your connections over all controllers. Preferably, pool connections to the same controller that are bound to a namespace:
• Example:
• You control 9 namespaces (ns1, ns2...ns9) and 3 controllers.
• A good practice from a performance perspective would be to:
• Send all connections for namespaces ns1, ns2 and ns3 to controller1.
• Send all connections for namespaces ns4, ns5 and ns6 to controller2.
• Send all connections for namespaces ns7, ns8 and ns9 to controller3.
• The advantage of this approach is that the connections are spread evenly over all controllers. Each namespace that needs to be referenced on a controller leads to a set of connections to the back-end.
• A bad practice would be to consult namespaces on random controllers. This would still work, but you will encounter performance degradation and you can encounter connection exhaustion on the back-end.
• If a controller fails, your code should be capable of temporarily rerouting traffic to another controller. If your environment was scaled properly, you should have made provisions for spare capacity.
If your application uses a lot of persistent connections and the data is slowly retrieved by your application (i.e. your application reads the data slower than Lattus can decode it from the back-end), match the amount of memory on your controllers to the expected number of connections. When a connection is open and the encoded data is not entirely retrieved by your application, the controller will still have a reference to a buffer in memory holding the encoded data (which is up to 6x the Superblock size). If your application leads to many of these semi-stale connections, the amount of memory inside the controller should be sufficient.
64 Lattus REST API User’s Guide
Chapter 4: Design ConsiderationsDesigning for Performance
Parallelism Reading different objects in multiple parallel threads leads to improved performance. The Lattus Client daemon is designed for parallelism. The amount of parallelism is determined by the number of connections your controllers and environment can handle.
Do not read the object in different parallel threads. Each read operation will lead to a process of fetching content from the storage back-end and decoding the fetched data. If this process is done multiple times for the same object at the same time, performance will adversely be affected as each thread will fighting for the same resource.
Avoid writing the same object from different threads. This is supported, but every object will be encoded and written to the back-end and only 1 object will be retained. For the writes that failed, metadata updates need to happen and data needs to be deleted from the back-end.
Partial Reads Keep in mind that a partial read triggers fetching and decoding of a superblock.
Example:
• Your storage policy uses 64 MiB superblocks.
• You stored a 1 GiB object and want to read 8 kiB from that object.
• This will lead to 64 MiB being fetched from the back-end, which leads to many times that amount of data being read from the back-end (depending on your storage policy).
Consider changing your superblocksize to better match your partial read size. However, keep the following in mind:
• Lowering the superblocksize lowers your overall performance.
• The minimum superblocksize is 2 MiB.
A valid design choice could be: Using a dedicated namespace with a smaller superblocksize for this type of file.
As such, make sure that your reads are superblock aligned in order to avoid having to fetch 2 superblocks for a single read.
Lattus REST API User’s Guide 65
Chapter 4: Design ConsiderationsDesigning for Scalability
Overwrites Overwrites are possible but keep in mind that these are costly operations. The object needs to be encoded, stored on the back-end and the original content has to be deleted.
Overwrites are done on an object level and currently do not support superblock aligned overwrites, which would only update a subset of the object (in case it entails multiple superblocks).
As such there is no performance benefit in only changing a couple of bytes in an object since the objects is rewritten in its entirety.
Caching If your application is reading the same (updated) content very frequently, consider deploying a read cache. This cache can be configured to update the content in the cache after it has been stored to the back-end.
Subsequent reads will benefit from the cache: their read latency will significantly go down. First byte read latency will go from hundreds of milliseconds to a few milliseconds.
Designing for Scalability
Namespace Limitations Namespaces typically have 2 limitations:
• Number of concurrent operations that can be done.
• Number of objects they can hold.
Maximum Number of Objects Per Namespace
Every object in a namespace consumes metadata. This metadata is stored in a MetaStore, which lives on a file system on top of an SSD. As such, the maximum number of objects that can be stored is limited by the size of this SSD. Lattus doesn't support spanning a namespace over multiple SSDs.
In order to identify how many objects can be stored in a namespace, one should determine the number of superblocks per object and the
66 Lattus REST API User’s Guide
Chapter 4: Design ConsiderationsDesigning for Scalability
size of the object metadata. Typically every superblock of an object consumes 200 bytes. We recommend measuring MetaStore consumption for your application (when the object name gets longer, more metadata is required).
When designing for the number of objects, take this limitation in consideration. There are 2 approaches:
1 When the namespace is exhausted, the application starts using a new namespace which lives one a different MetaStore (using different SSDs).
2 Your application is capable of handling multiple namespaces and balancing writes into these namespaces based upon their fill-level. In the early phase, when a single MetaStore is sufficient, your application writes to this one namespace. When your application exceeds a specified limit, configure your application so that it will start writing to a second namespace but still uses the first namespace but based upon consumption.
A. Example:
a Start with a single namespace (ns1) with maximum 1,000,000,000 superblocks. Your objects typically are 1 GiB in size. With a SuperBlocksize of 32 MiB, you need 32 superblocks per object. This means you can write 31,250,000 million objects to this namespace.
b After 9 months 400,000,000 superblocks (or 12.5 million objects) have been written to ns1 and you decide to extend your Lattus with an additional MetaStore capable of handling 1,500,000,000 superblocks (commercially available SSDs are still obeying Moore's Law). This implies you can write 46,875,000 objects to a namespace (ns2) managed by this MetaStore.
c Your application will write 600,000,000 / (600,000,000 + 1,500,000,000) = 28.5% of its writes to ns1 and 1,500,000,000 / (1,500,000,000 + 600,000,000) = 71.4% of its writes to ns2.
d 10 months later, you extend your Lattus with an additional MetaStore and now you balance over 3 namespaces. Let's assume that ns1 is filled for 90%, ns2 is filled for 40% and ns3 is still empty, but can take 2,000,000,000 superblocks. Writes will happen according to the following distribution:
1 100,000,000 / (100,000,000 + 900,000,000 + 2,000,000,000)= 3.3% to ns1.
Lattus REST API User’s Guide 67
Chapter 4: Design ConsiderationsDesigning for Scalability
2 900,000,000 / (100,000,000 + 900,000,000 + 2,000,000,000)= 30% to ns2.
3 2,000,000,000 / (100,000,000 + 900,000,000 + 2,000,000,000)= 66.6% to ns3.
B. Other mapping methods which guarantees proportional spreading of metadata over MetaStores can also be considered.
Keep in mind that moving an object from one namespace to another not only requires rewriting the metadata but also the data underneath (which drives data retrieval, decoding and encoding the object and finally storing the encoded data on the back-end).
Concurrent Operations
The MetaStore managing object metadata for a namespace is limited in the number of object operations it can handle. These operations are:
1 put
2 get
3 delete
4 list
5 repair
6 monitoring
When using appropriate hardware (dedicated SSDs and sufficient memory for caching) we recommend to never exceed 1000 of these operations (put, get, delete, list combined together) per second per MetaStore.
Connection Locality When reviewing the previous mentioned performance considerations related to connection management, one can easily see that accessing your data from everywhere introduces sub-optimal use of your Lattus.
For optimal scalability your application should be capable of localizing reads and writes for the same namespace to the same controller, or at least attempt to pursue this goal.
68 Lattus REST API User’s Guide
Chapter 5Policy Operations
Create Policy
A POST/PUT request creates a policy with certain parameters and returns the GUID of this policy. As the name of the policy can't be chosen by the user, a POST/PUT on a /policy subdirectory suffices.
On successful creation of a policy, the return status is “201 Created”, and the metadata of the new policy is returned.
If a policy with the same parameters already exists, the return status is “200 OK”, and the metadata of the existing policy is returned.
If for a certain parameter no value is passed, the default is used. The default values for REST are the following:
• Spread-Width: 16
• Safety: 4
• Hierarchy-Rules: true, true, true
• Max-Superblock-Size: 32 MiB
• Safety-Strategy: “RepairSpread”,”DynamicSafety”
• N-Messages: 4096
Lattus REST API User’s Guide 69
Chapter 5: Policy OperationsCreate Policy
http / text REQUEST /manage/policy
PUT /manage/policy HTTP/1.1
Host: my.example.com
Date: _date_
Accept: text/plain
Content-Type: text/plain
Content-Length: 0
X-Ampli-Spread-Width: 16
X-Ampli-Safety: 4
X-Ampli-Hierarchy-Rules: [ false , true , true ]
X-Ampli-Max-Superblock-Size: 33554432
X-Ampli-Safety-Strategy: [ "RepairSpread" , "DynamicSafety" ]
X-Ampli-N-Messages: 4096
X-Ampli-Full-Copy: false
or:PUT /manage/policy?meta=http HTTP/1.1Host: my.example.comDate: _date_Content-Length: 0X-Ampli-Spread-Width: 16X-Ampli-Safety: 4X-Ampli-Hierarchy-Rules: [ false , true , true ]X-Ampli-Max-Superblock-Size: 33554432X-Ampli-Safety-Strategy: [ "RepairSpread" , "DynamicSafety" ]X-Ampli-N-Messages: 4096X-Ampli-Full-Copy: false
RESPONSE /manage/policy
HTTP/1.1 201 CreatedDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24
70 Lattus REST API User’s Guide
Chapter 5: Policy OperationsCreate Policy
Content-Type: text/plain;charset=UTF-8Content-Length: 0X-Ampli-Id: "d27a3bc66f7e3c8e957cdaa01d5eacc1"X-Ampli-Spread-Width: 16X-Ampli-Safety: 4X-Ampli-Hierarchy-Rules: [ false , true , true ]X-Ampli-Max-Superblock-Size: 33554432X-Ampli-Safety-Strategy: [ "RepairSpread" , "DynamicSafety" ]X-Ampli-N-Messages: 4096X-Ampli-Full-Copy: false
json REQUEST /manage/policy
PUT /manage/policy HTTP/1.1Host: my.example.comDate: _date_Accept: application/jsonContent-Type: application/jsonContent-Length: _length_{"Spread-Width":16,"Safety":4,"Hierarchy-Rules":[ false , true , true ],"Max-Superblock-Size":33554432,"Safety-Strategy":[ "RepairSpread" , "DynamicSafety" ],"N-Messages":4096"Full-Copy":false}
or:PUT /manage/policy?meta=json HTTP/1.1Host: my.example.comDate: _date_Content-Length: _length_{"Spread-Width":16,
Lattus REST API User’s Guide 71
Chapter 5: Policy OperationsCreate Policy
"Safety":4,"Hierarchy-Rules":[ false , true , true ],"Max-Superblock-Size":33554432,"Safety-Strategy":[ "RepairSpread" , "DynamicSafety" ],"N-Messages":4096"Full-Copy":false}
RESPONSE /manage/policy
HTTP/1.1 201 CreatedDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/json;charset=UTF-8Content-Length: _length_{"Id":"9adea14a85004c44883301dca8864b99","Spread-Width":16,"Safety":4,"Hierarchy-Rules":[false,true,true],"Max-Superblock-Size":33554432,"Safety-Strategy":["RepairSpread","DynamicSafety"],"N-Messages":4096"Full-Copy":false}
xml REQUEST /manage/policy
PUT /policy HTTP/1.1Host: my.example.comDate: _date_Accept: application/xmlContent-Type: application/xmlContent-Length: _length_<?xml version="1.0" encoding="UTF-8"?>
72 Lattus REST API User’s Guide
Chapter 5: Policy OperationsCreate Policy
<Policy xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Spread-Width>16</Spread-Width><Safety>4</Safety><Hierarchy-Rules>false true true</Hierarchy-Rules><Max-Superblock-Size>33554432</Max-Superblock-Size><Safety-Strategy>RepairSpread DynamicSafety</Safety-Strategy><N-Messages>4096</N-Messages><Full-Copy>false</Full-Copy></Policy>
or:PUT /manage/policy?meta=xml HTTP/1.1Host: my.example.comDate: _date_Content-Length: _length_<?xml version="1.0" encoding="UTF-8"?><Policy xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Spread-Width>16</Spread-Width><Safety>4</Safety><Hierarchy-Rules>false true true</Hierarchy-Rules><Max-Superblock-Size>33554432</Max-Superblock-Size><Safety-Strategy>RepairSpread DynamicSafety</Safety-Strategy><N-Messages>4096</N-Messages><Full-Copy>false</Full-Copy></Policy>
RESPONSE /manage/policy
HTTP/1.1 201 CreatedDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/xml;charset=UTF-8Content-Length: _length_<?xml version="1.0" encoding="UTF-8"?>
Lattus REST API User’s Guide 73
Chapter 5: Policy OperationsGet Policy
<Policy xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Id>9adea14a85004c44883301dca8864b99</Id><Spread-Width>16</Spread-Width><Safety>4</Safety><Hierarchy-Rules>false true true</Hierarchy-Rules><Max-Superblock-Size>33554432</Max-Superblock-Size><Safety-Strategy>RepairSpread DynamicSafety</Safety-Strategy><N-Messages>4096</N-Messages><Full-Copy>false</Full-Copy></Policy>
Get Policy
A GET request on /policy/<policy id> returns the configuration details of the policy with <policy id> as ID. A HEAD request yields the same response as a GET request, except that the server does not return a message-body in the response.
http / text REQUEST /manage/policy/<policy id>
GET /manage/policy/d27a3bc66f7e3c8e957cdaa01d5eacc1 HTTP/1.1Host: my.example.comDate: _date_Accept: text/plain
or:GET /manage/policy/d27a3bc66f7e3c8e957cdaa01d5eacc1?meta=http HTTP/1.1Host: my.example.comDate: _date_
74 Lattus REST API User’s Guide
Chapter 5: Policy OperationsGet Policy
RESPONSE /policy/<policy id>
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: 0X-Ampli-Spread-Width: 16X-Ampli-Safety: 4X-Ampli-Hierarchy-Rules: [ false , true , true]X-Ampli-Max-Superblock-Size: 33554432X-Ampli-Safety-Strategy: [ "RepairSpread" , "DynamicSafety" ]X-Ampli-N-Messages: 4096X-Ampli-Full-Copy: false
json REQUEST /manage/policy/<policy id>
GET /manage/policy/d27a3bc66f7e3c8e957cdaa01d5eacc1 HTTP/1.1Host: my.example.comDate: _date_Accept: application/json
or:GET /manage/policy/d27a3bc66f7e3c8e957cdaa01d5eacc1?meta=json HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /policy/<policy id>
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/json;charset=UTF-8
Lattus REST API User’s Guide 75
Chapter 5: Policy OperationsGet Policy
Content-Length: _length_{"Id":"9adea14a85004c44883301dca8864b99","Spread-Width":16,"Safety":4,"Hierarchy-Rules":[ false , true , true ],"Max-Superblock-Size":33554432,"Safety-Strategy":[ "RepairSpread" , "DynamicSafety" ],"N-Messages":4096"Full-Copy":false}
xml REQUEST /manage/policy/<policy id>
GET /manage/policy/d27a3bc66f7e3c8e957cdaa01d5eacc1 HTTP/1.1Host: my.example.comDate: _date_Accept: application/xml
or:GET /manage/policy/d27a3bc66f7e3c8e957cdaa01d5eacc1?meta=xml HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/policy/<policy id>
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Length: _length_Content-Type: application/xml;charset=UTF-8<?xml version="1.0" encoding="UTF-8"?><Policy xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Id>9adea14a85004c44883301dca8864b99</Id>
76 Lattus REST API User’s Guide
Chapter 5: Policy OperationsList Policies
<Spread-Width>16</Spread-Width><Safety>4</Safety><Hierarchy-Rules>false true true</Hierarchy-Rules><Max-Superblock-Size>33554432</Max-Superblock-Size><Safety-Strategy>RepairSpread DynamicSafety</Safety-Strategy><N-Messages>4096</N-Messages><Full-Copy>false</Full-Copy></Policy>
List Policies
A GET performed on /policy directly returns a list of all policy ids.
A HEAD request yields the same response as a GET request, except that the server does not return a message-body in the response.
http / text REQUEST /manage/policy/
GET /manage/policy/ HTTP/1.1Host: my.example.comDate: _date_Accept: text/plain
or:GET /manage/policy/?list=http HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/policy/
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24
Lattus REST API User’s Guide 77
Chapter 5: Policy OperationsList Policies
Content-Type: text/plain;charset=UTF-8Content-Length: _length_"d27a3bc66f7e3c8e957cdaa01d5eacc1""9f5fb1b3de6e314ca093cd613d6b383a"
json REQUEST /manage/policy/
GET /manage/policy/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/json
or:GET /manage/policy/?list=json HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/policy/
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/json;charset=UTF-8Content-Length: _length_["d27a3bc66f7e3c8e957cdaa01d5eacc1","9f5fb1b3de6e314ca093cd613d6b383a"]
xml REQUEST /manage/policy/
GET /manage/policy/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/xml
78 Lattus REST API User’s Guide
Chapter 5: Policy OperationsDelete Policy
or:GET /manage/policy/?list=xml HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/policy/
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/xml;charset=UTF-8Content-Length: _length_<Policies xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Id>d27a3bc66f7e3c8e957cdaa01d5eacc1</Id><Id>9f5fb1b3de6e314ca093cd613d6b383a</Id></Policies>
Delete Policy
Deleting policies is not possible.
Lattus REST API User’s Guide 79
Chapter 5: Policy OperationsDelete Policy
80 Lattus REST API User’s Guide
Chapter 6Namespaces
User Operations
List Namespaces
A GET can be performed on /namespace/ directly and returns a list of all namespace names the current authenticated user has (any) permissions on. An administrator is able to retrieve the list of all namespace names, regardless of the permissions on the namespaces.
A HEAD request yields the same response as a GET request, except that the server does not return a message-body in the response.
http/text REQUEST /namespace/
GET /namespace/ HTTP/1.1Host: my.example.comDate: _date_ Accept: text/plain
or:GET /namespace/?list=http HTTP/1.1
Lattus REST API User’s Guide 81
Chapter 6: NamespacesList Namespaces
Host: my.example.comDate: _date_
RESPONSE /namespace/
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: _length_"VOL"
json REQUEST /namespace/
GET /namespace/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/json
or:GET /namespace/?list=json HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /namespace/
HTTP/1.1 200 OK Date: _date_Server: Quantum-Lattus/3.0.3-ba79602e871550668391a673fa57589cf531497eContent-Type: application/json;charset=UTF-8Content-LengHTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/json;charset=UTF-8Content-Length: _length_["VOL",]
82 Lattus REST API User’s Guide
Chapter 6: NamespacesList Objects in a Namespace
xml REQUEST /namespace/
GET /namespace/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/xml
or:GET /namespace/?list=xml HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /namespace/
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/xml;charset=UTF-8Content-Length: _length_<Namespaces xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Name>VOL</Name></Namespaces>
List Objects in a Namespace
A GET can be performed on /namespace/<namespace name> and returns a list of objects in the root directory of a specific namespace (see the Object API for details).
A HEAD request yields the same response as a GET request, except that the server does not return a message-body in the response.
http/text REQUEST /namespace/<namespace name>
GET /namespace/VOL HTTP/1.1
Lattus REST API User’s Guide 83
Chapter 6: NamespacesList Objects in a Namespace
Host: my.example.comDate: _date_ Accept: text/plain
or:GET /namespace/VOL?list=http HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /namespace/<namespace name>
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: _length_"file1""file2"
json REQUEST /namespace/<namespace name>
GET /namespace/VOL HTTP/1.1Host: my.example.comDate: _date_Accept: application/json
or:GET /namespace/VOL?list=json HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /namespace/<namespace name>
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/json;charset=UTF-8Content-Length: _length_["file1","file2"
84 Lattus REST API User’s Guide
Chapter 6: NamespacesList Objects in a Namespace
]
xml REQUEST /namespace/<namespace name>
GET /namespace/VOL HTTP/1.1Host: my.example.comDate: _date_Accept: application/xml
or:GET /namespace/VOL?list=xml HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /namespace/<namespace name>
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/xml;charset=UTF-8Content-Length: _length_<Directory-Entriesxmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Name>file1</Name><Name>file2</Name></Directory-Entries>
Lattus REST API User’s Guide 85
Chapter 6: NamespacesCreate Namespace
Admin Operations
Create Namespace
A POST/PUT request creates a NameSpace. Creating a NameSpace requires the following parameters:
• Name: Unique name of the NameSpace (string)
• Policy-Id: Unique policy GUID that represents the protection policy (string)
If the namespace is successfully created, the return status is "201 Created". It then returns the metadata of the created Namespace.
If a namespace with the same name already exists, the return status is "403 Forbidden", and no metadata is returned.
http/text REQUEST /manage/namespace
PUT /manage/namespace HTTP/1.1Host: my.example.comDate: _date_Accept: text/plainContent-Type: text/plainContent-Length: 0X-Ampli-Name: "VOL"X-Ampli-Policy-Id: "d27a3bc66f7e3c8e957cdaa01d5eacc1"X-Ampli-Syncstore-Id:"a5069e6108ed48de933904a1d87eb925"X-Ampli-Master-Node-Id: 4X-Ampli-Verification-Interval: 15552000X-Ampli-Verification-Target-Date: 15552000X-Ampli-Small-Files-Policy-Id: "7c76bb4ca67f4e07a2a47db280c2a32f"X-Ampli-Small-Files-Threshold: 50000
or:PUT /manage/namespace?meta=http HTTP/1.1Host: my.example.com
86 Lattus REST API User’s Guide
Chapter 6: NamespacesCreate Namespace
Date: _date_Content-Length: 0X-Ampli-Name: "VOL"X-Ampli-Policy-Id: "d27a3bc66f7e3c8e957cdaa01d5eacc1"X-Ampli-Syncstore-Id: "a5069e6108ed48de933904a1d87eb925"X-Ampli-Master-Node-Id: 4X-Ampli-Verification-Interval: 15552000X-Ampli-Verification-Target-Date: 15552000X-Ampli-Small-Files-Policy-Id: "7c76bb4ca67f4e07a2a47db280c2a32f"X-Ampli-Small-Files-Threshold: 50000
RESPONSE /manage/namespace
HTTP/1.1 201 CreatedDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: 0X-Ampli-Id: "fbee9730697b43458e705d10bc6b8c94"X-Ampli-Name: "VOL"X-Ampli-Policy-Id: "d27a3bc66f7e3c8e957cdaa01d5eacc1"X-Ampli-Syncstore-Id: "a5069e6108ed48de933904a1d87eb925"X-Ampli-Master-Node-Id: 4X-Ampli-Namespace-Location-Id: 0X-Ampli-Verification-Interval: 15552000X-Ampli-Verification-Target-Date: _epoch_time_X-Ampli-Creation-Date: _epoch_time_X-Ampli-Small-Files-Policy-Id: "7c76bb4ca67f4e07a2a47db280c2a32f"X-Ampli-Small-Files-Threshold: 50000
json REQUEST /manage/namespace
PUT /manage/namespace HTTP/1.1Host: my.example.comDate: _date_Accept: application/json
Lattus REST API User’s Guide 87
Chapter 6: NamespacesCreate Namespace
Content-Type: application/json Content-Length: _length_
{"Name":"VOL""Policy-Id":"d27a3bc66f7e3c8e957cdaa01d5eacc1" "Syncstore-Id":"a5069e6108ed48de933904a1d87eb925","Master-Node-Id":4,"Verification-Interval":15552000,"Verification-Target-Date":15552000,"Small-Files-Policy-Id":"68484ee04c57449abf1dffcb214a322b","Small-Files-Threshold":50000}
or:PUT /manage/namespace?meta=json HTTP/1.1Host: my.example.comDate: _date_Content-Length: _length_
{"Name":"VOL""Policy-Id":"d27a3bc66f7e3c8e957cdaa01d5eacc1""Syncstore-Id":"a5069e6108ed48de933904a1d87eb925","Master-Node-Id":4,"Verification-Interval":15552000,"Verification-Target-Date":15552000,"Small-Files-Policy-Id":"68484ee04c57449abf1dffcb214a322b","Small-Files-Threshold":50000}
RESPONSE /manage/namespace
HTTP/1.1 201 CreatedDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/json;charset=UTF-8Content-Length: _length_{"Id":"fbee9730697b43458e705d10bc6b8c94""Name":"VOL"
88 Lattus REST API User’s Guide
Chapter 6: NamespacesCreate Namespace
"Policy-Id":"d27a3bc66f7e3c8e957cdaa01d5eacc1""Syncstore-Id":"a5069e6108ed48de933904a1d87eb925""Master-Node-Id":4"Namespace-Location-Id":0"Verification-Interval":15552000"Verification-Target-Date":_epoch_time_"Creation-Date":_epoch_time_,"Small-Files-Policy-Id":"68484ee04c57449abf1dffcb214a322b","Small-Files-Threshold":50000}
xml REQUEST /manage/namespace
PUT /manage/namespace HTTP/1.1Host: my.example.comDate: _date_Accept: application/xml Content-Type: application/xml Content-Length: _length_
<?xml version="1.0" encoding="UTF-8"?><Namespace xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Name>VOL</Name><Policy-Id>d27a3bc66f7e3c8e957cdaa01d5eacc1</Policy-Id><Syncstore-Id>a5069e6108ed48de933904a1d87eb925</Syncstore-Id><Master-Node-Id>4</Master-Node-Id><Verification-Interval>15552000</Verification-Interval><Verification-Target-Date>_epoch_time_</Verification-Target-Date><Small-Files-Policy-Id>c81af402571b45c19096617c5a8fe461</Small-Files-Policy-Id><Small-Files-Threshold>50000</Small-Files-Threshold></Namespace>
or:PUT /manage/namespace?meta=xml HTTP/1.1Host: my.example.comDate: _date_
Lattus REST API User’s Guide 89
Chapter 6: NamespacesCreate Namespace
Content-Length: _length_
<?xml version="1.0" encoding="UTF-8"?><Namespace xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Name>VOL</Name><Policy-Id>d27a3bc66f7e3c8e957cdaa01d5eacc1</Policy-Id><Syncstore-Id>a5069e6108ed48de933904a1d87eb925</Syncstore-Id><Master-Node-Id>4</Master-Node-Id><Verification-Interval>15552000</Verification-Interval><Verification-Target-Date>_epoch_time_</Verification-Target-Date><Small-Files-Policy-Id>c81af402571b45c19096617c5a8fe461</Small-Files-Policy-Id><Small-Files-Threshold>50000</Small-Files-Threshold></Namespace>
RESPONSE /manage/namespace
HTTP/1.1 201 CreatedDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/xml;charset=UTF-8Content-Length: _length_<?xml version="1.0" encoding="UTF-8"?><Namespace xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Id>fbee9730697b43458e705d10bc6b8c94</Id><Name>VOL</Name><Policy-Id>d27a3bc66f7e3c8e957cdaa01d5eacc1</Policy-Id><Syncstore-Id>a5069e6108ed48de933904a1d87eb925</Syncstore-Id><Master-Node-Id>4</Master-Node-Id><Namespace-Location-Id>0</Namespace-Location-Id><Verification-Interval>15552000</Verification-Interval><Verification-Target-Date>_epoch_time_</Verification-Target-Date><Creation-Date>_epoch_time_</Creation-Date>
90 Lattus REST API User’s Guide
Chapter 6: NamespacesUpdate Namespace
<Small-Files-Policy-Id>c81af402571b45c19096617c5a8fe461</Small-Files-Policy-Id><Small-Files-Threshold>50000</Small-Files-Threshold></Namespace>
Update Namespace
With a PUT request, you can also modify a namespace. However, at this time you can modify only the encryption policy.
http/text To enable encryption for a namespace/bucket using a specific encryption policy, set the desired encryption policy ID.
To disable encryption, set the encryption policy to “none”.
REQUEST /manage/namespace/<namespace name>
PUT /manage/namespace/<namespace name> HTTP/1.1Host: my.example.comDate: _date_Accept: text/plainContent-Type: text/plainContent-Length: 0X-Ampli-Encryption-Policy: "1"
or:PUT /manage/namespace/<namespace name>?meta=http HTTP/1.1Host: my.example.comDate: _date_Content-Length: 0X-Ampli-Encryption-Policy: "1"
RESPONSE /manage/namespace/<namespace name>
HTTP/1.1 200 OK
Lattus REST API User’s Guide 91
Chapter 6: NamespacesUpdate Namespace
Date: _date_Server: Quantum-Lattus/3.0.0Content-Type: text/plain;charset=UTF-8Content-Length: 0X-Ampli-Id: "fbee9730697b43458e705d10bc6b8c94"X-Ampli-Name: _namespace name_X-Ampli-Policy-Id: "d27a3bc66f7e3c8e957cdaa01d5eacc1"X-Ampli-Syncstore-Id: "a5069e6108ed48de933904a1d87eb925"X-Ampli-Master-Node-Id: 4X-Ampli-Namespace-Location-Id: 0X-Ampli-Verification-Interval: 15552000X-Ampli-Verification-Target-Date: _epoch_time_X-Ampli-Creation-Date: _epoch_time_X-Ampli-Small-Files-Policy-Id: "7c76bb4ca67f4e07a2a47db280c2a32f"X-Ampli-Small-Files-Threshold: 50000X-Ampli-Encryption-Policy: "1"X-Ampli-Encryption-Scheme: "AES-256-CTR"
Note: The ‘‘X-Ampli-Encryption-Scheme header cannot be modified or user specified. Only one scheme is currently supported: AES-256-CTR.
json To enable encryption for a namespace/bucket using a specific encryption policy, set the desired encryption policy ID.
To disable encryption, set the encryption policy to “null”.
REQUEST /manage/namespace/<namespace>
PUT /manage/namespace/<namespace name> HTTP/1.1Host: my.example.comDate: _date_Accept: application/jsonContent-Type: application/jsonContent-Length: _length_{"Encryption-Policy":"1"}
92 Lattus REST API User’s Guide
Chapter 6: NamespacesUpdate Namespace
or:PUT /manage/namespace/<namespace name>?meta=json HTTP/1.1Host: my.example.comDate: _date_Content-Length: _length_{"Encryption-Policy":"1"}
RESPONSE /manage/namespace
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0Content-Type: application/json;charset=UTF-8Content-Length: 471{"Id":"8960a887ce4b441ebc22f9843877a355","Name":"VOL","Policy-Id":"b0d8e3dd5b674ef1b3bd09185198c270","Syncstore-Id":"701d5929a46f4bd49e8aa7627ecd62d8","Master-Node-Id":1,"Namespace-Location-Id":0,"Verification-Interval":15552000,"Verification-Target-Date":1385465338,"Creation-Date":1369913338,"Small-Files-Policy-Id":"796e6685ef534a88945071b9b3222d80","Small-Files-Threshold":1048576,"Encryption-Policy":"1","Encryption-Scheme":"AES-256-CTR"}
xml To enable encryption for this namespace/bucket, using a specific encryption policy: <Encryption-Policy>1</Encryption-Policy>
To disable encryption for this name space/bucket: <Encryption-Policy></Encryption-Policy> or simply <Encryption-Policy/>
Lattus REST API User’s Guide 93
Chapter 6: NamespacesUpdate Namespace
REQUEST /manage/namespace<name space>
PUT /manage/namespace/<namespace name> HTTP/1.1Host: my.example.comDate: _date_Accept: application/xmlContent-Type: application/xmlContent-Length: _length_<?xml version="1.0" encoding="UTF-8"?><Namespace xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Encryption-Policy>1</Encryption-Policy></Namespace>
or:PUT /manage/namespace/<namespace name>?meta=xml HTTP/1.1Host: my.example.comDate: _date_Content-Length: _length_<?xml version="1.0" encoding="UTF-8"?><Namespace xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Encryption-Policy>1</Encryption-Policy></Namespace>
RESPONSE /manage/namespace
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0Content-Type: application/xml;charset=UTF-8Content-Length: _length_<?xml version="1.0" encoding="UTF-8"?><Namespace xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Id>8960a887ce4b441ebc22f9843877a355</Id><Name>VOL</Name><Policy-Id>b0d8e3dd5b674ef1b3bd09185198c270</Policy-Id><Syncstore-Id>701d5929a46f4bd49e8aa7627ecd62d8</Syncstore-Id><Master-Node-Id>1</Master-Node-Id><Namespace-Location-Id>0</Namespace-Location-Id>
94 Lattus REST API User’s Guide
Chapter 6: NamespacesGet Namespace
<Verification-Interval>15552000</Verification-Interval><Verification-Target-Date>1385465338</Verification-Target-Date><Creation-Date>1369913338</Creation-Date><Small-Files-Policy-Id>796e6685ef534a88945071b9b3222d80</Small-Files-Policy-Id><Small-Files-Threshold>1048576</Small-Files-Threshold><Encryption-Policy>1</Encryption-Policy><Encryption-Scheme>AES-256-CTR</Encryption-Scheme></Namespace>
Get Namespace
A GET request on /manage/namespace/<namespace name> returns (depending on the query parameters) a list of objects in the root directory of a specific namespace or the metadata of that specific namespace. See the Object API for details.
A HEAD request yields the same response as a GET request, except that the server does not return a message-body in the response.
http/text REQUEST /manage/namespace/<namespace name>
GET /namespace/VOL?meta= HTTP/1.1Host: my.example.comDate: _date_ Accept: text/plain
or:GET /namespace/VOL?meta=http HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/namespace/<namespace name>
HTTP/1.1 200 OKDate: _date_
Lattus REST API User’s Guide 95
Chapter 6: NamespacesGet Namespace
Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: 0X-Ampli-Id: "fbee9730697b43458e705d10bc6b8c94"X-Ampli-Name: "VOL"X-Ampli-Policy-Id: "d27a3bc66f7e3c8e957cdaa01d5eacc1"X-Ampli-Syncstore-Id: "a5069e6108ed48de933904a1d87eb925"X-Ampli-Master-Node-Id: 4X-Ampli-Namespace-Location-Id: 0X-Ampli-Verification-Interval: 15552000X-Ampli-Verification-Target-Date: _epoch_time_X-Ampli-Creation-Date: _epoch_time_X-Ampli-Small-Files-Policy-Id: "c81af402571b45c19096617c5a8fe461"X-Ampli-Small-Files-Threshold: 50000
json REQUEST /manage/namespace/<namespace name>
GET /manage/namespace/VOL?meta= HTTP/1.1Host: my.example.comDate: _date_Accept: application/json
or:GET /manage/namespace/VOL?meta=json HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/namespace/<namespace name>
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/json;charset=UTF-8Content-Length: _length_{"Id":"fbee9730697b43458e705d10bc6b8c94""Name":"VOL""Policy-Id":"d27a3bc66f7e3c8e957cdaa01d5eacc1"
96 Lattus REST API User’s Guide
Chapter 6: NamespacesGet Namespace
"Syncstore-Id":"a5069e6108ed48de933904a1d87eb925""Master-Node-Id":4"Namespace-Location-Id":0"Verification-Interval":15552000"Verification-Target-Date":_epoch_time_X-Ampli-Creation-Date: _epoch_time_X-Ampli-Small-Files-Policy-Id: "c81af402571b45c19096617c5a8fe461"X-Ampli-Small-Files-Threshold: 50000}
xml REQUEST /manage/namespace/<namespace name>
GET /manage/namespace/VOL?meta= HTTP/1.1Host: my.example.comDate: _date_Accept: application/xml
or:GET /manage/namespace/VOL?meta=xml HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/namespace/<namespace name>
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Length: _length_Content-Type: application/xml;charset=UTF-8<?xml version="1.0" encoding="UTF-8"?><Namespace xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Id>fbee9730697b43458e705d10bc6b8c94</Id><Name>VOL</Name><Policy-Id>d27a3bc66f7e3c8e957cdaa01d5eacc1</Policy-Id><Syncstore-Id>a5069e6108ed48de933904a1d87eb925</Syncstore-Id><Master-Node-Id>4</Master-Node-Id><Namespace-Location-Id>0</Namespace-Location-Id><Verification-Interval>15552000</Verification-Interval>
Lattus REST API User’s Guide 97
Chapter 6: NamespacesList Namespaces
<Verification-Target-Date>_epoch_time_</Verification-Target-Date>X-Ampli-Creation-Date: _epoch_time_X-Ampli-Small-Files-Policy-Id: "c81af402571b45c19096617c5a8fe461"X-Ampli-Small-Files-Threshold: 50000</Namespace>
List Namespaces
A GET performed on /manage/namespace/ returns a list of all namespaces, regardless of the permissions on the namespaces and whether the client side GET request has a digest user specified.
A HEAD request yields the same response as a GET request, except that the server does not return a message-body in the response.
http/text REQUEST /manage/namespace/
GET /manage/namespace/ HTTP/1.1Host: my.example.comDate: _date_ Accept: text/plain
or:GET /manage/namespace/?list=http HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/namespace/
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: _length_"VOL""VOL2"
98 Lattus REST API User’s Guide
Chapter 6: NamespacesList Namespaces
json REQUEST /manage/namespace/
GET /namespace/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/json
or:GET /manage/namespace/?list=json HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/namespace/
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/json;charset=UTF-8Content-Length: _length_["VOL","VOL2",]
xml REQUEST /manage/namespace/
GET /manage/namespace/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/xml
or:GET /manage/namespace/?list=xml HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/namespace/
HTTP/1.1 200 OKDate: _date_
Lattus REST API User’s Guide 99
Chapter 6: NamespacesDelete Namespace
Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/xml;charset=UTF-8Content-Length: _length_<Namespaces xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Name>VOL</Name><Name>VOL2</Name></Namespaces>
Delete Namespace
REQUEST /manage/namespace/<namespace name>
DELETE /manage/namespace/VOL HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/namespace/<namespace name>μ
HTTP/1.1 204 No ContentDate: _date_Server: Amplidata-AmpliStor/2.5.0-1f206c99fa6c1df2070e69741a6c0c6d52353ec9Content-Length: 0
100 Lattus REST API User’s Guide
Chapter 6: NamespacesGET/HEAD
Capacity Reporting
GET/HEAD
To retrieve a batched list of namespace capacity reports, perform a GET on either:
• /manage/capacity/namespace
• /s3manage/capacity/bucket
To retrieve a batched list of syncstore id's, a GET can be performed on either:
• /manage/capacity/syncstore/
• /s3manage/capacity/syncstore/
To retrieve the capacity report of a specific syncstore, perform a GET on either:
• /manage/capacity/syncstore/<syncstore id>
• /s3manage/capacity/syncstore/<syncstore id>
A HEAD request returns the same response as a GET request, but the server does not return a message-body in the response.
List Namespace Capacity Reports
By default, only the first 50 namespace capacity reports are returned. This limit can be adjusted to X by appending the query parameter "limit=X" to the URL.
Note: The maximum value for this limit is 1,023. Higher values will result in an HTTP 500 Error for AXR or in at most 1000 entries for S3.
Lattus REST API User’s Guide 101
Chapter 6: NamespacesList Namespace Capacity Reports
The following example url can be used:
/manage/capacity/namespace/?limit=500
In addition, the namespace from which the list should start can be specified using "marker=Y". For example, the following URL could be used to list 4 namespace capacity reports starting from namespace "test": /manage/user?marker=test&limit=4. An additional query parameter, "include_marker=false" can be added, which results in the marker being not included in the list.
json REQUEST /manage/capacity/namespace/
GET /manage/capacity/namespace/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/json
or:GET /manage/capacity/namespace/?list=json HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/capacity/namespace/
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Type: application/json;charset=UTF-8Content-Length: 1201Pragma: no-cacheCache-Control: no-cacheExpires: Thu, 01 Jan 1970 00:00:00 GMT{"name":"NS1","start_date":1400154451.88,"last_update":1400154452.01,"current_policy_id_stats":[{"Id":"8e8b395a79d941e7b0d9a0e9476babeb","Statistics":{"nr_objects_ok":4,"nr_superblocks_ok":4,"nr_objects_repair":0,"nr_superblocks_repair":0,
102 Lattus REST API User’s Guide
Chapter 6: NamespacesList Namespace Capacity Reports
"nr_objects_delete":0,"nr_superblocks_delete":0,"nr_objects_unverified":0,"nr_superblocks_unverified":0,"capacity_frontend_ok":30788,"capacity_frontend_repair":0,"capacity_frontend_delete":0,"capacity_frontend_objects_unverified":0,"capacity_frontend_superblocks_unverified":0,"disksafety_objects":[ {"disk safety":4, "count":4} ],"disksafety_superblocks":[ {"disk safety":4, "count":4} ],"disksafety_objects_offline":[ {"disk safety":4, "count":4} ],"disksafety_superblocks_offline":[ {"disk safety":4, "count":4} ],"disksafety_objects_decommissioned":[ {"disk safety":4, "count":4} ],"disksafety_superblocks_decommissioned":[ {"disk safety":4, "count":4} ],"policy_stats_hashtbl":{"nr" : {"object" : {"ok" : 4,"all" : 4,"repair" : 0,"change policy" : 0,"unverified" : 0,"disk safety normal" : { "4" : 4 },"disk safety offline" : { "4" : 4 },"disk safety decommissioned" : { "4" : 4 },"needs conversion" : 0 },"part" : {"ok" : 4,"all" : 4,"repair" : 0,"change policy" : 0,"unverified" : 0,"disk safety normal" : { "4" : 4 },"disk safety offline" : { "4" : 4 },"disk safety decommissioned" : { "4" : 4 },"needs conversion" : 0 },"superblock" : {
Lattus REST API User’s Guide 103
Chapter 6: NamespacesList Namespace Capacity Reports
"ok" : 4,"all" : 4,"repair" : 0,"change policy" : 0,"unverified" : 0,"disk safety normal" : { "4" : 4 },"disk safety offline" : { "4" : 4 },"disk safety decommissioned" : { "4" : 4 },"needs conversion" : 0 } },"capacity" : {"object" : {"ok" : 30788,"all" : 30788,"repair" : 0,"change policy" : 0,"unverified" : 0,"disk safety normal" : { "4" : 30788 },"disk safety offline" : { "4" : 30788 },"disk safety decommissioned" : { "4" : 30788 },"needs conversion" : 0 },"part" : {"ok" : 30788,"all" : 30788,"repair" : 0,"change policy" : 0,"unverified" : 0,"disk safety normal" : { "4" : 30788 },"disk safety offline" : { "4" : 30788 },"disk safety decommissioned" : { "4" : 30788 },"needs conversion" : 0 },"superblock" : {"ok" : 30788,"all" : 30788,"repair" : 0,"change policy" : 0,"unverified" : 0,"disk safety normal" : { "4" : 30788 },"disk safety offline" : { "4" : 30788 },"disk safety decommissioned" : { "4" : 30788 },"needs conversion" : 0 }}}}
104 Lattus REST API User’s Guide
Chapter 6: NamespacesList Namespace Capacity Reports
},{"Id":"e10bf033967e46baa88909d521cdd963","Statistics":{"nr_objects_ok":5,"nr_superblocks_ok":201,"nr_objects_repair":0,"nr_superblocks_repair":0,"nr_objects_delete":0,"nr_superblocks_delete":0,"nr_objects_unverified":0,"nr_superblocks_unverified":0,"capacity_frontend_ok":13309621000,"capacity_frontend_repair":0,"capacity_frontend_delete":0,"capacity_frontend_objects_unverified":0,"capacity_frontend_superblocks_unverified":0,"disksafety_objects":[ {"disk safety":4, "count":5} ],"disksafety_superblocks":[ {"disk safety":4, "count":201} ],"disksafety_objects_offline":[ {"disk safety":4, "count":5} ],"disksafety_superblocks_offline":[ {"disk safety":4, "count":201} ],"disksafety_objects_decommissioned":[ {"disk safety":4, "count":5} ],"disksafety_superblocks_decommissioned":[ {"disk safety":4, "count":201} ],"policy_stats_hashtbl":{"nr" : {"object" : {"ok" : 5,"all" : 5,"repair" : 0,"change policy" : 0,"unverified" : 0,"disk safety normal" : { "4" : 5 },"disk safety offline" : { "4" : 5 },"disk safety decommissioned" : { "4" : 5 },"needs conversion" : 0 },"part" : {"ok" : 5,"all" : 5,"repair" : 0,
Lattus REST API User’s Guide 105
Chapter 6: NamespacesList Namespace Capacity Reports
"change policy" : 0,"unverified" : 0,"disk safety normal" : { "4" : 5 },"disk safety offline" : { "4" : 5 },"disk safety decommissioned" : { "4" : 5 },"needs conversion" : 0 },"superblock" : {"ok" : 201,"all" : 201,"repair" : 0,"change policy" : 0,"unverified" : 0,"disk safety normal" : { "4" : 201 },"disk safety offline" : { "4" : 201 },"disk safety decommissioned" : { "4" : 201 },"needs conversion" : 0 }},"capacity" : {"object" : {"ok" : 13309621000,"all" : 13309621000,"repair" : 0,"change policy" : 0,"unverified" : 0,"disk safety normal" : { "4" : 13309621000 },"disk safety offline" : { "4" : 13309621000 },"disk safety decommissioned" : { "4" : 13309621000 },"needs conversion" : 0 },"part" : {"ok" : 13309621000,"all" : 13309621000,"repair" : 0,"change policy" : 0,"unverified" : 0,"disk safety normal" : { "4" : 13309621000 },"disk safety offline" : { "4" : 13309621000 },"disk safety decommissioned" : { "4" : 13309621000 },"needs conversion" : 0 },"superblock" : {"ok" : 13309621000,"all" : 13309621000,"repair" : 0,"change policy" : 0,
106 Lattus REST API User’s Guide
Chapter 6: NamespacesList Namespace Capacity Reports
"unverified" : 0,"disk safety normal" : { "4" : 13309621000 },"disk safety offline" : { "4" : 13309621000 },"disk safety decommissioned" : { "4" : 13309621000 },"needs conversion" : 0 }}}}} ],"old_policy_id_stats":[ ],"object_name_length_stats":{"name": "object_name_length_stats","average": 12.777778,"variance": 18.839506,"count": 9,"min": 5.000000,"max": 20.000000 }
• version: version of the format in which the data is stored. This information is of no further use for the user.
• name: name of the monitored name space
• start_date: date and time when the last monitor crawl was started
• last_update: last update of the cached data
• current_policy_id_stats: dict with policy guid as key and value the following information:
• nr_objects_ok: number of healthy objects
• nr_superblocks_ok: number of healthy superblocks
• nr_objects_repair: number of objects that have at least one superblock in repair status
• nr_superblocks_repair: number of superblocks that need repair
• nr_objects_delete: number of objects that need to be deleted
• nr_superblocks_delete: number of superblocks that still need to be deleted
• nr_objects_unverified: number of unverified objects. Every object is verified each 180 days if it is still a healthy object. If this verification would not have taken place within these 180 days, the object is considered as unverified.
Lattus REST API User’s Guide 107
Chapter 6: NamespacesList Namespace Capacity Reports
• nr_superblocks_unverified: number of unverified superblocks. This is similar as the unverified objects, but for superblocks.
• capacity_frontend_ok: the sum of the sizes of the objects put that are healthy, expressed in bytes
• capacity_frontend_repair: the sum of the sizes of the objects put that need to be repaired, expressed in bytes
• capacity_frontend_delete: the sum of the sizes of the objects put that still need to be deleted, expressed in bytes
• capacity_frontend_objects_unverified: the sum of the sizes of the objects put that is still unverified, expressed in bytes
• capacity_frontend_superblocks_unverified: the sum of the sizes of the superblocks put that is still unverified, expressed in bytes
• disksafety_objects: a dict with keys going from ‘disksafety - spread width’ to disksafety giving for all these values the number of objects that have that disk safety, taking into account the ABANDONED blockstores.
• disksafety_superblocks: a dict with keys going from ‘disksafety - spread width’ to disksafety giving for all these values the number of superblocks that have that disk safety, taking into account the ABANDONED blockstores
• disksafety_objects_offline: a dict with keys going from ‘disksafety - spread width’ to disksafety giving for all these values the number of objects that have that disk safety, taking into account the ABANDONED and OFFLINE blockstores
• disksafety_superblocks_offline: a dict with keys going from ‘disksafety - spread width’ to disksafety giving for all these values the number of superblocks that have that disk safety, taking into account the ABANDONED and OFFLINE blockstores
• disksafety_objects_decommissioned: a dict with keys going from ‘disksafety - spread width’ to disksafety giving for all these values the number of objects that have that disk safety, taking into account the ABANDONED, OFFLINE and DECOMMISSIONED blockstores
• disksafety_superblocks_decommissioned: a dict with keys going from ‘disksafety - spread width’ to disksafety giving for all these values the number of superblocks that have that disk
108 Lattus REST API User’s Guide
Chapter 6: NamespacesList Namespace Capacity Reports
safety, taking into account the ABANDONED,OFFLINE and DECOMMISSIONED blockstores
• policy_stats_hashtbl: overview with the statistics of all objects, parts, and superblocks with that policy as their target. This is a dict which contains two main sections, “nr” and “capacity”, respectively the number of items and the taken disk space of the items. These two main sections show the information by objects, parts, and superblocks.
Note: The Lattus parts are not exactly the same as the S3 parts. In S3, there is a flat structure of the parts, in Lattus you have a tree structure, where the main object (or storage object) consists of parts and each part can consist of child-parts.
• nr: gives you an overview of the number of items (objects, parts, superblocks)
• capacity: gives you an overview of the taken disk space, split in objects, parts, and superblocks. The indicated disk spaces are expressed in bytes.
• ok: all healthy items, which don’t have either label REPAIR, CHANGE_POLICY, or UNVERIFIED
• all: all items
• repair: all items which need a repair action
• change policy: items for which a new policy is selected, but which are not yet encoded with the new policy
• unverified: items which are unverified. Every item is verified each 180 days if it is still a healthy object. If this verification would not have taken place within these 180 days, the item is considered as unverified.
• disk safety normal: dict which takes the healthy blockstores into account. The disk safety is the key and the number of items/disk space is the value. This dict can contain multiple key/value pairs.
• disk safety offline: dict which takes the offline blockstores into account.
• needs conversion: remaining items/data volume that need conversion. This is a decreasing value and is ideally 0.
Lattus REST API User’s Guide 109
Chapter 6: NamespacesList Syncstores
• disk safety decommissioned: dict which takes the decommissioned blockstores into account.
• old_policy_id_stats: is no longer in use as of Lattus 3.4.0, and is replaced by the parameter change policy. It was used to report on the statistics of objects in a name space which were not yet encoded with the target policy. This parameter is still in use for compatibility reasons.
• object_name_length_stats: statistics about the length of the object names in the name space.
• average: average object name length
• count: number of object to calculate the average length
• variance: variance of the length
• max: length of the longest object name
• min: length of the shortest object name
• name: name of the statistic
You can see that there are two policy ID statistices. This occurs when the namespace uses a policy which has also an enabled small files policy.
List Syncstores
json REQUEST /manage/capacity/syncstore/
GET /manage/capacity/syncstore/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/json
or:GET /manage/capacity/syncstore/?list=json HTTP/1.1Host: my.example.comDate: _date_
110 Lattus REST API User’s Guide
Chapter 6: NamespacesGet Syncstore Capacity Report
RESPONSE /manage/capacity/syncstore
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Type: application/json;charset=UTF-8Content-Length: 78Pragma: no-cacheCache-Control: no-cacheExpires: Thu, 01 Jan 1970 00:00:00 GMT["160dfb46b98c481f9d18b3d98041017d","825a2f8627334b3bb8f480f63ef7f663"]
Get Syncstore Capacity Report
json REQUEST /manage/capacity/syncstore/<syncstore id>
GET /manage/capacity/syncstore/<syncstore id> HTTP/1.1Host: my.example.comAccept: application/json
or:GET /manage/capacity/syncstore/<syncstore id>?meta=json HTTP/1.1Host: my.example.com
RESPONSE /manage/capacity/syncstore/<syncstore id>
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/_revisionContent-Type: application/json;charset=UTF-8Content-Length: 58{"Id":"167da646596f4f83b6cb979cfd4c2afb","Count":5000}
Lattus REST API User’s Guide 111
Chapter 6: NamespacesGet Syncstore Capacity Report
112 Lattus REST API User’s Guide
Chapter 7Object Operations
Create or Overwrite Object
Use the PUT operation to add objects to a namespace. The Lattusenvironment always stores the complete object. An error code will be returned if a failure is encountered during the upload process.
By default the object is overwritten every time when a PUT action is processed on an existing object ID.
If multiple PUT actions are processed simultaneously on the same object, then the last successful PUT will define the final content of the object.
A POST/PUT request creates an object and returns the metadata of the object. On successful creation of an object, the return status is “201 Created”. On successful overwrite, the return status is “200 OK”.
For HTTP, the default is to allow overwrite. If any existing object should not be overwritten, the header “If-None-Match: *” can be added. See the If-None-Match sections for details.
Lattus also supports the concept of subdirectories. To create a file object in a subdirectory of a namespace, its parent directory must already have been created. Parent directories are not automatically created. See Create Directory Object on page 123 for more information.
Lattus REST API User’s Guide 113
Chapter 7: Object OperationsTransfer with Content-Length
Overwriting objects preserves creation date and preserves existing custom metadata.
There are two modes for uploading files:
• Chunked mode
• Non-Chunked mode
Chunked mode is used when the object size is not known up front. Data streaming can be achieved by uploading multiple chunks one after another. Non-chunked mode expects the content length of the object to be known up front.
Transfer with Content-Length
http REQUEST /namespace/<namespace name>/<object name>
PUT /namespace/namespace1/object1 HTTP/1.1Host: my.example.comDate: _date_ Accept: text/plainContent-Type: binary/octet-streamContent-Length: _object_length_
_object_length_bytes_of_data_or:PUT /namespace/namespace1/object1?meta=http HTTP/1.1Host: my.example.comDate: _date_Content-Type: binary/octet-streamContent-Length: _object_length_
_object_length_bytes_of_data_
RESPONSE /namespace/<namespace name>/<object name>
HTTP/1.1 201 CreatedDate: _date_
114 Lattus REST API User’s Guide
Chapter 7: Object OperationsTransfer with Content-Length
Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: 0Last-Modified: _date_ETag: "8a6547849714fa7f021a923e391a87a8"X-Ampli-Name: "object1"X-Ampli-Id: "6a4f803b1efc48ad9a93ab5e1ee8380e"X-Ampli-Namespace-Id: "9777e0eef27c44b7929806a8c42f1684"X-Ampli-Kind: "File"X-Ampli-Policy-Id: "d684bbd656004c50a29db47b67ae9666"X-Ampli-Size: _object_length_X-Ampli-Crc32: 3827519356X-Ampli-Status: OKX-Ampli-Creation-Date: _epoch_time_X-Ampli-Modification-Date: _epoch_time_X-Ampli-Verification-Date: _epoch_time_
json REQUEST /namespace/<namespace name>/<object name>
PUT /namespace/namespace1/object1 HTTP/1.1Host: my.example.comDate: _date_Accept: application/jsonContent-Type: binary/octet-streamContent-Length: _object_length_
_object_length_bytes_of_data_
or:PUT /namespace/namespace1/object1?meta=json HTTP/1.1Host: my.example.comDate: _date_Content-Type: binary/octet-streamContent-Length: _object_length_
_object_length_bytes_of_data_
RESPONSE /namespace/<namespace name>/<object name>
HTTP/1.1 201 Created
Lattus REST API User’s Guide 115
Chapter 7: Object OperationsTransfer with Content-Length
Date: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/json;charset=UTF-8Content-Length: _length_Last-Modified: _date_ETag: "8a6547849714fa7f021a923e391a87a8"{"Name": "object1""Id": "6a4f803b1efc48ad9a93ab5e1ee8380e""Namespace-Id": "9777e0eef27c44b7929806a8c42f1684""Kind":"File","Policy-Id": "d684bbd656004c50a29db47b67ae9666""Size": _object_length_"Crc32": 3827519356"Status": OK"Creation-Date": _epoch_time_"Modification-Date": _epoch_time_"Verification-Date": _epoch_time_}
xml REQUEST /namespace/<namespace name>/<object name>
PUT /namespace/namespace1/object1 HTTP/1.1Host: my.example.comDate: _date_Accept: application/xmlContent-Type: binary/octet-streamContent-Length: _object_length_
_object_length_bytes_of_data_
or:PUT /namespace/namespace1/object1?meta=xml HTTP/1.1Host: my.example.comDate: _date_Content-Type: binary/octet-streamContent-Length: _object_length_
_object_length_bytes_of_data_
116 Lattus REST API User’s Guide
Chapter 7: Object OperationsTransfer Without Content-Length
RESPONSE /namespace/<namespace name>/<object name>
HTTP/1.1 201 CreatedDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/xml;charset=UTF-8Content-Length: _length_Last-Modified: _date_ETag: "8a6547849714fa7f021a923e391a87a8"<Object xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Id>6a4f803b1efc48ad9a93ab5e1ee8380e"</Id><Name>object1</Name><Namespace-Id>9777e0eef27c44b7929806a8c42f1684"</Namespace-Id><Kind>File</Kind><Policy-Id>d684bbd656004c50a29db47b67ae9666</Policy-Id><Size>_object_length_</Size><Crc32>3827519356</Crc32><Status>OK</Status><Creation-Date>_epoch_time_</Creation-Date><Modification-Date>_epoch_time_</Modification-Date><Verification-Date>_epoch_time_</Verification-Date></Object>
Transfer Without Content-Length
The HTTP/1.1 specification provides a way to to PUT an object of which the size is not known beforehand, by repeatedly sending chunks of a number of bytes of content, and a final chunk of size 0 bytes.
For this purpose, the Transfer-Encoding header exists. See Transfer-Encoding on page 119 below for details.
Request Headers For the PUT of an object several request headers can be specified to:
• make the PUT conditional, or
Lattus REST API User’s Guide 117
Chapter 7: Object OperationsTransfer Without Content-Length
• modify the way in which the object content is streamed to the server.
Note: The examples below also work for other meta types than "meta=http".
If-Match This header is supported for the PUT of an object, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.24 "If-Match".
See also ETag on page 141.
The subsections contains an example: the ETag we received from the server for object "/namespace/namespace1/object1" in a previous response to a PUT request or GET request was "19d0a529e39347f4823c6e22f1160825". We wish to overwrite the object, but only if it has not been changed in the mean time. In this scenario, the object has been changed in the mean time, so the ETag no longer matches, and the server responds with a 412 error response.
REQUEST /namespace/<namespace name>/<object name>
PUT /namespace/namespace1/object1?meta=http HTTP/1.1Host: my.example.comIf-Match: "19d0a529e39347f4823c6e22f1160825" Content-Length: 19Expect: 100-continue
RESPONSE /namespace/<namespace name>/<object name>
HTTP/1.1 412 Precondition FailedDate: _date_Server: Quantum-Lattus/3.0.3-ba79602e871550668391a673fa57589cf531497e
If-Modified-Since This header is supported for PUT of an object, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.25 "If-Modified-Since". If this precondition is not satisfied, the response will be "412 Precondition Failed".
118 Lattus REST API User’s Guide
Chapter 7: Object OperationsTransfer Without Content-Length
See also the Last-Modified response header.
If-None-Match This header is supported for PUT of an object, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.26 "If-None-Match".
See also ETag on page 141.
The subsections contain an example where the object should only be put if no object with the specified name exists, and in this case, the object already exists.
REQUEST /namespace/<namespace name>/<object name>
PUT /namespace/namespace1/object1?meta=http HTTP/1.1Host: my.example.com If-None-Match: * Content-Length: 19Expect: 100-continue
RESPONSE /namespace/<namespace name>/<object name>
HTTP/1.1 412 Precondition FailedDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24
If-Unmodified-Since This header is supported for PUT of an object, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.28 "If-Unmodified-Since". If this precondition is not satisfied, the response will be "412 Precondition Failed".
See also the Last-Modified response header.
Transfer-Encoding Please see RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", sections 3.6.1 "Chunked Transfer Coding" and 19.4.6 "Introduction of Transfer-Encoding" for more information. The subsections include an example.
REQUEST /namespace/<namespace name>/<object name>
PUT /namespace/namespace1/object1 HTTP/1.1
Lattus REST API User’s Guide 119
Chapter 7: Object OperationsTransfer Without Content-Length
Host: my.example.comDate: _date_ Accept: text/plainContent-Type: binary/octet-streamTransfer-Encoding: chunked
1e50_7760_bytes_of_data_
a30_2608_bytes_of_data_
0
RESPONSE /namespace/<namespace name>/<object name>
HTTP/1.1 201 CreatedDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: 0Last-Modified: _date_ETag: "8a6547849714fa7f021a923e391a87a8"X-Ampli-Name: "object1"X-Ampli-Id: "6a4f803b1efc48ad9a93ab5e1ee8380e"X-Ampli-Namespace-Id: "9777e0eef27c44b7929806a8c42f1684"X-Ampli-Kind: "File"X-Ampli-Policy-Id: "d684bbd656004c50a29db47b67ae9666"X-Ampli-Size: _object_length_X-Ampli-Crc32: 3827519356X-Ampli-Status: OKX-Ampli-Creation-Date: _epoch_time_X-Ampli-Modification-Date: _epoch_time_X-Ampli-Verification-Date: _epoch_time_
120 Lattus REST API User’s Guide
Chapter 7: Object OperationsResponse Headers
Response Headers
ETag
This header is returned in responses to GET, HEAD, PUT and POST requests for objects, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.19 "ETag".
An ETag uniquely identifies a specific state of an object. Whenever the content or custom metadata of an object is modified, the ETag changes with it. An ETag returned in a response to a request can be used in subsequent requests with an If-Match header or an If-None-Match header to conditionally retrieve or update an object.
Last-Modified
This header is returned in responses to GET, HEAD, PUT and POST requests for objects, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.29 "Last-Modified".
The Last-Modified value returned in a response is equal to the "Modification-Date" property of the object. Whenever the content or custom metadata of an object is modified, the "Modification-Date" property of the object is updated. A Last-Modified date value returned in a response to a request can be used in subsequent requests with an If-Modified-Since header or an If-Unmodified-Since header to conditionally retrieve or update an object.
An ETag is more accurate than the Last-Modified date, because the Last-Modified date is only accurate to the second.
Create Object with Generated Name
It is possible to create an object whose name is generated. The generated name will be the guid of the put object.
To do so, you have to add the X-Ampli-Generate-Name header to a PUT the request.
Lattus REST API User’s Guide 121
Chapter 7: Object OperationsUsing Curl to Upload a File
PUT /namespace/<namespace-name> HTTP/1.1...X-Ampli-Generate-Name: GUID...
Currently, only one generation method (GUID) is supported.
The client knows the generated name by looking at the X-Ampli-Name field in the response. The X-Ampli-Id field also contains the same name, however this is only the case for the GUID method so clients should always inspect the X-Ampli-Name field.
The response is a 201 Created, identical to what is shown above for a normal PUT/POST of an object.
Using Curl to Upload a File
$ curl --verbose --fail --upload-file /myfiles/test2.txt "http://myhost.amplidata.com:8080/namespace/Documents/myfiles/myfavoritefile1.txt"* About to connect() to myhost.amplidata.com port 8080 (#0)* Trying www.quantum.com... connected* Connected to myhost.amplidata.com (127.0.0.1) port 8080 (#0)....
The unique ID of the object is set to /Documents/myfiles/myfavoritefile1.txt in this example.
In the response message no other ID is returned as the ID required to receive the object's content is the object's name set during the PUT process.
122 Lattus REST API User’s Guide
Chapter 7: Object OperationsCreate an Object with Generated Name
Create an Object with Generated Name
It is possible to have Lattus generate the name automatically when storing a new object.
This is indicated by adding the X-Ampli-Generate-Name header to a PUT request with the namespace directory as base URL.
PUT /namespace/<namespace-name> HTTP/1.1...X-Ampli-Generate-Name: GUID...
Currently only one generation method (GUID) is supported. This generates a Globally Unique Identifier (128 bit) for the object which can be used as a name for that object in the namespace.
You can verify the generated name by checking the X-Ampli-Name field in the response message.
The X-Ampli-Id field contains the same name as the X-Ampli-Name field when using this automated GUID generation method. In case you are not using this automated method always inspect the X-Ampli-Name field.
The response output returns a "201 Created" message, identical to the standard PUT/POST request of an object.
Create Directory Object
To create a directory object in a subdirectory of a namespace, its parent directory must already have been created. Parent directories are not automatically created.
http REQUEST /namespace/<namespace name>/<directory object name>
PUT /namespace/namespace1/directory1?meta=http HTTP/1.1Host: my.example.com
Lattus REST API User’s Guide 123
Chapter 7: Object OperationsCreate Directory Object
X-Ampli-Kind: Directory
RESPONSE /namespace/<namespace name>/<directory object name>
HTTP/1.1 201 CreatedDAV: 1,3Date: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: 0Last-Modified: _date_ETag: "16ab1e9936f92a7494e417099937b4f8"X-Ampli-Id: "e45ee50dba0440259575bb04c4217fa3"X-Ampli-Name: "directory1"X-Ampli-Namespace-Id: "b42cf6efc9f94d3585372877828fc4e6"X-Ampli-Kind: "Directory"X-Ampli-Policy-Id: "034daa52fd22402f83be5a760b6a47cd"X-Ampli-Size: 0X-Ampli-Crc32: 4294967295X-Ampli-Status: "OK"X-Ampli-Creation-Date: _epoch_time_X-Ampli-Modification-Date: _epoch_time_X-Ampli-Verification-Date: _epoch_time_
json REQUEST /namespace/<namespace name>/<directory object name>
PUT /namespace/namespace1/directory1?meta=json HTTP/1.1Host: my.example.comX-Ampli-Kind: Directory
RESPONSE /namespace/<namespace name>/<directory object name>
HTTP/1.1 201 CreatedDAV: 1,3Date: _date_
124 Lattus REST API User’s Guide
Chapter 7: Object OperationsCreate Directory Object
Server: Quantum-Lattus/3.0.3-ba79602e871550668391a673fa57589cf531497eContent-Type: application/json;charset=UTF-8Content-Length: _length_ Last-Modified: _date_ETag: "16ab1e9936f92a7494e417099937b4f8"
{"Id":"e45ee50dba0440259575bb04c4217fa3", "Name":"directory1","Namespace-Id":"b42cf6efc9f94d3585372877828fc4e6", "Kind":"Directory","Policy-Id":"034daa52fd22402f83be5a760b6a47cd", "Size":0,"Crc32":4294967295, "Status":"OK""Creation-Date":_epoch_time_ "Modification-Date":_epoch_time_ "Verification-Date":_epoch_time_}
xml REQUEST /namespace/<namespace name>/<directory object name>
PUT /namespace/namespace1/directory1?meta=xml HTTP/1.1Host: my.example.comX-Ampli-Kind: Directory
RESPONSE /namespace/<namespace name>/<directory object name>
HTTP/1.1 201 CreatedDAV: 1,3Date: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/xml;charset=UTF-8Content-Length: _length_Last-Modified: _date_ETag: "16ab1e9936f92a7494e417099937b4f8"<?xml version="1.0" encoding="UTF-8"?><Object xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Id>e45ee50dba0440259575bb04c4217fa3</Id>
Lattus REST API User’s Guide 125
Chapter 7: Object OperationsCustom Metadata
<Name>directory1</Name><Namespace-Id>b42cf6efc9f94d3585372877828fc4e6</Namespace-Id><Kind>Directory</Kind><Policy-Id>034daa52fd22402f83be5a760b6a47cd</Policy-Id><Size>0</Size><Crc32>4294967295</Crc32><Status>OK</Status><Creation-Date>_epoch_time_</Creation-Date><Modification-Date>_epoch_time_</Modification-Date><Verification-Date>_epoch_time_</Verification-Date></Object>
Curl Example curl -v -X MKCOL http://127.0.0.1/namespace/namespace1/a/b
Custom Metadata
Custom metadata can be associated with existing directory objects and file objects. All custom metadata field names have prefix "Custom-Meta-".
All custom metadata field names are case-insensitive and are converted automatically to lowercase (ASCII byte range 65..90 are converted to 97..122) for internal tracking, e.g "Id" can be referred to as "id", "Id", "iD", or "ID" in HTTP header names, in json object field names and in xml tags. Therefore, a custom metadata field name such as "Custom-Meta-User" will be stored as "user" and show up in the response message as "Custom-Meta-user".
Setting or modifying custom metadata updates the following two attributes of the object:
• the Lattus Modification-Date attribute.
• the Last-Modified and ETag headers.
126 Lattus REST API User’s Guide
Chapter 7: Object OperationsCustom Metadata
Note: The recommended maximum size for object metadata is 1000 bytes.Custom metadata is not encrypted in the backend database, nor in the blockstores (tlogs).
Set Custom Metadata
http REQUEST /namespace/<namespace name>/<object name>?update=set
PUT /namespace/namespace1/object1?update=set&meta=http HTTP/1.1Host: my.example.comDate: _date_ Content-Length: 0X-Ampli-Custom-Meta-User: "jdoe"X-Ampli-Custom-Meta-Group: "users"
RESPONSE /namespace/<namespace name>/<object name>?update=set
HTTP/1.1 200 OKDAV: 1,3Date: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: 0Last-Modified: _date_ETag: "2191d1429936ea1aae67962a56110c83"X-Ampli-Id: "f5c104cba97b42139448c80c217f143c"X-Ampli-Name: "namespace1/object1"X-Ampli-Namespace-Id: "011d9d4d92cc456b91d69e4e441be915"X-Ampli-Kind: "File"X-Ampli-Policy-Id: "814d15ed2c4240b2b918c63514718c4c"X-Ampli-Size: 3X-Ampli-Crc32: 3827519356X-Ampli-Status: "OK"
Lattus REST API User’s Guide 127
Chapter 7: Object OperationsCustom Metadata
X-Ampli-Creation-Date: _epoch_time_X-Ampli-Modification-Date: _new_epoch_time_X-Ampli-Verification-Date: _epoch_time_X-Ampli-Custom-Meta-group: "users"X-Ampli-Custom-Meta-user: "jdoe"
json REQUEST /namespace/<namespace name>/<object name>?update=set
PUT /namespace/namespace1/object1?update=set&meta=json HTTP/1.1Host: my.example.comDate: _date_Content-Length: _length_
{"Custom-Meta-User":"jdoe", "Custom-Meta-Group:"users"}
RESPONSE /namespace/<namespace name>/<object name>?update=set
HTTP/1.1 200 OKDAV: 1,3Date: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: _length_Last-Modified: _date_ETag: "2191d1429936ea1aae67962a56110c83"{"Id":"f5c104cba97b42139448c80c217f143c","Name":"namespace1/object1","Namespace-Id":"011d9d4d92cc456b91d69e4e441be915","Kind":"File","Policy-Id":"814d15ed2c4240b2b918c63514718c4c","Size":3,"Crc32":3827519356,"Status":"OK","Creation-Date":_epoch_time_,
128 Lattus REST API User’s Guide
Chapter 7: Object OperationsCustom Metadata
"Modification-Date":_new_epoch_time_,"Verification-Date":_epoch_time_,"Custom-Meta-group":"users","Custom-Meta-user":"jdoe"}
xml REQUEST /namespace/<namespace name>/<object name>?update=set
PUT /namespace/namespace1/object1?update=set&meta=xml HTTP/1.1Host: my.example.comDate: _date_Content-Length: _length_
<Object xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Custom-Meta-User>jdoe</Custom-Meta-User><Custom-Meta-Group>users</Custom-Meta-Group></Object>
RESPONSE /namespace/<namespace name>/<object name>?update=set
HTTP/1.1 200 OKDAV: 1,3Date: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/xml;charset=UTF-8Content-Length: _length_Last-Modified: _date_ETag: "2191d1429936ea1aae67962a56110c83"<Object xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Id>f5c104cba97b42139448c80c217f143c</Id><Name>namespace1/object1</Name><Namespace-Id>011d9d4d92cc456b91d69e4e441be915</Namespace-Id><Kind>File</Kind>
Lattus REST API User’s Guide 129
Chapter 7: Object OperationsCustom Metadata
<Policy-Id/>814d15ed2c4240b2b918c63514718c4c</Policy-Id><Size>3</Size><Crc32>3827519356</Crc32><Status>OK</Status><Creation-Date>_epoch_time_</Creation-Date><Modification-Date>_new_epoch_time_</Modification-Date><Verification-Date>_epoch_time_</Verification-Date><Custom-Meta-group>users</Custom-Meta-group><Custom-Meta-user>jdoe</Custom-Meta-user></Object>
Apply Updates to Existing Custom Metadata
In the examples below, we start from an object that has the following custom metadata fields:
Custom-Meta-group: "users" Custom-Meta-user: "jdoe"
http REQUEST /namespace/<namespace name>/<object name>?update=apply
PUT /namespace/namespace1/object1?update=apply&meta=http HTTP/1.1Host: my.example.comDate: _date_ Content-Length: 0X-Ampli-Custom-Meta-User: "john" X-Ampli-Custom-Meta-Color: "red"
RESPONSE /namespace/<namespace name>/<object name>?update=apply
HTTP/1.1 200 OKDAV: 1,3Date: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: 0Last-Modified: _date_ETag: "8a6547849714fa7f021a923e391a87a8"
130 Lattus REST API User’s Guide
Chapter 7: Object OperationsCustom Metadata
X-Ampli-Id: "f5c104cba97b42139448c80c217f143c"X-Ampli-Name: "namespace1/object1"X-Ampli-Namespace-Id: "011d9d4d92cc456b91d69e4e441be915"X-Ampli-Kind: "File"X-Ampli-Policy-Id: "814d15ed2c4240b2b918c63514718c4c"X-Ampli-Size: 3X-Ampli-Crc32: 3827519356X-Ampli-Status: "OK"X-Ampli-Creation-Date: _epoch_time_X-Ampli-Modification-Date: _new_epoch_time_X-Ampli-Verification-Date: _epoch_time_X-Ampli-Custom-Meta-color: "red"X-Ampli-Custom-Meta-group: "users"X-Ampli-Custom-Meta-user: "john"
json REQUEST /namespace/<namespace name>/<object name>?update=apply
PUT /namespace/namespace1/object1?update=apply&meta=json HTTP/1.1Host: my.example.comDate: _date_Content-Length: _length_
{"Custom-Meta-User":"john", "Custom-Meta-Color":"red"}
RESPONSE /namespace/<namespace name>/<object name>?update=apply
HTTP/1.1 200 OKDAV: 1,3Date: _date_Server: Quatnum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/json;charset=UTF-8Content-Length: 0Last-Modified: _date_ETag: "8a6547849714fa7f021a923e391a87a8"
Lattus REST API User’s Guide 131
Chapter 7: Object OperationsCustom Metadata
{"Id":"f5c104cba97b42139448c80c217f143c","Name":"namespace1/object1","Namespace-Id:"011d9d4d92cc456b91d69e4e441be915","Kind":"File","Policy-Id":"814d15ed2c4240b2b918c63514718c4c","Size":3,"Crc32":3827519356,"Status":"OK","Creation-Date":_epoch_time_,"Modification-Date":_new_epoch_time_,"Verification-Date":_epoch_time_,"Custom-Meta-color":"red","Custom-Meta-group":"users","Custom-Meta-user":"john"}
xml REQUEST /namespace/<namespace name>/<object name>?update=apply
PUT /namespace/namespace1/object1?update=apply&meta=xml HTTP/1.1Host: my.example.comDate: _date_Content-Length: _length_
<Object xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Custom-Meta-User>john</Custom-Meta-User><Custom-Meta-Color>red</Custom-Meta-Color></Object>
RESPONSE /namespace/<namespace name>/<object name>?update=apply
HTTP/1.1 200 OKDAV: 1,3Date: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/xml;charset=UTF-8
132 Lattus REST API User’s Guide
Chapter 7: Object OperationsList Objects
Content-Length: 0Last-Modified: _date_ETag: "8a6547849714fa7f021a923e391a87a8"<Object xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Id>f5c104cba97b42139448c80c217f143c</Id><Name>namespace1/object1</Name><Namespace-Id>011d9d4d92cc456b91d69e4e441be915</Id><Kind>File</Kind><Policy-Id>814d15ed2c4240b2b918c63514718c4c</Policy-Id><Size>3</Size><Crc32>3827519356</Crc32><Status>OK</Status><Creation-Date>_epoch_time_</Creation-Date><Modification-Date>_new_epoch_time_</Modification-Date><Verification-Date>_epoch_time_</Verification-Date><Custom-Meta-color>red</Custom-Meta-color><Custom-Meta-group>users</Custom-Meta-group><Custom-Meta-user>john</Custom-Meta-user></Object>
List Objects
A GET can be performed on /namespace/<namespace name>/<object name>.
A HEAD request yields the same response as a GET request, except that the server does not return a message-body in the response.
List Objects
The media type of the list to be returned. By default, only the first 50 items are returned.
You can add one of the following parameters to the URL:
Lattus REST API User’s Guide 133
Chapter 7: Object OperationsList Objects
http / text REQUEST /namespace/<namespace name>/<directory object name>/
GET /namespace/namespace1/directory1/ HTTP/1.1Host: my.example.comDate: _date_ Accept: text/plain
or:GET /namespace/namespace1/directory1/?list=http HTTP/1.1Host: my.example.com
Parameter Explanation Example URL
"limit=X" Use this query parameter to adjust the limit of the items to X (maximum 1000).Note: Any value higher than
1000 will result in 1000 entries.
"/namespace/namespace1/directory1/?limit=500"
With this URL, the first 500 itemsare returned.
"marker=Y" Use this query parameter to define from which marker onward the list should start.
"/namespace/namespace1/directory1/?marker=objectc&limit=4&meta=xml"
With this URL, the first 4 objects will be returned in xml format, starting with "objectc".
"include_marker=false" Use this parameter, to exclude the marker from the list.
"/namespace/namespace1/directory1/?marker=objectc&limit=4&include_marker=false&meta=xml"
With this URL, the first 4 objects will be returned in xml format, start ing with the first object after "objectc".
134 Lattus REST API User’s Guide
Chapter 7: Object OperationsList Objects
Date: _date_
RESPONSE /namespace/<namespace name>/<directory object name>/
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: text/plain;charset=UTF-8Content-Length: _length_"object1""object2"
json REQUEST /namespace/<namespace name>/<directory object name>/
GET /namespace/namespace1/directory1/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/json
or:GET /namespace/namespace1/directory1/?list=json HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /namespace/<namespace name>/<directory object name>/
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/json;charset=UTF-8Content-Length: _length_["object1","object2"]
Lattus REST API User’s Guide 135
Chapter 7: Object OperationsList Objects
xml REQUEST /namespace/<namespace name>/<directory object name>/
GET /namespace/namespace1/directory1/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/xml
or:GET /namespace/namespace1/directory1/?list=xml HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /namespace/<namespace name>/<directory object name>/
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Type: application/xml;charset=UTF-8Content-Length: _length_<Directory-Entriesxmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Name>object1</Name><Name>object2</Name></Directory-Entries>
Using Curl to List the Objects in a Namespace
curl --verbose --fail -H "Accept: application/xml" "http://127.0.0.1:8080/namespace/examplenamespace/"
136 Lattus REST API User’s Guide
Chapter 7: Object OperationsMove Object
Move Object
Moving an object, renames the object. This request is a limited implementation of the move command of the WebDAV specification of RFC 4918, section 9.9.
You can only rename objects inside a name space and when the object is a file. When moving a file to another directory, the destination directory must exist upfront.
You can not rename files, nor move them to another name space.
Prerequisites A user can only move objects with the proper permissions: DELETE, READ, and WRITE
Response /namespace/namespace1/source_object
HTTP/1.1 201 CreatedDate: _date_Server: Amplidata-AmpliStor/2.1.0-1f206c99fa6c1df2070e69741a6c0c6d52353ec9
Extra Information The Depth request header is optional. The only value allowed at this time is “infinity”.
The Destination request header is required and should specify the full destination path, including the new object name. The directory component of the destination path to move the object must exist. If it does not exist, the request fails with the HTTP response 409 Conflict.
The Overwrite request header is required and can have value T for true or F for false. At this moment T is not supported and results in the HTTP response 405 Method Not Allowed.
Lattus REST API User’s Guide 137
Chapter 7: Object OperationsGet Object Content
Get Object Content
Complete Content REQUEST /namespace/<namespace name>/<object name>
GET /namespace/namespace1/object1 HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /namespace/<namespace name>/<object name>
HTTP/1.1 200 OKDate: _date_Server: Quantum-Lattus/2.1.0-1f206c99fa6c1df2070e69741a6c0c6d52353ec9Content-Type: text/plainContent-Length: _length_Last-Modified: _date_ETag: "c0af7bfb6616414d89fad0552274a722"X-Ampli-Id: "c0af7bfb6616414d89fad0552274a722"X-Ampli-Name: "namespace/namespace1/object1"X-Ampli-Namespace-Id: "9ec81b3cfd714655adb9a9fcf20d277c"X-Ampli-Policy-Id: "31784b34443440a89a0dab38563a5937"X-Ampli-Size: _length_X-Ampli-Crc32: 3827519356X-Ampli-Status: "OK"X-Ampli-Creation-Date: _epoch_time_X-Ampli-Modification-Date: _epoch_time_X-Ampli-Verification-Date: _epoch_time__length_bytes_of_object_content_
Request Headers Please note that the examples below also work for other meta types than "meta=http".
If-Match This header is supported for PUT of an object, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.24 "If-Match". If this
138 Lattus REST API User’s Guide
Chapter 7: Object OperationsGet Object Content
precondition is not satisfied, the response will be "412 Precondition Failed".
See also the ETag response header.
The subsections contain an example: the ETag we received from the server for object "/namespace/namespace1/object1" in a previous response to a range GET request was "19d0a529e39347f4823c6e22f1160825". We wish to GET the entire content of the object, but only if it has not been changed in the mean time. In this scenario, the object has been changed in the mean time, so the ETag no longer matches, and the server responds with a 412 error response.
REQUEST /namespace/<namespace name>/<object name>
GET /namespace/namespace1/object1?meta=http HTTP/1.1Host: my.example.comIf-Match: "19d0a529e39347f4823c6e22f1160825"
RESPONSE /namespace/<namespace name>/<object name>
HTTP/1.1 412 Precondition FailedDate: _date_Server: Quantum-Lattus/3.0.3-ba79602e871550668391a673fa57589cf531497e
If-Modified-Since This header is supported for GET and HEAD of an object, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.25 "If-Modified-Since". If this precondition is not satisfied, the response will be "304 Not Modified".
See also the Last-Modified response header.
If-None-Match This header is supported for GET and HEAD of an object, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.26 "If-None-Match". If this precondition is not satisfied, the response will be "304 Not Modified". See also #Last-Modified.
See also the ETag response header.
Lattus REST API User’s Guide 139
Chapter 7: Object OperationsGet Object Content
If-Unmodified-Since This header is supported for GET and HEAD of an object, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.28 "If-Unmodified-Since". If this precondition is not satisfied, the response will be "412 Precondition Failed". See also the Last-Modified response header.
Range The Range header is supported for GET of object content, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.35 "Range".
An example, for a file containing the text "AmplidataAmplistor":
REQUEST
GET /namespace/namespace1/object1 HTTP/1.1Host: my.example.comRange: bytes=10-14,5-9,0-4,15-,-10
REPLY
HTTP/1.1 206 Partial ContentAccept-Ranges: bytesContent-Length: 611Content-Type: multipart/byteranges;boundary=999200ad8891450f8f9aedb1245c7bde768e308403964649a8f16f1bf5e630a0Last-Modified: _date_ETag: "cbd1130013be42d2a9af04bbb2805905"X-Ampli-Id: "cbd1130013be42d2a9af04bbb2805905" X-Ampli-Name: "object1"X-Ampli-Namespace-Id: "2b130b116dc74308a6a40e148dda5e12" X-Ampli-Policy-Id: "b349b3b886b242229305827cfeb2b178"X-Ampli-Size: 19X-Ampli-Crc32: 3768369899X-Ampli-Status: "OK"X-Ampli-Creation-Date: _epoch_time_X-Ampli-Modification-Date: _epoch_time_ X-Ampli-Verification-Date: _epoch_time_
140 Lattus REST API User’s Guide
Chapter 7: Object OperationsGet Object Content
--999200ad8891450f8f9aedb1245c7bde768e308403964649a8f16f1bf5e630a0Content-Range: bytes 10-14/19
Ampli--999200ad8891450f8f9aedb1245c7bde768e308403964649a8f16f1bf5e630a0Content-Range: bytes 5-9/19
data--999200ad8891450f8f9aedb1245c7bde768e308403964649a8f16f1bf5e630a0Content-Range: bytes 0-4/19
Ampli--999200ad8891450f8f9aedb1245c7bde768e308403964649a8f16f1bf5e630a0Content-Range: bytes 15-18/19
Stor--999200ad8891450f8f9aedb1245c7bde768e308403964649a8f16f1bf5e630a0Content-Range: bytes 9-18/19
AmpliStor--999200ad8891450f8f9aedb1245c7bde768e308403964649a8f16f1bf5e630a0--
Response Headers ETag
This header is returned in responses to GET, HEAD, PUT and POST requests for objects, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.19 "ETag".
Lattus REST API User’s Guide 141
Chapter 7: Object OperationsGet Object Metadata
An ETag uniquely identifies a specific state of an object. Whenever the content or custom metadata of an object is modified, the ETag changes with it. An ETag returned in a response to a request can be used in subsequent requests with an If-Match header or an If-None-Match header to conditionally retrieve or update an object.
Last-Modified
This header is returned in responses to GET, HEAD, PUT and POST requests for objects, as specified in RFC 2616, "Hypertext Transfer Protocol - HTTP/1.1", section 14.29 "Last-Modified".
The Last-Modified value returned in a response is equal to the "Modification-Date" property of the object. Whenever the content or custom metadata of an object is modified, the "Modification-Date" property of the object is updated. A Last-Modified date value returned in a response to a request can be used in subsequent requests with an If-Modified-Since header or an If-Unmodified-Since header to conditionally retrieve or update an object.
An ETag is more accurate than the Last-Modified date, because the Last-Modified date is only accurate to the second.
Using Curl to Get an Object
curl --verbose --fail
"http://127.0.0.1:8080/namespace/examplenamespace/firstobject" >
/root/mydownloadedfile
Get Object Metadata
http / text REQUEST /namespace/<namespace name>/<object name>?meta=http
GET /namespace/namespace1/object1?meta=http HTTP/1.1Host: my.example.com
142 Lattus REST API User’s Guide
Chapter 7: Object OperationsGet Object Metadata
Date: _date_
RESPONSE /namespace/<namespace name>/<object name>?meta=http
HTTP/1.1 200 OKDAV: 1,3Date: __date__Server: Amplidata-AmpliStor/unknown-2bb0ba7170b333cd4e9f7f9d94d4fb8d450194b9Content-Type: text/plain;charset=UTF-8Content-Length: 0Last-Modified: __date__ETag: "4059eea7457257b679aae2e2e6c55aff"X-Ampli-Id: "38e618a671334338be2b060f414fafee"X-Ampli-Name: "object1"X-Ampli-Namespace-Id: "830a1890c94e46bfa86d82e4cf42123f"X-Ampli-Kind: "File"X-Ampli-Policy-Id: "cfd345302fae4bc6824f15fc6cfd603b"X-Ampli-Size: 4229X-Ampli-Crc32: 3552406836X-Ampli-Status: "OK"X-Ampli-Creation-Date: 1329472463X-Ampli-Modification-Date: 1329472463X-Ampli-Verification-Date: 1329472463
json REQUEST /namespace/<namespace name>?meta=json
GET /namespace/VOL HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /namespace/<namespace name>?meta=json
HTTP/1.1 200 OKDAV: 1,3Date: __date__Server: Amplidata-AmpliStor/unknown-2bb0ba7170b333cd4e9f7f9d94d4fb8d450194b9Content-Type: application/json;charset=UTF-8
Lattus REST API User’s Guide 143
Chapter 7: Object OperationsGet Object Metadata
Content-Length: 323Last-Modified: __date__ETag: "4059eea7457257b679aae2e2e6c55aff"{"Id":"38e618a671334338be2b060f414fafee","Name":"object1","Namespace-Id":"830a1890c94e46bfa86d82e4cf42123f","Kind":"File","Policy-Id":"cfd345302fae4bc6824f15fc6cfd603b","Size":4229,"Crc32":3552406836,"Status":"OK","Creation-Date":1329472463,"Modification-Date":1329472463,"Verification-Date":1329472463}
xml REQUEST /namespace/<namespace name>/<object name>?meta=xml
GET /namespace/namespace1/object1?meta=xml HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /namespace/<namespace name>/<object name>?meta=xml
HTTP/1.1 200 OKDAV: 1,3Date: __date__Server: Amplidata-AmpliStor/unknown-2bb0ba7170b333cd4e9f7f9d94d4fb8d450194b9Content-Type: application/xml;charset=UTF-8Content-Length: 538Last-Modified: __date__ETag: "4059eea7457257b679aae2e2e6c55aff"<?xml version="1.0" encoding="UTF-8"?><Object xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Id>38e618a671334338be2b060f414fafee</Id><Name>object1</Name>
144 Lattus REST API User’s Guide
Chapter 7: Object OperationsDelete Object
<Namespace-Id>830a1890c94e46bfa86d82e4cf42123f</Namespace-Id><Kind>File</Kind><Policy-Id>cfd345302fae4bc6824f15fc6cfd603b</Policy-Id><Size>4229</Size><Crc32>3552406836</Crc32><Status>OK</Status><Creation-Date>1329472463</Creation-Date><Modification-Date>1329472463</Modification-Date><Verification-Date>1329472463</Verification-Date></Object>
Delete Object
REQUEST /namespace/<namespace name>/<object name>
DELETE /namespace/namespace1/object1 HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /namespace/<namespace name>/<object name>
HTTP/1.1 204 No ContentDate: _date_Server: Quantum-Lattus/3.0.0-1526584800240429ab180380ccfa8aa6ceda7b24Content-Length: 0
Using Curl to Delete an Object
curl -f -v -X DELETE http://127.0.0.1:8080/namespace/test-4-1/test.txt
User Management Operations
Lattus REST API User’s Guide 145
Chapter 7: Object OperationsAdding A user
Adding A user
To add a new user, the following fields must be specified:
• name: the user name. Needs to be unique
• secret: the password
The following fields are optional:
• Default-policy: the default policy to use when this user creates a bucket
• Default-small-files-policy: the default small files policy to use when this user creates a bucket
• Default-small-files-threshold: the default threshold to use to select either the regular or the small files policy when this user creates a bucket
Default-small-files-policy and default-small-files-threshold need to be specified together: either both or not at all. If an optional field is not specified, it is set to None.
http / text REQUEST /manage/user
POST /manage/user HTTP/1.1Host: my.example.comX-Ampli-Name: "my_user_name"X-Ampli-Secret: "my_secret"X-Ampli-Default-Policy: "cadb732d2fdd4486be9b05dbe4b51d14"X-Ampli-Default-Small-Files-Policy: "66da5c8f321d4dd28e3da264c42d04b7"X-Ampli-Default-Small-Files-Threshold: 25000Accept:plain/text
or:POST /manage/user?meta=http HTTP/1.1Host: my.example.comX-Ampli-Name: "my_user-name"X-Ampli-Secret: "my_secret"
146 Lattus REST API User’s Guide
Chapter 7: Object OperationsAdding A user
X-Ampli-Default-Policy: "cadb732d2fdd4486be9b05dbe4b51d14"X-Ampli-Default-Small-Files-Policy: "66da5c8f321d4dd28e3da264c42d04b7"X-Ampli-Default-Small-Files-Threshold: 25000
RESPONSE /manage/user
HTTP/1.1 201 CreatedDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Type: text/plain;charset=UTF-8Content-Length: 0X-Ampli-Name: "my_user_name"X-Ampli-Id: "36"X-Ampli-Status: "ACTIVE"X-Ampli-Seq: 0X-Ampli-Num-S3-Namespaces: 0X-Ampli-Default-Policy: "cadb732d2fdd4486be9b05dbe4b51d14"X-Ampli-Default-Small-Files-Policy: "66da5c8f321d4dd28e3da264c42d04b7"X-Ampli-Default-Small-Files-Threshold: 25000
json REQUEST /manage/user
POST /manage/user HTTP/1.1Host: my.example.comContent-Type: application/jsonAccept: application/jsonContent-Length: 205{"Name":"my_user_name","Secret":"my_secret","Default-Policy":"cadb732d2fdd4486be9b05dbe4b51d14","Default-Small-Files-Policy":"66da5c8f321d4dd28e3da264c42d04b7","Default-Small-Files-Threshold":10000}
or:POST /manage/user?meta=json HTTP/1.1
Lattus REST API User’s Guide 147
Chapter 7: Object OperationsAdding A user
Host: my.example.comContent-Length: 205{"Name":"my_user_name","Secret":"my_secret","Default-Policy":"cadb732d2fdd4486be9b05dbe4b51d14","Default-Small-Files-Policy":"66da5c8f321d4dd28e3da264c42d04b7","Default-Small-Files-Threshold":10000}
RESPONSE /manage/user
HTTP/1.1 201 CreatedDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Type: application/json;charset=UTF-8Content-Length: 245{"Name":"my_user_name","Id":"41","Status":"ACTIVE","Seq":0,"Num-S3-Namespaces":0,"Default-Policy":"cadb732d2fdd4486be9b05dbe4b51d14","Default-Small-Files-Policy":"66da5c8f321d4dd28e3da264c42d04b7","Default-Small-Files-Threshold":10000}
xml REQUEST /manage/user
POST /manage/user HTTP/1.1Host: my.example.comContent-Type: application/xmlAccept: application/xmlContent-Length: 410<?xml version="1.0" encoding="UTF-8"?><User xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Name>my_user_name</Name>
148 Lattus REST API User’s Guide
Chapter 7: Object OperationsAdding A user
<Secret>my_secret</Secret><Default-Policy>cadb732d2fdd4486be9b05dbe4b51d14</Default-Policy><Default-Small-Files-Policy>66da5c8f321d4dd28e3da264c42d04b7</Default-Small-Files-Policy><Default-Small-Files-Threshold>50000</Default-Small-Files-Threshold></User>
or:POST /manage/user?meta=xml HTTP/1.1Host: my.example.comContent-Length: 410<?xml version="1.0" encoding="UTF-8"?><User xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Name>my_user_name</Name><Secret>my_secret</Secret><Default-Policy>cadb732d2fdd4486be9b05dbe4b51d14</Default-Policy><Default-Small-Files-Policy>66da5c8f321d4dd28e3da264c42d04b7</Default-Small-Files-Policy><Default-Small-Files-Threshold>50000</Default-Small-Files-Threshold></User>RESPONSE /manage/userHTTP/1.1 201 CreatedDate: _date_Server: Amplidata-AmpliStor/_revision-Content-Type: application/xml;charset=UTF-8Content-Length: 403<?xml version="1.0" encoding="UTF-8"?><User><Name>my_user_name</Name><Id>47</Id><Status>ACTIVE</Status><Seq>0</Seq><Num-S3-Namespaces>0</Num-S3-Namespaces><Default-Policy>cadb732d2fdd4486be9b05dbe4b51d14</Default-Policy>
Lattus REST API User’s Guide 149
Chapter 7: Object OperationsList Users
<Default-Small-Files-Policy>66da5c8f321d4dd28e3da264c42d04b7</Default-Small-Files-Policy><Default-Small-Files-Threshold>50000</Default-Small-Files-Threshold></User>
List Users
By default, only the first 50 users are returned. This limit can be adjusted to X by appending the query parameter "limit=X" to the URL. The maximum value for this limit is 1000, higher values will result in at most 1000 entries.
For example, the following URL could be used: /manage/user?limit=500.
In addition, the user from which the list should start can be specified using "marker=Y".
For example, the following URL could be used to list 4 users starting from user "test": /manage/user?marker=test&limit=4.
An additional query parameter, "include_marker=false" can be added, which results in the marker being not included in the list.
http / text REQUEST /manage/user/
GET /manage/user/ HTTP/1.1Host: my.example.comDate: _date_Accept: text/plain
or:GET /manage/user/?list=http HTTP/1.1Host: my.example.comDate: _date_
150 Lattus REST API User’s Guide
Chapter 7: Object OperationsList Users
RESPONSE /manage/user/
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Type: text/plain;charset=UTF-8Content-Length: _length_"fred""erik"
json REQUEST /manage/user/GET /manage/user/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/json
or:GET /manage/user/?list=json HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/user/
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Type: application/json;charset=UTF-8Content-Length: _length_["fred","erik"]
xml REQUEST /manage/user/
GET /manage/user/ HTTP/1.1Host: my.example.comDate: _date_Accept: application/xml
Lattus REST API User’s Guide 151
Chapter 7: Object OperationsGET User
or:GET /manage/user/?list=xml HTTP/1.1Host: my.example.comDate: _date_
RESPONSE /manage/user/
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Type: application/xml;charset=UTF-8Content-Length: _length_<Users xmlns="http://www.amplidata.com/amplistor/api/rest/1/amplistor.xsd"><Name>fred</Name><Name>erik</Name></Users>
GET User
A GET can be performed on /manage/user directly, on /manage/user/<username> or on /manage/user_by_id/<id>. The first returns a list of all users names, the second the details of a specific user, identified by his user name, and the third the details of a specific user, identified by his user id.
Listing by id is not supported.
If a user has certain optional fields which are currently set to None, they will be absent from the http/json/xml output.
A HEAD request yields the same response as a GET request, except that the server does not return a message-body in the response
http / text REQUEST /manage/user/<username>
GET /manage/user/my_user_name HTTP/1.1Host: my.example.com
152 Lattus REST API User’s Guide
Chapter 7: Object OperationsGET User
Accept: text/plain
or:GET /manage/user/my_user_name?meta=http HTTP/1.1Host: my.example.com
RESPONSE /manage/user/<username>
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Type: text/plain;charset=UTF-8Content-Length: 0X-Ampli-Name: "my_user_name"X-Ampli-Id: "48"X-Ampli-Status: "ACTIVE"X-Ampli-Seq: 0X-Ampli-Num-S3-Namespaces: 0X-Ampli-Default-Policy: "cadb732d2fdd4486be9b05dbe4b51d14"X-Ampli-Default-Small-Files-Policy: "66da5c8f321d4dd28e3da264c42d04b7"X-Ampli-Default-Small-Files-Threshold: 50000
json REQUEST /manage/user/<username>
GET /manage/user/my_user_name HTTP/1.1Host: my.example.comAccept: application/json
or:GET /manage/user/my_user_name?meta=json HTTP/1.1Host: my.example.com
RESPONSE /manage/user/<username>
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Type: application/json;charset=UTF-8Content-Length: 245{
Lattus REST API User’s Guide 153
Chapter 7: Object OperationsGET User
"Name":"my_user_name","Id":"48","Status":"ACTIVE","Seq":0,"Num-S3-Namespaces":0,"Default-Policy":"cadb732d2fdd4486be9b05dbe4b51d14","Default-Small-Files-Policy":"66da5c8f321d4dd28e3da264c42d04b7","Default-Small-Files-Threshold":50000}
xml REQUEST /manage/user/<username>
GET /manage/user/my_user_name HTTP/1.1Host: my.example.comAccept: application/xml
or:GET /manage/user/my_user_name?meta=xml HTTP/1.1Host: my.example.com
RESPONSE /manage/user/<username>
HTTP/1.1 200 OKDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Type: application/xml;charset=UTF-8Content-Length: 402<?xml version="1.0" encoding="UTF-8"?><User><Name>my_user_name</Name><Id>48</Id><Status>ACTIVE</Status><Seq>0</Seq><Num-S3-Namespaces>0</Num-S3-Namespaces><Default-Policy>cadb732d2fdd4486be9b05dbe4b51d14</Default-Policy><Default-Small-Files-Policy>66da5c8f321d4dd28e3da264c42d04b7</Default-Small-Files-Policy>
154 Lattus REST API User’s Guide
Chapter 7: Object OperationsDelete A User
<Default-Small-Files-Threshold>50000</Default-Small-Files-Threshold></User>
Delete A User
The user to delete can be identified by his user name ( URL /manage/user/<user name> ) or his id ( URL /manage/user_by_id/<user id> ).
REQUEST /manage/user/<username>
DELETE /manage/user/my_user_name HTTP/1.1Host: my.example.com
RESPONSE /manage/user/<username>
HTTP/1.1 204 No ContentDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Length: 0orREQUEST /manage/user/<username>DELETE /manage/user_by_id/my_id HTTP/1.1Host: my.example.comRESPONSE /manage/user/<username>HTTP/1.1 204 No ContentDate: _date_Server: Amplidata-AmpliStor/_revision_Content-Length: 0
Lattus REST API User’s Guide 155
Chapter 7: Object OperationsDelete A User
156 Lattus REST API User’s Guide
Chapter 8Authentication
Purpose
Lattus implements a security model in which an identified and authenticated user has certain user rights (permissions) on a namespace.
The different user rights are:
• Read
• Create
• Update
• Delete
• List
Lattus REST API User’s Guide 157
Chapter 8: AuthenticationHow It Works
How It Works
Using the Lattus command line interface, the storage administrator does the following:
• creates users with their credentials (password)
• assigns these users and their respective rights to a specific namespace
When the application, communicating over REST authenticates to the client daemon, it supplies the user and their credentials and a such access to the namespace is controlled.
The security rights are granted on a namespace basis, not on the objects that belong to this namespace
REST Authentication
Authentication is implemented by means of HTTP Digest authentication, as specified in RFC2617
This is how digest authentication is used:
• The request URL is inspected and determined for which namespace the request is. The namespace metadata is inspected to find the relevant credentials. Special care is taken to also map the “upload-form” to the correct namespace secrets.
• A 401 response is sent back.
• The client re-sends the request with an Authentication header.
• The server checks the content of that header against the relevant credentials.
158 Lattus REST API User’s Guide
Chapter 8: AuthenticationAPI Description
API Description
Users Users are identified by their unique user id. This int64 value is automatically assigned when creating a new user and cannot be changed. Each user id is linked to a user name (which must also be unique, deployment-wide), a password and a status, which can be ACTIVE, INACTIVE or DELETED.
User id 0 corresponds to a default user with user name “everyone”, which has no password. It corresponds to the unauthenticated user. Setting permissions on a path for this user allows unauthenticated users to perform operations.
Paths and Flags The previuosly mentioned permissions can be assigned, for each individual user, to a number of specific paths:
• root "/"
• manage "/manage"
• namespace root "/namespace"
• a namespace "/namespace/<namespace name>"
In addition to user permissions, the security system allows to enable certain global (i.e. not user specific) flags on each of these paths. These flags are:
• INHERIT: enabling the inherit flag on a path means that the permissions defined on this path are combined with the permission of the parent path.
Example:
If a user has permission READ on /namespace/VOL and permission CREATE on /namespace, this user will, if the INHERIT flag on path /namespace/VOL is not set, not be allowed to create new objects in /namespace/VOL. If the INHERIT flag is enabled on /namespace/VOL, this user will obtain effective rights of READ, CREATE on /namespace/VOL.
INHERIT also works in cascade, e.g. if inherit is enabled on /, /namespace and /namespace/VOL, then the effective permissions on /namespace/
Lattus REST API User’s Guide 159
Chapter 8: AuthenticationAPI Description
VOL are those combined over all three paths. The cascade works top -> down and stops at the first level where the INHERIT flag is not set.
Q-Shell Interface for User Management
Add a New User with Username, Credentials and Status
The status can be ACTIVE (default) or INACTIVE)In [7]: q.dss.manage.addUser(?Definition: q.dss.manage.addUser(self, userName, credentials, status='ACTIVE', nodeIP='127.0.0.1', port=23510)Documentation:
Add a user.Parameters:- userName: string The username of the user.- credentials: string The secret keyword of the user.- status: string The user status, one of: ACTIVE, INACTIVE.- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact.
Get User Information by Referring to its Username
Setting showDeleted to True will also show the user if it has status DELETED:In [8]: q.dss.manage.showUser(?Definition: q.dss.manage.showUser(self, userName, showDeleted=False, nodeIP='127.0.0.1', port=23510)Documentation:
Show a user by name.Parameters:- userName: string The username of the user.- showDeleted: bool Show deleted users or not.- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact.
160 Lattus REST API User’s Guide
Chapter 8: AuthenticationAPI Description
Getting User Information by Referring to its ID
In [9]: q.dss.manage.showUserById(?Definition: q.dss.manage.showUserById(self, userId, showDeleted=False, nodeIP='127.0.0.1', port=23510)Documentation:
Show a user by id.
Parameters:
- userId: int The userid of the user.- showDeleted: bool Show deleted users or not.- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact.
List All Users
Setting showDeleted to True will include users that have status DELETED:In [10]: q.dss.manage.listUsers(?Definition: q.dss.manage.listUsers(self, showDeleted=False, nodeIP='127.0.0.1', port=23510)Documentation:
List all users.
Parameters:
- showDeleted: bool Show deleted users or not.- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact.
Modifying User Information with a UserName
In [11]: q.dss.manage.changeUser(?
Lattus REST API User’s Guide 161
Chapter 8: AuthenticationAPI Description
Definition: q.dss.manage.changeUser(self, userName, credentials=None, status=None, nodeIP='127.0.0.1', port=23510)Documentation:
Change a user.
Parameters:
- userName: string The username of the user.- credentials: string The new secret of the user, None for no change.- status: string The new status of the user, one of: ACTIVE, INACTIVE, None for no change.- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact.
Modify User Information by Referring to its ID
In [13]: q.dss.manage.changeUserById(?
Definition: q.dss.manage.changeUserById(self, userId, userName=None, credentials=None, status=None, nodeIP='127.0.0.1', port=23510) Documentation:
Change a user by id.
Parameters:
- userId: int The userId of the user.- userName: string The new userName of the user, None for no change.- credentials: string The new secret of the user, None for no change.- status: string The new status of the user, one of: ACTIVE, INACTIVE, None for no change.- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact.
162 Lattus REST API User’s Guide
Chapter 8: AuthenticationAPI Description
Deleting a User
In [14]: q.dss.manage.deleteUser(?Definition: q.dss.manage.deleteUser(self, userName, nodeIP='127.0.0.1', port=23510)Documentation:
Delete a user.
Parameters:
- userName: string The new userName of the user, None for no change.- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact.
Q-shell Interface for Assignment of Rights to a Specific Path
Get the Permission Flags for a Specific Path
In [15]: q.dss.manage.showPermissionFlags(?Definition: q.dss.manage.showPermissionFlags(self, path, nodeIP='127.0.0.1', port=23510)Documentation:
Show the permission flags for a specified path.
Parameters:
- path: string The path for which to show the permission flags: "/", "/manage", "/namespace", or "/namespace/<namespace-name>".- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact. Returns: permission flags
Lattus REST API User’s Guide 163
Chapter 8: AuthenticationAPI Description
Setting the Permission Flags for a Specific Path
In [21]: q.dss.manage.setPermissionFlags(?
Definition: q.dss.manage.setPermissionFlags(self, path, permissionFlags, nodeIP='127.0.0.1', port=23510)Documentation:
Set the permission flags for a specified path.
Parameters:
- path: string The path for which to set the permission flags: "/", "/manage", "/namespace", or "/namespace/<namespace-name>".- permissionFlags: [string] A list of permission flags. Currently allowed values: "Inherit".- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact. Returns: permission flags
Get the permissions for a specific user on a path
In [18]: q.dss.manage.showPermissions(?
Definition: q.dss.manage.showPermissions(self, path, userName, nodeIP='127.0.0.1', port=23510)Documentation:
Show the user access permissions of a user for a specified path.
Parameters:
- path: string The path for which to show the permissions: "/", "/manage", "/namespace", or "/namespace/<namespace-name>".
164 Lattus REST API User’s Guide
Chapter 8: AuthenticationAPI Description
- userName: string The username of the user, or "everyone" for the permissions for special user "everyone".- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact. Returns: permissions
Set the permissions for a specific user on a path
In [20]: q.dss.manage.setPermissions(?Definition: q.dss.manage.setPermissions(self, path, userName, permissions, nodeIP='127.0.0.1', port=23510)Documentation:
Set the user access permissions of a user for a specified path.
Parameters:
- path: string The path for which to set the permissions: "/", "/manage", "/namespace", or "/namespace/<namespace-name>".- userName: string The username of the user, or "everyone" for the permissions for special user "everyone".- permissions: [string] A list of permissions. Currently allowed values: "Read", "Create", "Delete", "List", "Update".- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact. Returns: permissions
Getting the permission-settings for a path (combines flags, permissions, paths)
q.dss.manage.showPermissionSettings(self, path, nodeIP='127.0.0.1', port=23510) Documentation:
Show the permission settings for a specified path.
Lattus REST API User’s Guide 165
Chapter 8: AuthenticationREST Authentication in Detail
Parameters:
- path: string The path for which to show the permission settings: "/", "/manage", "/namespace", or "/namespace/<namespace-name>".- nodeIP: string The IP of the Storage Daemon to contact.- port: int The port of the Storage Daemon to contact. Returns: permission flags
REST Authentication in Detail
WWW-Authenticate header
This is the header the server sends back to the client in the 401.
An example:WWW-Authenticate: Digestrealm="Amplidata-AmpliStor/unknown-59b0f5a6e4709f566a5e8464153401ff76964cb9",qop=auth, nonce="246a5bf426a77dc28c195ce0373a9d2c",opaque="7e81b3abf2a84373b087cec73aca62c1"
It contains the following parameters:
• realm: an unique string identifying the server
• qop: this tells the client that we want digest authentication
• nonce: a hex string that is calculated as a hash (see the RFC)
• opaque: another hex string chosen at random at the start of the session
Authorization header This is the header the client sends in the subsequent re-send of the request. An example:
WWW-Authenticate: Digest
166 Lattus REST API User’s Guide
Chapter 8: AuthenticationREST Authentication in Detail
realm="Amplidata-AmpliStor/unknown-59b0f5a6e4709f566a5e8464153401ff76964cb9", qop=auth, nonce="246a5bf426a77dc28c195ce0373a9d2c",opaque="7e81b3abf2a84373b087cec73aca62c1"
It contains the following parameters:
• username: user name, used for validating the response
• realm: has to be identical as in WWW-Authenticate
• nonce: has to be identical as in WWW-Authenticate
• uri: has to be the request URI
• qop: has to be auth
• nc: 8 digit hex string counting up for each new request within the authorization session, this has to be 1 in the first and incremented each time
• cnonce: a random 8 byte string chosen by the client
• response: The client calculates the response by means of a formula based on request-method uri nonce username secret realm qop nc and cnonce. As all these fields are also available the server can afterwards do the same calculation and see if it matches.
• opaque: has to be identical as in WWW-Authenticate
The server will change the nonce at times during the session, because a nonce has a limited lifetime. The server sends a STALE flag to the client to indicate that the nonce is stale.
Curl Examples Get
curl --digest -u user:rw_1 -v http://localhost:8080/namespace/VOL/whatever.bin -o/tmp/1.bin
Put
curl --digest -u user:rw_1 -v http://localhost:8080/namespace/VOL/1.bin--upload-file /tmp/1.bin
Lattus REST API User’s Guide 167
Chapter 8: AuthenticationGetting Started with Authentication
Both commands assume that a user rw_1 with write rights on namespaceVOL was created.
Getting Started with Authentication
Out of the Box Authentication
When freshly installed, following settings are active:
• no users are created, only the "everyone" account is present
• unauthenticated users can LIST on /
• unauthenticated users have no rights on /manage
• unauthenticated users have all (update, read, create, list, delete) rights on /namespace
Before you Start Prevent an Unauthenticated User From Having Rights on all Namespaces
q.dss.manage.setPermissions('/namespace','everyone',[])
Allowing an Admin User to Create Namespaces and Polcies
Create a User to Administer Namespaces and Policies
q.dss.manage.addUser('admin','pass4admin')
Give the Admin User Rights on /manage
q.dss.manage.setPermissions("/manage","admin",['CREATE','LIST','READ','UPDATE','DELETE'])
The admin user will have no rights on the /namespace path and as such no rights on namespaces underneath.
168 Lattus REST API User’s Guide
Chapter 8: AuthenticationGetting Started with Authentication
Add a Namespace
You can do this over REST using the admin user or using the q-shell. Let's assume, for the remainder of this guide, that you created a namespace called mynamespace.
Create a User that Will Have All Rights on This Namespace
q.dss.manage.addUser('namespaceadmin','pass4namespaceadmin')
Set the Permissions for the Namespaceadmin User
q.dss.manage.setPermissions("/namespace/mynamespace","namespaceadmin",['CREATE',' LIST','READ','DELETE','UPDATE'])
Allowing Another User to Only Read Data from a Specific Namespace
Create a User That Only Has Read Rights on This Namespace
q.dss.manage.addUser('namespacereader','pass4namespacereader')
Set the Permissions for the Namespacereader User
q.dss.manage.setPermissions("/namespace/mynamespace","namespacereader",['READ'])
Inheriting Permissions Permissions can be set only at specific levels. Those levels are:
• /
• /namespace
• /manage
• /namespace/mynamespace (where “mynamespace” is your own self-created namespace)
The inherit flag dictates that the level where it is set "inherits" all permissions that are set at the level above it.
For example:
• You have three users, one of which is the "everyone" user.
Lattus REST API User’s Guide 169
Chapter 8: AuthenticationLimitations
• The users have permissions at "/namespace".
• You set the inherit flag at /namespace/mynamespace.
In this case the permissions set at /namespace will also apply to /namespace/mynamespace.
In [7]: q.dss.manage.showPermissionSettings('/namespace/mynamespace')
Out[7]: {u'Flags': [u'INHERIT'], u'User-Permissions': {}}
Tips and Tricks If you're using a browser to validate authentication, you can switch between users (and force re-authentication) by specifying the username in the URL.
Example: http://[email protected]:8080/manage/namespace/
The logfile of the clientdaemon is your first resource for troubleshooting authentication issues.
The value of both the username and the password of that user can be up to 256 complex characters (A-Z, a-z, 0-9 and/or special characters like #, @, !, etc.).
Limitations
• At most 5,000,000 users can be created (as many as the number of supported namespaces). This limitation applies only if you are upgrading from a clean installation of Lattus 3.2.0 or higher. If you upgraded from Lattus 3.1.x or earlier, there is a limit of 5000 users.
Note: With the five million user limitation, a new directory structure for the blockstores is introduced. After the upgrade to 3.4.0 the new directory structure is in place, the existing data is automatically moved into the new structure. This process is periodic and is spread over 180 days.
170 Lattus REST API User’s Guide
Chapter 8: AuthenticationLimitations
• The maximum size of the username and credentials is 256 bytes
• We recommend limiting the number of users that have rights on a single namespace to 64
• Authentication is currently not implemented in the Microsoft .net interface
Lattus REST API User’s Guide 171
Chapter 8: AuthenticationLimitations
172 Lattus REST API User’s Guide
Chapter 9Return Codes and
Troubleshooting
Return Codes
Lattus follows the standard HTTP/1.1 status code definitions. The following table describes a list of potential status codes:
TypeHTTP Description Message HTTP/1.1 Status Code Description
SUCCESS Continue 100 The client SHOULD continue with its request.
SUCCESS OK 200 The request was successfully executed.
SUCCESS Object Created 201 The request was successfully executed and the object is created.
REDIRECTION Not Modified 304 The precondition is not satisfied.
Lattus REST API User’s Guide 173
Chapter 9: Return Codes and TroubleshootingReturn Codes
ERROR Bad Request 400 The request could not be understood by the server due to malformed syntax. The client SHOULD NOT repeat the request without modifications.
ERROR Unauthorized 401 The request failed because of lack of permissions.
ERROR Forbidden 403 The Server understood the request, but refuses to fulfill it.
ERROR Object Not Found 404 The server has not found anything matching the Request-URI.
ERROR Conflict 409 Operation aborted because of a conflict.
ERROR Precondition Failed 412 The precondition given in the request header field evaluated to false when tested on the server.
ERROR Request Entity Too Large
413 The requested data (object, customer data, etc.) is larger than the server is able to process.
ERROR Metastore <ID> is full 500 The server encountered an internal error.
TypeHTTP Description Message HTTP/1.1 Status Code Description
174 Lattus REST API User’s Guide
Chapter 9: Return Codes and TroubleshootingReturn Codes
ERROR Internal Server Error 501 The server encountered an internal error.
FAILURE HTTP Version Not Supported
505 The server does not support the HTTP protocol version that was used in the request message.
ERROR Insufficient Storage 507 No more free space on the blockstores.
TypeHTTP Description Message HTTP/1.1 Status Code Description
Lattus REST API User’s Guide 175
Chapter 9: Return Codes and TroubleshootingReturn Codes
176 Lattus REST API User’s Guide
Chapter 10Microsoft .net Bindings
Note: There is currently no support for authentication when using the .net bindings
AmplistorClientLib Class Documentation
AmplistorClientLib.Types.AmpliObject Class Reference - Public Attributes
• string Id
• string Name• string NamespaceId• ObjectKind Kind • string PolicyId• long Size • long Crc32• string Status• long CreationDate • long ModificationDate• long VerificationDate
Lattus REST API User’s Guide 177
Chapter 10: Microsoft .net BindingsAmplistorClientLib Member Function Documentation
AmplistorClientLib.Types.AmpliPolicy Class Reference - Public Attributes
• const string REPAIR_SPREAD = "RepairSpread" • const string DYNAMIC_SAFETY = "DynamicSafety"• string Id• long SpreadWidth• bool SpreadWidthSpecified• long Safety• bool SafetySpecified • string HierarchyRules • long MaxSuperblockSize• bool MaxSuperblockSizeSpecified• string SafetyStrategy• long NMessages• bool NMessagesSpecified
AmplistorClientLib Member Function Documentation
Policy Operations Policy Creation
AmpliPolicy AmplistorClientLib.AmplistorClient.CreatePolicy (AmpliPolicy policyConfiguration)
Creates a new policy in the Lattus StoragePool.
Parameters:
• policyConfiguration: Configuration for the new policy. All fields of the configuration object are optional.
Returns:
The full policy configuration as stored during the creation.
178 Lattus REST API User’s Guide
Chapter 10: Microsoft .net BindingsAmplistorClientLib Member Function Documentation
Getting Policy Information
AmpliPolicy AmplistorClientLib.AmplistorClient.GetPolicy (string id)
Gets the policy configuration with id id from the LattusStoragePool. Parameters:
• id: id of the desired namespace.
Returns:
The full policy configuration as stored during the creation.
Listing Policies
IList<string> AmplistorClientLib.AmplistorClient.ListPolicies ()
Gets the policies on the LattusStoragePool.
Returns:
A list of policies on the Lattus StoragePool.
Namespace Operations Namespace Creation
AmpliNamespace AmplistorClientLib.AmplistorClient.CreateNamespace (AmpliNamespace namespaceConfig)
Creates a new namespace in the Lattus StoragePool.
Parameters:
• namespaceConfig: Configuration for the new namespace. Except for the Name & PolicyId fields, all fields of the configuration object are optional.
Returns:
The full namespace configuration as stored during the creation.
Lattus REST API User’s Guide 179
Chapter 10: Microsoft .net BindingsAmplistorClientLib Member Function Documentation
Listing Namespaces
IList<string> AmplistorClientLib.AmplistorClient.ListNamespaces ()
Gets the namespaces on the LattusStoragePool.
Returns:
A list of namespaces on the Lattus.
Getting Namespace Information
AmpliNamespace AmplistorClientLib.AmplistorClient.GetNamespace (string name)
Gets the namespace configuration with name name from the LattusStoragePool.
Parameters:
Name: name of the desired namespace.
Returns:
The full namespace configuration as stored during the creation.
Deleting Namespaces
void AmplistorClientLib.AmplistorClient.DeleteNamespace (string name)
Removes a complete namespace and its content from the LattusStoragePool.
Parameters:
Name: name of the namespace to be removed.
180 Lattus REST API User’s Guide
Chapter 10: Microsoft .net BindingsAmplistorClientLib Member Function Documentation
Object Operations Streambased Operations
Getting an object
Stream AmplistorClientLib.AmplistorClient.GetObject (string ampliNamespace, IList< string > pathEntries, string objectName)
Gets a read-only stream to a stored object in the Lattusin the specified namespace in the specified directory.
Parameters:
• ampliNamespace: Lattus namespace in which the specified directory needs to be located.
• pathEntries: The different sub-directory entries concatendated as a full path to the desired directory.
• objectName: The name of the saved object.
Returns:
A read-only stream to the saved object.
Uploading an Object
AmpliObject AmplistorClientLib.AmplistorClient.StoreObject (Stream sourceStream, string ampliNamespace, string ampliFilename)
Creates or updates an object in the LattusStoragePool using a stream.
Parameters:
• sourceStream: Stream to upload to the Lattus StoragePool.
• ampliNamespace: Lattus namespace in which the streamed object will be stored.
• ampliFilename: Lattus object name
Returns:
Detailed information about the stored stream.
Lattus REST API User’s Guide 181
Chapter 10: Microsoft .net BindingsAmplistorClientLib Member Function Documentation
Non-Streambased Operations
Uploading an Object
AmpliObject AmplistorClientLib.AmplistorClient.StoreObject (string sourceFilename, string ampliNamespace, string ampliFilename)
Creates or updates an object in the Lattus StoragePool using the locally accessible sourceFilename.
Parameters:
• sourceFilename: Local file to store on the Lattus StoragePool.
• ampliNamespace: Lattus namespace in which the file will be stored.
• ampliFilename: Lattus object name
Returns:
Detailed information about the stored object.
Creating a Directory
AmpliObject AmplistorClientLib.AmplistorClient.CreateDirectory (string ampliNamespace, IList< string > pathEntries)
Creates the specified directory path on the Lattus StoragePool in the specified namespace. Parameters:
• ampliNamespace: Lattus namespace in which the specified directory needs to be located.
• pathEntries: The different sub-directory entries concatendated as a full path to the desired directory.
Returns:
Detailed information about the created directory.
Deleting an object
void AmplistorClientLib.AmplistorClient.DeleteObject (string ampliNamespace, IList< string > pathEntries, string objectName)
Removes a stored object with name objectNme from the LattusStoragePool in the specified namespace in the specified directory.
182 Lattus REST API User’s Guide
Chapter 10: Microsoft .net BindingsAmplistorClientLib Member Function Documentation
Parameters:
• ampliNamespace: Lattus namespace in which the specified directory needs to be located.
• pathEntries: The different sub-directory entries concatendated as a full path to the desired directory.
• objectName: The name of the saved object.
Downloading an object
void AmplistorClientLib.AmplistorClient.DownloadObject (string ampliNamespace, IList< string > pathEntries, string objectName, string targetFilename)
Downloads a stored object from the Lattus in the specified namespace in the specified directory to a targetFilename.
Parameters:
• ampliNamespace: Lattus namespace in which the specified directory needs to be located.
• pathEntries: The different sub-directory entries concatendated as a full path to the desired directory.
• objectName: The name of the saved object.
• targetFilename: The local accessible path to the file to which the object needs to be saved.
Listing the Contents of a Directory
IList<string> AmplistorClientLib.AmplistorClient.ListObjects (string ampliNamespace)
Convenience method to get up to 50 object names in the root directory of the specified namespace.
Parameters:
AmpliNamespace: Lattus namespace in which the root directory needs to be located.
Lattus REST API User’s Guide 183
Chapter 10: Microsoft .net BindingsAmplistorClientLib Member Function Documentation
Returns:
A list of directory entries.IList<string> AmplistorClientLib.AmplistorClient.ListObjects (string ampliNamespace, IList< string > pathEntries, int? limit, string marker)
Gets the first limit object names found after marker in the specified directory of the specified namespace.
Parameters:
• ampliNamespace: Lattus namespace in which the specified directory needs to be located.
• pathEntries: The different sub-directory entries concatendated as a full path to the desired directory.
• limit: A value between 1 and 1000 indicating the number of items to fetch. If null is passed, the limit parameter is not passed through to the Lattus REST interface and which will make the Amplistor® return a default (50) maximum amount of entries.
• marker: Sets the start position at marker + 1. If null is passed, the marker parameter is not passed through to the Lattus REST interface which will make the Lattus start at the beginning of the supplied directory.
Returns:
a list of directory entries.
Getting Object information
AmpliObject AmplistorClientLib.AmplistorClient.ObjectDetails (string ampliNamespace, IList< string > pathEntries, string objectName)
Gets detailed information about a stored object in the Lattusin the specified namespace in the specified directory.
Parameters:
• ampliNamespace: Lattus namespace in which the specified directory needs to be located.
184 Lattus REST API User’s Guide
Chapter 10: Microsoft .net BindingsGetting Started
• pathEntries: The different sub-directory entries concatendated as a full path to the desired directory.
• objectName: The name of the saved object.
Returns:
Detailed information about a stored object.
Various Operations Managing the Streaming Buffersize
int AmplistorClientLib.AmplistorClient.BufferSize [get, set]
Gets or Sets the buffersize when streaming objects to and from the Lattus StoragePool. The default is 1 MiB.
Getting Started
Getting the .net Bindings
The latest version for 3.3.0 can be downloaded at https://confluence.amplidata.com/login.action;jsessionid=74BDF8F5058500D6B241D445944287EB?os_destination=%2Fnotpermitted.action%3Fversion%3D1%26modificationDate%3D1329472414000.
The md5 fingerprint of this library is A840B8E750846F59F926A4C46961DF66
Create a Policy and Namespace
Create both a default policy and default namespace to use with Lattus.
Create a Lattus Connection Object
AmplistorClientLib.AmplistorClient.AmplistorClient (string host, int port)
Creates a connection object to a Lattus host.
Lattus REST API User’s Guide 185
Chapter 10: Microsoft .net BindingsGetting Started
Parameters:
• host:Lattus hostname or ipaddress.
• port: Port on which the Lattus REST interface is listening.
Upload an Object The following examples assume you have created a form with an uploadbutton:
using System;using System.Collections.Generic;using System.ComponentModel;using System.Data; using System.Drawing; using System.IO;using System.Linq;using System.Text;using System.Windows.Forms;
using AmplistorClientLib;
namespace ampli_upload_app{public partial class FormUploadApp : Form{public FormUploadApp(){InitializeComponent();}
private void uploadbutton_Click(object sender, EventArgs e){string filename = textBox.Text;Stream filestream = File.OpenRead(filename);AmplistorClient client = new AmplistorClient("192.168.12.206", 8080);client.StoreObject(filestream, "testnamespace", "uploadedobject.txt");}}
186 Lattus REST API User’s Guide
Chapter 11Working With REST
This chapter describes how to start the q-shell, set up a test environment, begin developing, and resetting the environment.
Starting the q-shell sudo /opt/qbase3/qshell
Setting up a Test Environment
q.dsstestframework.setUp(enableAXR=True)
Expected output of the command:In [1]: q.dsstestframework.setUp(enableAXR=True)Out[1]: <_pm__factory._LocalEnvironment object at 0x31a59d
Starting with Development
When you setup the test-environment, a default 'test' namespace is created, which you can use without any authentication settings.
Example listing all the namespaces in the system using curl:curl http://[VM_IP_ADDRESS]:8080/namespace -H "Accept: text/plain"
Output on a freshly installed system will be:
"test"
Lattus REST API User’s Guide 187
Chapter 11: Working With REST
Resetting the Environment
q.dsstestframework.tearDown()
q.dsstestframework.setUp(enableAXR=True)
Caution: A reset will restart all services and remove all data from your environment.
188 Lattus REST API User’s Guide
Chapter 12Working with Lattus S3
The S3 Web services interface can be used to store and retrieve data from locations on the Web. This interface gives developers access to a data storage infrastructure that is scalable, reliable, fast, and inexpensive.
Starting the q-shell
sudo /opt/qbase3/qshell
Setting up an S3 Test Environment
q.dsstestframework.setUp(enableS3=True)
Expected output of the command:In [1]: q.dsstestframework.setUp()Out[1]: <_pm__factory._LocalEnvironment object at 0x31a59d
Lattus REST API Guide 189
Chapter 12: Working with Lattus S3Enabling S3
The memory address will differ from environment to environment.
Enabling S3
Adding a User for S3 Authentication
You need to create a user that will be used to authenticate:q.dss.manage.addUser('s3user','s3pass')q.dss.manage.setPermissions("/manage","s3user",["Read", "Create", "Delete","List", "Update"])
Configure the Client Daemon to Listen on its S3 Interface
1 Open the configuration file located at /opt/qbase3/cfg/dss/clientdaemons/0.cfg (where “0” is the GUID for the machine).
2 Change following entry to the config section using an editor: (Large S becomes lowercase s)
[S3]
address = 127.0.0.1:7070
into
[s3]
address = 127.0.0.1:7070
3 Save the cfg file and restart the client daemon using Q-shell:
q.dss.clientdaemons.restart()
Updating the HTTPS Proxy
AmpliStor is using a standard Web Proxy Server (Pound) to offer secure transmission services between the client and the S3 rest server. Incoming HTTPS S3 requests are decrypted and proxied onto the client daemon listening on localhost for incoming S3 requests.
You will need to tell the Web Proxy Server where it can find its pem file. A file with the .pem extension is used to create the private key. Pem stands for privacy enhanced mail.
190 Lattus REST API Guide
Chapter 12: Working with Lattus S3Enabling S3
Store your pem file in: /opt/qbase3/cfg/pound/server.pem (after creating the appropriate subdirectory)
Start the HTTPS proxy:
sudo /etc/init.d/pound restart
Note: By default, HTTPS is disabled and we will connect to the S3 service on localhost on port 7070 (by using the s3cmd proxy settings)
Defining the Policy for S3 Bucket Creation
Since S3 doesn't have the concept of storage policy, we need to provide a method to define what policy to use when creating an S3 bucket. There are 2 options.
When the Test environment has been created, a policy is created by default. You need to identify the unique policy-GUID in order to enable the S3 interface. This is done with following command:
In Q-shell:
mypolicy = q.dss.manage.listPolicies().keys()[0]
Setting a Storage Pool-Wide Policy for Creating S3 Buckets
When you select this option, you can define a single storage policy for creating ALL your S3 buckets.
Execute following command:
q.dss.manage.changeEnvironment(policyID=mypolicy)
Note: Once you associate a default S3 policy with your storage pool, you cannot remove this default policy anymore. The only thing you can do is change the default S3 policy.
Setting a Per-User Policy for Creating S3 Buckets
When you select this option, you can define a storage policy per user. When the authenticated user creates a bucket, the default policy assigned to that user will be used.
Lattus REST API Guide 191
Chapter 12: Working with Lattus S3Testing Your Setup
q.dss.manage.changeUser(userName='s3user', policyID=mypolicy)
Testing Your Setup
We installed the tool s3cmd by default on the machine and we created in the Amplidata home directory a configfile for it, called .s3cfg
Note: Quantum installed s3cmd version 1.1.0 beta3 since the 0.9 GA version contains several bugs.
Open this file with an editor and supply following fields:
• access_key (the user that you created)
• secret_key (the password you supplied for that user)
Note: The cfg file already has some default values that match these examples but modify them to match your setup. Keep in mind that s3cmd is very sensitive about spaces in the configuration file. It will show authentication or broken pipe errors if there is an additional space after access_key or secret_key.In these examples, we are assuming that the commands will be executed as the Amplidata user. As such, the s3cmd tool will use the configuration file in the home directory of the Lattus user account.
s3cmd will by default connect to s3.amazonaws.com. To prevent needing to update your hosts file for every bucket you add, we preinstalled and preconfigured dsnmasq (see http://www.thekelleys.org.uk/dnsmasq/doc.html)
Make sure that you have modified the resolv.conf file of this VM, so that a dns-lookup uses localhost (on which this dns server is running).
sudo vim /etc/resolv.conf
Make sure that your resolv.conf file starts with following line:
nameserver 127.0.0.1
192 Lattus REST API Guide
Chapter 12: Working with Lattus S3Testing Your Setup
Check that the s3.amazonaws.com FQDN resolves to the correct IP:
ping <bucketname>.s3.amazonaws.com
You can now execute following commands to create, list and delete buckets and to put, get and delete objects in these buckets:
Create a New Bucket s3cmd mb s3://mynewbucket
Bucket 's3://mynewbucket/' created
List Buckets s3cmd ls
2012-09-14 12:45 s3://mynewbucket
Put a File dd if=/dev/zero of=/home/amplidata/1MB bs=1M count=1
s3cmd put /home/amplidata/1MB s3://mynewbucket/file_s3_1
/home/amplidata/1MB -> s3://mynewbucket/file_s3_1 [1 of 1] 1048576 of 1048576 100% in 0s 3.85 MB/s done
Get a File s3cmd get s3://mynewbucket/file_s3_1 1MB.get.13
s3://mynewbucket/file_s3_1 -> 1MB.get.13 [1 of 1] 1048576 of 1048576 100% in 0s 11.85 MB/s done
List Files in a Bucket s3cmd ls s3://mynewbucket2012-09-14 09:20 99258469 s3://mynewbucket/file12012-09-14 09:15 99258469 s3://mynewbucket/file1-u2012-09-14 09:21 99258469 s3://mynewbucket/file22012-09-14 09:22 99258469 s3://mynewbucket/file32012-09-14 09:22 99258469 s3://mynewbucket/file42012-09-14 09:23 99258469 s3://mynewbucket/file52012-09-14 09:23 99258469 s3://mynewbucket/file62012-09-14 09:26 99258469 s3://mynewbucket/file7
Lattus REST API Guide 193
Chapter 12: Working with Lattus S3Rebooting an S3 Test environment
List All Objects in All Buckets
s3cmd la2012-09-14 09:20 99258469 s3://mynewbucket/file12012-09-14 09:15 99258469 s3://mynewbucket/file1-u2012-09-14 09:21 99258469 s3://mynewbucket/file22012-09-14 09:22 99258469 s3://mynewbucket/file32012-09-14 09:22 99258469 s3://mynewbucket/file42012-09-14 09:23 99258469 s3://mynewbucket/file52012-09-14 09:23 99258469 s3://mynewbucket/file62012-09-14 09:26 99258469 s3://mynewbucket/file72012-09-14 09:20 99258469 s3://mynewbucket/file12012-09-14 09:15 99258469 s3://s3test/testfile12012-09-14 09:21 99258469 s3://s3test/testfile2
Delete File From a Bucket
s3cmd del s3://mynewbucket/file_s3_1
File s3://mynewbucket/file_s3_1 deleted
Rebooting an S3 Test environment
When an S3 Test environment is rebooted, the data is persisted but the services are not automatically started. These should be started in the following sequence:q.manage.arakoon.start()q.dss.storagedaemons.start()q.dss.maintenanceagents.start()q.dss.clientdaemons.start()
194 Lattus REST API Guide
Chapter 12: Working with Lattus S3Tearing Down an S3 Test Environment
Tearing Down an S3 Test Environment
You can start from scratch by issuing following command from the q-shell (sudo /opt/qbase3/qshell):
q.dsstestframework.tearDown()
This command will remove all your data and stop all services.
Note: As a result of this action, you will have to reconfigure S3.
Getting Support
When escalating issues to your Quantum contact, make sure to provide following information:
Output of Following qshell Command
q.dss.manage.getRevisionInformation()
This will look like this:
version: 3.2.0, branch: default, revision:
379a00490bde08f5a1a456853aaa176df16cb5fd, compile_time: 19/06/2013 14:00:44 UTC'
Clientdaemon Log File This file is located under:
/opt/qbase3/var/log/dss/clientdaemons/
Lattus REST API Guide 195
Chapter 12: Working with Lattus S3Getting Support
196 Lattus REST API Guide