programming on windows 8.1: the new stream and storage paradigm (raffaele rialdi)
DESCRIPTION
Looking at the Windows 8.1 development platform, the streams and storage management are totally different from the past. Streams classes have changed, files and folders management is radically different and a new set of classes exist in the WinRT library to support the Windows Store application model and the new asynchronous paradigm. After a brief overview of asynchronous pattern in WinRT, the session will dig into the new streams and storage APIs showing practical examples of use for modern Windows Store applications.TRANSCRIPT
Premium community conference on Microsoft technologies itcampro@ itcamp14#
The new stream and storage paradigm
twitter: @raffaeler
email: [email protected]
blog: http://www.iamraf.net
Programming on Windows 8.1:
Premium community conference on Microsoft technologies itcampro@ itcamp14#
Huge thanks to our sponsors & partners!
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• ActivateInstance API to create objects
• Native types
• IVector<T> and IMap<T, K>
• HRESULT for errors
.NET Framework Projection magics
• Familiar new keyword
• Familiar BCL types
• List<T> and Dictionary<T, K>
• Exceptions
WinRT .NET projection
But projections and mappings can’t fix all the frictions
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• The need to enforce the async paradigm
– “fast and fluid” means a better UX
–Blocking apps are crappy!
• Solutions:
– async/await
–Promise pattern
The asynchronous problem
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• async / await language feature is easy
• Promise is a ‘continuation’
Async quick recap
var content = await FileIO.ReadTextAsync(storageFile);
lblResult.Text = result;
var content = File.ReadAllText(path);lblResult.Text = result;
Callback
Sync
Async
create_task(FileIO::ReadTextAsync(storageFile)).then( [ = ] (String^ content) {lblResult->Text = result;
}
C++ Promise
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• The need to enforce a sandbox
–Protecting the system and user’s resources from the Apps
• A sandbox is substantially a process with low privileges
–App process is created with an very poor token
– Leverage the Integrity Levels
–A high number of APIs cannot be used
• sockets, System.IO, Streams, http api, …
The security problem
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• System.IO
– Let you browse/navigate the disk
– Let you access absolute path
– Use the old-school synchronous pattern
• Storage API
– Must restrict the access to few well-known locations
• Can access arbitrary locations picked from the user
– Must never block the main thread
• Security and Async are two requirements not allowing mapping or conversion
The I/O APIs have both those problems
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Application Package– The place where EXE and DLL lives in
• Using MEF or an IoC? This is your code repository
– Xaml Uri: "ms-appx:///"
• If the manifest specified special folders …… and the Store certification approved it
– KnownFolder contains the special folders• Documents, Pictures, Music, etc.
– Note: Documents capability is hidden in VS and its usage is permitted only to company accounts
What can an app access to?
Windows.ApplicationModel.Package.Current.InstalledLocation
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• ApplicationData.Current.LocalFolder– Local assets, no storage limits
– Xaml Uri: "appdata:///local/"
• ApplicationData.Current.RoamingFolder– Synced content, limited in size
• ApplicationData.RoamingStorageQuota (100K)
– Xaml Uri: "ms-appdata:///roaming/"
• ApplicationData.Current.TempFolder– Use only for short duration operations
– Xaml Uri: "ms-appdata:///temp/"
Application-specific folders
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Files shipped with the package–Mark them as 'content' in Visual Studio
– "Copy Always" ensure fresh updated content
– From XAML use "ms-appx:///" or "/"
– From code use • Windows.ApplicationModel.Package.Current.
InstalledLocation
• var img = new Uri("ms-appx:///Assets/Logo.png");– Uri as only absolute in WinRT
Resources
<Image Source="/Assets/Logo.png" Height="100"/><Image Source="ms-appx:///Assets/Logo.png" Height="100"/>
Premium community conference on Microsoft technologies itcampro@ itcamp14#
DEMO:A LAP AROUND STORAGE API
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• There are still missing methods in some classes of the WinRT API
• .NET users can use Extension Methods in the shared code to fill the gap– Add a method which is identical to the one
available in Win8.1
• Trick! Extension methods do not prevail over real ones– Win8.1 will execute the real WinRT one
– WP8.1 will execute the extension method
Windows Phone 8.1 API differences
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Do you remember the security constraints?
– Poor token, low integrity level, restricted APIs
• Sometimes we need to bypass the limitations
– Accessing a file outside the ones owned by the App
• Pickers use the Runtime Broker to bypass the limitation
– Accesses the resources for the requesting App
– User is always in control of the Broker requests
Pickers
Premium community conference on Microsoft technologies itcampro@ itcamp14#
Pickers and the Runtime Security Broker
Kernel services
WinRT
RuntimeSecurity Broker
COM / Win32 filtered Complete COM / Win32
Component
Deviceaccess
PickerHost
…
Component
Broker UIProcess
AppContainerProcess
Restricted tokenLow Integrity Level
Standard tokenMedium Integrity Level
Premium community conference on Microsoft technologies itcampro@ itcamp14#
WORKING WITH STREAMS
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Streams can block the client code
–Until you read/write, you will never know
• Working in chunks makes sense
– Files, network, … typically send/receive chunks
• Buffering limits the use of async to loading or writing the buffer
• WinRT use a low level buffering concept
– IBuffer interface representing byte arrays
The need of buffering
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• It's made only of Capacity and Length– Creation via Buffer class (no read/write methods)– Write access is possible via
• WindowsRuntimeBuffer.Create• Low level access via COM interface
– IBufferByteAccess (C++ only)
– Conversions/mappings by:
IBuffer and friends
WinRT Type .NET Type .NET WinRT WinRT .NET
IBuffer Byte[] AsBuffer, CopyTo ToArray, CopyTo
IBuffer Byte N/A GetByte
IBufferStreamMemoryStream
AsStream,GetWindowsRuntimeBuffer AsStream
WindowsRuntimeBufferExtensions
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• DataReader can read IBuffer
• IInputStream and IOutputStream– Are backed up by IBuffer based buffers
IBuffer is used from other APIs
byte[] blob = Utiltities.CreateSampleBlob();byte[] blob2 = new byte[blob.Length];IBuffer buffer = blob.AsBuffer();using (DataReader reader = DataReader.FromBuffer(buffer)){
reader.ReadBytes(blob2);}Debug.Assert(Utiltities.IsSameDataAndDifferentReference(blob, blob2));
using (DataReader reader = DataReader.FromBuffer(buffer)){
buffer2 = reader.ReadBuffer((uint)blob.Length);blob2 = buffer2.ToArray();
}Debug.Assert(!buffer.IsSameData(buffer2));
WindowsRuntimeBufferExtensions
Premium community conference on Microsoft technologies itcampro@ itcamp14#
Streams hierarchy
IInputStreamReferenceIRandomAccessStream
Reference
IAsyncOperationWithProgress<IBuffer, uint>ReadAsync(IBuffer buffer,
uint count,InputStreamOptions options);
IAsyncOperationWithProgress<uint, uint> WriteAsync(IBuffer buffer);
IAsyncOperation<bool> FlushAsync();
string ContentType {get; }
IRandomAccessStream CloneStream();IInputStream GetInputStreamAt(
ulong position);IOutputStream GetOutputStreamAt(
ulong position);void Seek([In] ulong position);
bool CanRead { get; }bool CanWrite { get; }ulong Position { get; }ulong Size { get; set; }
Sequential access interfaces
IInputStream IOutputStream
IRandomAccessStream
IRandomAccessStreamWithContentType
IClosable(IDisposable)
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Maintain internally an IBuffer• LoadAsync load the buffer• ReadXYZ methods hit the buffer, not the stream!
– They can be safely synchronous
DataReader puts IBuffer and streams together
Stream
I/OLatency
0 Size
LoadAsync LoadAsync LoadAsync
IBuffer
DataReader
ReadBooleanReadByte
ReadBytes…
var file = await Package.Current.InstalledLocation.GetFileAsync(@"Assets\Logo.png");
using (var stream = await file.OpenReadAsync()){
using (var reader = new DataReader(stream)){
await reader.LoadAsync((uint)stream.Size);Debug.WriteLine("0x" +
reader.ReadByte().ToString("x2"));} // stream is disposed here ...// ... unless reader.DetachStream() is not called
}
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• Writes are against the IBuffer
• The stream is written only at StoreAsync time
• FlushAsync can be used to flush the underlying stream
DataWriter use the same schema
Stream
I/OLatency
0 Size
StoreAsync StoreAsync StoreAsync
IBuffer
DataWriter
WriteBooleanWriteByte
WriteBytes…
// ... using (var writer = new DataWriter(stream)){
writer.WriteInt32(1);written = await writer.StoreAsync();writer.DetachStream();
}// ...
Premium community conference on Microsoft technologies itcampro@ itcamp14#
WinRT Type .NET Type .NET WinRT WinRT .NET
IInputStream Stream AsInputStream AsStreamForRead
IOutputStream Stream AsOutputStream AsStreamForWrite
IRandomAccessStream Stream AsRandomAccessStream AsStream
Converting streams between .NET and WinRTbyte[] blob = Utiltities.CreateSampleBlob();byte[] blob2 = new byte[blob.Length];
using (var memStream = new MemoryStream()){
await memStream.WriteAsync(blob, 0, blob.Length);memStream.Seek(0, System.IO.SeekOrigin.Begin);
using (var reader = new DataReader(memStream.AsInputStream())){
await reader.LoadAsync((uint)blob.Length);reader.ReadBytes(blob2);
}}Debug.Assert(Utiltities.IsSameDataAndDifferentReference(blob, blob2));
Premium community conference on Microsoft technologies itcampro@ itcamp14#
TIPS
Premium community conference on Microsoft technologies itcampro@ itcamp14#
• RandomAccessStream static class expose helper methods to copy a stream
–CopyAsync, CopyAndCloseAsync
• C++ will take advantage of async/await
– they makes a huge difference!
• IBufferByteAccess
Useful tips to remember
Microsoft::WRL::ComPtr<Windows::Storage::Streams::IBufferByteAccess>GetByteBuffer(Windows::Storage::Streams::IBuffer^ buffer)
{Microsoft::WRL::ComPtr<Windows::Storage::Streams::IBufferByteAccess> ibba;Microsoft::WRL::ComPtr<IUnknown> unknown(reinterpret_cast<IUnknown*>(buffer));unknown.As<Windows::Storage::Streams::IBufferByteAccess>(&ibba);return ibba;
}
HRESULT Buffer([out] byte** value);
Premium community conference on Microsoft technologies itcampro@ itcamp14#
Q & A