filesystem abstraction with flysystem
DESCRIPTION
These are the slide for my talk about Abstract Filesystems I did at Laracon 2014.TRANSCRIPT
FILESYSTEMABSTRACTION
WITH FLYSYSTEMBy @FrankDeJonge
WHAT IS FILESYSTEM ABSTRACTION?Bridging the gap between filesystems so the storage engine
becomes an implementation detail.
FILESYSTEMABSTRACTION 101
What are filesystems?How do we use filesystems?Moving to remote filesystems.Abstraction saves the day.
WHAT IS A FILESYSTEM?
LOGIC & STRUCTURE
WHAT IS A FILESYSTEM?Logic to structure and organize pieces of data in a storage
system.
WHAT'S IN A FILESYSTEM?FilesDirectoriesMetadata
COMMON METADATALocation / PathTimestampsPermissions / Ownership
FILE SPECIFICMime-typeFile sizeContents
DERIVED INFORMATION / PATHINFO():FilenameBasenameExtensionDirnameMime-type
FILESYSTEM STRUCTURE
STRUCTURE TYPES:Nested or Linear
src/
Filesystem.php
AdapterInterface.php
tests/
Filesystem.php
AdapterInterface.php
src/Filesystem.php
src/AdapterInterface.php
tests/Filesystem.php
tests/AdapterInterface.php
NESTED FILESYSTEMSAre the most commonHave directorieswhich have files and more directories... which have more files and directories... which have more files and directories.We're really used to this structure.
LINEAR FILESYSTEMAre like key/value storesdo have files, of courserequire a different approachdon't really have directories.
DIRECTORIES ARE A MYTH
WHAT CAN WE DO WITHA FILESYSTEM?
Several operations:
writereadupdatedelete
move / renamecopylistinspect
HANDLING FILESYSTEMS WITH PHP, A WORLDOF HURT.
In vanilla PHP for local files:
$result = file_put_contents($location, $contents);
$contents = file_get_contents($location);
unlink($location);
$timestamp = mtime($location);
$mimetype = mime_content_type($location); // DEPRECATED
$mimetype = (new Finfo(FILEINFO_MIME_TYPE))->file($location);
LISTING FILESglob($location . '/*'); // ?
scandir($location); // ?
readdir(opendir($location)); // ?
array_map('normalize_fileinfo',
iterator_to_array(new DirectoryIterator($location))); // ?
array_map('normalize_fileinfo',
iterator_to_array(new RecursiveIteratorIterator(
new RecursiveDirectoryIterator($location)
))); // ?
Memory problems? Use streams:
$resource = fopen($rLocation, 'r+');
$handle = fopen($location, 'w+');
while ( ! feof($resource)) {
fwrite($handle, fread($resource, 1024), 1024);
}
$result = fclose($handle);
fclose($resource);
WORKING WITH OTHERFILESYSTEMS.
BECAUSE REASONS.
SOME REASONS WHY:We want to scale (horizontally).... so, we want stateless apps.We don't have to serve files ourselves.Share files across multiple application.We want to enable a nice experience to our users which wasnot possible otherwise.
SOME CONSEQUENCESWe can't use built-in PHP function anymore.We'll have to start using API'sWe'll have to depend on third party code.
COMPARING INTEGRATIONS: WRITING FILESAWS/S3:
$options = [
'Body' => $contents,
'ContentType' => 'plain/text',
'ContentLength' => mb_strlen($contents),
];
$result = $awsClient->putObject($options);
Dropbox:
$result = $dropboxClient->uploadFileFromString(
$location,
$contents,
WriteMode::add());
Rackspace:
$result = $rackspaceClient->uploadObject($location, $contents);
WebDAV:
$result = $client->request('PUT', $location, $contents);
FTP:
$stream = tmpfile();
fwrite($stream, $contents);
ftp_fput($connection, $destination, $stream, FTP_BINARY);
fclose($stream);
DOWNSIDESDifferent APISupport different kinds of input (string vs stream support)Different results for EVERY operation
CHOOSING A FILESYSTEM ...can create technical deptcan create vendor lock-inrequire you to invest in them
SOLUTION?
USE AN ABSTRACTFILESYSTEM.
WHAT IS AN ABSTRACT FILESYSTEM?Logic and structure that enable a generalized API to work with
different filesystems.
WHAT DOES A ABSTRACT FILESYSTEM DO?It bridges the gap between different filesystem
implementations/packages and normalizes responses.
INTRODUCING:
FLYSYSTEMMANY FILESYSTEMS, ONE API
FLYSYSTEM IS:A filesystem abstraction layerLike a DBAL for files.Like illuminate/filesystem on steroids.
PACKED WITH SUPERPOWERS:Well tested codeEasy APICachable metadataEasy stream handling
FIRST LOOK AT FLYSYSTEMuse League\Flysystem\Adapter\Local;
$adapter = new Local(__DIR__.'/path/to/dir');
$fs = new Filesystem($adapter);
BASIC FS OPERATIONS:$fs->write($location, $contents);
$fs->read($location);
$fs->update($location, $contents);
$fs->listContents($location, $recursive);
$fs->delete($location);
$fs->getTimestamp($location);
USING AWS S3use League\Flysystem\Adapter\AwsS3;
$adapter = new AwsS3($client, 'bucket-name');
$fs = new Filesystem($adapter);
BASIC FS OPERATIONS:$fs->write($location, $contents);
$fs->read($location);
$fs->update($location, $contents);
$fs->listContents($location, $recursive);
$fs->delete($location);
$fs->getTimestamp($location);
USING FTPuse League\Flysystem\Adapter\Ftp;
$adapter = new Ftp($ftpSettings);
$fs = new Filesystem($adapter);
BASIC FS OPERATIONS:$fs->write($location, $contents);
$fs->read($location);
$fs->update($location, $contents);
$fs->listContents($location, $recursive);
$fs->delete($location);
$fs->getTimestamp($location);
MANY FILESYSTEMS
ONE API
FLYSYSTEM AND STREAMS:$resource = $dropbox->readStream($location);
$awsS3->writeStream($destination, $resource);
FLYSYSTEM IS A GATEWAY DRUG FOR...... DRY code... centralized problem domain handling... easily testable FS interactions... reduced technical dept... lower development costs... having less to learn.
TESTING FILESYSTEM INTERACTIONSUSING FLYSYSTEM
Becomes easier.Becomes more reliable.Might even become fun again.
DEPENDING ON FLYSYSTEMclass Author
{
public function __construct(FilesystemInterface $filesystem)
{
$this->filesystem = $filesystem;
}
public function getBio()
{
return $this->filesystem->read($this->getBioLocation());
}
}
TESTING THE DEPENDANT CODEpublic function testGetBio()
{
$expected = 'Some Bio';
$fsMock = Mockery::mock('League\Flysystem\FilesystemInterface');
$fsMock->shouldReceive('read')
->with('james-doe-bio.txt')
->andReturn($expected);
$autor = new Author($fsMock);
$this->assertEquals($expected, $author->getBio());
}
BONUS: this makes tests run super fast because you don't haveto wait for blocking filesystem operations (slow).
IMPROVED STABILITY OF DEPENDANT CODEClass should have one reason to change... changing filesystems isn't one of them.Most code shouldn't have to care where files are stored.Flysystem enables classes to be unaware of this.
OPENING UP TO NEW POSSIBILITIESUsing Dropbox for uploadsEase development by deferring filesystem choice.Painless pivots during application lifecycle.Nice to build a package on.
FLYSYSTEM INTEGRATIONSLaravel Integration:https://github.com/GrahamCampbell/Laravel-FlysystemSymfony Bundle:https://github.com/1up-lab/OneupFlysystemBundleBackup Manager:https://github.com/heybigname/backup-managerCartalyst Media:http://cartalyst.com
CRAZY FLYSYSTEM ADAPTERSNOTE: some might be urban myths, I haven't seen them all.
FlickrYoutubeReddit
THE LEAGUE OF EXTRAORDINARY PACKAGES
HTTP://GITHUB.COM/THEPHPLEAGUE/FLYSYSTEM
THANKS FOR LISTENING!Find me on twitter @FrankDeJonge