![Page 1: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/1.jpg)
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Usman Shakeel – Amazon Web Services
Ryan Russell – Thinkbox Software
Cloud Rendering at Scale with
Thinkbox Deadline on AWS
![Page 2: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/2.jpg)
Visual Effects and
Animation1
Who is using AWS for Rendering?
2
3
Theme Parks
Gaming
Marketing Departments4
5
6
Manufacturing
Bio-Informatics &
Drug Design
7 Engineering and Architecture Design
![Page 3: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/3.jpg)
VFX/Animation Rendering - workflow components
CompostingModeling Backend
Rendering
Asset Management
Collaboration and Task Management
![Page 4: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/4.jpg)
Boutique Studios
• No or very little on-premise capacity
• Limited Licenses
• Project based work from customers
(large studios)
• Very tight budgets
• Some projects need very large scale
(more compute)
• No or very little infrastructure
optimization
• Content Security (mandated by their customers)
• Mostly All-in Cloud Workflows
A tale of two customers
Large VFX/Animation Houses/Studios
• A decent on-premise investment
• Unlimited Licenses
• Project based work from different Studio projects
• Tight Deadlines and producers decide the budgets
• Often under immense pressure for higher quality (more compute) and faster renders (more compute)
• Mastered the on-premise infrastructure optimization
• Content Security
• Hybrid Workflows
![Page 5: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/5.jpg)
They both ask us the same thing …
The Cloud Scale
The ability to spin up thousands of cores on-demand
Turning CapEx into OpEx
No need to invest in infrastructure that may not be needed
after projects are finished
The most up-to-date configurations
Latest CPU and memory configurations
Large bandwidth between instances
Fast Storage options
A project based “disposable” infrastructure
Flexible (Cloud Elastic) Licensing
![Page 6: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/6.jpg)
They both tell us the same thing …
=< $0.01per core/hour
Access to thousands of
cores whenever needed
No upfront investments in infrastructure
Easier collaboration
Ecosystem of
software providers
Access to large memory configs to do 6K/10K renders
Project based “disposable” infrastructure
![Page 7: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/7.jpg)
Rendering in the Cloud - State of the Union
Scale at a very cheap price
• EC2 Spot
– Spot Fleet
– Termination notification
– Further Automation with Lambda
• What is large scale?
– 10s/100s/1000s/10000s cores
![Page 8: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/8.jpg)
Rendering in the Cloud - State of the Union
Data/Content Migration
• Aspera, Signiant, File Catalyst, Expedat
Storage
• Shared File Systems
– Amazon Elastic File System (EFS)
– Intel Lustre
• Storage Cache for Hybrid Workloads
– Avere vFXT
– Penzura
– Netapp Cloud-onTap
![Page 9: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/9.jpg)
Rendering in the Cloud - State of the Union
Licensing at Cloud Scale
• SaaS
• AWS Marketplace/Usage based licensing
![Page 10: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/10.jpg)
Rendering in the Cloud - State of the Union
3D Modeling in the Cloud
• GPU based EC2 instances
• Terdici PCoIP, AWS Appstream, Nice, Frame, Otoy
• Windows and Linux Support (VNC+VirtualGL)
![Page 11: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/11.jpg)
Rendering in the Cloud - State of the Union
Content Security
• AWS alignment with the latest MPAA cloud based application
guidelines for content security – August 2015
• Amazon VPC private endpoint for S3 – enables a true private
workflow capability
• Encryption & key management capabilities using Amazon Key
management Service (KMS)
• Amazon Glacier Vault for high-value media/originals
![Page 12: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/12.jpg)
Introduction to Deadline
• What is Deadline?
– Cross-platform render farm management.
– Manages on-premise, Cloud-based, or hybrid render farms of all sizes.
– Supports over 70 content creation applications out of the box.
and more!
– Used in VFX, Design, Broadcast, AEC (Architecture, Engineering,
Construction), Education, etc.
![Page 13: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/13.jpg)
Introduction to Deadline
• What is Deadline used for?
– Distribute frames of an animation
sequence across multiple nodes.
– Perform other types of processing, including particle simulations, Quicktime
movie generation, file conversion, etc.
– Distribute regions of a still frame
across multiple nodes (Jigsaw).
![Page 14: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/14.jpg)
Use Deadline to automate your pipeline!
• Dependencies
– Allows jobs to start automatically.
– Job, frame, script, and asset
dependencies.
• Custom plugins
– Write Custom plugins to perform
any type of custom processing.
– Control anything with an API
(command line, scripting, etc).
![Page 15: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/15.jpg)
Use Deadline to automate your pipeline!
• Event triggers
– Automate tasks when jobs enter certain states.
• Update internal pipeline mgmt software when a job starts and finishes.
• Do some post processing on the rendered output after a job finishes (Draft).
• Draft
– Compositing Python module that is
included with Deadline.
– Automate compositing operations on rendered images.
• Quicktime movie generation (multiple movies at different resolutions).
• File conversion (convert EXRs to JPGs before downloading for review).
![Page 16: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/16.jpg)
Deadline Components
• Deadline Repository
– Consists of a MongoDB database and a shared file server.
– One Repository per render farm.
• Deadline Client
– Includes the Slave and Monitor applications (in addition to others).
– Slaves run on the render nodes to control the rendering process.
– Monitor is used to view render jobs and render nodes.
– Client applications connect directly to the database and file server.
![Page 17: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/17.jpg)
Rendering with Deadline in AWS Cloud
• Rendering in AWS Cloud with Deadline 7
– AWS can be an extension of your on-premise farm, or it can host your
entire render farm.
– Think of it as a remote office that you connect to over VPN.
![Page 18: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/18.jpg)
Rendering with Deadline in AWS Cloud
• What tools are currently available?
– Amazon Cloud Plugin for Deadline:
• View and control existing AWS instances
directly from the Deadline Monitor.
– Deadline VMX:
• Automates the launching / termination of
AWS instances based on jobs in queue.
– Deadline AWS Cloud Wizard:
• Set up a basic Deadline render farm in
the AWS Cloud in about 30 minutes.
![Page 19: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/19.jpg)
Deadline 8 makes Cloud rendering easier!
• Deadline 8
– Currently accepting beta requests (more info later).
– Introduces Usage Based Licensing for metered licensing.
– New Proxy Server application removes VPN requirement.
![Page 20: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/20.jpg)
Deadline 8 makes Cloud rendering easier!
• Usage Based Licensing
– Render nodes pull metered licenses from Thinkbox’s Cloud-based
license server.
– Usage is tracked per minute.
– Bulk minutes will be available via Thinkbox’s online store.
– Store will eventually host 3rd party licensing (Nuke, Vray, etc).
– Let’s check out the store!
![Page 21: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/21.jpg)
Deadline 8 makes Cloud rendering easier!
![Page 22: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/22.jpg)
Deadline 8 makes Cloud rendering easier!
• Deadline Proxy Server
– Facilitates communication between the Deadline applications and the
Deadline Database and Repository.
– Implements REST over HTTP(s), and can be combined with Nginx or
Apache for SSL security, removing the requirement of a VPN.
![Page 23: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/23.jpg)
Deadline 8 hybrid render farm
![Page 24: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/24.jpg)
Deadline 8 hybrid render farm
Local and Cloud render nodes in
the same render farm
Local and Cloud render nodes
working on the same job
Connected to Proxy Server
![Page 25: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/25.jpg)
What’s coming in the future?
• Deadline VMX Improvements
– Improved budget and cap flexibility.
• Budgets based on amount to spend
per day, per month, etc.
• Budgets based on number of concurrent AWS instances.
– Support for Amazon EC2 Spot Instances.
• Spot Instances are a great alternative for VFX rendering resources.
• Render more frames across more nodes concurrently at a reduced cost.
• If Spot Instances are terminated while rendering, Deadline can recover
those frames and assign to other render nodes.
![Page 26: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/26.jpg)
What’s coming in the future?
• AWS CloudFormation Templates
– Create an entire Deadline render farm infrastructure on AWS Cloud.
– Create AWS Cloud render nodes to connect to an existing on-premise
Deadline render farm.
• Asset Transferring
– Build asset awareness into Deadline jobs.
– Jobs won’t render until all assets are available.
– Missing assets will be automatically transferred.
![Page 27: AWS September Webinar Series - Visual Effects Rendering in the AWS Cloud with ThinkBox](https://reader033.vdocument.in/reader033/viewer/2022050613/586f73c71a28ab10258b58f3/html5/thumbnails/27.jpg)
Further Reference
• Join the Deadline 8 beta!
– Email [email protected] to join the beta.
– Free usage based licensing during the beta.
• Check out M&E solutions on AWS at:
http://aws.amazon.com/digital-media/solutions/
• Questions?