who watches the watchers?
DESCRIPTION
Security and asset integrity are becoming an increasingly important part of the asset management world. Detecting both intentional and unintentional asset or metadata modification is an important part of ensuring the overall integrity of an asset system. This talk will discuss how Pixar cross-check and cross-index their Perforce asset data and metadata with Pixar's Templar System so as to detect both intentional and unintentional file and metadata modification or tampering.TRANSCRIPT
#
Mark HarrisonTech Lead, Asset ManagementPixar
Who Watches the Watchers?
#
Who Can You Trust?
#
#
#
• Data• Metadata (file sizes, checksums, owners, etc)
Who Can You Trust(With Your Perforce Assets)?
#
• Very Good Track Record• P4 verify• P4 checkpoint• No Known Undetected Failures!
Trust Perforce!
#
• What if it does mess up?– Undetected error?– Metadata goes odd?
• It’s your job on the line!
Don’t Trust Perforce!
#
• People are Great!• People hardly ever make mistakes!• People are always looking out for your best
interests!
Trust People!
#
• Some People are Bad!• Some, just incompetent!• Some, both!• How can you tell? It doesn’t matter!• Intentional Data/Metadata Corruption
– Bad guy made a src code mod• Example, modded code, blame on msundy
– Crazy man put his picture in a film
Don’t Trust People!
#
• P4 Verify?– Good but not good enough– Intentional hacking– Plain old Bug
Trust, But Verify!
#
• “Who shall guard the self-same guardians?”• Need to have two systems that cross-check and
validate• Each system serves as a check on the other
– Catches (we hope!!) bugs and other oddities– Catches (we hope!!) intentional data modification
• Is two enough?– Down that path lies insanity
Quis custodiet ipsos custodes?
#
• We mirror all Perforce metadata (including checksums, sizes) in the Templar database
• Perforce is still the “system of record”• But, we can see if the system of record ever
changes• A Bad Guy will have to subvert both Perforce and
Templar to do his evil deeds
The Templar Approach
#
• Templar database initially mirrored Perforce data and metadata for “offline verify”
• Basically a “p4 verify” that bypassed the P4D– Didn’t cause a slowdown
• Takeaway:– Have a good abstraction, it can be used in multiple
places
Side Effect of Previous Work
#
SQL> desc p4_files
• ID generic asset id (234363343)
• P4PATH p4 path ("//ts3/myfile.jpg")
• HEADVER the head revision (3)
• REPOS repository name ("ts3")
SQL> desc p4_versions
• ID generic asset id (234363343)
• VERSIONOF this revision is a version of file x
• P4PATH p4 path ("//ts3/myfile.jpg")
• VERSION version number ("3")
• ISDELETED true if deleted version
• REPOS repository name ("ts3”)
Database Structure
#
SQL> desc asset_metadata
• ID generic asset id (234363343)
• md5sum checksum as per P4
• filesize filesize as per P4
• etc…
• For each asset ID, we store the metadata we care about
• Not just P4, but other asset types as well• All keyed by asset ID• Updates are by P4 checkin trigger
Database Structure (cont.)
#
• So far, things are good• Never detected a P4 error• Never detected an intentional modification• (Maybe the bad guys are just that good!)• But we can correlate our results with the Backup
System Metadata• All Systems Agree!!
Results
#
• Good to know that our systems are working• Good to know that our colleagues seem
trustworthy.• We don’t have to trust each other to like each
other.
Conclusion
#
Mark HarrisonTech Lead, Data ManagementPixar Animation StudiosMark is in charge of Pixar’s Data Management Group where he has a 50 year charter to store all data and metadata related to the Studio's feature films and related work.
Prior to that he lived in China and was the Chief Software Architect of the China Internet. His software supported the growth from 200K users to the current base of 350M users.
He studied Computer Science, worked in Texas, and wrote a couple of computer books.
(Speaker photo here)
#
RESOURCESCD Report: info.perforce.com/cd-report
White Paper: perforce.com/white-paper-link
Template: perforce.com/template