engineering and use of large formal specifications alastair reid … · huge value in being able to...

©2017ArmLimited

EngineeringandUseofLargeFormalSpecifications AlastairReid

ArmResearch

@alastair_d_reid

©2017ArmLimited�2

More Less

Data

Performance

MachineLearning

InternetofThings

SmartHomes

SelfDrivingCars

SocialMedia

Bugs

Crashes

Dataloss

Datacorruption

Dataleaks/theft

DDoSattacks

Cyber-Physicalattacks


BetterProgrammingLanguages

BetterSystemDesign

HardwareSecurity

Enforcement

AutomaticTest

Generation

FuzzTesting

ExploitDetection

BetterBug

Finding

FormalVerification

Legal/Regulatory


Specification

Specification Specification

Specification

Specification


What(formal)specificationsdoweneed?

Libraries:stdio.h,OpenGL,…

Languages:C,C++,ML,Javascript,Verilog,…

Network:TCP/IP,OAuth,DNS,TLS,WiFi,…

Filesystems:FAT32,NTFS,ext4,…

OSes:Posix/Linuxsystemcall,Linuxdevicedriver,KVM,UEFI,…

Hardware:CPU,PCIe,AMBA,NIC,…


Criticalpropertiesofspecifications

Scope-Completeness

-Notabstractingoutcriticaldetail

Applicability-Versionagnostic

-Vendoragnostic

Trustworthiness


OvercomingtheSpecificationBottleneck

CreatingformalspecificationsTestingspecificationsGettingbuyinUsingspecificationsFormalvalidationofspecificationsMakingyourspecificationspublic

“TrustworthySpecificationsoftheARMv8-Aandv8-Marchitecture,”FMCAD2016“EndtoEndVerificationofARMprocessorswithISAFormal,”CAV2016“Whoguardstheguards?FormalValidationofARMv8-MSpecifications,”OOPSLA2017“ISASemanticsforARMv8-A,,RISC-V,andCHERI-MIPS,”POPL2019

https://alastairreid.github.io/papers/


Creating formal specificationsTesting specificationsGetting buy in

“Trustworthy Specifications of the ARM v8-A and v8-M architecture,” FMCAD 2016


CreatingSpecifications


Pseudocode


ARMPseudocode

~40,000lines

-32-bitand64-bitmodes

-All4encodings:Thumb16,Thumb32,ARM32,ARM64

-Allinstructions(>1300encodings)

-All4privilegelevels(User,Supervisor,Hypervisor,SecureMonitor)

-BothSecuritymodes(Secure/NonSecure)

-MMU,Exceptions,Interrupts,Privilegechecks,Debug,TrustZone,…


Statusatthestart

- Nolanguagespec- Notools(parser,typechecker)- Incomplete(around15%missing)- Unexecuted,untested- Seniorarchitectsbelievedthatanexecutablespecwas

- Impossible- Notuseful- Lessreadable- Lesscorrect


ArchitecturalConformanceSuite

Processorarchitecturalcompliancesign-off

Large• v8-A32,000testprograms,billionsofinstructions• v8-M3,500testprograms,>250millioninstructions

Thorough• Testsdarkcornersofspecification

Hardtorun• Requiresadditionaltestinginfrastructure

©2017ArmLimited�15 ©2017ArmLimited

ProgressintestingArmspecification

- Doesnotparse,doesnottypecheck

- Can’tgetoutofreset

- Can’texecutefirstinstruction

- Can’texecutefirst100instructions

- …

- Passes90%oftests

- Passes99%oftests

- …

0

50

100


Measuringarchitecturecoverageoftests

Untested: op1*op2 == -3.0, FPCR.RND=-Inf


Creating a Virtuous Cycle

ARMSpec


Lessonslearnedaboutengineeringaspecification

Specificationscontainbugs

Hugevalueinbeingabletorunexistingtestsuites

-Needtobalanceagainstbenefitsofnon-executablespecs

Findwaystoprovidedirectbenefittootherusersofspec

-Theywilldosomeofthetesting/debuggingforyou

-Theywillsupportgettingyourchanges/specadoptedasmasterspec

-CreatesVirtuousCycle


Using Specifications

“End to End Verification of ARM processors with ISA Formal,” CAV 2016


VerificationofImplementations- BoundedModelChecking- Testing(GoldenReference)- DeductiveReasoning

VerificationofClients- FormallyverifyingOScode/etc.- VerifyingCompilers/Linkers

Generation- Testsuites(Concolic)- Simulators- PeepholeOptimisations- BinaryTranslators

Documentation- GeneratePDF/HTML- Interactivespecifications

SpecificationExtension- Testing/Exploration

StaticAnalysis- Abstractinterpretationofbinaries- Decompilationofbinaries- Reverseengineeringtools

InstrumentedExecution- MeasureCoverage- DrivingFuzzTesting


FormallyvalidatingARMprocessors-usinganexistingtool

ARM Specification

ARM Processor

TranslatetoVerilog

VerilogModelChecker


Checkinganinstrucpon

ADD


Checkinganinstrucpon

ADDCMP LDR STR BNE

Context


LessonsLearnedfromvalidatingprocessors

Veryeffectivewaytofindbugsinimplementations

Formallyvalidatingimplementationiseffectiveatfindingbugsinspec

-Trytofindmostofthebugsinyourspecbeforeyoustart

Hugevalueinbeingabletousespectovalidateimplementations

-Helpsgetformalspecificationadoptedaspartofofficialspec

Spec


Formal Validation of Specifications

“Who guards the guards? Formal Validation of ARM v8-M Specifications” OOPSLA 2017


OneSpecificationtorulethemall?

ArchitectureSpec

ComplianceTests

Processors

ReferenceSimulator


Rule JRJC Exit from lockup is by any of the following: • A Cold reset. • A Warm reset. • Entry to Debug state. • Preemption by a higher priority processor exception.



StateChangeXEventAEventB

StateChangeCEventD

R



StateChangeXEventAEventB

StateChangeCEventD

R

RuleR:X→A∨B∨C∨D


StateChange Exit from lockup Fell(LockedUp)

Event A Cold reset Called(TakeColdReset)

Event A Warm reset Called(TakeReset)

StateChange Entry to Debug state Rose(Halted)

Event Preemption by a higher priority processor exception

Called(ExceptionEntry)


“EyeballCloseness”Rule JRJC

Exit from lockup is by any of the following: • A Cold reset. • A Warm reset. • Entry to Debug state. • Preemption by a higher priority processor exception.

Fell(LockedUp)→Called(TakeColdReset)∨Called(TakeReset)∨Rose(Halted)∨Called(ExceptionEntry)


Rule VGNW Entry to lockup from an exception causes • Any Fault Status Registers associated with the exception

to be updated. • No update to the exception state, pending or active. • The PC to be set to 0xEFFFFFFE. • EPSR.IT to become UNKNOWN.

In addition, HFSR.FORCED is not set to 1.

OutofdateMisleading

AmbiguousUntestable


Counterexample

v8-M Spec

Rules

Convert Z3SMTSolver

+

~10,000lines ~1,000,000lines


LessonsLearnedfromvalidatingspecifications

Redundancyessentialfordetectingerrors

-Detectedsubtlebugsinsecurity,exceptions,debug,…

-FoundbugsinEnglishprose

Needsetof‘orthogonal’properties

-Invariants,Securityproperties,Reachabilityproperties,etc.

Eyeballcloseness

Neededtotranslatespecificationtoanotherlanguagetoletususeothertools


Making your specificationpublic


PublicreleaseofmachinereadableArmspecification

Enableformalverificaponofsotwareandtools

Machinereadable

Releases:

v8.2(4/2017)

v8.3(10/2017)

v8.4(6/2018)

v8.5(9/2018)hups://developer.arm.com/products/architecture/a-profile/explorapon-toolshups://github.com/alastairreid/mra_toolshups://github.com/herd/herdtools7/blob/master/herd/libdir/aarch64.cat


CambridgeUniversitySpecs/Tools

From“ISASemanticsforARMv8-A,,RISC-V,andCHERI-MIPS,”POPL2019UsedwithpermissionofREMSGroup,CambridgeUniversity




x86(ACL2)

Missing?




x86(ACL2)

Missing?

ACL2

Missing?


Work in Progress:Security of Architecture Specifications


Validatingsecurityofprocessorarchitectures

Scope

-Hardware-basedSecurityEnforcement(HSE)Mechanisms

-Confidentiality,Integrity,Availability

Challenges

-CompositionalAttacks

-CyclicdependenciesbetweenHSEs

-Microarchitecturalstorage/timingchannels


TheSpecificationBottleneck:ModellingRealWorldArtifacts

-Trustworthiness,ScopeandApplicability

-SignificantEngineeringEffort

-Importanceofsharingspecificationsacrossmanyusers

Spec


Thanks

Alasdair Armstrong (Cambridge U.)Alex Chadwick (ARM)Ali Zaidi (ARM)Anastasios Deligiannis (ARM)Anthony Fox (Cambridge U.)Ashan Pathirane (ARM)Belaji Venu (ARM)Bradley Smith (ARM)Brian Foley (ARM)Curtis Dunham (ARM)David Gilday (ARM)David Hoyes (ARM)David Seal (ARM)Daniel Bailey (ARM)Erin Shepherd (ARM)Francois Botman (ARM)

George Hawes (ARM)Graeme Barnes (ARM)Isobel Hooper (ARM)Jack Andrews (ARM)Jacob Eapen (ARM)Jon French (Cambridge U.)Kathy Gray (Cambridge U.)Krassy Gochev (ARM)Lewis Russell (ARM)Matthew Leach (ARM)Meenu Gupta (ARM)Michele Riga (ARM)Milosch Meriac (ARM)Nigel Stephens (ARM)Niyas Sait (ARM)Peng Wang (ARM)

Peter Sewell (Cambridge U.)Peter Vrabel (ARM)Richard Grisenthwaite (ARM)Rick Chen (ARM)Simon Bellew (ARM)Thomas Grocutt (ARM)Will Deacon (ARM)Will Keen (ARM)Wojciech Meyer (ARM)(and others)

ThankYou!Danke!Merci!谢谢!ありがとう!Gracias!Kiitos!


@alastair_d_reid

“TrustworthySpecificationsoftheARMv8-Aandv8-Marchitecture,”FMCAD2016“EndtoEndVerificationofARMprocessorswithISAFormal,”CAV2016“Whoguardstheguards?FormalValidationofARMv8-MSpecifications,”OOPSLA2017“ISASemanticsforARMv8-A,,RISC-V,andCHERI-MIPS,”POPL2019

https://alastairreid.github.io/papers/

engineering and use of large formal specifications alastair reid … · huge value in being able to...

Documents