open system firmware from...fast boot time demanded by some work loads coreboot/linuxboot osf...
TRANSCRIPT
![Page 1: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/1.jpg)
![Page 2: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/2.jpg)
Open System Firmware From Scratch
Jonathan Zhang, Firmware Engineer, FacebookDavid Hendricks, Firmware Engineer, Facebook
OPEN SYSTEMS FIRMWARE
![Page 3: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/3.jpg)
Agenda● What is FB doing? Why?● The build system design● How do we enable our partners and the community?
OPEN SYSTEMS FIRMWARE
![Page 4: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/4.jpg)
Use cases of OSF in FB• Enabling OSF on compute/storage server fleets• Deploying OSF on networking systems (minipack)• Deploying OSF on telecom devices (Open Cellular)• Deploying OSF on embedded devices
![Page 5: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/5.jpg)
●Open source○ Transparent○ Operational efficiency, reactive vs. proactive for issues
●Modern architecture○ Simple and portable, leverages Linux○ Easy to add features and customization
●Familiarity○ Reuse runtime components○ Leverage engineering resources
● Fast boot time○ Demanded by some work loads
coreboot/LinuxBoot OSF motivation
![Page 6: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/6.jpg)
FB OSF architecture• Put a kernel+initramfs in boot ROM
![Page 7: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/7.jpg)
FB OSF architecture• Put a kernel+initramfs in boot ROM• Do minimal silicon init and jump to
LinuxBoot as early as possible
![Page 8: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/8.jpg)
FB OSF architecture• Put a kernel+initramfs in boot ROM• Do minimal silicon init and jump to
LinuxBoot as early as possible• Use Linux to boot Linux
![Page 9: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/9.jpg)
FB OSF architecture• Put a kernel+initramfs in boot ROM• Do minimal silicon init and jump to
LinuxBoot as early as possible• Use Linux to boot Linux• Add features and tools as needed
![Page 10: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/10.jpg)
FB OSF architecture• Put a kernel+initramfs in boot ROM• Do minimal silicon init and jump to
LinuxBoot as early as possible• Use Linux to boot Linux• Add features and tools as needed• Debug, build, and deploy on our schedule.
![Page 11: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/11.jpg)
FB OSF architecture• Put a kernel+initramfs in boot ROM• Do minimal silicon init and jump to
LinuxBoot as early as possible• Use Linux to boot Linux• Add features and tools as needed• Debug, build, and deploy on our schedule.• Transparent and reproducible
![Page 12: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/12.jpg)
FB OSF architecture• Put a kernel+initramfs in boot ROM• Do minimal silicon init and jump to
LinuxBoot as early as possible• Use Linux to boot Linux• Add features and tools as needed• Debug, build, and deploy on our schedule.• Transparent and reproducible• Bring modern, open-source development
practices to firmware
![Page 13: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/13.jpg)
Agenda● What is FB doing? Why?● The build system design● How do we plan to enable our partners and the community
OPEN SYSTEMS FIRMWARE
![Page 14: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/14.jpg)
Build stepsBuild steps
initramfs source
initramfs binary
kernel source
Linuxboot payload
coreboot source
Silicon vendor binary blobs
firmware image
![Page 15: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/15.jpg)
Components -- build environment●coreboot cross toolchain build for elf● <COREBOOT_ROOT>/util/crossgcc
●Tool chain for other components○ GCC version needs to be no older than 5.0.0
●Use docker to ensure identical build environment
![Page 16: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/16.jpg)
Components - initramfs (FB solution)● go○ Modern systems software programming language with pointers,
concurrency, IPC, rich libraries, etc.○ Built-in testing facilities○ https://golang.org/
● u-root○ A fully Go userland with Linux bootloaders○ Can be source-based like Perl Linux, or binary like Busybox ○ https://github.com/u-root/u-root/
● Binaries○ Can include C binaries such as flashrom, dmidecode, etc.
![Page 17: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/17.jpg)
Components - Linux kernel●Same kernel used in runtime environment
○ Production-quality drivers, familiar interfaces○ Minimized configuration for size considerations
●Kconfig○ Intuitive and familiar configuration system○ `make menuconfig`
![Page 18: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/18.jpg)
Components -- coreboot●Kconfig
○ Same as Linux●Firmware Support Package (FSP)
○ Encapsulates Platform Initialization (PI) modules for general consumption
○ Used by coreboot, u-boot, Slim Bootloader, etc.○ Detailed at another presentation: “Open System Firmware support
for multi socket OCP platforms - coreboot POC on Tiogapass and Skylake SP”
●Flexible - Able to integrate vboot and other features●Robust community
![Page 19: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/19.jpg)
Components -- SIP binary blobs● IFD● ME● Microcode
![Page 20: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/20.jpg)
Components -- Put them together
Build EnvironmentGCC crossgcc tool -- gcc crossgcc tool -- acpica other crossgcc tools
Build Environment Linuxboot (build scripts, build config, patches, configs)
gpt other go modules
go u-root
Initramfs
other binaries
flashrom
kernel
Kernel
other 3rdparty modules
Coreboot vboot
Coreboot
blobs
![Page 21: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/21.jpg)
Build configuration file●JSON format describing:○ What components are included○ Where to get a component○ Public repos, Internal repos, tar balls, etc.○ Which snapshot for a component○ The directory structure
![Page 22: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/22.jpg)
Configurations●ChromeOS Viral Product Data (VPD) binary blob
● https://chromium.googlesource.com/chromiumos/platform/vpd/
●FSP configurations●Boot log verbose level.●Version info:
● Overall version -- RO● Internal versions -- RO
![Page 23: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/23.jpg)
Facebook Linuxboot Repo● Includes:a. Build scripts.b. Build configurations.c. Kernel Kconfig, coreboot configd. Patches (in rare case)●Will be upstreamed●Build steps:a. Clone Linuxboot repo.b. cd linuxbootc. PLATFORM=<platform> ./build.sh
![Page 24: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/24.jpg)
Scenarios● Q: Given an image, how is it build?
● A: Run vpd tool to get internal_Versions VPD variable.■ Q: Given a target, how do I debug?
● A: Boot the target into u-root shell or OS shell, run vpd command to get VPD variables.
● A: run flashrom command to dump the image.■ Q: Given build info, how do I replicate the build?
● Clone Linuxboot repo according to the build info, and build
![Page 25: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/25.jpg)
Agenda● What is FB doing? Why?● The build system design● How do we plan to enable our partners and the community?
OPEN SYSTEMS FIRMWARE
![Page 26: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/26.jpg)
OCP platform submission
EVT
Partner Collaboration Partner Collaboration
DVT
Partner Collaboration
PVT
Community Involvement
MP
OCP Platform SubmissionCommunity Enablement
Hardware Design Spec
Open System Firmware
![Page 27: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/27.jpg)
Partner Collaboration -- pre OCP submission
Facebook developers
Partner developers
Upstream Repos(u-root, coreboot, etc.)
Facebook internal Repos
Private shared github Repos
New code
New code
![Page 28: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/28.jpg)
Community Enablement -- post OCP submission
●FB is a community member.●All pieces are downloadable and redistributable○ Linuxboot, coreboot code○ Silicon Vendor binaries
●CIT enabled●OCP members can customize for their infrastructure as
needed:○ community support + professional support
![Page 29: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/29.jpg)
Test framework -- ConTest●Continuous, on-demand, per-diff tests●Plug-ins to accommodate test infrustructures.●Will be open-sourced
![Page 30: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/30.jpg)
Call to Action• OCP OSF project○ https://www.opencompute.org/projects/open-system-fir
mware• OSF community ○ Linuxboot: https://www.linuxboot.org/○ u-root: https://github.com/u-root/u-root○ coreboot: https://www.coreboot.org/• Play with off-the-shelf OCP platform (such as Winwynn
TiogaPass)
![Page 31: Open System Firmware From...Fast boot time Demanded by some work loads coreboot/LinuxBoot OSF motivation. FB OSF architecture •Put a kernel+initramfs in boot ROM. FB OSF architecture](https://reader030.vdocument.in/reader030/viewer/2022040609/5ec9d923233be5791b227426/html5/thumbnails/31.jpg)