many things · to have a windows hvm for gaming, you must have: • a dedicated amd gpu. by...

25
Qubes OS Many things Many things related to QubesOS Author: Neowutran

Upload: others

Post on 17-Aug-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

Qubes OS

Many things

Many things related toQubesOS

Author:Neowutran

Page 2: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

Contents1 Wiping VM 2

1.1 Low level storage technologies . . . . . . . . . . . . . . . . . . 21.1.1 Must read . . . . . . . . . . . . . . . . . . . . . . . . . 21.1.2 TL;DR of my understanding of the issue . . . . . . . . 21.1.3 Things that could by implemented by QubesOS . . . . 2

2 Create a Gaming HVM 22.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22.2 Prerequise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.3 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.4 Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.5 IOMMU Group . . . . . . . . . . . . . . . . . . . . . . . . . . 42.6 GRUB modification . . . . . . . . . . . . . . . . . . . . . . . . 42.7 Patching stubdom-linux-rootfs.gz . . . . . . . . . . . . . . . . 52.8 Pass the GPU . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.10 Bugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3 Create a Linux Gaming HVM, integrated with QubesOS 73.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 Hardware used . . . . . . . . . . . . . . . . . . . . . . . . . . 73.3 Main steps summary . . . . . . . . . . . . . . . . . . . . . . . 7

3.3.1 Detailled steps . . . . . . . . . . . . . . . . . . . . . . 83.3.2 Using a kernel provided by debian . . . . . . . . . . . . 8

3.4 Xorg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.4.1 Pulseaudio . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.5 Final notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4 Nitrokey and QubeOS 12

5 Recovery: Mount disk 12

6 Disposable VM 136.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

6.1.1 References . . . . . . . . . . . . . . . . . . . . . . . . . 146.1.2 What is a disposable VM? . . . . . . . . . . . . . . . . 14

6.2 Playing online video . . . . . . . . . . . . . . . . . . . . . . . 146.3 Web browsing . . . . . . . . . . . . . . . . . . . . . . . . . . . 156.4 Manipulating untrusted files/data . . . . . . . . . . . . . . . . 16

1

Page 3: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

6.5 Mounting LVM image . . . . . . . . . . . . . . . . . . . . . . 176.6 Replace sys-* VM . . . . . . . . . . . . . . . . . . . . . . . . . 186.7 Replace some AppVMs . . . . . . . . . . . . . . . . . . . . . . 18

7 Building a new QubesOS package 187.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187.2 Goal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187.3 The software . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197.4 Packaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197.5 Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2

Page 4: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

1 Wiping VMBefore starting this section, it should be noted that I don’t have a prior lowlevel knownledge of data wiping. This section is some thinkings about howcan we assure that a file have been deleted. And in our context, the goalwould be to wipe out a LVM image.

1.1 Low level storage technologies1.1.1 Must read

• Wei.pdf

• how-can-i-reliably-erase-all-information-on-a-hard-drive

• https://www.sans.org/blog/spin-stand-microscopy-of-hard-disk-data/

• https://www.vidarholen.net/ vidar/overwriting_hard_drive_data.pdf

• https://wiki.archlinux.org/index.php/Securely_wipe_disk

1.1.2 TL;DR of my understanding of the issue

Magnetic disk have non physical way of wiping data that are considered asrelatively reliable.

SSD are a nightmare to wipe anything, and in most of the cases, it isimpossible.

1.1.3 Things that could by implemented by QubesOS

TODO.Links to the ongoing discussion on the subjects. But basically my idea:

encryption, and store the encryption key on a specific physical device thatcan be logically destroyed (HDD ?), or that can be easily physically destroyed(MicroSD ?)

TODO

2 Create a Gaming HVM

2.1 ReferencesEverythings needed is referenced here

3

Page 5: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

• Usefull technical details

• Reddit thread of what is needed for GPU passthrough

• Solution to have more than 3Go of RAM in the Windows HVM

• Some old references

2.2 PrerequiseYou have a functional Windows HVM (Windows 7 or Windows 10). The"how to" for this part can be found on the Qubes OS documentation andhere: Usefull github comment. However, few tips:

• Do a backup (clone VM) of the Windows HVM BEFORE starting toinstall QWT (installing QWT is not required)

2.3 HardwareTo have a Windows HVM for gaming, you must have:

• A dedicated AMD GPU. By dedicated, it means: it is a secondaryGPU, not the GPU used to display dom0. Nvidia GPU are not sup-ported (or maybe with a lot of tricks).

• A really fast disk (M.2 disk)

• A lot of RAM

• A dedicated screen

• Dedicated gaming mouse and keyboard

In my case:

• Secondary GPU: AMD RX580

• Primary GPU: Some Nvidia trash, used for dom0

• 32Go of RAM. 12Go of RAM will be dedicated for the Windows HVM

• A fast M.2 disk

4

Page 6: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

2.4 ChecklistShort list of things to do to make the GPU passthrough work:

• You verified and confirmed that the secondary GPU is alone in itsIOMMU Group

• In dom0, you edited the file /etc/default/grub or /boot/efi/EFI/qubes/xen.cfgto allow PCI hiding for your secondary GPU, and regenerated the grubif needed

• You have patched stubdom-linux-rootfs.gz to allow to have more than3Go of RAM for your HVM

2.5 IOMMU GroupWarning: I am far from understanding the IOMMU group. Check onlinereferences on that subject. It seems that you can only do a successfullGPU passthough if you can passthrough everything that is in the IOMMUGroup of the GPU. Also, you can’t see your IOMMU Group when youare using Xen (the information is hidden from dom0). So, what I did: Ibooted from a Linux Mint Live USB. In the grub I enabled the IOMMU(iommu=1 iommu_amd=on), and then displayed the folder structure of/sys/kernel/iommu_group

tree /sys/kernel/iommu_group

My secondary GPU was alone in its IOMMU group.

2.6 GRUB modificationYou must hide your secondary GPU from dom0. To do that, you have toedit the GRUB. In a dom0 Terminal, type:

qvm-pci

Then find the devices id for your secondary gpu. In my case, it is dom0:0a_00.0and dom0:0a_00.1. Edit /etc/default/grub, and add the PCI hiding

GRUB_CMDLINE_LINUX="... rd.qubes.hide_pci=0a:00.0,0a:00.1 "

then regenerate the grub

grub2-mkconfig -o /boot/grub2/grub.cfg

5

Page 7: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

2.7 Patching stubdom-linux-rootfs.gzFollow the instructions here:github.com/QubesOS/qubes-issues/issues/4321

Copy-paste of the comment:

This is caused by the default TOLUD (Top of Low Usable DRAM) of3.75G provided by qemu not being large enough to accommodate the largerBARs that a graphics card typically has. The code to pass a custom max-ram-below-4g value to the qemu command line does exist in the libxl_dm.cfile of xen, but there is no functionality in libvirt to add this parameter. It ispossible to manually add this parameter to the qemu commandline by doingthe following in a dom0 terminal:

mkdir stubrootcp /usr/lib/xen/boot/stubdom-linux-rootfs

stubroot/stubdom-linux-rootfs.gz↪→

cd stubrootgunzip stubdom-linux-rootfs.gzcpio -i -d -H newc --no-absolute-filenames <

stubdom-linux-rootfs↪→

rm stubdom-linux-rootfsnano init

Before the line "#$dm_args and $kernel are separated with \x1b to allowfor spaces in arguments." add:

SP=$'\x1b'dm_args=$(echo "$dm_args" \| sed "s/-machine\\${SP}xenfv/-machine\

\\${SP}xenfv,max-ram-below-4g=3.5G/g")

Then execute:

find . -print0 | cpio --null -ov \--format=newc | gzip -9 > ../stubdom-linux-rootfssudo mv ../stubdom-linux-rootfs /usr/lib/xen/boot/

Note that this will apply the change to all HVMs, so if you have anyother HVM with more than 3.5G ram assigned, they will not start without theadapter being passed through. Ideally to fix this libvirt should be extended topass the max-ram-below-4g parameter through to xen, and then a calculationadded to determine the correct TOLUD based on the total BAR size of thePCI devices are being passed through to the vm.

6

Page 8: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

2.8 Pass the GPUIn qubes settings for the windows HVM, go to the "devices" tab, pass the IDcorresponding to your AMD GPU. (in my case, it was 0a:00.0 and 0a:00.1)And check the option for "nostrict reset" for those 2. In some case, you mightalso need to set the "permissive" flag to true (But I didn’t need that with theRX 580):

qvm-pci attach windows-hvm dom0:0a_00.0 -o permissive=True -ono-strict-reset=True↪→

qvm-pci attach windows-hvm dom0:0a_00.1 -o permissive=True -ono-strict-reset=True↪→

2.9 ConclusionDon’t forget to install the GPU drivers, you can install the official one fromAMD website, no modification or trick to do. Nothing else is required to makeit work (in my case at least, once I finish to fight to find those informations).If you have issues, you can refer to the links in the first sections. If it doesn’twork and you need to debug more things, you can go deeper.

• Virsh (start, define, ...)

• /etc/libvirt/libxl/

• xl

• /etc/qubes/templates/libvirt/xen/by-name/

• /usr/lib/xen/boot/

• virsh -c xen:/// domxml-to-native xen-xm /etc/libvirt/libxl/...

I am able to play games on my windows HVM with very good perfor-mances. And safely.

2.10 BugsThe AMD GPUs have a bug when used in HVM: each time you will rebootyour windows HVM, it will get slower and slower. It is because the AMDGPUs is not correctly resetted when you restart your windows HVM Twosolutions for that:

• Reboot your computer

7

Page 9: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

• In the windows HVM, use to windows option in the system tray to"safely remove devices", remove your GPU. Restart the HVM.

This bug is referenced somewhere, but lost the link and too lazy to searchfor it.

3 Create a Linux Gaming HVM, integratedwith QubesOS

3.1 GoalsWe want a qube with the following characteristics:

• Standalone linux from a template

• With GPU passthough

• Full Qubes integration

• Able to play video games from Steam

3.2 Hardware usedThe same as for my article "Create a Gaming HVM".

Prerequise You already followed my article "Create a Gaming HVM"

3.3 Main steps summary1. Create a standalone qube based on a debian template

2. Install the GPU drivers and Steam

3. Make the qube bootable without a kernel provided by dom0

4. Create a Xorg configuration file for your screen and GPU

5. Install your favorite Windows Manager and some Xorg input depen-dencies

6. Create specific pulseaudio config for muliseat Xorg

8

Page 10: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

3.3.1 Detailled steps

Create a new standalone based on a debian template (I used the debianminimal template, but it is a better idea to use the debian template). Theninstall the required dependencies for Steam and the AMD GPU

dpkg --add-architecture i386apt updateapt install pciutils steam firmware-linux-nonfree

libgl1-mesa-dri xserver-xorg-video-amdgpu libgl1:i386mesa-vulkan-drivers mesa-vulkan-drivers:i386

↪→

↪→

3.3.2 Using a kernel provided by debian

You must run the kernel provider by debian, because we will use some non-default kernel module for the GPU driver. Just follow the doc: managing-vm-kernel.

HVM In the qube configuration, set it to HVM, without kernel and do allthe PCI passthough you want. I do 3 PCI passthough:

• GPU: VGA

• GPU: Audio

• A USB controller. I plug my gaming mouse and keyboard on it

3.4 XorgNow Xorg and Pulseaudio. From XKCD:

9

Page 11: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

Things you need to install:

• The Xorg input driver to support your mouse and keyboard

• A pulseaudio gui client

• Your favorite Windows Manager

In my case, it is:

apt install xserver-xorg-input-kbd xserver-xorg-input-libinputxserver-xorg-input-mouse pavucontrol i3↪→

Then create a xorg configuration file for your GPU and screen. My file,named "AOC.conf":

Section "ServerLayout"Identifier "Gaming"Screen 0 "AMD AOC" Absolute 0 0EndSection

10

Page 12: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

Section "Device"Identifier "AMD"Driver "amdgpu"# The BusID value will change after each qube reboot.BusID "PCI:0:8:0"EndSection

Section "Monitor"Identifier "AOC"VertRefresh 60# https://arachnoid.com/modelines/Modeline "1920x1080" 172.80 1920 2040 2248 2576 1080 1081 1084

1118↪→

EndSection

Section "Screen"Identifier "AMD AOC"Device "AMD"Monitor "AOC"EndSection

We can’t know what is the correct BusID before the qube is started. Andit change after each reboot. So let’s write a script — named "xorgX1.sh" —that update this configuration file with the correct value, then start a binaryon the Xorg X screen nř1.

#!/bin/bash

binary=${1:?binary required}

# Find the correct BusID of the AMD GPU, then set it in theXorg configuration file↪→

pci=$(lspci | grep "VGA" | grep "AMD/ATI" | cut -d " " -f 1 |cut -d ":" -f 2 | cut -d "." -f 1 | cut -d "0" -f 2)↪→

sed -i "s/PCI:0:[0-9]:0/PCI:0:$pci:0/g" /home/user/AOC.conf

# Start the Xorg server for the X screen number 1.# The X screen nř0 is already used for QubesOS integrationstartx "$binary" -- :1 -config /home/user/AOC.conf

You can launch you favorite Windows Manager like that

11

Page 13: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

sudo ./xorgX1.sh /usr/bin/i3

Now... another issue: You don’t have sound for anything that is not onXorg server X screen nř0.

3.4.1 Pulseaudio

So you need to configure pulseaudio for Xorg multiseat. The archlinux doc-umentation explain that very well: Xorg multiseat Use the option with-out system-mode deamon and adapt it to qube: Add the following line to/etc/pulse/qubes-default.pa

load-module module-native-protocol-tcp auth-ip-acl=127.0.0.1

Then add this config for root:

mkdir /root/.pulseecho "default-server = 127.0.0.1" > /root/.pulse/client.conf

The sound was buggy/laggy on my computer. So tried to find a workaroundby playing with pulseaudio settings. It was more or less random tries, so Ican’t really explain it: In /etc/pulse/daemon.conf add the following lines:

default-fragments = 60default-fragment-size-msec = 1high-priority = norealtime-scheduling = nonice-level = 18

In /etc/pulse/qubes-default.pa change

load-module module-udev-detect

to

load-module module-udev-detect tsched=0

3.5 Final notesIt is working, integrated with QubesOS, and good enough to play video games(I am playing / played on it: FFXIV, Tomb raider, Avorion, Don’t Starve,. . . ). However, it is a bit slower than on a Windows HVM.

12

Page 14: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

3.6 References• Archlinux: PulseAudio

• Archlinux: PulseAudio/Troubleshooting

4 Nitrokey and QubeOSTo use a nitrokey on QubeOS, a USB passhrough is required. This means,you need to have a sys-usb VM. This is mentioned in the Qubes Documen-tation.

Note, you cannot pass through devices from dom0(in other words: a USB VM is required).

If you are using a USB keyboard, the sys-usb VM is not installed by default.If you are using a USB keyboard, you have 2 options:

• Create a sys-usb VM and assign a USB Controller to it.

• If you can’t assign a USB Controller (ex: You only have 1 on yourcomputer and can’t buy another), then buy and use a PS/2 Keyboard.

5 Recovery: Mount disk#!/bin/bash

######################################################### Make a LVM image appear in qvm-block# Create a disposable VM# Attach the image to the newly created disposable VM# Wait until the disposable VM is destroyed# Remove the LVM image from the qvm-block list######################################################

image=${1?Image file is required, example"/dev/qubes_dom0/vm-debian-9-tmp-root"}↪→

dvm=${2?DVM template name is required, example:"fedora-29-dvm"}↪→

dev=$(basename $(readlink "$image"))qubesdb-write/qubes-block-devices/$dev/desc "$image"

13

Page 15: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

list_before=$(qvm-ls | cut -d " " -f1 | sort)qvm-run -v --dispvm=$dvm --service qubes.StartApp+xterm &sleep 5list_after=$(qvm-ls | cut -d " " -f1 | sort)diff=$(comm -3 <(echo "$list_before") <(echo "$list_after"))qvm-block attach $diff dom0:$devwaitqubesdb-rm /qubes-block-devices/$dev/

Then mount the volume.

mount /dev/xxxx /xxxxx

In some case, the volume is partitionned, so you need to mount the par-tition. First, print the partition table

sudo parted /dev/XXXXXXXXXX unit B print

Example:

$:sudo parted /dev/windows-vg/vg-game-root unit B printModel: Linux device-mapper (thin) (dm)Disk /dev/dm-138: 314572800000BSector size (logical/physical): 512B/512BPartition Table: msdosDisk Flags:

Number Start End Size TypeFile system Flags↪→

1 1048576B 105906175B 104857600B primaryntfs boot↪→

2 105906176B 314571751423B 314465845248B primary ntfs

Then mount the partition you want

sudo mount -o loop,offset=105906176 -t ntfs /dev/XXXXXXXXXX/mnt/↪→

6 Disposable VMI found some interesting ideas on Reddit, Qubes OS Official Documentation,Qubes OS mailling list, Qubes OS unofficial documentation, and I wrotesome. The main goal of this document is to provide examples of what ispossible to be done with disposableVM.

14

Page 16: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

6.1 Introduction6.1.1 References

• disposablevm

• disposablevm-customization

6.1.2 What is a disposable VM?

DisposableVM are created for a specific task, so it doesn’t contains data froma previous use. Strong points:

• It doesn’t contains any previous data — unless you added personnaldata in the templateVM, but that is a very bad idea —, so only thedata from the current task can be exploited by a malware.

• If it get infected by a malware, the malware cannot persist.

You want to use DisposableVM everytime you can.

6.2 Playing online videoThe following bash function will:

1. Create a new disposable VM

2. Run the "youtube-dl" on the given URL parameter

3. Launch the downloaded video with the default video player

4. Destroy the disposable VM once the video is over

yt(){qvm-run-vm --dispvm youtube-dl "$1" -q --exec 'xdg-open {}'

}

Examples:

# Download a youtube videoyt e07q7afCakc

# Download any videoyt https://tube.neowutran.ovh/videos/watch/b66d6a30-3d8e-41b7- ⌋

aa74-43e462bcb643↪→

If you want to download/play video that weight more than 1.8Go, youneed to increate the Private Storage Max Size of the DisposableVM Template.

15

Page 17: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

6.3 Web browsingLike nearly everyone, I have some plugins installed on my browser. So itrequire to have the plugins / config installed directly in the TemplateVM.

1. Create a new ‘web’ disposableVM Template

2. In the ‘web’ template, configure your browser and install the pluginsyou need.

And now you can use your newly create DisposableVM to browse theweb. However, you probably open web page in many VM (from email, rss,chat room, ...). So you need a configuration to automatically open web pagein a disposable VM.

Create the file ‘ /.local/share/applications/browser-dvm.desktop’ withthe following content:

[Desktop Entry]Encoding=UTF-8Name=BrowserVMExec=qvm-open-in-vm @dispvm:web %uTerminal=falseX-MultipleArgs=falseType=ApplicationCategories=Network;WebBrowser;MimeType=x-scheme-handler/unknown;x-scheme-handler/about;text/ ⌋

html;text/xml;application/xhtml+xml;application/xml;applic ⌋

ation/vnd.mozilla.xul+xml;application/rss+xml;application/ ⌋

rdf+xml;image/gif;image/jpeg;image/png;x-scheme-handler/ht ⌋

tp;x-scheme-handler/https;

↪→

↪→

↪→

↪→

Due to what I count as a bug, you also need to edit dom0. Add thefollowing line to the file ‘/etc/qubes-rpc/policy/qubes.OpenURL’

# Syntax information:# @dispvm means "The default disposableVM Template"# @dispvm:XXX means "The disposableVM Template XXX"@anyvm @dispvm:web allow

Set it as default and test it.

xdg-settings set default-web-browser browser-dvm.desktopxdg-open https://www.qubes-os.org

16

Page 18: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

If you have a lot of VM and want to apply this configuration to most ofthem, it can be automated instead of doing that manually for every vm. Indom0, create a new bash file ‘browser.sh’

cat <<EOF > ~/.local/share/applications/browser-dvm.desktop[Desktop Entry]Encoding=UTF-8Name=BrowserVMExec=qvm-open-in-vm @dispvm:web %uTerminal=falseX-MultipleArgs=falseType=ApplicationCategories=Network;WebBrowser;MimeType=x-scheme-handler/unknown;x-scheme-handler/about;text/ ⌋

html;text/xml;application/xhtml+xml;application/xml;applic ⌋

ation/vnd.mozilla.xul+xml;application/rss+xml;application/ ⌋

rdf+xml;image/gif;image/jpeg;image/png;x-scheme-handler/ht ⌋

tp;x-scheme-handler/https;

↪→

↪→

↪→

↪→

EOF

xdg-settings set default-web-browser browser-dvm.desktop

Then adapt and execute the following command:

# For every VM, except "web", "vpn-internal" and"anon-whonix", execute the content of the file"./browser.sh"

↪→

↪→

qvm-run -all --exclude vpn-internal --exclude web --excludeanon-whonix "$(cat ./browser.sh)"↪→

I know there is a tool more "specialized" for this kind of task, called "Salt",but I found it quite complex to use for what I want to do. For anyone readingthis with knowledge about Salt, what is the correct way of doing that?

Security notice Like every template VM, if the template VM is compro-mised, every disposableVM / appVM based on it are compromised too.

6.4 Manipulating untrusted files/dataSame idea as for the web example, set the default application for XXXX tobe: create a new disposableVM and open it inside.

Create the file ‘ /.local/share/application/pdf-dvm.desktop’

17

Page 19: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

[Desktop Entry]Encoding=UTF-8Name=PdfVMExec=qvm-open-in-dvm %uTerminal=falseX-MultipleArgs=falseType=ApplicationCategories=PdfViewer;MimeType=application/pdf

Set it as default and test it.

xdg-mime default pdf-dvm.desktop application/pdfxdg-open ./test.pdf

6.5 Mounting LVM imageSource

#!/bin/bash

######################################################### Make a LVM image appear in qvm-block# Create a disposable VM# Attach the image to the newly created disposable VM# Wait until the disposable VM is destroyed# Remove the LVM image from the qvm-block list######################################################

image=${1?Image file is required, example"/dev/qubes_dom0/vm-debian-tmp-root"}↪→

dvm=${2?DVM template name is required, example: "fedora-dvm"}dev=$(basename $(readlink "$image"))qubesdb-write /qubes-block-devices/$dev/desc "$image"list_before=$(qvm-ls | cut -d " " -f1 | sort)qvm-run -v --dispvm=$dvm --service qubes.StartApp+xterm &sleep 5list_after=$(qvm-ls | cut -d " " -f1 | sort)diff=$(comm -3 <(echo "$list_before") <(echo "$list_after"))qvm-block attach $diff dom0:$devwaitqubesdb-rm /qubes-block-devices/$dev/

18

Page 20: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

6.6 Replace sys-* VMUsing static DisposableVMs for sys-*

6.7 Replace some AppVMsFor any VM that doesn’t need to persistantly store new data, you can replaceit with a static disposableVM. That way, every time you launch it, it iscompletly clean.

One example from my setup: A qube that have a specific settings (browserconfiguration, net-vm, firewall and some other), but never need to store newdata.

7 Building a new QubesOS package

7.1 References• QubesOS minisubmit video (QubesOS - 3mdeb), by Frédéric Pierret

• qubes-skeleton

7.2 GoalThe goal will be to create simple Qube software that do the following:

• Provide a binary named "qvm-druide-antidote"

• The binary will create a new disposableVM, and send file to it

• The disposableVM will open the software "Antidote" from "Druide" ’(itis a proprietary spellchecker) on the file it received

• The user fix the spelling issue and close "Antidote"

• The corrected file is sent back to the original VM

We will only explain the "packaging" and "building" part. My repository:qubes-app-linux-druide-antidote

19

Page 21: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

7.3 The softwareTo be run on the "client" vm:

#!/bin/bashFILE=${1?file required}TMPFILE=$(mktemp --tmpdir druide-antidote-XXXXXXXX)BACKUP=$(mktemp --tmpdir druide-antidote-backup-XXXXXXXX)cp "$FILE" "$BACKUP"echo "Backup file is: $BACKUP" >&2

cat "$FILE"exec >&-cat > "$TMPFILE"[ -s "$TMPFILE" ] && mv "$TMPFILE" "$FILE"

To be run on the "server" vm:

#!/bin/bash

FILE=$(mktemp --tmpdir druide-antidote-XXXXXXXX)

cat > "$FILE"/opt/Druide/Antidote10/Application/Bin/Antidote10 -f "$FILE"

2> /dev/null > /dev/null↪→

cat "$FILE"

rm -f "$FILE"

7.4 PackagingI want it to be:

• compatible with the following templates:

– Fedora– Debian– Archlinux

• easily reusable for other software that use this kind of client-server RPC

For the easily reusable part, lots of packaging variables come from the README.md.Read the file "vars.sh" and "archlinux/PKGBUILD" to understand better.PKGBUILD:

20

Page 22: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

pkgname=(qubes-$(./vars.sh name)-vm)pkgver=$(cat version)pkgrel=$(cat rel)arch=(x86_64)pkgdesc=$(./vars.sh description)url=$(git remote get-url origin)license=(GPL3)depends=()makedepends=(pandoc git)

build() {ln -s "$srcdir"/../ "$srcdir/src"

}check(){src/tests/all

}package() {cd srcmake install-vm DESTDIR="$pkgdir/"

}

vars.sh:

#!/bin/bash

name(){head -n 1 README.md | cut -d ' ' -f 2 | cut -d '(' -f 1 | cut

-c5- | tr [:upper:] [:lower:]↪→

}

description(){awk '/DESCRIPTION/,!//' ./README.md | tail -n +3 | awk

'1;/=======/{exit}' | head -n -3↪→

}

variable=${1:?Variable name is required. name, description orcamelname}↪→

case "$variable" in"name"):name;;

"description"):

21

Page 23: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

description;;

"camelname"):name | sed 's/^\([A-Za-z0-9]\)\([A-Za-z0-9]\+\)-\([A-Za-z0-9 ⌋

]\)\([A-Za-z0-9]\+\)$/\U\1\L\2\U\3\L\4/g'↪→

;;"summary"):description | head -n 1;;

*):echo "This variable doesn't exist" >&2exit 1

;;esac

For the building flow, it start from the template packaging (./debian/, ./arch-linux/, ./rpm/), then the build script of the template will call the Makefile

7.5 BuildingI learned everything in this section from the minisubmit video I referencedbefore.

We need to build this package using the standard qubes builder. So themost important things are in the next file named "app-builder.conf".

• How to build a QubesOs package that doesn’t come from the officialrepository

• How to build only that package and not everything related to QubesOS

app-builder.conf :

# vim: ft=make ts=4 sw=4

GIT_BASEURL ?= https://github.comGIT_PREFIX ?= QubesOS/qubes-NO_SIGN ?= 1#BRANCH ?= release4.0

BACKEND_VMM=xen

DIST_DOM0 ?= fc32DISTS_VM ?= fc32+minimal buster+minimal bullseye+minimal

22

Page 24: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

COMPONENTS ?= \app-linux-druide-antidote \builder \builder-debian \builder-rpm

NO_CHECK = app-linux-druide-antidoteGIT_URL_app_linux_druide_antidote = https://github.com/neowutr ⌋

an/qubes-app-linux-druide-antidote↪→

BRANCH_app_linux_druide_antidote = master

BUILDER_PLUGINS = builder-rpm builder-debianBUILDER_PLUGINS += mgmt-salt

DISTS_VM += archlinux+minimalCOMPONENTS += builder-archlinuxBUILDER_PLUGINS += builder-archlinux

app-builder.sh

#!/bin/bashBASE="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null

2>&1 && pwd )"↪→

directory=$BASE/qubes-buildersudo rm -Rf "$directory"sudo dnf install wget make git qubes-gpg-splitgit clone "https://github.com/QubesOS/qubes-builder.git"

key1=$(curl -s https://keys.qubes-os.org/keys/qubes-master-sig ⌋

ning-key.asc | sha512sum | cut -d " " -f1)

↪→

↪→

key2=$(sha512sum /usr/share/qubes/qubes-master-key.asc | cut-d " " -f 1)↪→

if [ "$key1" != "$key2" ]; thenecho "CRITICAL SECURITY FAILURE: qubes master signing key is

not the same on different source (local and official qubesos website)" >&2

↪→

↪→

exit 1fi

23

Page 25: Many things · To have a Windows HVM for gaming, you must have: • A dedicated AMD GPU. By dedicated, it means: it is a secondary GPU, not the GPU used to display dom0. Nvidia GPU

gpg --import /usr/share/qubes/qubes-master-key.ascecho "Check the key, if it is good for you, set the trust to 5

and exit"↪→

echo "fpr" | gpg --edit-key0x427F11FD0FAA4B080123F01CDDFA1A3E36879494↪→

wget https://keys.qubes-os.org/keys/qubes-developers-keys.ascgpg --import qubes-developers-keys.ascrm qubes-developers-keys.asc

commit_data=$(cd "$directory" && git tag -v $(git describe)2>&1 | grep "gpg: ")↪→

echo "$commit_data"echo "$commit_data" | tail -n 1 | grep "Good signature from "success=$?

if (( $success == 1 )); thenecho "CRITICAL SECURITY FAILURE: last commit from

qubes-builder is not signed with an approved gpg key" >&2↪→

exit 1fi

echo "Does this seems good to you ?"read trash

cp ./app-builder.conf $directory/builder.conf

cd "$directory"make get-sources

make install-depsmake qubes-vm

24