squashing the beast into a 60mb cage · tml, #libreoffice-dev, irc.freenode.net. background: one...

47
Squashing the beast into a 60MB cage Tor Lillqvist <[email protected]> tml, #libreoffice-dev, irc.freenode.net

Upload: others

Post on 20-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Squashing the beast into a 60MB cage

Tor Lillqvist <[email protected]>

tml, #libreoffice-dev, irc.freenode.net

Page 2: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Background: One single executable in an iOS

app. No own shared libs

Page 3: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Repeat: all non-system code has to be in one

executable

Page 4: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

App Store rules: “iOS App binary files can be

as large as 2 GB”

Page 5: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

“but”

Page 6: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

“the executable file cannot exceed 60 MB”

Page 7: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

We have one test iOS app: TiledLibreOffice

Page 8: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

(which is a simple viewer for Writer docs)

Page 9: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

At first, the TiledLibreOffice

executable was ~90MB

Page 10: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

(optimised build, no debug information or symbols in the file)

Page 11: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

A third had to go without loss of functionality

Page 12: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Obviously there is a lot of code that gets linked

in but never will get called at run-time

Page 13: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

But we don't want to sprinkle ugly ifdefs all

over the place if we don't have to

Page 14: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Only in as few key places as possible

Page 15: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Largest code reduction: ICU data

Page 16: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

(“Internationalisation Components for

Unicode”)

Page 17: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Normally, ICU data is present as constant data

in code segment

Page 18: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

When building ICU one has the option to use a

data file instead

Page 19: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

This data file needs to be memory-mapped in and passed to a single ICU

call

Page 20: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Saving from ICU data: 23MB. Still lots to go

Page 21: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Locale data tables

Page 22: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Desktop LibreOffice includes data for all locales we know of

Page 23: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

… but no need to do that in an iOS app

Page 24: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Introduce --with-locales configure-time option

Page 25: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Restricts what locales have data compiled in

Page 26: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Even better would be to use data files instead of constant data in code

Page 27: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

… but that can be complicated

Page 28: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Our Japanese and Chinese “dictionaries”

are large

Page 29: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Luckily simply structured, so can use memory-

mapped data files instead

Page 30: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Use generated data files instead of generated

code for OOXML custom shape presets

Page 31: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Split UNO components into smaller ones by refactoring factory

methods

Page 32: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

More aggressive ifdeffing-out of code irrelevant on mobile

platforms

Page 33: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

(for instance: to bypass code for desktop-style help, a11y features or

extensions)

Page 34: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Charset/encoding conversion tables in sal: Optionally bin obscure

ones

Page 35: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Tell compiler to optimise harder: -Oz

Page 36: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Unfortunately, somewhat fragile, compiler bugs?

Page 37: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Link-time optimisation? Not feasible: Linker grew

to 40 GB in one hour before I lost patience

Page 38: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Non-issue: Unreferenced functions. Linker is

smart, we use -dead_strip

Page 39: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Note: Don't make assumptions based on

Linux experience

Page 40: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Apple's object file format, executable file format,

and toolchain are different

Page 41: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

How to find stuff to get rid of?

Page 42: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Inspect the linker map, workdir/

TiledLibreOffice.map

Page 43: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Use the bin/ios-mapfile-statistics script

Page 44: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Oh, and after the squashing spree, the

size of TiledLibreOffice was 43MB

Page 45: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Thanks to CloudOn for funding this work

Page 46: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

FIN

Page 47: Squashing the beast into a 60MB cage · tml, #libreoffice-dev, irc.freenode.net. Background: One single executable in an iOS app. No own shared libs. Repeat: all non-system code has

Collabora

● Collabora Ltd.– Leading Open Source Consultancy

– 8 years of experience. 90+ People.

● Collabora Productivity Ltd.

– Dedicated to Enterprise LibreOffice

– Provides Level-3 support (code issues) to all Novell / SUSE LibreOffice clients

– Architects of Microsoft OpenXML filters