Advanced memory allocation in Go

Joris BonnefoyDevOps @ OVH@devatoria

Before we get started...

Keep it simple, readable

● Avoid premature optimizations○ Sometimes, readability and logicalness are better than performances

● Use go tools in order to point out real problems○ pprof

○ gcflags

Introduction tomemory management

Virtual and physical memory

● When you launch a new process, the kernel

creates its address space

● In this address space, the process can

allocate memory

● For the process, its address space looks like

a big contiguous memory space

● For two identical processes, (logical)

addresses can be the same○ Depending on the compiler, the architecture, ...

Virtual and physical memory

● Each process has its own “memory

sandbox” called its virtual address

space

● Virtual addresses are mapped to

physical addresses by the CPU (and

the MMU component) using page

tables

● Per-process virtual space size is

4GB on 32-bits system and 256TB

on 64-bits one

Virtual and physical memory

● Virtual address space is split into 2 spaces○ Kernel space

○ User mode space (or process address space)

● Split depends on operating system

configuration

How is user space managed? (C example)

Stack vs. Heap

The stack● On the top of the process address space

○ Grows down

● Last In First Out design○ Cheap allocation cost

● Only one register is needed to track content○ Stack Pointer (SP register)

● Calling a function pushes a new stack frame

onto the stack

● Stack frame is destroyed on function return

● Each thread in a process has its own stack○ They share the same address space

● Stack size is limited, but can be expanded○ Until a certain limit, usually 8MB

The heap

● Grows up

● Expensive allocation cost○ No specific data structure

● Complex management

● Used to allocate variables that must

outlive the function doing the allocation

● Fragmentation issues○ Contiguous free blocks can be merged

Memory allocationin Go

Like threads, goroutines have their own stack.

<= Go 1.2 - Segmented stacks

● Discontiguous stacks

● Grows incrementally

● Each stack starts with a segment (8kB in Go 1.2)

● When stack is fulla. another segment is created and linked to the stack

b. stack segment is removed when not used anymore (stack is shrinked)

● Stacks are doubly-linked list

<= Go 1.2 - Segmented stacks

<= Go 1.2 - Hot split issue

>= Go 1.3 - Copying stacks

● Contiguous stacks

● Each stack starts with a size of 2kB (since Go 1.4)

● When stack is fulla. a new stack is created, with the double size of the previous one

b. content of the old one is copied to the new one

c. pointers are re-adjusted

d. old one is destroyed

● Stack is never shrinked

>= Go 1.3 - Copying stacks

Benchmarks

Efficient memoryallocation (and compileroptimizations)

Reminders

Heap

(de)allocation

is expensive

Stack

(de)allocation

is cheap

Reminders

Go manages memory automatically

Reminders

Go prefers allocation on the stack

Go functionsinlining

Go functions inlining

● The code of the inlined function is inserted at the place of each call to this function

● No more assembly CALL instruction

● No need to create function stack frame● Binary size is increased because of the possible repetition of assembly instructions

● Can be disabled using //go:noinline comment just before the function declaration

Go escapeanalysis system

Escape analysis

● Decides whether a variable should be allocated on the heap or on the stack

● Creates a graph of function calls in order to track variables scope

● Uses tracking data to pass checks on those variables○ Those checks are not explicitly detailed in Go specs

● If checks pass, the variable is allocated on the stack (it doesn’t escape)

● If at least one check fails, the variable is allocated on the heap (it escapes)

● Escape analysis results can be checked at compile time using○ go build -gcflags '-m' ./main.go

One basic rule (not always right…)

If a variable has its address taken,

that variable is a candidate for allocation on the heap

Closure calls

Closure calls are not analyzed

Assignments to slices and maps

A map can be allocated on the stack,

but any keys or values inserted into the map will escape

Flow through channels

A variable passing through a channel

will always escape

Interfaces

Interfaces can lead to escape

when a function of the given interface is called

(because the compiler doesn’t know

what the function is doing with its arguments)

And a lot of other cases...

● Go escape analysis is very simple and not so smart

● Some issues are opened to improve it

Conclusion

Keep it simple, readable

● Avoid premature optimizations○ Sometimes, readability and logicalness are better than performances

● Use go tools in order to point out real problems○ pprof

○ gcflags

Remember the basics

● Pointers are only useful if you directly manipulate variable value○ Most of the time, a copy of the value is sufficient

● Closures are not always sexy○ Do not overuse them just to overuse them

● Manipulate arrays when possible○ Slices are cool, but arrays too... :)

Thank youfor listening!

References

● https://segment.com/blog/allocation-efficiency-in-high-performance-go-services/● https://en.wikipedia.org/wiki/Stack-based_memory_allocation● http://gribblelab.org/CBootCamp/7_Memory_Stack_vs_Heap.html● https://dave.cheney.net/2014/06/07/five-things-that-make-go-fast● https://blog.cloudflare.com/how-stacks-are-handled-in-go/● http://www.tldp.org/LDP/tlk/mm/memory.html● http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory/● http://agis.io/2014/03/25/contiguous-stacks-in-go.html● https://medium.com/@felipedutratine/does-golang-inline-functions-b41ee2d743fa● https://docs.google.com/document/d/1CxgUBPlx9iJzkz9JWkb6tIpTe5q32QDmz8l0BouG0Cw/preview