advanced programming & c++ language · the heap (cont…) 11 when a dynamically allocated...

Dr. Miri (Kopel) Ben-NissanAriel University2018

Advanced Programming & C++ Language

Introduction to Memory Management

~6~

© Miri Ben-Nissan

Stack & Heap2

The memory a program uses is typically divided into four different areas: The code area, where the

compiled program sits in memory.

The global (data) area, where global variables are stored.

The heap, where dynamically allocated variables are allocated from.

The stack, where parameters and local variables are allocated from.

© Miri Ben-Nissan

3

© Miri Ben-Nissan

The stack4

The stack is a container that holds other variables. It is a last-in, first-out (LIFO) structure. It is a fixed-size chunk of sequential memory

addresses. All the item are allocated in advanced and used according to the program’s needs.

A register (a small piece of memory) in the CPU is used as the stack pointer. The stack pointer keeps track of where the top of the stack currently is.

© Miri Ben-Nissan

The stack – handing function calls5

When a function is called:1. Push return address: The address of the instruction beyond the function

call is pushed onto the stack. 2. Return value: Room is made on the stack for the function’s return type. This

is just a placeholder for now. 3. Jump: The CPU jumps to the function’s code. 4. Function’s frame in stack: The current top of the stack is held in a special

pointer called the stack frame. Everything added to the stack after this point is considered “local” to the function.

5. Passing arguments: All function arguments are placed on the stack. 6. Execute: The instructions inside of the function begin executing. 7. Local variables are pushed onto the stack as they are defined.

© Miri Ben-Nissan

The stack – handing function calls (cont…)

6 When the function terminates, the following steps happen:1. Push return value: The function’s return value is copied

into the placeholder that was put on the stack for this purpose.

2. Clean local frame: Everything after the stack frame pointer is popped off. This destroys all local variables and arguments.

3. Use return value: The return value is popped off the stack and is assigned as the value of the function. If the value of the function isn’t assigned to anything, no assignment takes place, and the value is lost.

4. Reconstruct current frame: The address of the next instruction to execute is popped off the stack, and the CPU resumes execution at that instruction.

© Miri Ben-Nissan

The stack – handing function calls (cont…)

7

© Miri Ben-Nissan

The stack – stack overflow

8

The stack has a limited size, and consequently can only hold a limited amount of information.

Stack overflow is generally the result of allocating too many variables on the stack, and/or making too many nested function calls.

Overflowing the stack generally causes the program to crash.

© Miri Ben-Nissan

The stack (cont…)

9

The stack has advantages and disadvantages:Memory allocated on the stack stays in scope as long as

it is on the stack. It is destroyed when it is popped off the stack.

All memory allocated on the stack is known at compile time. Consequently, this memory can be accessed directly through a variable.

The stack is relatively small, thus it is generally not a good idea to allocate large arrays, structures, and classes, as well as perform heavy recursion.

© Miri Ben-Nissan

The heap10

The heap (also known as the “free store”) is a large pool of memory used for dynamic allocation.

In C++, when you use the new operator to allocate memory, this memory is assigned from the heap.

int *pValue = new int;

int *pArray = new int[10];

Because the precise location of the memory allocated is not known in advance, the memory allocated has to be accessed indirectly — which is why new returns a pointer.

© Miri Ben-Nissan

The heap (cont…)

11

When a dynamically allocated variable is deleted, the memory is “returned” to the heap and can then be reassigned as future allocation requests are received.

The heap has advantages and disadvantages: Allocated memory stays allocated until it is specifically

deallocated (beware memory leaks).Dynamically allocated memory must be accessed

through a pointer.Because the heap is a big pool of memory, large arrays,

structures, or classes should be allocated here.

© Miri Ben-Nissan

The heap (cont…)

12

© Miri Ben-Nissan

Memory Management13

The stack is managed by the compiler. We can manage only the heap allocated

memory. What does it mean “managing memory”?

Allocate memory. Free memory.Manipulate addresses (pointers).Reuse of free memory.

© Miri Ben-Nissan

Memory Management14

Memory management is one of the most fundamental areas of computer programming.

Knowing the abilities and limitations of your memory manager is critical for effective programming.

Memory management is usually divided into three areas:

HardwareOperating systemApplication

© Miri Ben-Nissan

Physical memory Hard disk(s)

Virtual Memory Manager

Win32 APIs

Win32 Applications

Hardware

Application

15

Memory Management (cont…)

Operating

System

© Miri Ben-Nissan

Memory Management (cont…)

16

Hardware memory management Electronic devices that actually store data.

This includes devices like RAM and memory caches.

Operating system memory management Efficiency memory allocation and reuse for processes. Virtual Memory Memory protection, Memory sharing, Security, etc.

Application memory management Efficient memory allocation techniques. Reuse of not used allocated memory. Consider CPU overhead.

© Miri Ben-Nissan

1) Hardware Memory Management17

© Miri Ben-Nissan

1) Hardware Memory Management (cont…)

18

Computers have several different types of memory. This memory is often viewed as a hierarchy as shown below.

© Miri Ben-Nissan

1) Hardware Memory Management (Cont…)

19

CPUCache

(fast)

memory

Main

(slow)

memory

Reg

file

Word

Line

Cache is transparent to user;

transfers occur automatically

© Miri Ben-Nissan


20

© Miri Ben-Nissan


21

The cache memory boost’s the speed of accessing memory, and managed entirely by the hardware.

There are two types of addresses into the memory: logic address and physical address. Processes uses logical addresses. The mapping of logic address into the physical addresses is

done inside the hardware, and programmed by the operating system.

There are some types of Hardware Accelerators that makes those operations as well as math operations etc’ much faster.

© Miri Ben-Nissan

2) OS Memory Management22

//upload.wikimedia.org/wikipedia/commons/0/06/Operating_system_placement-he.svg

© Miri Ben-Nissan

Virtual Memory23

“virtual memory” is a technique in which the computer simulates having large amounts of contiguous physical memory.

A paging table is responsible for moving segments of virtual memory into physical memory as necessary. If the amount of memory demanded by all running

processes exceeds the available physical memory (RAM), the paging table stores low-priority processes on the hard drive in the page file, which is much slower than RAM.

2) OS Memory Management (cont…)

© Miri Ben-Nissan

2) OS Memory Management - Virtual Memory (cont…)

24

When a program is started (a web browser or a word processor), it runs in its own process. A process contains its own "virtual" memory space and resources.

Its memory is "virtual" because memory the process thinks is at address 0x12345678 may actually be at address 0x65f7a678 in physical memory.

© Miri Ben-Nissan

The paging table

moves segments of

virtual memory into

physical memory as

needed to provide

more memory to

running processes.

25


© Miri Ben-Nissan


26

© Miri Ben-Nissan


27

The virtual memory is divided into two parts: user-spaceand system space.

The user space is one global area in which windows runs the user-mode applications. Each user-mode application gets a block of virtual addresses.

The system space is the portion of the address space in which the OS and kernel-mode drivers reside. It cannot be directly accessed by the application.

© Miri Ben-Nissan


28

Virtual memory is useful for many reasons:

The process cannot access other process' memory,Each page can have different protection settings

(read-only or read-write, kernel-mode-only), and Inactive memory regions of the process can be

"paged out" (stored) to the pagefile and be retrieved by the operating system when needed. This is also done when the system is low on physical memory.


© Miri Ben-Nissan

3) Application Memory Management29

The user has the ability to manage its applications’ memory usage by himself.Data Structures.Memory Pools.Overloading dynamic memory functionality.Garbage Collections.Recycling.

© Miri Ben-Nissan

3) Application Memory Management (Cont…)

30

When talking about application memory management, we usually deal with the following problems that the programmer should deal with:Premature free or dangling pointer.Memory leaks.External fragmentation.Poor locality of reference. Inflexible design. Interface complexity.


© Miri Ben-Nissan


31

Premature free / Dangling pointer:class Sample

{int *ptr;

public:Sample(int i) { ptr = new int(i); }~Sample() { delete ptr; }void PrintVal() { cout<<"The value is "<<*ptr; }

};

void SomeFunc(Sample x){cout << "Say i am in someFunc " << endl;}

int main(){Sample s1 = 10;SomeFunc(s1);s1.PrintVal(); // dangling pointer}


© Miri Ben-Nissan


32 Memory Leaks:


© Miri Ben-Nissan


33 External Fragmentation:


© Miri Ben-Nissan


34

Poor locality of reference

Also known as the principle of locality. The use of data elements within relatively close storage locations. The phenomenon that the collection of the data locations referenced

in a short period of time in a running computer often consists of relatively well predictable clusters.

for i in 0..n

for j in 0..m

for k in 0..p

C[i][j] = C[i][j] + A[i][k] * B[k][j];


© Miri Ben-Nissan


35

The larger matrix can be divided into evenly-sized sub-matrices, so that the smaller blocks can be referenced (multiplied) several times while in memory.

for (ii = 0; ii < SIZE; ii += BLOCK_SIZE)

for (kk = 0; kk < SIZE; kk += BLOCK_SIZE)

for (jj = 0; jj < SIZE; jj += BLOCK_SIZE)

for (i = ii; i < ii + BLOCK_SIZE && i < SIZE; i++)

for (k = kk; k < kk + BLOCK_SIZE && k < SIZE; k++)

for (j = jj; j < jj + BLOCK_SIZE && j <SIZE; j++)

C[i][j] = C[i][j] + A[i][k] * B[k][j];

A block can be used several times before moving on, so that it is moved in and out of memory less often. In addition, elements with consecutive memory addresses tend to be pulled up the memory hierarchy together.


© Miri Ben-Nissan


36 Inflexible design

Memory managers can also cause severe performance problems if they have been designed with one use in mind, but are used in a different way.

These problems occur because any memory management solution tends to make assumptions about the way in which the program is going to use memory, such as typical block sizes, reference patterns, or lifetimes of objects.

If these assumptions are wrong, then the memory manager may spend a lot more time doing bookkeeping work to keep up with what's happening.


© Miri Ben-Nissan

37 Interface complexity

When one module uses one memory management toolkit and another module uses a different one, it is complicated to pass arguments between those modules.


© Miri Ben-Nissan

Managed and Unmanaged Systems38

Memory management can be automatically managed by the programming language, or either manually managed by the user.

© Miri Ben-Nissan

Manual memory management39

The programmer has direct control over when memory may be recycled.Usually this is either by explicit calls to heap

management functions (for example, malloc/free in C), or by language constructs that affect the stack (such as local variables).

© Miri Ben-Nissan

Manual memory management (Cont…)

40

The advantages of manual memory management are:

It can be easier for the programmer to understand exactly what is going on;

Works well in simple situations;Can be ideal for small, constrained problems,

especially where the total memory required is close to the total memory available.

© Miri Ben-Nissan


41

The disadvantages of manual memory management are:The programmer must write a lot of code to do

repetitive bookkeeping of memory;Memory management must form a significant part of

any module interface;Manual memory management typically requires

more memory overhead per object;Memory management bugs are common.

© Miri Ben-Nissan

Manual memory management (Cont…)42

Bugs in memory management implementations:Grows over timeOut of memoryGPF / Bus ErrorData CorruptionPerformance (Thrashing)


© Miri Ben-Nissan


43

Causes of those bugs are usually:Premature FREE (Dangling pointer).Memory leaks. Fragmentation.Poor locality.

SolutionsDebugging Application (such as Purify).Garbage Collection (Automatic Memory

Management).

© Miri Ben-Nissan


44

The following languages use mainly manual memory management in most implementations, although many have conservative garbage collection extensions: Algol; C; C++; COBOL; Fortran; Pascal.

© Miri Ben-Nissan

Automatic memory management45

A service, either as a part of the language or as an extension, that automatically recycles memory that a program would not otherwise use again.

Automatic memory managers often known as garbage collectors, or simply collectors.

Usually do their job by recycling blocks that are unreachable from the program variables (that is, blocks that cannot be reached by following pointers).

© Miri Ben-Nissan

Automatic memory management (Cont…)

46 There are two aspects to Automatic Memory Management:AllocationSubdividing the large blocks that the memory manager

receives from the operating system into blocks suitable to the application.

Static/Dynamic.RecyclingPhysical memory is a limited resourceVirtual memory is a trade-offObject-oriented Languages and Dynamic Languages.

© Miri Ben-Nissan


47

The advantages of automatic memory management are:The programmer is freed to work on the actual

problem;Module interfaces are cleaner;There are fewer memory management bugs;Memory management is often more efficient.

© Miri Ben-Nissan


48

The disadvantages of automatic memory management are:

Memory may be retained because it is reachable, but won't be used again;

Automatic memory managers (currently) have limited availability.

When a garbage collector bug does arise, since it is a result of a failure of a runtime system, rather than the application programmer’s own code, it can be difficult to diagnose and repair.

© Miri Ben-Nissan


49

Most modern languages use mainly automatic memory management: BASIC, JavaTM, JavaScriptTM, Perl, the PostScript® language, Python, Smalltalk, etc.

© Miri Ben-Nissan

50

The user has the ability to control the usage of its application’s memory even in managed (automatic) systems.

One example is the Alignment of data.


© Miri Ben-Nissan

Alignment51

What is the size of the following struct?

struct MixedData

{

char Data1;

short Data2;

int Data3;

char Data4;

};


© Miri Ben-Nissan


53

Data structure alignment is the way data is arranged and accessed in computer memory.

It consists of two separate but related issues: data alignment and data structure padding.

Data alignment means putting the data at a memory offset equal to some multiple of the word size, which increases the system's performance due to the way the CPU handles memory.

Alignment (cont…)

© Miri Ben-Nissan

Alignment (cont…)

54

To align the data, it may be necessary to insert some meaningless bytes between the end of the last data structure and the start of the next, which is data structure padding.

If the highest and lowest bytes in a datum are not within the same memory word the computer must split the datum access into multiple memory accesses.

Additionally the size of the structure must be such that in an array of the structures all the structures are correctly aligned in memory so there may be padding bytes at the end of the structure too.


© Miri Ben-Nissan

Alignment (cont…)

57

The MixedData struct, after compilation in 32-bit x86 machine:

struct MixedData

{

char Data1;

short Data2;

int Data3;

char Data4;

};

© Miri Ben-Nissan

Alignment (cont…)

58

The sizeof a struct is compiler and processor dependent for reasons of both architectural limitations and efficiency.

Some processors are unable to read or write multibytevalues from certain locations. Almost always they cannot read them from odd addresses.

Frequently they cannot read them unless the address is a multiple of the size of the value - so a four-byte long can only be read if the address is a multiple of four, and a two-byte short can only be read if the address is a multiple of two. Single-byte values can be addressed at any location.

© Miri Ben-Nissan

Alignment (cont…)

59

Another example:

struct example

{

char c1;

short s1;

char c2;

long l1;

char c3;

};

//[c1][P]

//[s1][s1]

//[c2][P][P][P]

//[l1][l1][l1][l1]

//[c3][P][P][P]

advanced programming & c++ language · the heap (cont…) 11 when a dynamically allocated...

Documents