Prof. Boris Obsieger

Faculty of EngineeringUniversity of Rijeka

CROATIA

© 2006 by Boris Obsieger, All rights reserved.Noncommercial use in unchanged form is allowed only.

Improved method for solving linear equations system

Gauss’s elimination

The disadvantage of Gauss’s elimination procedure is that the complete solution is finding at once, after all coefficients of equations are determined.

Such approach in solving linear equations system disables any considerable improvement.

Improved method for solving linear equations system

One coefficient of equation (or limited group of coefficients) is used in calculation immediately after its determining and discard after that.

Quantity of stored data dynamically changes and occupies les than 25% of memory used by Gauss’s elimination procedure.

When matrix of the system is sparse, the only selected number of unknowns can be calculated. In that case, the required memory can be even less than those required by iterative methods.

Although the number of numerical operations is not changed, the reduction of memory requirements can dramatically speed up calculation. The reason of that is reduced number of data swaps between fast RAM and slow external memory (hard disk).

The proposed algorithm is simple as those based on Gauss’s method and can be easily implemented in any computer program.

Basic principles of proposed strategywe will explained on the trivial problem

a1+a2+a3+….=?

Basic principles

1. Calculate and store all coefficients

2. Summation

1. Calculate and store one coefficient

2. Add to partial SUM

3. Repeat from step 1.

All at once One by one

Summation strategies

23

32

15

43

+ 22

=135

23

+ 32

= 55

+ 15

= 70

+ 43

=113

+ 22

=135

1. Calculate and store all coefficients

2. Summation

1. Calculate and store one coefficient

2. Add to partial SUM

3. Repeat from step 1.

A1 SUM

15 43 22 135 23 32

A2 A3 A4 A5

Used memory (6 cells)

22 113135 43 70 15 55 32 23

A SUM

Used memory(2 cells)

4 Additions 4 Additions equal

All at once One by one

2 Memory cells used 6 Memory cells used different

Init

iali

se

Set

A(i

) C

lose

END

Ni ...1

)(ASUMSUM i

All at once

OUTPUT SUM

SUM=0

Ni ... 1

)()(A ifi

i

i

Cal

cula

te

SU

M

Init

iali

se

Set

an

d a

dd

on

e b

y on

e

Clo

se

END

One by one

i

OUTPUT SUM

ASUMSUM

Ni ... 1

)(A if

SUM=0 equal

2 Memory cells used N+1 Memory cells used different

equal

similar

equal

N Additions N Additions equal

Solving linear equations system

All at once One by one

Solving strategies

a11 a12 a13 a14 a15 b1

a21 a22 a23 a24 a25 b2

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

Use coefficients of equation k in calculation

END

Output results

Read or calculate coefficients of equation k

For equation k=1 to N

One by one

Use coefficients of row k in forward substitution

END

For equation k=1 to N

Output results

Back-substitution

Sub

stit

utio

n

Read or calculate coefficients of equation k

For equation k=1 to N

All at once (GAUSS)

equal

similar

equal

a11 a12 a13 a14 a15 b1

a21 a22 a23 a24 a25 b2

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

a11 a12 a13 a14 a15 b1

a21 a22 a23 a24 a25 b2

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

1 c12 c13 c14 c15 c1

0 c22 c23 c24 c25 c2

0 c32 c33 c34 c35 c3

0 c42 c43 c44 c45 c4

0 c52 c53 c54 c55 c5

1 d12 d13 d14 d15 d1

0 1 d23 d24 d25 d2

0 0 1 d34 d35 d3

0 0 0 1 d45 d4

0 0 0 0 1 d5

1 0 0 0 0 x1

0 1 0 0 0 x2

0 0 1 0 0 x3

0 0 0 1 0 x4

0 0 0 0 1 x5

Set equation system at once

Required memory can’t be completely utilised

Eliminate unknown xk in all equations after equation k

Eliminate last N-k unknowns xk in equation k

Substitution

Back-substitution

a11 a12 a13 a14 a15 b1

a21 a22 a23 a24 a25 b2

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

Repeat substitution

GAUSS

Used 18 memory cells

0 1 c23 c24 c25 c2

1 c12 c13 c14 c15 c1

0 0 1 c34 c35 c3

0 0 0 c54 c55 c5

0 0 0 c44 c45 c4

With memory utilisation

N 2

N 2/2

Executed substitution cycles

0 N

Use

d m

emor

y ce

lls

0

N 2/4

Without memory utilisation

Utilised

Used 30 memory cells

Without memory utilisation

One by one

?

GAUSS

Up to N 2 memory cells used

N 3/3 Additions and multiplications

Stored in memory

0 1 c23 c24 c25 c2

1 c12 c13 c14 c15 c1

0 0 1 c34 c35 c3

0 0 0 c54 c55 c5

0 0 0 c44 c45 c4

a11 a12 a13 a14 a15 b1

a21 a22 a23 a24 a25 b2

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5= Calculate and store coefficients

Adding equation 1

Repeat with next equation

a11 a12 a13 a14 a15 b1

a21 a22 a23 a24 a25 b2

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

1 c12 c13 c14 c15 c1

Used 5 memory cells

Divide equation 1 by a11

One by one

a11 a12 a13 a14 a15 b1

a21 a22 a23 a24 a25 b2

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5= Eliminate first unknown x1 in eq. 2

Eliminate unknown x2 in equation 1

Calculate coefficients

Adding equation 2

Repeat with next equation

a21 a22 a23 a24 a25 b2

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

1 c12 c13 c14 c15 c1

0 c22 c23 c24 c25 c2

Used 8 memory cells

0 1 d23 d24 d25 d2

1 0 d13 d14 d15 d1 Divide equation 2 by c22

0 1 d23 d24 d25 d2

1 0 d13 d14 d15 d1

a11 a12 a13 a14 a15 b1

a21 a22 a23 a24 a25 b2

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5= Eliminate first two unknowns x1 and x2 in equation 3

Eliminate unknown x2 in equations 1 and 2

Calculate coefficients

Adding equation 3

Repeat with next equation

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

Used 9 memory cells

0 c32 c33 c34 c35 c30 0 d33 d34 d35 d30 0 1 e34 e35 e3

0 1 0 e24 e25 e2

1 0 0 e14 e15 e1 Divide equation 3 by d33

1 0 0 e14 e15 e1

0 1 0 e24 e25 e2

0 0 1 e34 e35 e3

0 1 0 0 f25 f2

1 0 0 0 f15 f1

a11 a12 a13 a14 a15 b1

a21 a22 a23 a24 a25 b2

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5= Eliminate first k-1 unknowns in equation k

Eliminate unknown xk in equations before equation k

Calculate coefficients

Adding equation k=4

Repeat with next equation

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

0 0 1 0 f35 f3

Used 8 memory cells

Divide equation k by ekk

0 c42 c43 c44 c45 c40 0 d43 d44 d45 d40 0 0 e44 e45 e40 0 0 1 f45 f4

0 0 0 1 f45 f4

0 0 1 0 f35 f3

0 1 0 0 f25 f2

1 0 0 0 f15 f1

Eliminate first N-1 unknowns in equation N

Eliminate unknown xN in equations before equation N

Calculate coefficients

Adding last equation (N=5)

Divide equation k by fNN

Output results

a11 a12 a13 a14 a15 b1

a21 a22 a23 a24 a25 b2

a31 a32 a33 a34 a35 b3

a41 a42 a43 a44 a45 b4

a51 a52 a53 a54 a55 b5

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=

x1+ x2+ x3+ x4+ x5=a51 a52 a53 a54 a55 b5x1+ x2+ x3+ x4+ x5=

Used 5 memory cells

0 0 0 0 f55 f50 0 0 0 1 x5

1 0 0 0 0 x1

0 1 0 0 0 x2

0 0 1 0 0 x3

0 0 0 1 0 x4

0 1 0 c24 c25 c2

1 0 0 c14 c15 c1

0 0 1 c34 c35 c3

Stored in memory

a51 a52 a53 a54 a55 a5

a41 a42 a43 a44 a45 b4

Will be added one by one

N 2

N 2/2

Executed substitution cycles

0 N

Use

d m

emor

y ce

lls

0

N 2/4

GAUSS(utilised)

One by one

Significant memory utilisation is possible

One by oneGAUSS

Used memory up to

1 Gb

E x a m p l e

N=10.000 equations

DOUBLE PRECISION

Used memory up to

250 Mb

One by one

Virt

ual R

AM

239

Mb

0 10.000Introduced equations

RA

M (

512

Mb)

OS and program

Data in RAM

Free RAM

Free virtual RAM on HD

GAUSSU

sed

mem

ory

(up

to 9

54 M

B)

OS and program

Swaped data

Dana in RAM

0 10.000

Free virtual RAM on HD

Virt

ual R

AM

RA

M (

512

Mb)

Executed substitution cycles

GAUSS

Up to N 2 memory cells used

N 3/3 Additions and multiplications

One by one

Up to N 2/4 memory cells useddifferent

N 3/3 Additions and multiplicationsequal

0 1 0 c24 c25 c2

1 0 0 c14 c15 c1

0 0 1 c34 c35 c3

Used 9 memory cellsUsed 18 memory cells

a51 a52 a53 a54 a55 a5

a41 a42 a43 a44 a45 b4

Will be added one by one

0 1 c23 c24 c25 c2

1 c12 c13 c14 c15 c1

0 0 1 c34 c35 c3

0 0 0 c54 c55 c5

0 0 0 c44 c45 c4

GAUSS (All at once) One by one

1. Allocate memory, set initial values

3. Eliminate first k-1 unknowns in eq. k

5. Eliminate unknown xk in all equations before equation k

6. Output results

2. Calculate coefficients for equation k

For equations k=1 to N

4. Divide equation k by ckk

7. Free allocated memory

1. Allocate memory, set initial values

3. Eliminate last k-1 unknowns in eq. k

4. Eliminate unknown xk in all equations after equation k

6. Output results

2. Calculate coefficients for equation k

For equations k=1 to N

3. Divide row k by ckk

7. Free allocated memory

For k=1 to N

For k=N-1 to 1

Init

iali

se

Cal

cula

te Eliminate unknown xk in all

equations after equation k

Allocate memory and set initial values

For k=1 to N

GAUSS (All at once)

Set coefficients of equation k

For k=1 to N

Set

A(i

,j)

Clo

se

END

Free allocated memory

Output results

For k=N-1 to 1

Eliminate last N-k unknowns in equation k

Init

iali

se

Eliminate first k-1 unknowns in equation k

Allocate memory and set initial values

One by one

Set coefficients of equation k

For k=1 to N

Clo

se

END

Free allocated memory

Output results

Eliminate unknown xk in all equations before equation k

Set

an

d a

dd

equ

atio

ns

one

by

one

equal

Variable - up to N 2/4 memory cellsN 2 Memory cells used different

equal

similar

equal

N 3/3 Additions and multiplicationsN 3/3 Additions and multiplications equal

STOP

coef

fici

ents

of

kth e

quat

ion

elim

inat

ion

of x

1 to

xk

1

in k

th e

quat

ion

savi

ng

solu

tion

di

vidi

ng k

th

equa

tion

wit

h a k

k el

imin

atio

n of

xk

in f

irst

k1

equ

atio

ns

1 to1 Nkj

1 to1 ki

i

kjikijij cccc

j

1 to1 Nkj

kkkjkj aac /

j

1 to1 kr

1 to Nkj

j rjkrkjkj caaa

r

One by one

1 to1 Nj

Nk to1

),( jkfakj

j

k

Ni to1

1 Nii cx

j

STOP

coef

fici

ents

of

equa

tion

s sy

stem

Nik to1

1 to1 Ni

111 kNikiNiN aaaa

k

Gauss

Nk to1

k

Ni to1

1 Nii ax

j

1 to1 Nj

Ni to1

),( jifaij

j

i

1 to1 Nkj

Nki to1

i

kjikijij aaaa

1 to1 Nkj

kkkjkj aaa /

j

i

savi

ng

sol

utio

n di

vidi

ng k

th

equa

tion

wit

h a k

k

elim

inat

ion

of x

k in

equ

atio

ns k

+1 to

N

elim

inat

ion

of x

i+1 t

o x N

in e

quat

ions

i=1…

N1

j

equal

equal

similar

sim

ilar

equal

)(addeq_ini N

1 to1 Nj

j

Ni to1

)(addeq a

i

),(][ jifja

) (addeq_end

Use coefficients of one equation in calculation

Allocate memory for solving equation system

Free allocated memory

Read or calculate coefficients of one equation

END

One by one

Init

iali

se

Cal

cula

te

Clo

se

Application

Improved method for solving linear equations system

One coefficient of equation (or limited group of coefficients) is used in calculation immediately after its determining and discard after that.

Quantity of stored data dynamically changes and occupies les than 25% of memory used by Gauss’s elimination procedure.

When matrix of the system is sparse, the only selected number of unknowns can be calculated. In that case, the required memory can be even less than those required by iterative methods.

Although the number of numerical operations is not changed, the reduction of memory requirements can dramatically speed up calculation. The reason of that is reduced number of data swaps between fast RAM and slow external memory (hard disk).

The proposed algorithm is simple as those based on Gauss’s method and can be easily implemented in any computer program.

1

2

3

4

5