cs 584 l assignment. systems of linear equations l a linear equation in n variables has the form l a...
TRANSCRIPT
![Page 1: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/1.jpg)
CS 584CS 584
Assignment
![Page 2: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/2.jpg)
Systems of Linear EquationsSystems of Linear Equations
A linear equation in n variables has the form
A set of linear equations is called a system. A solution exists for a system iff the solution
satisfies all equations in the system. Many scientific and engineering problems
take this form.
a0x0 + a1x1 + … + an-1xn-1 = b
![Page 3: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/3.jpg)
Solving Systems of EquationsSolving Systems of Equations
Many such systems are large.
– Thousands of equations and unknowns
a0,0x0 + a0,1x1 + … + a0,n-1xn-1 = b0
a1,0x0 + a1,1x1 + … + a1,n-1xn-1 = b1
an-1,0x0 + an-1,1x1 + … + an-1,n-1xn-1 = bn-1
![Page 4: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/4.jpg)
Solving Systems of EquationsSolving Systems of Equations
A linear system of equations can be represented in matrix form
a0,0 a0,1 … a0,n-1 x0 b0
a1,0 a1,1 … a1,n-1 x1 b1
an-1,0 an-1,1 … an-1,n-1 xn-1 bn-1
=
Ax = b
![Page 5: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/5.jpg)
Solving Systems of EquationsSolving Systems of Equations
Solving a system of linear equations is done in two steps:
– Reduce the system to upper-triangular
– Use back-substitution to find solution These steps are performed on the system in
matrix form.
– Gaussian Elimination, etc.
![Page 6: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/6.jpg)
Solving Systems of EquationsSolving Systems of Equations
Reduce the system to upper-triangular form
Use back-substitution
a0,0 a0,1 … a0,n-1 x0 b0
0 a1,1 … a1,n-1 x1 b1
0 0 … an-1,n-1 xn-1 bn-1
=
![Page 7: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/7.jpg)
Reducing the SystemReducing the System
Gaussian elimination systematically eliminates variable x[k] from equations k+1 to n-1.
– Reduces the coefficients to zero This is done by subtracting a appropriate
multiple of the kth equation from each of the equations k+1 to n-1
![Page 8: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/8.jpg)
Procedure GaussianElimination(A, b, y) for k = 0 to n-1
/* Division Step */for j = k + 1 to n - 1 A[k,j] = A[k,j] / A[k,k]y[k] = b[k] / A[k,k]A[k,k] = 1
/* Elimination Step */for i = k + 1 to n - 1 for j = k + 1 to n - 1
A[i,j] = A[i,j] - A[i,k] * A[k,j] b[i] = b[i] - A[i,k] * y[k] A[i,k] = 0endfor
endforend
![Page 9: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/9.jpg)
Parallelizing Gaussian Elim.Parallelizing Gaussian Elim.
Use domain decomposition
– Rowwise striping Division step requires no communication Elimination step requires a one-to-all
broadcast for each equation. No agglomeration Initially map one to to each processor
![Page 10: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/10.jpg)
![Page 11: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/11.jpg)
![Page 12: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/12.jpg)
![Page 13: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/13.jpg)
Communication AnalysisCommunication Analysis
Consider the algorithm step by step Division step requires no communication Elimination step requires one-to-all bcast
– only bcast to other active processors
– only bcast active elements Final computation requires no
communication.
![Page 14: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/14.jpg)
Communication AnalysisCommunication Analysis
One-to-all broadcast
– log2q communications
– q = n - k - 1 active processors Message size
– q active processors
– q elements required
T = (ts + twq)log2q
![Page 15: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/15.jpg)
Computation AnalysisComputation Analysis
Division step
– q divisions Elimination step
– q multiplications and subtractions Assuming equal time --> 3q operations
![Page 16: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/16.jpg)
Computation AnalysisComputation Analysis
In each step, the active processor set is reduced by one resulting in:
2/)1(3
11
0
nnCompTime
knCompTimen
k
![Page 17: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/17.jpg)
Can we do better?Can we do better?
Previous version is synchronous and parallelism is reduced at each step.
Pipeline the algorithm Run the resulting algorithm on a linear
array of processors. Communication is nearest-neighbor Results in O(n) steps of O(n) operations
![Page 18: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/18.jpg)
Pipelined Gaussian Elim.Pipelined Gaussian Elim.
Basic assumption: A processor does not need to wait until all processors have received a value to proceed.
Algorithm– If processor p has data for other processors,
send the data to processor p+1– If processor p can do some computation using
the data it has, do it.– Otherwise, wait to receive data from processor
p-1
![Page 19: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/19.jpg)
![Page 20: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/20.jpg)
![Page 21: CS 584 l Assignment. Systems of Linear Equations l A linear equation in n variables has the form l A set of linear equations is called a system. l A solution](https://reader034.vdocument.in/reader034/viewer/2022051621/56649ea45503460f94ba85c4/html5/thumbnails/21.jpg)
ConclusionConclusion
Using a striped partitioning method, it is natural to pipeline the Gaussian elimination algorithm to achieve best performance.
Pipelined algorithms work best on a linear array of processors.
– Or something that can be linearly mapped Would it be better to block partition?
– How would it affect the algorithm?