CS 290H Administrivia: April 2, 2008CS 290H Administrivia: April 2, 2008

• Course web site: www.cs.ucsb.edu/~gilbert/cs290

• Join the email (Google) discussion group!! (see web site)

• Homework 1 is due next Monday (see web site)

• Reading in Davis: • Review Ch 1 (definitions). • Skim Ch 2 (you don't have to read all the code in detail). • Read Chapter 3 (sparse triangular solves).

• I have a few copies of Davis (at a discount :)

Compressed Sparse Matrix StorageCompressed Sparse Matrix Storage

• Full storage: • 2-dimensional array.• (nrows*ncols) memory.

31 0 53

0 59 0

41 26 0

31 41 59 26 53

1 3 2 3 1

• Sparse storage: • Compressed storage by

columns (CSC).• Three 1-dimensional arrays.• (2*nzs + ncols + 1) memory.• Similarly, CSR.

1 3 5 6

value:

row:

colstart:

Matrix – Matrix Multiplication: C = A * BMatrix – Matrix Multiplication: C = A * B

C(:, :) = 0;

for i = 1:n

for j = 1:n

for k = 1:n

C(i, j) = C(i, j) + A(i, k) * B(k, j);

• The n3 scalar updates can be done in any order.

• Six possible algorithms: ijk, ikj, jik, jki, kij, kji

(lots more if you think about blocking for cache).

• Goal is O(nonzero flops) time for sparse A, B, C.

• Even time = O(n2) is too slow!

Organizations of Matrix MultiplicationOrganizations of Matrix Multiplication

Outer product: for k = 1:n C = C + A(:, k) * B(k, :)

Inner product: for i = 1:n for j = 1:n C(i, j) = A(i, :) * B(:, j)

Column by column: for j = 1:n for k where B(k, j) 0 C(:, j) = C(:, j) + A(:, k) * B(k, j)

Barriers to O(flops) work

- Inserting updates into C is too slow

- n2 loop iterations cost too much if C is sparse

- Loop k only over nonzeros in column j of B

- Use sparse accumulator (SPA) for column updates

Sparse Accumulator (SPA)Sparse Accumulator (SPA)

• Abstract data type for a single sparse matrix column

• Operations:• initialize spa O(n) time & O(n) space

• spa = spa + (scalar) * (CSC vector) O(nnz(spa)) time

• (CSC vector) = spa O(nnz(spa)) time

• spa = 0 O(nnz(spa)) time

• … possibly other ops

Sparse Accumulator (SPA)Sparse Accumulator (SPA)

• Abstract data type for a single sparse matrix column

• Operations:• initialize spa O(n) time & O(n) space

• spa = spa + (scalar) * (CSC vector) O(nnz(spa)) time

• (CSC vector) = spa O(nnz(spa)) time

• spa = 0 O(nnz(spa)) time

• … possibly other ops

• Standard implementation (many variants):• dense n-element floating-point array “value”• dense n-element boolean array “is-nonzero”• linked structure to sequence through nonzeros

CSC Sparse Matrix Multiplication with SPACSC Sparse Matrix Multiplication with SPA

B

= x

C A

for j = 1:n C(:, j) = A * B(:, j)

SPA

gather scatter/accumulate

All matrix columns and vectors are stored compressed except the SPA.

Graphs and Sparse MatricesGraphs and Sparse Matrices: Cholesky factorization: Cholesky factorization

10

13

2

4

5

6

7

8

9

10

13

2

4

5

6

7

8

9

G(A) G+(A)[chordal]

Symmetric Gaussian elimination:for j = 1 to n add edges between j’s higher-numbered neighbors

Fill: new nonzeros in factor

1. Preorder• Independent of numerics

2. Symbolic Factorization• Elimination tree

• Nonzero counts

• Supernodes

• Nonzero structure of L

3. Numeric Factorization• Static data structure

• Supernodes use BLAS3 to reduce memory traffic

4. Triangular Solves

Symmetric positive definite systems: Symmetric positive definite systems: A = LLA = LLTT

O(#flops)

O(#nonzeros in L)

}O(#nonzeros in A), almost

O(#flops)