using hpc to advance water desalination by electrodialysis...using hpc to advance water desalination...

Using HPC To Advance Water Desalination By

Electrodialysis

Clara DruzgalskiDepartment of Mechanical Engineering

Stanford University

Water Desalination

Distillation

Reverse Osmosis

Electrodialysis

Electrodialysis: Industrial

Electrodialysis water treatment plants in Barcelona, Spain produce 257 million liters of water per day.

Abrera (2007) 200 million litersSant Boi del Llobregat (2009) 57 million liters

Credit: Sant Boi del Llobregat

Electrodialysis: Applications

GrayWhite Black

Portable water treatment

Salt production Biomedical analysis: lab-on-a-chip devices

Electrodialysis

Model Problem

Channel Height 10-6 meters

Smallest Feature 10-9 meters

Applied voltage 1-3 Volts

Example Dimensional Values

Model Problem: Experiments

Well-described by 1D theory

Electroconvective chaos: 1D theory no

longer predictive

“

Should we use a commercial code like Comsol Multiphysics or build

a high performance code from scratch?

?

Commercial Software

◎ Commercial codes often use artificial smoothening for numerical robustness. This dissipates small structures generated by turbulent and chaotic fluid motion.

◎ Commerical codes must be general enough to handle a wide variety of problems, but this limits the user’s ability to take advantage of crucial time-saving algorithms

Commercial Software

Custom HPC Software

EKaos a high performance direct numerical simulation code that simulates electrokinetic chaos.

◎ No artificial smoothening

◎ Over 100 times faster than Comsol on a single node in 2D.

EKaos

2D EKaos SimulationConcentration

Charge Density

Experimental Observation

Joeri C. de Valença, R. Martijn Wagterveld, Rob G. H. Lammertink, and Peichun Amy TsaiPhys. Rev. E 92, 031003(R) – Published 8 September 2015

Simulation vs. Experiment

Experiment:De Valenca, et. al.

Simulation:Davidson, et. al.

Submitted to

Scientific Reports

2D EKaos: Current-Voltage

16

2D EKaos: Current-Voltage

Qualitative matching with experiment

17

3D EKaos Simulation

165 million mesh pointsThat’s over 1 billion degrees of freedom

11 terabytes of dataPer simulation

100,000 time stepsTo reach converged statistics

Each 3D EKaos simulation…

“

Why is a simulation of just one small section of a desalination channel so

computationally expensive?

?

The computational cost is determined by the range of relevant length and time

scales that must be resolved.

AlgorithmDetails

The mathematical details behind a high performance code

Governing EquationsSpecies Conservation:

Navier-Stokes:

Gauss’s Law:

c+ Concentration of cation

c- Concentration of anion

ϕ Electric potential

u Velocity vector

P Pressure 22

y

x

Governing EquationsSpecies Conservation:

Navier-Stokes:

Gauss’s Law:

23

y

x

Reservoir:

Boundary Conditions

Membrane:

Periodic in x and z directions

Dimensionless ParametersParameter Description Range Value

ϵ Screening length, EDL size 10-6 – 10-3 10-3

Δϕ Applied voltage 20-120 120

κ Electrohydrodynamic coupling const. O(1) 0.5

c0+ Cation concentration at membrane >1 2

Sc Schmidt number 103 103

24

Spatial Discretization

25

◎ EKaos: 2D and 3D Direct numerical simulation (DNS)

◎ 3D has over 165 million spatial grid points

◎ Staggered mesh configuration

◎ Non-uniform mesh is used in the membrane-normal direction to handle sharp gradients

◎ Discretization: 2nd order central finite difference scheme

Time IntegrationSpecies Conservation

Navier-Stokes

Gauss’s Law

26

Time IntegrationSpecies Conservation

2nd Order Implicit Scheme

Semi-Implicit: 1st order

27

Time IntegrationIterative Algorithm

δ-form

Linearization

28

Time IntegrationIterative Algorithm

δ-form

Linearization

29

Time IntegrationEquation in δ-form

Remove Directional Coupling

Move non-stiff terms to left hand side

30

Time IntegrationEquation in δ-form

Remove Directional Coupling

Move non-stiff terms to left hand side

31

Time IntegrationEquation in δ-form

Analytical substitution using Gauss’s Law

Remove Directional Coupling

Time IntegrationFinal Equation

• Left hand side operator is linear and now only involves local coupling between δc+ and δc-

• We need to solve for u*, v*, w*, P*, and ϕ* at each iteration

33

Pseudo-spectral SolverConservation of momentum

Pressure equation

Gauss’s Law

34

By taking advantage of the geometry andusing physical insight we were able to:

1. Design operators that reduced thematrix bandwidth

2. Use fast and robust math libraries suchas LAPACK and FFTW

3. Reduce communication cost acrossprocessors by designing the algorithmwith parallelization in mind.

Conclusions◎Developed EKaos: a parallel 3D DNS code to simulate electroconvective chaos.

◎Developed a numerical algorithm for efficiently solving the coupled Poisson-Nernst-Planck and Navier-Stokes equations

◎Improved prediction of mean current density that has been observed in experiments

◎Comparison of 2D and 3D simulations show qualitative similarities, but quantitative differences

◎Electroconvective chaos can generate structures similar to turbulence.

36

Thanks!Any questions?

You can find me at:cdruzgal@stanford.edu