using hpc to advance water desalination by electrodialysis...using hpc to advance water desalination...
TRANSCRIPT
Using HPC To Advance Water Desalination By
Electrodialysis
Clara DruzgalskiDepartment of Mechanical Engineering
Stanford University
Water Desalination
Distillation
Reverse Osmosis
Electrodialysis
Electrodialysis: Industrial
Electrodialysis water treatment plants in Barcelona, Spain produce 257 million liters of water per day.
Abrera (2007) 200 million litersSant Boi del Llobregat (2009) 57 million liters
Credit: Sant Boi del Llobregat
Electrodialysis: Applications
GrayWhite Black
Portable water treatment
Salt production Biomedical analysis: lab-on-a-chip devices
Electrodialysis
Model Problem
Channel Height 10-6 meters
Smallest Feature 10-9 meters
Applied voltage 1-3 Volts
Example Dimensional Values
Model Problem: Experiments
Well-described by 1D theory
Electroconvective chaos: 1D theory no
longer predictive
“
Should we use a commercial code like Comsol Multiphysics or build
a high performance code from scratch?
?
Commercial Software
◎ Commercial codes often use artificial smoothening for numerical robustness. This dissipates small structures generated by turbulent and chaotic fluid motion.
◎ Commerical codes must be general enough to handle a wide variety of problems, but this limits the user’s ability to take advantage of crucial time-saving algorithms
Commercial Software
Custom HPC Software
EKaos a high performance direct numerical simulation code that simulates electrokinetic chaos.
◎ No artificial smoothening
◎ Over 100 times faster than Comsol on a single node in 2D.
EKaos
2D EKaos SimulationConcentration
Charge Density
Experimental Observation
Joeri C. de Valença, R. Martijn Wagterveld, Rob G. H. Lammertink, and Peichun Amy TsaiPhys. Rev. E 92, 031003(R) – Published 8 September 2015
Simulation vs. Experiment
Experiment:De Valenca, et. al.
Simulation:Davidson, et. al.
Submitted to
Scientific Reports
2D EKaos: Current-Voltage
16
2D EKaos: Current-Voltage
Qualitative matching with experiment
17
3D EKaos Simulation
165 million mesh pointsThat’s over 1 billion degrees of freedom
11 terabytes of dataPer simulation
100,000 time stepsTo reach converged statistics
Each 3D EKaos simulation…
“
Why is a simulation of just one small section of a desalination channel so
computationally expensive?
?
The computational cost is determined by the range of relevant length and time
scales that must be resolved.
AlgorithmDetails
The mathematical details behind a high performance code
Governing EquationsSpecies Conservation:
Navier-Stokes:
Gauss’s Law:
c+ Concentration of cation
c- Concentration of anion
ϕ Electric potential
u Velocity vector
P Pressure 22
y
x
Governing EquationsSpecies Conservation:
Navier-Stokes:
Gauss’s Law:
23
y
x
Reservoir:
Boundary Conditions
Membrane:
Periodic in x and z directions
Dimensionless ParametersParameter Description Range Value
ϵ Screening length, EDL size 10-6 – 10-3 10-3
Δϕ Applied voltage 20-120 120
κ Electrohydrodynamic coupling const. O(1) 0.5
c0+ Cation concentration at membrane >1 2
Sc Schmidt number 103 103
24
Spatial Discretization
25
◎ EKaos: 2D and 3D Direct numerical simulation (DNS)
◎ 3D has over 165 million spatial grid points
◎ Staggered mesh configuration
◎ Non-uniform mesh is used in the membrane-normal direction to handle sharp gradients
◎ Discretization: 2nd order central finite difference scheme
Time IntegrationSpecies Conservation
Navier-Stokes
Gauss’s Law
26
Time IntegrationSpecies Conservation
2nd Order Implicit Scheme
Semi-Implicit: 1st order
27
Time IntegrationIterative Algorithm
δ-form
Linearization
28
Time IntegrationIterative Algorithm
δ-form
Linearization
29
Time IntegrationEquation in δ-form
Remove Directional Coupling
Move non-stiff terms to left hand side
30
Time IntegrationEquation in δ-form
Remove Directional Coupling
Move non-stiff terms to left hand side
31
Time IntegrationEquation in δ-form
Analytical substitution using Gauss’s Law
Remove Directional Coupling
Time IntegrationFinal Equation
• Left hand side operator is linear and now only involves local coupling between δc+ and δc-
• We need to solve for u*, v*, w*, P*, and ϕ* at each iteration
33
Pseudo-spectral SolverConservation of momentum
Pressure equation
Gauss’s Law
34
By taking advantage of the geometry andusing physical insight we were able to:
1. Design operators that reduced thematrix bandwidth
2. Use fast and robust math libraries suchas LAPACK and FFTW
3. Reduce communication cost acrossprocessors by designing the algorithmwith parallelization in mind.
Conclusions◎Developed EKaos: a parallel 3D DNS code to simulate electroconvective chaos.
◎Developed a numerical algorithm for efficiently solving the coupled Poisson-Nernst-Planck and Navier-Stokes equations
◎Improved prediction of mean current density that has been observed in experiments
◎Comparison of 2D and 3D simulations show qualitative similarities, but quantitative differences
◎Electroconvective chaos can generate structures similar to turbulence.
36
Thanks!Any questions?
You can find me at:[email protected]