analysis and visualization with yt · pkdgrav pluto ramses rockstar sdf stream http stream grids...

Analysis and Visualization with yt

Matthew TurkUIUC School of Information Sciences

UIUC Astronomy

yt-project.orghub.yt

Layers of Representation

Bit


Bit

Data


Bit

Model

Data

yt-project.org

Volumetric analysis and visualization

Jill Naiman, AJ Christensen, Kalina Borkiewiczytini.com

the yt project

● Python-based (C, Cython, etc)

● Community developed

○ NumFOCUS FSA

○ Code of Conduct

○ Governance structure

○ 100+ contributors

○ Volumetric and non-spatial data

● Used in nearly 300 papers

● Grids, particles, octrees, and unstructured meshes

● Arbitrary geometric representations

● Minimize time to inquiry

Ingestion

Ingestion Representation

Ingestion Representation Analysis

Ingestion Representation Analysis Visualization

Ingestion Representation Analysis Visualization

Community

Project MembersKenza Arraki

Corentin CadiouBrian Crosby

Bili DongHilary Egan

Nathan GoldbaumCameron Hummels

Suoqing JiAllyson Julian

Ben KellerKacper Kowalik

Sam LeitnerAlex LindsayChris Malone

Andrew MyersJill NaimanJeff OishiBrian O'SheaDouglas RuddAnthony ScopatzSam SkillmanStephen SkoryBritton SmithCasey StarkMatthew TurkJohn WiseMichael ZingaleJohn ZuHone

About 100 contributors...

yt

Data

Some Simulation


Bit

Model

Data

ARTARTIO

AthenaCarpetCastro

ChomboEagleEnzo

ExodusIIFITS

FLASHGadget

Gadget-FOFGasoline

GDF

MaestroMOABNyxOWLSOWLS-SubfindPKDGravPLUTORAMSESRockstarSDFStream HTTPStream GridsStream OctreeStream ParticlesStream Unstructured

Some Simulation

Some Simulation

Selector

Selected

Selected

Data


Bit

Model

Data

Data

Low-Level Operations

Data

Chunk Chunk Chunk

Chunk Chunk Chunk

Chunk Chunk Chunk


Selected

ChunkChunk

ChunkChunk Chunk Chunk Chunk

ChunkChunk

Chunk


Chunk Chunk Chunk

Chunk Chunk Chunk

Chunk Chunk Chunk


Chunk

Particles

Cells

Elements

Chunk Chunk Chunk

Chunk Chunk Chunk

Chunk Chunk Chunk



Bit

Model

Data

● Coordinate Handling

○ Cartesian

○ Cylindrical

○ Spherical (geographic, tomographic)

● Symbolic Units

● Derived fields

○ Dependency calculation

○ Arithmetic and spatial

Data Representation

“Fields”

Representation of state.

“Fields”


● Name● Units● Context● Prescription

“Fields”


Primitive

“Fields”


Primitive

Derived

“Fields”

Arithmetic Operations E =mv2

2

“Fields”

@derived_field("energy", units="erg")def energy(field, data): E = 0.5 * (data["mass"] * data["velocity_magnitude"]**2) return E

Arithmetic Operations

“Fields”

div V = δV

x +δV

y +δV

z

δx δy δzSpatial Operations

“Fields”

@derived_field("velocity_divergence", units="1/s", validators=[ValidateSpatial(1)])def divergence(field, data): dx = data["index", "dx"] dy = data["index", "dy"] dz = data["index", "dz"] f = data["velocity_x"][sl_r,1:-1,1:-1]/ds f -= data["velocity_x"][sl_l ,1:-1,1:-1]/ds f += data["velocity_y"][1:-1,sl_r,1:-1]/ds f -= data["velocity_y"][1:-1,sl_l ,1:-1]/ds f += data["velocity_z"][1:-1,1:-1,sl_r]/ds f -= data["velocity_z"][1:-1,1:-1,sl_l ]/ds return f

Spatial Operations

Mid-Level Operations

x =


x =

m s-1 m/s


m s-1 m/sx =

Symbolic manipulation and pragmatic ontologies

Semantically-meaningful Data

m s-1 m/sx


=

Semantically-meaningful Data

m s-1 m/sx =


43 primitive fields

363 derived fields

35 distinct units

2.5 primitive fields per derived

High-Level Operations

Chunk

High-Level Operations

Chunk Chunk Chunk

Chunk Chunk Chunk

Chunk Chunk Chunk

Imaging and volume tools

● Volumetric segmentation

○ Parallel

○ Irregular resolution data

● Marching cubes

● Ray-tracing

○ Radiative transfer

○ Volume rendering

● Rasterization / pixelization

○ Coordinate systems

○ Discretization

Location

Tim

e

Turk et al 2009

Density

Te

mp

era

ture

Disk

Simulation

Disk

Simulation

Analysis

libyt

Hsi-Yu Schive

First Order

f(x,y,z)

Alex Lindsay, Andrew Myers

Second Order


Approximate Second Order


Compressed bitmap indices


Spatial Tree


Lang & Turk, 2016

Community

● Not the biggest, not the smallest, but active

○ 375 on the “users” mailing list

○ 115 on the “dev” mailing list

● Code

○ Peer review

○ Mentorship

○ Continuous testing system

● How can we increase diversity?

● How can we foster careers?

● How can we lower barriers?

Community

What are our core values?

Technically Easy

Technically Challenging

Socially EasySocially Challenging

Functional

HumanitarianDetrimental

Problematic

product versus project


“the thing”


“the people”

Thank you.

[email protected]

http://yt-project.org/

http://dxl.ncsa.illinois.edu/

http://sites.google.com/site/matthewturk/

Three Options:

$ docker pull xarthisius/ythub-jupyter

$ docker run --rm -ti -p 8888:8888 xarthisius/ythub-jupyter

$ conda install -c conda-forge yt

yt-project.org and click on “Get yt”

analysis and visualization with yt · pkdgrav pluto ramses rockstar sdf stream http stream grids...

Documents