type-safe off-heap memory for scaladownloads.typesafe.com/website/presentations/...type-safe...
TRANSCRIPT
![Page 1: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/1.jpg)
Type-safe Off-heap Memory for ScalaDenys Shabalin, LAMP/EPFL
![Page 2: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/2.jpg)
Off-heap memory: memory which is allocated and managedoutside of garbage collected heap.
![Page 3: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/3.jpg)
Why?You want to handle large data in-memoryGC does not meet your latency requirementsYou want to share memory with native code
![Page 4: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/4.jpg)
State of the off-heap
![Page 5: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/5.jpg)
Direct byte bufferscase class Point(x: Int, y: Int)val point = Point(10, 20)
// alocatingval bb = java.nio.ByteBufer.allocateDirect(size)bb.putInt(0, p.x)bb.putInt(4, p.y)
// readingval x = bb.getInt(0)val y = bb.getInt(4)val point = Point(x, y)
![Page 6: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/6.jpg)
Direct byte buffers: issueslow-level apiat most 2GB per bufferbound checking affects performance
![Page 7: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/7.jpg)
sun.misc.Unsafecase class Point(x: Int, y: Int)val point = Point(x = 10, y = 20)
// alocatingval unsafe = sun.misc.Unsafe.getUnsafe()val addr = unsafe.allocateMemory(size)unsafe.putInt(addr, p.x)unsafe.putInt(addr + 4L, p.y)
// readingval x = unsafe.getInt(addr)val y = unsafe.getInt(addr + 4L)val point = Point(x, y)
![Page 8: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/8.jpg)
sun.misc.Unsafe: issueseven lower level apilack of memory safetymemory leaks
![Page 9: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/9.jpg)
JNI/JNA interop with C codestruct point { int x; int y; }
JNIEXPORT jlong JNICALL Offheap_allocate(JNIEnv *env, jobject jpoint) { struct *point = (*point) malloc(sizeof(point)); ... return (jlong) point;}
JNIEXPORT jobject JNICALL Offheap_read(JNIEnv *env, jlong address) { ...}
![Page 10: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/10.jpg)
JNI/JNA: issuesas low-level as it getsnon-trivial amount of boilerplateJNI calls limit JIT optimizationslack of memory safetymemory leakscode distribution is complicated
![Page 11: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/11.jpg)
State of the off-heapVery low-level.
![Page 12: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/12.jpg)
What is a Memory?Any subtype of trait:
trait Memory { def allocate(size: Size): Addr def copy(from: Addr, to: Addr, size: Size) def getChar(addr: Addr): Char def getByte(addr: Addr): Byte .... def putChar(addr: Addr, value: Char): Unit def putByte(addr: Addr, value: Byte): Unit ...}
![Page 13: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/13.jpg)
What is a Memory?Single interface, many implementations:
NativeMemory (Unsafe-based)ByteBufferMemory...
![Page 14: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/14.jpg)
Why not just use unsafedirectly?
It might go away in the future. Some thin abstraction layerlets us swap implementations without changing client code.
![Page 15: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/15.jpg)
MemoryBest of Unsafe and ByteBuffers:
versions with x64 and x32 addressingsafety is optionalautomatic resource cleanupeasily implementable interfacestill low-level
![Page 16: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/16.jpg)
Offheap classes
![Page 17: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/17.jpg)
@data classesJust like case classes only off-heap.
@data class Point(x: Int, y: Int)
val memory = NativeMemory()val point = Point(10, 20)(at = memory)
![Page 18: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/18.jpg)
@data classesJust like case classes only off-heap.
@data class Point(x: Int, y: Int)
implicit val memory = NativeMemory()val point = Point(10, 20)
![Page 19: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/19.jpg)
@data classes// field accesspoint.x + point.y
// pattern matchingval Point(x, y) = point
// copy on writeval point2 = point.copy(x = 42)
// nice toStringpoint.toString == "Point(10, 20)"
![Page 20: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/20.jpg)
@enum classesTagged unions with straightforward syntax.
@enum class Figureobject Figure { @data class Point(x: Float, y: Float) @data class Circle(center: Point, radius: Float) @data class Segment(start: Point, end: Point)}
![Page 21: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/21.jpg)
@enum classes// implicit upcastsval fig: Figure = Figure.Circle(Figure.Point(10, 20), 30)
// type testsfig.is[Figure.Circle]
// explicit downcastsval circle = fig.as[Figure.Circle]
// pattern matchingfig match { case Figure.Circle(center, r) => }
// nice toStringfig.toString == "Figure.Circle(Figure.Point(10.0, 20.0), 30.0)"
![Page 22: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/22.jpg)
Offheap arraysLooks and feels just like the standard ones.
implicit val memory = NativeMemory()var arr = Array(1, 2, 3)
// bound-checked indexed accessarr(0) == 1arr(1) == 2arr(2) == 3arr(3) // throws OutOfBoundsException
// mappingval arr2 = arr.map(_ * 2)
// iteratingarr2.foreach(println)
![Page 23: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/23.jpg)
So we've got memory, whatabout management?
![Page 24: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/24.jpg)
Regions are the answer.
![Page 25: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/25.jpg)
Region-based memoryDelimited scopes with constant-time allocation & clean-up.
implicit val pool = Pool(NativeMemory())
Region { implicit r => val point = Point(10, 20)}
![Page 26: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/26.jpg)
Region-based memoryObjects are accessible as long as Region is open.
implicit val pool = Pool(NativeMemory())
var point: Point = _Region { implicit r => point = Point(10, 20)}point.x // throws InaccessibleRegionException
![Page 27: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/27.jpg)
Does it have to be scoped?No, open-ended regions are also supported.
val region = Region.open...region.close
![Page 28: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/28.jpg)
Do I have to close the region?No, it will be automatically closed once finalized.
val region = Region.open...
![Page 29: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/29.jpg)
What about performance?
![Page 30: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/30.jpg)
BinaryTree benchmark
![Page 31: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/31.jpg)
How does it work?
![Page 32: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/32.jpg)
Macros all the way.@data and @enum are macro annotations.
// lets look at expansion of @data@data class Point(x: Int, y: Int)
![Page 33: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/33.jpg)
Macros all the way.Checked mode desugaring.
class Point(ref: Ref) extends AnyVal { def x: Int = ref.memory.getInt(ref.addr) def y: Int = ref.memory.getInt(ref.addr) ...}object Point { def apply(x: Int, y: Int)(implicit m: Memory): Point = { val addr = m.allocate(8) m.putInt(addr, x) m.putInt(addr + 4L, y) new Point(Ref(addr, m)) }}
![Page 34: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/34.jpg)
Macros all the way.Unchecked mode desugaring.
class Point(addr: Long) extends AnyVal { def x: Int = unsafe.getInt(addr) def y: Int = unsafe.getInt(addr) ...}object Point { def apply(x: Int, y: Int)(implicit m: Memory): Point = { val addr = m.allocate(8) unsafe.putInt(addr, x) unsafe.putInt(addr + 4L, y) new Point(addr) } ...}
![Page 35: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/35.jpg)
Macros all the wayArray operations are blackbox macros.
arr.map(_ * 2)
![Page 36: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/36.jpg)
Macros all the way{ val narr = Array.uninit[int](arr.length) var p = narr.ref.addr + Memory.sizeof[size] arr.foreach { v: int => mme.putInt(p, v * 2) p += Memory.sizeof[Int] } narr}
![Page 37: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/37.jpg)
Macros all the way{ val narr = Array.uninit[int](arr.length)(mem) var p = narr.ref.addr + Memory.sizeof[Size]
{ val len = mem.getLong(arr.ref.addr) var p2 = arr.ref.addr + Memory.sizeof[Size] val bound = p2 + len * Memory.sizeOf[Int] while (p2 < bound) { mem.putInt(p, mem.getInt(p2) * 2) p += Memory.sizeof[Int] p2 += Memory.sizeof[Int] } }
narr}
![Page 38: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/38.jpg)
Macros all the way{ val narr = Array.uninit[int](arr.length)(mem) var p = narr.ref.addr + 8
{ val len = mem.getLong(arr.ref.addr) var p2 = arr.ref.addr + 8 val bound = p2 + len * 4 while (p2 < bound) { mem.putInt(p, mem.getInt(p2) * 2) p += 4 p2 += 4 } }
narr}
![Page 39: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/39.jpg)
Efficient memory poolingmachinery
![Page 40: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/40.jpg)
![Page 41: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/41.jpg)
Can I start using it today?Not yet, but experimental 0.1 release coming soon.
Source code is available today:
github.com/densh/scala-offheap
![Page 42: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/42.jpg)
Summaryscala-offheap is:
high-level and easy-to-use API to offheap memorywith optional memory safetyand deterministic performance
![Page 43: Type-safe Off-heap Memory for Scaladownloads.typesafe.com/website/presentations/...Type-safe Off-heap Memory for Scala Denys Shabalin, LAMP/EPFL](https://reader033.vdocument.in/reader033/viewer/2022052720/5f09a24f7e708231d427c630/html5/thumbnails/43.jpg)
Questions?