Migrating to Scala 2.13Julien Richard-Foy, Scala Center

Stefan Zeiger, Lightbend

Scala 2.13

• Simplifying the collections

• Compiler performance• Modularizing the standard library

• User-friendliness

https://github.com/scala/scala-dev/issues/324

Scala 2.13 Roadmap

• Scala 2.13.0-M4 released one month ago during ScalaDays Berlin

• First milestone with the new collections library• Final milestone release: M5 (August 10)

• Minor API changes• Bug fixes

• Performance improvements• Compatibility improvements

Current Status

• Project that works on Scala 2.12

• Targeting Scala 2.13.0-M4 or higher• sbt build (for examples shown)

Starting Point

Prepare for migration

• Remove deprecated calls

• Many deprecated APIs were removed in 2.13 (e.g. JavaConversions)• A clean build without deprecations on 2.12 makes the migration easier

• build.sbt:

scalacOptions in Compile += "-deprecation"

Clean up on 2.12: Deprecations

• Compiler flags that modify the language are being removed in 2.13

• Only -Xsource from now on• Compile without the following flags:

• -Yno-adapted-args• -Xstrict-inference

• -Xfull-lubs• -Yoverride-objects

• -Yoverride-vars• -Yinfer-argument-types

• -Yvirtpatmat• See https://github.com/scala/scala/pull/6505

Clean up on 2.12: Compiler flags

• Argument adaptation cannot be turned off selectively any more

• -Yno-adapted-args removed• Dotty has more restrictive adaptation

• Goal: Always allow safe adaptation, prohibit unsafe (accidental) cases• Scala 2.14 will align with Dotty rules

• Automatic eta-expansion of zero-arg methods already disabled in 2.13 under -Xsource:2.14 (see https://github.com/scala/scala/pull/6475)

• But more aggressive eta-expansion of other methods

Clean up on 2.12: Argument adaptation / Auto-tupling

• scala-library-all has been removed

• Depend on the required modules individually• Parallel collections have been moved into a separate module

• Add a dependency to scala-parallel-collections on 2.13 and see https://github.com/scala/scala-parallel-collections/issues/22 for cross-building

• Not available for M4, will be published again for M5

• scala-xml is no longer a transitive dependency of scala-compiler• Not used by scaladoc anymore

• Depend on it directly if necessary

Clean up on 2.12: Modules

Build on 2.13

crossScalaVersions := Seq("2.13.0-M4", "2.12.6")

scalaVersion := crossScalaVersions.value.head

Cross-building: Basic setup

scalacOptions ++=(CrossVersion.partialVersion(scalaVersion.value) match {case Some((2, n)) if n >= 13 => Seq("-Xsource:2.14")case _ => Seq("-Yno-adapted-args")

})

Cross-building: Compiler options

• Expect some breakage due to the new collections library

• Singleton types and minor type inference changes can influence types and implicit resolution in corner cases

• Other small changes in source compatibility (see https://github.com/scala/scala-dev/issues/470)

• Try -Xsource:2.12

Build on 2.13

• The new collections library is mostly source compatible

• Many old methods and types are deprecated• e.g. Traversable, TraversableOnce, Stream, ...

• Some features could not be added back in a 2.12-compatible way• Instead scala-collection-compat provides the 2.13 syntax on 2.11 & 2.12

• https://github.com/scala/scala-collection-compat/

scala-collection-compat

• build.sbt:

libraryDependencies +="org.scala-lang.modules" %%

"scala-collection-compat" % "0.1.0"

• Old Scala code:xs.to[List]

• New Scala code:import scala.collection.compat._xs.to(List)

scala-collection-compat: Example

New collections

• This is not the topic of this talk, but in short:

• Simpler user-facing API (no CanBuildFrom)

• Correct operation implementations for non-strict collections• Simpler internal hierarchy (no Gen… types)

Goals of the collections redesign

• scala.Seq is now scala.collection.immutable.Seq• Same for IndexedSeq• Consistent with Set and Map

• Decide on a case-by-case basis which one to use• Use s.c.Seq or s.c.i.Seq explicitly when cross-building

Major collection incompatibilities: Immutable Seq

• Varargs use scala.Seq, so also immutable now

• Wrapped Java varargs pretend the array is immutable• Using the new s.c.i.ArraySeq• You can do the same with ArraySeq.unsafeWrapArray• A deprecated implicit conversion makes a copy of the array

Major collection incompatibilities: Immutable Seq

method copyArrayToImmutableIndexedSeq in class LowPriorityImplicits2 is deprecated (since 2.13.0): Implicit conversions from Array to immutable.IndexedSeq are implemented

by copying; Use the more efficient non-copying ArraySeq.unsafeWrapArray or an explicit toIndexedSeq call

• CanBuildFrom replaced by BuildFrom

• BuildFrom requires an instance of the source collection:trait BuildFrom[-From, -A, +C] {def fromSpecificIterable(from: From)(it: Iterable[A]): Cdef newBuilder(from: From): Builder[A, C]

}• Collection methods like flatMap no longer need it

• Overloaded to produce different results depending on source collection type• Used in methods like Future.sequence

Major collection incompatibilities: CanBuildFrom

• Factory allows target type-driven building (like CanBuild in 2.12)

• Used in Iterable.to• Like BuildFrom but does not use a source collection:

trait Factory[-A, +C] {def fromSpecific(it: IterableOnce[A]): Cdef newBuilder: Builder[A, C]}

• General rule: Use BuildFrom to rebuild with the best matching type of an existing source collection, otherwise use Factory

Major collection incompatibilities: CanBuildFrom

• Without CanBuildFrom, there is no breakOut

• Use an Iterator: xs.iterator.map(…).to(Vector)• Methods with BuildFrom or Factory can be called with a companion object

instead (via implicit conversion to BuildFrom / Factory):

• Old:val xs: List[Future[Int]] = …Future.sequence(xs)(breakOut, implicitly):Future[Vector[Int]]

• New:val xs: List[Future[Int]] = …Future.sequence(xs)(Vector, implicitly)

Major collection incompatibilities: breakOut

Doesn't actually work in 2.12

• Views in the new collections are reified Iterator operations

• View vs Iterator is similar to Stream vs Spliterator in Java 8 streams

• Views don't remember source collection types

• Use an explicit to… operation instead of force to build the desired type• mapValues and filterKeys on Map now return MapView

• It was always a lazy View-like object but pretended to be a Map• Add .toMap if necessary

Major collection incompatibilities: Views

• Only very simple collection implementations are source compatible

• General rule: Use two different source files in different source directories to cross-build

• Type hierarchy simplified

• Less boilerplate• All methods have alphabetic names, symbolic operators are aliases

• E.g. override concat instead of ++• Overloading instead of CanBuildFrom

Major collection incompatibilities: Custom collections

crossScalaVersions := Seq("2.13.0-M4", "2.12.6")

scalaVersion := crossScalaVersions.value.head

// Add src/main/scala-2.13+ for Scala 2.13 and newer// and src/main/scala-2.12- for Scala versions older than 2.13unmanagedSourceDirectories in Compile += {

val sourceDir = (sourceDirectory in Compile).valueCrossVersion.partialVersion(scalaVersion.value) match {

case Some((2, n)) if n >= 13 => sourceDir / "scala-2.13+"case _ => sourceDir / "scala-2.12-"

}}

Cross-building: Source directories

sbt already gives you:• scala• scala-2.12• scala-2.13.0-M4

• See https://github.com/scala/collection-strawman/wiki/FAQ for a more comprehensive list

Major collection incompatibilities

Automated migration

2.12 2.13Scalafix

manual work

Migrating an application Rules under development –more in M5

• plugins.sbt:addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.5.7")

• From your shell:$ sbt> scalafixEnable> scalafix github:scala/scala-collection-compat/NewCollections

• build.sbt:scalaVersion := "2.13.0-M4"

• And then:> ;reload ;compile

• Documentation: https://github.com/scala/scala-collection-compat

Running the migration rulesStep 1: Add sbt-scalafix to

your project

Step 2: Run the migration rule

Step 3: Update the scalaVersion

• Renamings• Stream(1, 2, 3).append(Stream(4, 5, 6))• LazyList(1, 2, 3).lazyAppendedAll(LazyList(4, 5, 6))

• Expression rewritings• xs.copyToBuffer(b)• b ++= xs

• The more complex the expression, the harder to implement the rewrite rule

Scope of the migration rulesHas a lazy head

• Custom collection implementations

• Advanced usage of CanBuildFrom• scala.Seq usage

• Comprehensive list of supported rewrites:https://github.com/scala/collection-strawman/wiki/FAQ

Not in the scope of the migration rules

2.112.12

2.112.122.13

Scalafix

scala-collection-compat

manual work

Migrating a library Rules not yet implemented

• "The Architecture of Scala 2.13’s Collections": https://docs.scala-lang.org/overviews/core/architecture-of-scala-213-collections.html

• Demo project for cross-building:https://github.com/szeiger/new-collections-demo

• FAQ: https://github.com/scala/collection-strawman/wiki/FAQ

• Previous talks on the new collections:• The new collections library for Scala 2.13 and Dotty

• The next() collections

• @julienrf• @StefanZeiger

Links