0.5.0 (2024-04-11)

We’re happy to announce the new release of Scala Native!

Check out the documentation at https://scala-native.readthedocs.io/

TL;DR

  • Not backward compatible with previous releases,

  • Added support for multithreading based on platform threads

  • Added support for targeting 32-bit architectures

  • Initial source level debugging support

  • Various changes to the build system. See “Build Integrator features” below.

  • Removed stub implementation for partially implemented Java Standard Library types

  • SIP-51 support: artifacts for every Scala standard library version

Supported Scala versions

Scala Binary Version Supported Scala Versions
2.12 2.12.14 ... 2.12.19
2.13 2.13.8 ... 2.13.13
3 3.1.2 ... 3.1.3
3.2.0 ... 3.2.2
3.3.0 ... 3.3.3
3.4.0 ... 3.4.1

Upon release of new Scala version (stable, or Scala 3 RC) version dependent artifacts would be published without a new release.


Commits since 0.4.17 (excluding backported commits): 299
Contributors: 21

New features

Multithreading support

Scala Native now allows to use system threads based on java.lang.Thread implementation, along with all the necessary primitives to work with concurrency:

  • synchronized blocks are now using object monitors following partially JVM semantics - these might be less performant when compared to JVM, but are going to achive improvements in the followup patch releases.

  • JVM compliant support for @volatile annotations

  • Configurable mostly JVM-compliant support for final fields semantics, see multithreading guide

  • Thread safe implementation of most java.util.concurrent types and methods, including atomics, thread pools, etc. See list of currently implemented primitives in our meta-issue on GitHub. Be aware that our Java Standard Library implementation might still contain thread-unsafe implementations of types not listed in the tracker.

  • Support for most of scala.concurrent and scala.collection.concurrent types.

  • Multithreading-aware implemenation of all garbage collections.

Support for 32-bit architectures

We’ve introduced preliminary, experimental support for targeting 32bit architectures, like ARMv7. To allow for representing variable-length integers in C bindings, we’ve introduced 2 new types scala.scalanative.unsafe.{Size, USize} which correspond to ssize_t, size_t in C. During linking these would be transformed into 32 or 64 bits longs integer. Multiple C standard library and POSIX bindings were adapted to use these new types.

Initial support for source level debugging

We’ve introduced initial support for generating source level debug informations, allowing to map code executed in the debugger to the original sources or to represent local variables. When executing on MacOS it also allows to obtain approximated source code lines in the exception stack traces (best effort, might not point to exact line, but rather to function definitions). Be aware that both Scala Native optimizer and LLVM optimizers can remove some of the optimized out debug informations. For best experience run with disabled optimizations:

  nativeConfig ~= { 
        .withSourceLevelDebuggingConfig(_.enableAll) // enable generation of debug informations
        .withOptimize(false)  // disable Scala Native optimizer
        .withMode(scalanative.build.Mode.debug) // compile using LLVM without optimizations
   }

See our debugging guide for more informations.

SIP-51 support: Drop forward binary compatibility of the Scala 2.13 standard library

We’re chaning how the Scala standard library artifacts are published. Previously each Scala Native version shipped with 1 artifact for Scala standard library based on latest version of Scala. However this approach was insufficient to fully cover changes in Scala 3 standard library and it’s 2 lines of releases: Scala LTS and Scala Next. To mittigate this issue and to prepare for unfreezing Scala 2 standard library we’re changing how artifacts are published. Now each release of Scala Native would contain N variants of Scala standard library with custom versioning in format <ScalaVersion>+<ScalaNativeVersion>, eg. 3.3.2+0.5.0.

Improved missing symbols reports

The new release introuduces a refactored machanism for tracking and reporting missing symbols in the linked code. Now when referring to not implemented Java standard library method, or a type from not cross-compiled library you would receive a detailed information about missing symbols containg its signature approximation and detailed stack trace of method calls leading to usage of missing symbol.

JVM service providers supports

We’re introducing a user configured support for Java Service Providers. Similarry to it’s JVM variant these would allow you to introduce implementations of Service Provider Interfaces however due to Ahead-of-time compilation constraint and to limit ammount of introduced dependencies only explicitly enabled implementations would be linked in the final binary. For more details refer to our guide

User build settings

You can now specify the basename of your executable or library.. Typically it will default to the project’s module name. In the following example, after running nativeLink you’ll find an myapp or myapp.exe file in the target directory of your projects (by default in the <project-dir>/target/<scala-version>/myapp)

lazy val myproj = project
  .in(file("."))
  .settings(
    nativeConfig ~= { c =>
      c.withBaseName("myapp")
    }
  )

Breaking changes

Broken backward compatibility

Scala Native 0.5.0 breaks backward binary compatibility with previous releases of Scala Native. Libraries published using version 0.4.x or older must be republished for Scala Native 0.5.x.

Scala Native Runtime

  • scala.scalanative.unsafe.Zone.apply by default uses a context function argument. For Scala 2 cross-compilation see Memory managment section

  • The idiomatic alternative for C void* is now Ptr[?](Scala 3) / Ptr[_] (Scala 2). Use unsafe.CVoidPtr for alias when defining bindings.

Java Standard Library

  • Removed harmfull stub types defined in multiple packages, for 0.4.x implementation see scala-native-java-stubs:

    • Removed java.security stubs - these were harmfull, by providing a false sense of security, especially in java.security.MessageDiggest.

    • Removed java.net.URL stubs - these had no working implementation for majority of features.

    • Removed java.reflect stubs - cannot be implemented to provide real reflective access.

POSIX bindings

  • There is a breaking change in the utsnameOps of the posixlib. Prior to version 0.4.x, this Ops provided an API that returned system properties as String for Ptr[utsname.utsname]. However, like other posixlib Ops, it has been changed to return a CArray instead of a String.

Replace time_t fields with timespec in POSIX sys/stat.scala

In order to support nanosecond resolution for java.nio.file.attribute.BasicFileAttributes, we introduce breaking changes to sys/stat struct.

Previously, stat struct was defined as following and you were able to access second resolution file stat fields st_atime, st_mtime and st_ctime by _7, _8 and _9.

type stat = CStruct13[
  dev_t, // st_dev
  dev_t, // st_rdev
  ino_t, // st_ino
  uid_t, // st_uid
  gid_t, // st_gid
  off_t, // st_size
  time_t, // st_atime
  time_t, // st_mtime
  time_t, // st_ctime
  blkcnt_t, // st_blocks
  blksize_t, // st_blksize
  nlink_t, // st_nlink
  mode_t // st_mode
]

Since 0.5.0, stat struct uses timespec for file stat fields. Therefore, you need to replace _7, _8 and _9 with _7._1, _8._1 and _9._1 to access those fields.

  type stat = CStruct13[
    dev_t, // st_dev
    dev_t, // st_rdev
    ino_t, // st_ino
    uid_t, // st_uid
    gid_t, // st_gid
    off_t, // st_size
    timespec, // st_atim or st_atimespec
    timespec, // st_mtim or st_mtimespec
    timespec, // st_ctim or st_ctimespec
    blkcnt_t, // st_blocks
    blksize_t, // st_blksize
    nlink_t, // st_nlink
    mode_t // st_mode
  ]

There is a helper implicit class statOps, which provides human-friendly field accessors like st_dev or st_rdev. It is recommended to use these fields from statOps rather than accessing fields by _N.

For example, import scala.scalanative.posix.timeOps._ and scala.scalanative.posix.sys.statOps.statOps, then you can get the last access time of a file by st_atime or st_atim field.

import scala.scalanative.unsafe._
import scala.scalanative.posix.sys.stat
import scala.scalanative.posix.timeOps._
import scala.scalanative.posix.sys.statOps.statOps

Zone { implicit z =>
  val filepath = c"/path/to/file"
  val stat = alloc[stat.stat]()
  stat.stat(filepath, stat)
  // directly get the last access time in second resolution
  val atime = stat.st_atime
  // get the last access time in second resolution from timespec
  val atimesec = stat.st_atim.tv_sec 
  // get access time in nanosecond resolution from timespec
  val atimensec = stat.st_atim.tv_nsec
}

Corrections to POSIX sys/utsname.scala

A number of defects have been corrected in sys/utsname.scala. These corrections required breaking changes to field definition. The change most noticeable to end users is likely to be that the uname object, holding implicit conversions, has been renamed to utsname.

A Test in UtsnameTest.scala shows on way of using the required CArray fields in the utsname structure as instances of Scala types.

Build integrator features

There are a few features to be used by build tool integrators that have changed.

  • The entrypoint for building projectsscala.scalanative.build.Build.buildnow takes an implicit ExecutionContext and returns Future[Path] instead of Path. Use Build.buildCached to use build-tool agnostic cache to skip linking if config and inputs have not changed.

  • Changes to scala.scalanative.build.Config:

    • Config.artifactPath The final artifact is now calculated for the integrator. No need to worry about the extension for Windows.

    • Now the baseName can be set by the developer if the module name is not desired.

    • Config.withTestConfig(true) for tests to allow a -test to be appended as before for test applications. The default is false for normal projects.

    • Config.withBaseDir(crossTarget) is a Path that needs to be set rather than workDir.

    • Config.workDir is now calculated from baseDir but is available for integrators as needed.

val nativeConfig = build.NativeConfig.empty
  withBaseName("myapp") // override config module name

val config = build.Config.empty
  .withLogger(logger)
  .withMainClass(mainClass)
  .withClassPath(classpath)
  .withBaseDir(crossTarget) // Path
  .withModuleName(module.name)
  .withTestConfig(testConfig)
  .withCompilerConfig(nativeConfig)

Other breaking changes:

  • Runtime environment variables to control the Garbage Collector are now aligned to match the Boehm GC as much as possible. In particular the first two variables are used on all GCs. The last one works on Boehm and Commix.

    • GC_INITIAL_HEAP_SIZE (was SCALANATIVE_MIN_HEAP_SIZE)

    • GC_MAXIMUM_HEAP_SIZE (was SCALANATIVE_MAX_HEAP_SIZE)

    • GC_NPROCS (was SCALANATIVE_GC_THREADS)

    • GC_TIME_RATIO (was SCALANATIVE_TIME_RATIO)

    • GC_FREE_RATIO (was SCALANATIVE_FREE_RATIO)

    • GC_STATS_FILE (was SCALANATIVE_STATS_FILE)

Deprecated definitions

Removed in this version

Ordered by version of Scala Native in which a deprecation was introduced.

  • Deprecated in 0.3.7

    • ScalaNativePlugin.scala ‘val AutoImport’.

  • Deprecated in 0.4.1

    • scala.scalanative.libc.signal.kill(pid, sig).
      Suggested replacement: kill(pid, sig) from POSIX signal.

    • scala.scalanative.libc.signal.SIGUSR1.
      Suggested replacement: SIGUSR1 from POSIX signal.

Introduced in this version

All newly deprecated declarations are subject to removal in the future.

  • posixlib unistd.scala ‘sethostname()’ is now deprecated because it is not part of the POSIX 2018 standard.

  • posixlib unistd.scala ‘vfork()’ is now deprecated because it was removed in the POSIX.1-2018 standard. Suggested replacement: ‘posix_spawn()’.

Contributors

Big thanks to everybody who contributed to this release or reported an issue!

  141  Wojciech Mazur
   65   LeeTibbert
   19   Rikito Taniguchi
   19   Eric K Richardson
   10   Dimi Racordon
    6   Natsu Kagami
    6	  Kirill A. Korinsky
    5   David Bouyssié
    4   kim / Motoyuki Kimura
    4   Anton Sviridov
    2   Mark Hammons
    2	  Lorenzo Gabriele
    2   João Costa
    1   Jamie Willis
    1   Yawen Guan
    1   yuly16
    1   Jarek Sacha
    2   Michel Davit
    1   Jakub Kozłowski
    1	  Claudio Bley

Merged PRs

v0.5.0 (2024-04-11)

Full Changelog

Merged pull requests (excluding 0.4.17 backports):

Scala Native sbt plugin

  • Partial fix #2765: Remove long deprecated declaration in ScalaNativePlugin #3004 (LeeTibbert)

  • nativeLinkReleaseFast and nativeLinkReleaseFull SBT tasks #3391 (keynmol)

  • fix: Use dedicated thread pool per nativeLink instead of using ExecutionContext.global #3725 (WojciechMazur)

Scala Native Compiler PLugin

Scala Native runtime

Scala Standard Library

Java Standard Library

Toolchain

Garbage Collectors

  • Fix #2921: Commix Heap.c now compiles with Clang 15.0.3 #2922 (LeeTibbert)

  • Fixes #3151: Standardize GC env vars #3152 (ekrich)

  • Install an EXC_BAD_ACCESS handler for safepoints on MacOS #3278 (natsukagami)

  • Multithreading support for Commix GC #3229 (WojciechMazur)

  • improvement/GC: Create a dedicated thread to invoke registered WeakReference handlers #3649 (WojciechMazur)

  • refactor: Synchronize naming of GC public API #3652 (WojciechMazur)

  • fix: Ensure to mark aligned fields by fixing MemoryLayout.referenceFieldsOffsets bug #3735 (WojciechMazur)

  • refactor/fix: Use object offsets instead of field indexes for precise object scanning #3736 (WojciechMazur)

  • improvement: Limit amount of compiled code, restrict glue layer only referenced files. #3849 (WojciechMazur)

Documentation

  • Fix #1826: Add documentation for GC settings #2910 (ekrich)

Versioning

Tests interface

  • Allow to prefetch debug info to mitigate spurious failures in the CI #3517 (WojciechMazur)

POSIX bindings

  • Fix #2707: Implement most of POSIX stddef.h #2709 (LeeTibbert)

  • Fix #2629, #2295: posix sockaddr_storage now implements specification #2630 (LeeTibbert)

  • Fix #2717: Complete POSIX errno.scala by providing errno variable #2721 (LeeTibbert)

  • Fix #2666: posix dirent constants are now parameterless methods #2668 (LeeTibbert)

  • Partial fix #2623: posixlib sys/socket sendto() & recvfrom() now succeed on Linux & Windows using IPv6 #2705 (LeeTibbert)

  • Fix #2626, #2623: handle socket fields sin6_len & sin_len, when present on OS #2734 (LeeTibbert)

  • Simplify & shorten additional posixlib socket code paths #2742 (LeeTibbert)

  • Fix #2738: Shorten posixlib uio codepaths & add UioTest #2741 (LeeTibbert)

  • Simplify & shorten posixlib netdb code paths #2743 (LeeTibbert)

  • Fix #2255: complete socket.h; simplify & shorten code paths #2766 (LeeTibbert)

  • Fix #2835: Simplify & shorten posixlib time code paths #2836 (LeeTibbert)

  • Fix #2832: A step towards posixlib using shared types #2833 (LeeTibbert)

  • Fix #2873, #2865: posix netdb is missing symbols #2881 (LeeTibbert)

  • Fix #2831: Towards a more complete posixlib unistd.scala #2882 (LeeTibbert)

  • Fix #2892: Implement posixlib sys/select pselect() #2895 (LeeTibbert)

  • Fix #2891: posixlib spawn is now implemented. #2894 (LeeTibbert)

  • Partial fix #2963: Add missing SIGTERM & kin to posixlib signal.scala #2964 (LeeTibbert)

  • Fix #2893: Implement posixlib wait.scala #2969 (LeeTibbert)

  • Rework many POSIX and C lib with new trait feature #2996 (ekrich)

  • Fix #3263: Fix failing in utsnameOps #3264 (mox692)

  • Fix #3276: Remove two major defects in posixlib utsname.scala #3280 (LeeTibbert)

  • Fix #3690: compiling posixlib StatTest no longer warns about use of tmpnam() #3703 (LeeTibbert)

  • Add chroot syscall #3822 (catap)

  • Add pledge and unveil #3823 (catap)

  • Expose scheduler policy #3831 (catap)

C standard library bindings