Simple Command Line Tools with Scala Native
I've been enjoying writing Scala lately, but there are some legitimate cases where being on the JVM just isn't the right fit. Despite liking Scala more than alternatives, I've had to turn to languages like Go and Node.JS to write simple command line tools. With the introduction of Scala Native, that all changes.
Scala Native allows you to write idiomatic Scala code, but instead of compiling down to the JVM or JavaScript, it compiles to LLVM bytecode, then platform-dependent machine code. This makes it perfect for short-lived programs that need a fast boot time, like command line tools.
The Tool
Pretty regularly, I find myself in a situation where a decision needs to be made, but no one particularly cares what the result is. This could be which game to play with a group of friends, where to go for lunch, or what to read next. This can be solved pretty easily with dice or coins, or even an app like Chwazi, but all options are a little slower than a command line tool.
So I set out to write a tool that, given a list of words, could pick one randomly.
Getting Started
To start our Scala Native project, after making sure we have our environment setup, we can generate a new project using giter8 via
mkdir randomizer
cd randomizer
sbt new scala-native/scala-native.g8
By default, the giter8 template generates a Hello World program. We can test that it works with sbt nativeLink, which compiles and links a binary file. This is a long process the first time you run it, taking over half a minute on my machine. After the process has finished, we can test that it works correctly by running the binary.
$ ./target/scala-2.11/randomizer-out
Hello, World!
Adding Libraries
While there are plenty of programs we can write with just Scala, most work gets done with libraries. When looking for a good command line arg parser in Scala, I came across Scopt, which as luck would have it, already compiles to Scala Native. You can see what they did to add compatibility to their project in this commit.
Because they have a Scala Native version, I assumed I could the library the same way as a normal Scala library. It's not quite the same, though, as adding
libraryDependencies += "com.github.scopt" %% "scopt" % "3.6.0"
to build.sbt leads this error:
[info] Compiling 1 Scala source to /Users/kevingreene/programming/randomizer/target/scala-2.11/classes...
[info] Linking (1041 ms)
[error] cannot link: @scopt.OptionDef
[error] cannot link: @scopt.OptionDef::action_scala.Function2_scopt.OptionDef
[error] cannot link: @scopt.OptionDef::text_java.lang.String_scopt.OptionDef
[error] cannot link: @scopt.OptionParser
[error] cannot link: @scopt.OptionParser::head_scala.collection.Seq_scopt.OptionDef
[error] cannot link: @scopt.OptionParser::init_java.lang.String
[error] cannot link: @scopt.OptionParser::opt_char_java.lang.String_scopt.Read_scopt.OptionDef
[error] cannot link: @scopt.OptionParser::parse_scala.collection.Seq_java.lang.Object_scala.Option
[error] cannot link: @scopt.Read
[error] cannot link: @scopt.Read$
[error] cannot link: @scopt.Read$::intRead_scopt.Read
[error] unable to link
[error] (*:nativeLinkNIR) unable to link
[error] Total time: 5 s, completed Jun 16, 2017 3:53:22 PM
This was my first road bump, and was fixed pretty easily with the help of my colleague Richard. Simply add another percentage sign.
libraryDependencies += "com.github.scopt" %%% "scopt" % "3.6.0"
Just as two percentage signs tells sbt to use the right version of the library, three percentage signs tells sbt to use the right target environment, either Scala Native or Scala.js. In otherwords, the first version is equivalent to
libraryDependencies += "com.github.scopt" % "scopt_2.11" % "3.6.0"
whereas the second is equivalent to
libraryDependencies += "com.github.scopt" % "scopt_native0.2_2.11" % "3.6.0"
Breaking Halfway Through
However, if the Scala Native compatible version didn't exist, we'd need to download and compile our dependencies as well.
In this case, while going through the steps for this blog again, I ran into a new issue. While Scopt has a version for Scala Native 0.2, there wasn't a version published for Scala Native 0.3. Luckily, it's not too hard for us to download and compile locally.
First, we clone the GitHub project.
$ git clone https://github.com/scopt/scopt/.
Then, we modify site.sbt, updating the version of the sbt-scala-native plugin to our desired version.
addSbtPlugin("org.scala-native" % "sbt-scala-native" % "OUR.NATIVE.VERSION")
Finally, run sbt publishLocal, which should push the artifact to our local Ivy cache. We can now continue on!
The Interface
Choosing random things in Scala is pretty easily. We can make an object Picker, with a single function choose, as follows
import scala.util.Random
object Picker {
def choose(objects: List[String], count: Int): List[String] = {
Random.shuffle(objects).take(count)
}
}
With this helper object done, we can implement the Main object, extending App, to complete our CLI.
import scopt.OptionParser
case class Config(count: Int = 1, objects: List[String] = List())
object Main extends App {
val parser = new OptionParser[Config]("randomizer") {
override def showUsageOnError = true
head("randomizer", "1.x")
arg[String]("<options>...").unbounded().optional().action((x, c) =>
c.copy(objects = c.objects :+ x)).text("Options to choose from")
help("help").text("prints this usage text")
}
parser.parse(args, Config()) match {
case Some(config) =>
Picker.choose(config.objects, config.count).foreach(println)
case None =>
// arguments are bad, error message will have been displayed
}
}
And it works!
$ ./target/scala-2.11/randomizer-out x y z
z
Fun with Random
But, if we keep testing this, we'll notice a distinct pattern.
$ ./target/scala-2.11/randomizer-out x y z
z
$ ./target/scala-2.11/randomizer-out x y z
z
$ ./target/scala-2.11/randomizer-out x y z
z
$ ./target/scala-2.11/randomizer-out x y z
z
$ ./target/scala-2.11/randomizer-out x y z
z
$ ./target/scala-2.11/randomizer-out x y z
z
$ ./target/scala-2.11/randomizer-out x y z
z
$ ./target/scala-2.11/randomizer-out x y z
z
$ ./target/scala-2.11/randomizer-out x y z
z
Something seemed to be off. Indeed, when I ran Random.nextInt(10) on a fresh Scala Native process, it always returned 4.
Digging in to the root cause, I was about at the point where I was ready to make an issue on the project, but less than an hour after I started the project, the same issue was reported.
Scala Native is definitely the bleeding edge, be prepared to run into cases where random isn't random at all.
While work is done on a reasonable default seed, we can fix our program's randomness by adding a Random instance, seeded with the current time. Picker should now look like this:
import scala.util.Random
object Picker {
val random = new Random(System.currentTimeMillis())
def choose(objects: List[String], count: Int): List[String] = {
random.shuffle(objects).take(count)
}
}
Useful Options
Right now our Randomizer is pretty simple but effective. I've used it a few times myself. But I noticed a few patterns, e.g. flipping a coin with ./randomizer-out Heads Tails. Additionally, I found myself wanting to select random numbers, select more from a list, and more.
Luckily, with Scopt, all that is pretty simple to add. The first thing to add is a mode, which we can split on for commands. By default, this should be choose.
case class Config(count: Int = 1, objects: List[String] = List(), mode: String = "choose")
Now we can use commands to branch to different functionality.
val parser = new OptionParser[Config]("randomizer") {
override def showUsageOnError = true
head("randomizer", "1.x")
// Uses the default of mode = "choose"
val flipConfig = Config(objects = List("Heads", "Tails"))
val rollConfig = Config(objects = List("1", "6"), mode = "roll")
cmd("flip").action((_, _) => flipConfig).
text("Flips a coin")
cmd("roll").action((_, _) => rollConfig).
text("Rolls a die")
cmd("choose").action((_, c) => c.copy(mode = "choose")).
text("choose picks from an unbounded number of strings.").
children(
opt[Int]('c', "count").action((x, c) =>
c.copy(count = x)).text("count determines how many items will be picked"))
cmd("roll").action((_, c) => c.copy(mode = "roll")).
text("roll picks a number between the min and max arguments.").
children(
arg[String]("<options>...").minOccurs(2).maxOccurs(2).required().action((x, c) =>
c.copy(objects = c.objects :+ x)).text("Min followed by Max"))
arg[String]("<options>...").unbounded().optional().action((x, c) =>
c.copy(objects = c.objects :+ x)).text("Options to choose from")
help("help").text("prints this usage text")
}
Now, when parsing, we need to pattern match against all the possible options.
parser.parse(args, Config()) match {
case Some(config) =>
config.mode match {
case "choose" =>
Picker.choose(config.objects, config.count).foreach(println)
case "roll" =>
println(Picker.roll(config.objects(0).toInt, config.objects(1).toInt))
}
case None =>
// arguments are bad, error message will have been displayed
}
So all that's left is to implement the Picker.roll function, like so
def roll(min: Int, max: Int): Int = {
random.nextInt(max + 1 - min) + min
}
After running sbt nativeLink again, we're at the point where we have a fairly flexible and useful cli randomizer!
$ ./target/scala-2.11/randomizer-out choose -c 2 Liz Cedric Dan Kevin
Liz
Cedric
$ ./target/scala-2.11/randomizer-out dice
3
$ ./target/scala-2.11/randomizer-out flip
Tails
$ ./target/scala-2.11/randomizer-out roll 1 10
7
Thoughts
On one hand, Scala Native presented a lot of challenges. There were some unexpected hiccups, and bugs in standard libraries.
On the other, it was a delight to have some libraries just work. I thought integrating Scopt would be a much more complicated process, but it was relatively painless after the %%% trick.
I don't plan on writing much Scala Native code in production on the current version, but the quick pace of development has me hopeful that it will be a compelling option in the near future.
All code is available on GitHub