http://anadoxin.org/blog

My Gradle workflow

Wed, 04 August 2021 :: #java :: #gradle :: #kotlin

When writing short-lived command line tools on JVM I often realise that when trying to run the application I prefer to simply switch to the terminal window and run the app from there, instead of doing it from the IDE. Maybe that's a thing I've inherited from years of using Vim to write all sorts of scripts, or maybe it's a way that allows for easier tinkering with the command line when testing the app -- whatever the case, much of my workflow depends on this requirement.

When writing tools in pure Java or Kotlin my only default choice for the build system is Gradle, because it's so ubiquitous (although recent polls show that it's actually Maven that's more popular, but I can't imagine anyone knows how to write Maven pom.xml's from scratch without using any templates). I've actually started to evaluate Gradle for Scala projects in order to ditch sbt, but I'm not sure yet if that's a good idea, so I'll just refrain from commenting on that :).

In order to launch the application that's using Gradle from the command line we really have to care about one thing: the classpath. All of the applications I've written are using thirdparty libraries, so in order to run the app, we need to take into account each library and its required JAR files. So, in order to run the app from the terminal we can consider these options:

  • Use ./gradlew run. This one is the easiest one and manages everything automatically -- just run this command and the app will be executed.

    But it has also drawbacks: one of them is an awkward method of specifying the command line arguments to your app, another one is that each time your small tool executes, a full gradle instance is executed first, which takes sometimes a few seconds of wasted time on every execution (and JVM is not exactly known for fast startup times),

  • Use build plugins like shadow. This plugin will add a shadowJar task to your gradle build. If you run it, it will create a "fat jar" executable with all your dependencies included. This jar will often be multiple megabytes in size (depending on the number of dependencies you're using), but the advantage is obvious -- you just need to use java -jar your-generated-shadow.jar command in order to run the app, and it'll magically execute without gradle's help.

    The disadvantage of this is that jar packaging takes time, so while it's very helpful when done just before the deployment step, it's still a PITA to wait for it each time we want to simply run another iteration of our app. So, as far as quick execution after development iteration is concerned, this is no better than the ./gradlew run method above.

  • Default tasks added by the application plugin like distZip or distTar have the same problem like the shadow plugin -- they take time, although they're probably faster than shadow, because they don't need to repackage (re-zip) the class files.

    The obvious disadvantage of this approach is that it copies lots of files (multiple megabytes) and compresses them with zip, only for us to unpack the archive and run the tool while disposing the intermediate zip file.

None of the methods work on the long run (I mean they work all right; but at the cost of time of the development iteration), so I guess we need to dive into Gradle a little bit. The method I've worked out in a few of my last tools is forcing Gradle to output the current classpath for the project, and writing a wrapper bash script to use this data while running the app directly with the java command.

So, let's say we have this build script (in .kts dialect) for our app:

import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
import java.io.*

plugins {
    kotlin("jvm") version "1.5.21"
    id("com.github.johnrengelman.shadow") version "5.2.0"

    application
}

group = "me.antek"
version = "1.0-SNAPSHOT"

repositories {
    mavenCentral()
}

dependencies {
    implementation("info.picocli:picocli:4.6.1")
    implementation("com.github.kittinunf.fuel:fuel:2.3.1")
    runtimeOnly("com.github.kittinunf.fuel:fuel-gson:2.3.1")
    implementation("com.google.code.gson:gson:2.8.7")
    implementation("com.sksamuel.hoplite:hoplite-core:1.4.4")
    implementation("com.sksamuel.hoplite:hoplite-toml:1.4.4")
    implementation("org.fusesource.jansi:jansi:2.3.4")
    implementation("org.jsoup:jsoup:1.14.1")
    implementation("org.apache.commons:commons-lang3:3.12.0")
    implementation("org.apache.commons:commons-text:1.9")
    testImplementation(kotlin("test"))
}

tasks.test {
    useJUnit()
}

tasks.withType<KotlinCompile>() {
    kotlinOptions.jvmTarget = "1.8"
}

application {
    mainClassName = "MainKt"
}

It declares the shadow plugin, but we won't use it here. It also declares multiple libraries, like:

  • PicoCLI -- a really cool library that acts as a "framework" for command line apps, handles arguments, does many things,
  • Fuel -- a HTTP client that also works on Android if needed,
  • Gson -- a Java JSON deserialization library,
  • Hoplite -- config file loader, it supports the TOML format which I like,
  • JAnsi -- console color support, works also on Windows terminal,
  • JSoup -- a Java HTML parser, when writing web scrapers this library is a must,
  • Commons-lang and commons-text -- auxiliary helper libraries with a pretty diverse set of functionalities.

So, our list of libraries is maybe not huge, but there's a few of them.

Let's ask Gradle to dump the current classpath somewhere. My workflow assumes that it dumps the classpath into the .deps.sh script:

tasks.register("deps") {
    val paths = configurations.runtimeClasspath.get().asPath

    FileOutputStream(".deps.sh").bufferedWriter().use { fw ->
        fw.write("DEPS=\"${paths}\"")
    }

    println("> Written .deps.sh")
}

tasks.getByName("build").dependsOn += "deps"
tasks.getByName("run").dependsOn += "deps"

This snippet, when appended to the build.gradle.kts will register the deps task, allowing it to be manually called by ./gradlew deps command. This task will dump current classpath into the .deps.sh file in the current directory. The deps task is then registered to be a prerequisite of the build and run tasks, so each time a build task is run, our deps task will run as well.

The generated .deps.sh file can look like this:

DEPS="/home/antek/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib-jdk8/1.5.21/6b3de2a43405a65502728047db37a98a0c7e72f0/kotlin-stdlib-jdk8-1.5.21.jar:/home/antek/.gradle/caches/modules-2/files-2.1/info.picocli/picocli/4.6.1/49a67ee4b4d9722fa60f3f9ffaffa72861c32966/picocli-4.6.1.jar:/home/antek/.gradle/caches/modules-2/files-2.1/com.github.kittinunf.fuel/fuel-gson/2.3.1/96753e785fe8bf92d5505ae66c9401db95d3e003/fuel-gson-2.3.1.jar:/home/antek/.gradle/caches/modules-2/files-2.1/com.github.kittinunf.fuel/fuel/2.3.1/c1b88ced2b60d50c519526d0a82463561afcbcd4/fuel-2.[...]

so it's a pretty much autogenerated file that is different for every user, as well as it depends on the minor versions of the libraries inside gradle's cache dir.

Now, in order to run the application, all we have to do is to source the .deps.sh file, and use it to set up a proper classpath. This is an example .run.sh file:

#!/usr/bin/bash

source $PWD/.deps.sh
java -cp $DEPS:build/classes/kotlin/main:build/classes/java/main MainKt "$@"

Running the .run.sh file should execute your application directly, without using any ./gradlew wrappers, and without copying any files, not to mention that it's also without re-zipping anything. Just build in your IDE, switch to terminal, and do ./.run.sh, it should magically work.

I've tested this on Linux and it works there. No idea about Windows. But if you're a Windows hacker, you'll probably figure it out ;)