bundled1

This is the simplest CLI model to implement. Our "bundled1" approach keeps all application authored content in the "src" folder and the application is bundled into a single output file using a webpack first build model.

Here are the main ingredients:

• .scala files to bundled together using sbt

• Other application .js source files, all of which will be bundled together.

• External dependencies from npm (nodejs based) NOT bundled into our application, they are loaded using nodejs's standard import module process.

The bundling/unbundled language applies to the .js files in our project. While it is possible to bundle even nodejs's packages, there is not much value in doing so.

scala

• src/main/scala/app:

  • cli.app - main cli app that prints out the command line parameters and contains some small facades to access chalk and table

We use the npm versions of chalk and table.

Dependencies

• scala:

  • scopt (scalajs version)

• pure javascript:

  • src/main/resources/ContentFormatting.js

  • src/main/resources/formatting.js

  • src/main/scala/app/messages.js

• facades:

  • io.scalajs.nodejs - comprehensive nodejs facade

• npm (left unbundled):

  • chalk - colorized terminal output

  • table - table output

Outputs

• bin:

  • cli.js - output cli executable (shell script)

  • cli.js.map - source map

• node_modules

  • chalk

  • table

Our core output cli.js will be built using webpack. webpack will take both the output of scalajs as well as the dependency*.js files and bundle them into a single output file.

Build config

This example requires both sbt and webpack to create the final outputs in the form we want. We could use sbt for some of the last processing steps but that would require some code and knowing more about sbt vs webpack configuration. Granted, both sbt programming and webpack configuration are complex topics, but we want to show how webpack fits into the overall build especially when there are alot of moving parts.

In the end, the build will be a webpack led build lifecycle. If you are not familiar with webpack, each section is explained below. It may look complicate but its easily turned into a javascript function that takes the name of your source directories, the name of your main .scala file and the naem of your output script. Most of the webpack configuration below can be considered boilerplate independent of the actual CLI application.

Here's our cli.webpack.config.js:

let webpack = require('webpack'),
    merge = require('webpack-merge'),
    nodeExternals = require('webpack-node-externals'),
    UglifyJSPlugin = require('uglifyjs-webpack-plugin'),
    fs = require("fs"),
    path = require("path"),
    util = require("util"),
    common, config;

const cli = "cli.js",
      outfiledir = path.resolve(__dirname, "bin"),
      outfile = path.resolve(outfiledir, cli);

const scalajsLoader = {
    test: /\.scala$/,
    use: [
        { loader: 'scalajs-loader' },
        {
            loader: "source-map-loader",
            options: { enforce: "pre" }
        }
    ]
};

common = {
    entry: path.resolve(__dirname, "src/main/scala/app/cli.scala"),
    output: {
        path: outfiledir,
        filename: cli
    },
    target: "node",
    devtool: "source-map",
    externals: [nodeExternals()],
    resolve: {
        alias: {
            Provided: path.resolve(__dirname, './src/main/resources'),
            app: path.resolve(__dirname, './src/main/scala/app')
        },
        modules: [path.resolve(__dirname, "./src/main/resources"), "node_modules"]
    },
    module: {
        rules: [
            scalajsLoader,
            {
                test: /\.js$/,
                exclude: [/(node_modules)/, /(target)/],
                use: [
                    {
                        loader: "source-map-loader",
                        options: { enforce: "pre" }
                    },
                    "babel-loader"]}]},
    plugins: [
        new webpack.BannerPlugin({banner:"#!/usr/bin/env node", raw: true}),
        function() {
            this.plugin('done', () => {
                fs.chmodSync(outfile, "755");
            })}]};

switch(process.env.npm_lifecycle_event) {
case 'afterscalajs':
    config = merge(common, {
        entry: path.resolve(__dirname, "target/scala-2.12/bundled1-fastopt.js")
    });
    break;

case 'build':
    scalajsLoader.use[0].options = {
        jsStage: "fullOptJS",
        clean: true
    };
    config = merge(common, {
        devtool: "source-map",
        plugins: [
            new UglifyJSPlugin({
                sourceMap: true,
                compress: true
            })]});
    break;

default:
    config = merge(common, {});
    break;
}

//console.log(util.inspect(config));

module.exports = config

There's alot here so lets take it in pieces.

Imports & vars

let webpack = require('webpack'),
    merge = require('webpack-merge'),
    nodeExternals = require('webpack-node-externals'),
    UglifyJSPlugin = require('uglifyjs-webpack-plugin'),
    fs = require("fs"),
    path = require("path"),
    util = require("util"),
    common, config;

const cli = "cli.js",
      outfiledir = path.resolve(__dirname, "bin"),
      outfile = path.resolve(outfiledir, cli);

const scalajsLoader = {
    test: /\.scala$/,
    use: [
        { loader: 'scalajs-loader' },
        {
            loader: "source-map-loader",
            options: { enforce: "pre" }
        }
    ]
};

Since the webpack config file is a nodejs module, we can require what we need at the top. We also set up some convenience variables that allow us easily modify the scalajsLoader loader configuration when we wish to "merge" our webpack config together for different targets. We will want to modify the scalajsLoader to reflect fastOptJS or fullOptJS based on whether we are building for full production or not.

Specify start of webpack graph

common = {
    entry: path.resolve(__dirname, "src/main/scala/app/cli.scala"),
    output: {
        path: outfiledir,
        filename: cli
    },
    target: "node",
    devtool: "source-map",
    externals: [nodeExternals()],
...

This specifies that webpack should assume the cli.scala file is the entry point for our CLI program. The final output will the executable script. Our target build is "node" which tells webpack not to bundle in nodejs standard libraries like "os" or "fs".

Our nodeExternals() is a function that returns all of the modules in the node_modules directory so that they are excluded from the bundle. We want to do this of course because we want the nodejs modules to remain in their default location and not bundled into our cli.js. If there were a reason to bundle them, say for example so that the script could be moved around to another location, we could bundle everything. However, some modules rely on the local filesystem for files (using the nodejs "fs" module) and these modules would not be pulled in through webpack directly. If you want to bundle nodejs dependencies (still excluding default nodejs builtin modules), you could whitelist the ones in nodeExternals() or remove the externals directive altogether.

Resolving modules

We used placeholders in our scala like app and Provided. When webpack loads the javascript file from scalajs, the module "requires" are still present and must be "traced" by webpack to find all dependencies. Webpack ultimately leaves them be, unresolved via the "externals" directive, or pulls them into the final bundle that it builds. Webpacks "resolve" is used to direct how app and Provider are resolved. It is a general alias capability, so you can actually be quite flexible in how modules paths are resolved to actual javascipt modules.

resolve: {
        alias: {
            Provided: path.resolve(__dirname, './src/main/resources'),
            app: path.resolve(__dirname, './src/main/scala/app')
        },
        modules: [path.resolve(__dirname, "./src/main/resources"), "node_modules"]
    },

The first two are straight aliases. The "resolve.modules" says that when a module is expressed as a module without local pathing information, search in src/main/resources as well as the standard locations such as "node_modules" for those modules. So require(MyModule) will be first searched in src/main/resources/MyModule.js and then in node_modules/MyModule` type of location using normal module resolution rules that nodejs uses (search for index.js, or look in package.json or look for the file directly with a .js extension, etc.)

Webpack loaders and build.sbt

scalajs-loader is the loader for .scala files. The scalajs-loader npm module provides a webpack loader that shells out to sbt to build. Hence, we need a build.sbt file which is quite simple since we are using webpack to resolve module imports and perform final processing:

enablePlugins(ScalaJSPlugin)

name := "bundled1"
scalaVersion := "2.12.2"
scalaJSModuleKind := ModuleKind.CommonJSModule
scalaJSUseMainModuleInitializer := true

resolvers += Resolver.jcenterRepo

libraryDependencies ++= Seq(
  "io.scalajs"  %%% "nodejs" % "0.4.0-pre5",
  "com.github.scopt" %%% "scopt" % "latest.version"
)

val postScalaJS = TaskKey[Unit]("postScalaJS", "Run webpack and then the program after scalajs.")

postScalaJS := {
  println("Running webpack then CLI command")
  "npm run afterscalajs"!;
  "bin/cli.js -a blah"!
}

Our project/* files are:

sbt.version=0.13.15

and

addSbtPlugin("org.scala-js" % "sbt-scalajs" % "0.6.15")

We also add a task that will be used during development cycles that shells out to npm to run webpack and run the CLI program with some program arguments.

js files in "target" are also excluded, which seems a bit strange at first since later we define target/scala-2.12/bundled1-fastOt.js" as an input into the webpack graph. Webpack already knows how to default load .js files. Our extra js loader is used to run babel over our .js files and pickup the sourcemaps which will created based on our .babelrc configuration

{
    presets: ['env'],
    sourceMaps: "inline",
    plugins: ["transform-object-rest-spread"]
}

Babel is essentially like scalajs, it is a transpiler. Since scalajs outputs standard .js already with its own source maps, we want to exclude the specific babel-oriented loader from reinterpreting the scalajs output file.

Plugins

This last webpack config says to add the shebang line to the output, cli.js. Then, an inline webpack module is specified which changes the execution mode and final name of the CLI program.

Merging for different nodejs "lifecycles"

switch(process.env.npm_lifecycle_event) {
case 'afterscalajs':
    config = merge(common, {
        entry: path.resolve(__dirname, "target/scala-2.12/bundled1-fastopt.js")
    });
    break;

case 'build':
    scalajsLoader.use[0].options = {
        jsStage: "fullOptJS",
        clean: true
    };
    config = merge(common, {
        devtool: "source-map",
        plugins: [
            new UglifyJSPlugin({
                sourceMap: true,
                compress: true
            })]});
    break;

default:
    config = merge(common, {});
    break;
}

//console.log(util.inspect(config));

module.exports = config

This section merges different configs together depending on the npm lifecyle. The npm lifecycle is based on the package.json "script" labels. There are some standard lifecycles that are "convention" vs required, and you are free to make up your own, which we did to some extent. You can see that for a production build, "build", we want to uglify the output. scalajs output in fullOptJS has already been optimized, but we need to optimize the webpack bundled .js. We could us the js version of GCC here or just uglify since it's easy to access and the bulk of our program is already optimized. It's probably good enough for a CLI. We do not really need to uglify but why not since our source files can be easily accessed elsewhere.

package.json

We never really specified our package.json file, which we need to do because need to declare our dependencies and ensure that when nodejs installs our CLI it knows how to do that. Also, package.json holds toplevel "scripts" to execute to run the build:

{
  "bin": {
    "cli": "./bin/cli.js"
  },
  "description": "Scalajs+nodejs CLI application",
  "name": "cli-app",
  "version": "0.1.0",
  "license": "MIT",
  "scripts": {
    "build": "webpack -p --progress --config cli.webpack.config.js",
    "dev": "webpack --progress --config cli.webpack.config.js",
    "afterscalajs": "webpack --progress --config cli.webpack.config.js",
    "afterscalajs-watch": "webpack --progress --config cli.webpack.config.js --watch",
    "clean": "sbt clean"
  },
  "dependencies": {
    "chalk": "^1.1.3",
    "table": "^4.0.1"
  },
  "devDependencies": {
    "babel-cli": "^6.24.0",
    "babel-core": "^6.8.0",
    "babel-loader": "^6.2.4",
    "babel-plugin-transform-object-rest-spread": "^6.23.0",
    "babel-preset-env": "^1.3.2",
    "scalajs-loader": "0.0.1",
    "source-map-loader": "^0.2.1",
    "uglifyjs-webpack-plugin": "^0.4.3",
    "webpack": "^2.5.1",
    "webpack-merge": "^4.1.0",
    "webpack-node-externals": "^1.5.4"
  }
}

The dependencies were declared saved using npm install --save-dev <dependency> or npm install --save <dependency> as appropriate.