Live website builds with `shake-watch`

TL;DR: You can make a blog with live-reload using our updates to Slick with our template.

At MigaMake we spend a lot of our attention on automation. Everything that can be automated should be automated. That’s not an only good rule but also a productive approach, thinking ahead about developers and users of our open sources solutions. Last couple of months we spent on polishing Slick project, a static website generation tool that utilizes Shake build system and allows to create sites in a more productive and repeatable manner with regular Haskell record types.

One of the most requested features by our front-end developers is the ability to see their changes on the fly, regardless of what’s changed – HTML template, CSS file or something in the Haskell code, the internals of site builder. Slick is a rather new project that follows a minimalistic approach. It doesn’t provide requested functionality out of the box and we had to define our own solution that helps to resolve this request.

There are several options to resolve this issue

  • use an already existing external tool to
  • use a well-known tool from the Haskell ecosystem
  • use non-conventional tool to emulate specific behavior

Shake Watch Solution

We wanted to maintain close coupling with Shake project and decided to use a set of well-known Haskell tools used for different cases.

1) For the static files used for templating and styling, we extended Shake with our small library shake-watch that track special directory where all templates and styles stored. For Slick- based projects this is usually site directory but of course, can be easily changed to something else. For tracking, we use fsnotify package as a tool that served well for many previous projects and identify a set of options, called WatchOptions similar to ShakeOptions. That allows integrating our library in a Shake-based project with configuration specific for a particular case.

2) We were able to connect GHCI interactive GHC rebuilds with Shake build system and our simplistic watch system that creates continuous rebuilding pipeline. Just a few months ago ghcid was released to the new version that able to track file changes, --restart/--reload key triggered if new files are added. That’s exactly what we want without the need to reinvent the wheel. We decided to integrate this feature with our pipeline. For the files that are supplementary to project we use tools also already available for use, creating a productive mashup for our needs.

Additionally, we use an internal caching feature provided by Shake system - the ability to create a local database with all required rules and update only those rules that are related to the recent changed in files.

To run the builder in a continuous rebuild mode you need to use functionality provided by shake-watch library:

  1. Move build rules to a separate function (something like buildRules)
  2. Move site builder to a separate function (something like runShakeBuilder)
  3. Set watch options for your needs
  4. Use provided runWatcher function, forward build rules and automatically start rebuilds on file change. Watcher needs to be started in separate thread and block for continuous execution.
main :: IO ()
main = do
  ..
  -- some other code

  -- customize Shake options
  let shOpts  = shakeOptions {shakeVerbosity = Quiet}
  -- customize watcher options by modifying default set
      wcOpts  = defaultWatchOps
                  { watchPath   = cwd
                  , includePath = cwd ++ "/site"
                  , remake      = True
                  }
  forkIO $
    runWatcher                       -- function to start watcher
      shOpts                         -- shake options
      wcOpts                         -- watch options
      (buildRules flags'')           -- function describing rules
      (runShakeBuilder shOpts flags) -- site builder function

  -- block execution to run watcher continuously
  forever $
    threadDelay 100000

Making those steps not only provides watching functionality but also encourage you to decompose your builder functionality into several smaller functions.

In shake-watch you can identify a set of useful options. For Slick users most important are

  • watch directory, place where lives Haskell source code
  • supplementary directory, place where lives website data - templates, html/css/js files, content files.

Live Reload Mode

For the Slick-based project, this library provides live-reload functionality that automatically updates static website generated output. To achieve this we implemented package slick-live that contains WebSocket server implementation and notification system. You can integrate it with already existing functionality simply by starting the server after

import           Slick.Serve
...

main :: IO ()
main =
  shakeArgs shakeOptions {shakeVerbosity = Chatty} $ do
  -- build ruls and actions ..
  liftIO $ void . forkIO $ do
    putStrLn $ "Running with Preview"
    serverStart "public" "127.0.0.1" 3030 serverHandler

Additionally, On a client-side you need to inject simplified javascript code that handles live r eloads (reading notifications from websocket server), that looks like this:

var ws = new WebSocket('ws://localhost:3030');
ws.onopen = function() {
    ws.send("Browser client connected!");
}
ws.onmessage = function (event) {
    console.log(event.data);
    window.location.reload(true);
};

put it inside template where you want rebuilds, usually templates that share functionality and embedded in each generated file, like a header. Customize the action flow that happening on a file change in Haskell source:

  withSocketsDo
    $ WS.runClient "127.0.0.1" 3030 "/" wsClient

Note, that not all browsers allow you to run unsecured WebSocket connection. So, you need to change the browser configuration

For example, in Firefox higher version 66, navigate about:config and look for network.websocket.allowInsecureFromHTTPS parameter and set it to true.

Benefits

  • low system footprint
  • reusing well-established solution
  • extending Shake build system with convenient tooling

How to run from template

You can start sample Slick project right away with all functionality available out of the box using this Stack template. Simply by running:

# stack new <name of the project> https://gitlab.com/migamake/slick-template/raw/master/simple-slick.hsfiles

If you already have Slick-based project and want to start using additional features. You’ll need to add in cabal dependencies:

and import Slick.Serve, Slic.Extra, Shake.Development.Watch packages to start using new functionality.

Conclusions

This small but very convenient library allows us to simplify development workflow, reduce the time our frontend developers spent on manual actions and provide almost instantaneous feedback for content writers.

During the development, we extended the functionality of Slick with useful functions and created a separate package with additional tools useful fo static website generation, called slick-extra.

This collection of tools slick-extra, slick-live and shake-watch significantly improved our development workflow and prepared foundation for future improvements.

  1. Shake - universal build system
  2. Slick - static website generator
  3. GHCid - release feature