How was decidables made?

The decidables project is a collection of explorable explanations of decision making: d′etectable, prospectαbλe, and diskountable. Each explorable is a statically-served website with interactive elements implemented as web components. The websites and component libraries are built from source using a toolchain running on Node.js and published to GitHub Pages and the npm registry respectively.

Note: If you want to see exactly how the project is built, nothing beats looking at the source in the decidables monorepo on GitHub!

Websites

The site for each explorable is published as HTML, SVG, CSS, and JavaScript. However, the content is authored in other related languages that facilitate the development process, and then compiled to the native web languages.

Text

The body text is authored in markdown which is lighter weight than HTML, so it makes for a better writing experience for substantive content. The content files also have front matter written in YAML containing metadata to guide their use. They are compiled into HTML using front-matter and remark, which provides a highly flexible ecosystem for working with markdown. For example, using remark-directive along with a custom plugin, remark-terminology, bespoke formatting is added for keywords (e.g. :key[Accuracy]), terminology (e.g. :term[fixation]), tools (e.g. :tool[remark]), and UI features (e.g. button:[Pause]).

Citations and references

Another custom remark plugin, remark-citeproc, is used to include citations (e.g. :cite[@Green1966]). The full set of references for a site are stored in BibTeX format. During compilation, citeproc-js and Citation.js are used to generate APA-formatted citations, and all of the cited works are collected for output into an APA-formatted reference list that is injected into a markdown page.

Page structure

The structure of the webpages, including headers, footers, and sidebars is written in the Embedded JavaScript templating language with YAML front matter. By combining HTML and JavaScript, this makes it easy to specify the organization of the pages a single time, and have it reused for every page on the site. Elements like “next” links are generated programmatically, since their target is different on each page. EJS is used to combine the templates with the content output from remark to generate the resulting HTML pages.

Styling

The styles for each site are written in SCSS syntax using the Bootstrap framework. Together these help to create clean consistent styling with much less effort than writing raw CSS from scratch. The SCSS is compiled to CSS using Dart Sass.

The look of any project depends in no small part on colors, fonts, icons, and other decorations. This project uses the Source super-family of fonts from Adobe Originals, which provides serif (Source Serif), sans-serif (Source Sans), and monospace (Source Code Pro) fonts that work together seamlessly and all have a large compliment of characters, weights, and styles (including variable font support).

Color is a powerful way to visually convey meaning, but finding a set of colors that meets the needs of a diverse audience is a challenge. This project relies on ColorBrewer for a set of distinct, saturated colors (9-class Set 1).

For icons, the project depends on Bootstrap Icons.

Script

The code for each site is written in modern JavaScript using ECMAScript modules and ES.NEXT features. This allows for clean, expressive coding using features like template literals and arrow functions. In order to support a wide range of browsers, Babel and rollup are used to compile this code to ES5 and an IIFE for execution in the browser, with more advanced language features supported by core-js and regenerator-runtime.

Publishing

All of the explorables are developed in the decidables monorepo which is a Git repository hosted on GitHub. When changes are pushed from a local working copy to the remote origin, this triggers a GitHub Action which deploys the updates by pushing them to the decidables.github.io repository from where they are staged to decidables.github.io.

Component libraries

The interactive elements in the explorable explanations are implemented as web components which are published in packages to npm.

Packages

Each explorable is encapsulated in a site package, and has two accompanying library packages: a math library exposes a single class of static methods that implement core calculations of the associated theory (for example, the subjective value function for cumulative prospect theory); and an elements library containing all of the associated web components. Those web components fall into three categories: components, which are individual interactive graphs, tasks, controls, etc…; equations, which are live visualizations of key equations; and examples, which combine components (and equations?) to form interconnected demonstrations. For example, d′etectable has its own detectable package, along with detectable-math, and detectable-elements.

In addition, there is a core decidables-elements package with basic UI components that are shared across the individual explorables.

Code

The code for the libraries, like the sites, is written in modern JavaScript using ECMAScript modules and ES.NEXT. Where appropriate, this code can be imported directly from source en masse or using deep imports. For direct use in browsers, it is also compiled to ES5 and bundled as both UMD and ESM using Babel and rollup, with advanced language features supported by core-js and regenerator-runtime.

Note: The issue of which library formats to provide seems to be both complicated and fraught these days. I’ve opted to provide multiple options: untranspiled source exposed by exports, a transpiled UMD bundle exposed by main, and a transpiled ESM bundle exposed by module. There are both minified and unminified versions of the bundles available.

Web components

To support the broadest possible usage, the interactive elements are implemented as native web components, as opposed to the proprietary components used in many frameworks. This means that they each define a custom element that can be used directly in HTML, once the appropriate library has been included (e.g. via a script tag). For example, an interactive rendering of ROC space can be used as easily as:
<roc-space interactive hr="0.75" far="0.25"></roc-space>

Our web components are implemented using Lit, which facilitates a declarative reactive approach based on tagged template literals.

Graphics

Most of our graphical components are specified as SVG. Advantages of this format include high resolution at any scale, persistent objects via the DOM, CSS styling, and event handling as in HTML. In order to ease the dynamic integration of data with SVG, we use D3 to directly bind data to the DOM. It comes in particularly handy when generating SVG paths from mathematically-defined curves, and for path transitions.

Note: Lit and D3 are not a natural fit. The canonical way to update the DOM in Lit is declaratively via tagged template literals. The canonical way to update a component in D3 is programmatically via selections. When using D3, we override the typical Lit approach, which works fine.

Statistics

Unlike, say R, JavaScript has almost no built in statistics functions. Fortunately, jStat provides just enough of the right sort of things, including a nice selection of statistical distributions, to get us where we need to go.

Publishing

All of the libraries, like the sites, are developed in the decidables monorepo which is a Git repository hosted on GitHub. Unlike the sites, the library packages are published to the npm registry using lerna with semantic versioning (semver) supported by the use of conventional commits. The libraries all reside in the @decidables namespace. For example, the detectable-elements package would be @decidables/detectable-elements.

Development tools

As much as possible, the development tooling for this project runs in the Node.js ecosystem. The fundamental advantage of this is that the tooling and the project itself are both running in the same language using the same infrastructure. This ecosystem works seamlessly across operating systems, and has an astounding variety of packages for web development. Indeed, I might go so far as to suggest that the Node/npm/GitHub ecosystem is one of mankind’s greatest works of collective engineering.

Linting

Web development requires the use of multiple languages, each with its own requirements and quirks. Layered on top of that, they can be used and formatted in a wide variety of ways. We use linting tools to statically analyze code in order to catch errors early and maintain consistency. We lint JavaScript with eslint, CSS with stylelint, markdown with remark-lint, and HTML with HTMLHint and Nu Html Checker.

Testing

Good software development practice includes automated testing. We are implementing testing of our web component libraries using Web Test Runner and the Open Web Components Testing Package. Each component has a corresponding test file with appropriate tests.

Task automation

Building libraries and sites with all of the tools described above involves a great deal of coordination and repetition. The streaming build system and task automation engine gulp is used to facilitate these processes. Using plugins, it allows us to flexibly compose different tools together in order to take files in a variety of different source formats and process them into files in a variety of different output formats. Since the structure is specified with JavaScript code, instead of an esoteric configuration language, the system has easily adapted over time to changing tools and needs.

Package management

Yarn is used for package and workspace management. It is fast at installing and updating dependencies, and does an excellent job of seamlessly handling our monorepo organization, where all of the sites and libraries are housed within a single overarching decidables package/repository.

Note: We are not making use of Yarn PNP. There are two main reasons for this. First, when experimenting with it, we’ve hit multiple inscrutable errors coming from the depths of our build, and it wasn’t obvious how to resolve those issues. Second, it seems to obfuscate file-system access to our dependencies. It is often helpful to browse the code in a package to understand exactly what it is doing. PNP makes this harder by removing node_modules and storing packages in cached zip files instead.

Version control

Software development involves a continual process of creating and revising many interrelated files. A version control system/source code management tool facilitates and empowers this process, while recording an accurate history of every step along the way. We use Git for this with a remote repository hosted on GitHub, which allows for highly distributed development.

We are using husky and commitlint to consistently use conventional commits. Using a standard format for git commit messages makes it easier to understand what is changing, and facilitates adherence to the principles of semantic versioning.

File editing

The text editor is a coder’s nearest and dearest tool. This project has mostly been developed using first Atom with its supporting packages and now VSCode with its supporting extensions. Both of these editors are implemented using JavaScript so they can directly load devtools in our tool chain. For example, the live linting in the editor uses the identical linter with the identical rules as we run in our build process.

Reference management

Literature references were collected and organized using Zotero and the Zotero Connector extension for Chrome. Reference data can be pulled in seamlessly from web pages, and then exported in BibTeX format for use in the project.