Whole Modules

When I first began making websites, I adopted the popular practice of organizing my projects by file type. Stylesheets would all live in a directory together, as would scripts. My projects tended to look like this:

styles/
    map.css
    layout.css
scripts/
    map.js
    home.js
map.html
index.html

Here, map.html uses both map.js and map.css. Making a change to one of these 3 files would often necessitate a change to the others. Effectively, they got worked on together, as a whole. Although it was easy to locate each file, having them in different directories obscured their relationship.

One evening, I attended a JavaScript meetup where the speaker (whose name I have forgotten) argued that if files change together, they should live together. This is obvious when you consider a directory to be a kind of module, a realization that had somehow escaped me. That night I stayed up late reorganising my project, horrified at how unmodular it now appeared. Continuing from the example above, I ended up with something like:

map/
    index.html (formerly map.html)
    map.js
    map.css
home.js
index.html
layout.css

The advantages of grouping related files together became apparent over time. The map/ directory could be moved around with no adjustment to internal references, such as the relative path of <script src="./map.js">. Also, map/ could be deleted without the risk of orphaning distant files.

But I did not learn my lesson. I went on to write my first Node.js application, and structured it thus:

src/
    auth.js
    validate.js
test/
    auth.js
    validate.js
server.js
README.md

In its defence, this structure demanded very little of the test runner. It was as simple as running the command

$ node test_runner.js test/

In every other way, however, it was suboptimal. I was doing my best to practice Test Driven Development, so a change to src/auth.js would usually coincide with a change to its test, test/auth.js. Having such closely related files so distant from each other was a source of friction and mistakes. Sometimes I would neglect to update a test simply because I forgot it existed. Out of sight, out of mind. And with test/ mirroring the directory structure of src/, tests had to reference their modules via long and brittle paths like ../../../src/path/to/module.js.

Realizing my folly, I collapsed the src/ and test/ directories, moving their contents into the application's root directory.

auth.js
auth.test.js
validate.js
validate.test.js
server.js
README.md

The test runner had to work a little harder to find the tests, but it rose to the challenge.

$ node test_runner.js ./*.test.js ./**/*.test.js

Having the test right next to the implementation made my life easier, and for several years I was happy with this structure. But then I discovered that some modern programming languages had taken this idea even further, providing a way to include tests alongside the implementation in the same file. Rust, a language 20 years younger than JavaScript, encourages modules like double.rs:

pub fn double(number: isize) -> isize {
    2 * number
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn positive() {
        assert_eq!(double(3), 6);
    }

    #[test]
    fn negative() {
        assert_eq!(double(-2), -3);
    }
}

Rust's built in test runner, cargo test, compiles the program (enabling the conditional mod tests block) and runs every #[test] function it finds.

$ cargo test
    Finished test [unoptimized + debuginfo] target(s) in 0.75s
     Running unittests src/double.rs

running 2 tests
test tests::positive ... ok
test tests::negative ... FAILED

---- tests::negative stdout ----
thread 'tests::negative' panicked at 'assertion failed: `(left == right)`
  left: `-4`,
 right: `-3`', src/double.rs:16:9

test result: FAILED. 1 passed; 1 failed; 0 ignored; ...

There are some clear benefits that come with Rust's approach to testing. For one, the tests are able to make use of the module's private functions and data structures, which would otherwise need to be exposed. That helps to keep module interfaces small. Another benefit is that a module can be worked on without the hassle of frequently switching editor tabs. I have found working within a single file for extended periods of time to be a more focused and enjoyable experience.

This is all well and good for Rust, a modern compiled language, but I wanted to use this approach with JavaScript, a decrepit scripting language. I recalled that Python, an even more decrepit scripting language, has a special __name__ variable that lets a module determine whether it is the entry point to the program. Porting double.rs to Python, we get double.py:

def double(number):
    return 2 * number

if __name__ == "__main__":
    if double(3) != 6:
        raise Exception("FAIL positive")
    if double(-2) != -3:
        raise Exception("FAIL negative")

double.py has an interesting duality. When imported as a module, it exhibits no side effects because __name__ is not "__main__". But when run from the command line, __name__ is "__main__" so the tests are executed. If any test fails, the process exits with a non-zero exit code.

$ python double.py || echo "Bug detected"
Traceback (most recent call last):
  File "double.py", line 8, in <module>
    raise Exception("FAIL negative")
Exception: FAIL negative
Bug detected

Now, imagine my excitement when I discovered JavaScript's import.meta.main property. Its value is true when the current module is the entry point, otherwise its value is false. At the time of writing, import.meta.main is only implemented by a few runtimes, including Deno. Though it is not part of any standard, there is a general consensus on what it means.

Let's use it to port double.py to double.js.

function double (number) {
    return 2 * number;
}

if (import.meta.main) {
    if (double(3) !== 6) {
        throw new Error("FAIL positive");
    }
    if (double(-2) !== -3) {
        throw new Error("FAIL negative");
    }
}

export default Object.freeze(double);

Notice that double.js and double.py share the same duality. Importing this module causes no side effects, even if the runtime does not implement import.meta.main. That is because import.meta.main will be undefined, a falsy value, in such runtimes. On the other hand, running this module with Deno executes the tests.

$ deno run double.js || echo "Bug detected"
error: Uncaught (in promise) Error: FAIL negative
        throw new Error("FAIL negative");
              ^
    at double.js:10:15
Bug detected

Deno, like other command line JavaScript runtimes, exits with a non-zero exit code in response to an uncaught exception or unhandled Promise rejection. Even when a test takes many turns to complete (perhaps because setTimeout or fetch is involved) it can always throw to fail loudly.

For want of an existing term, I have been calling this technique whole modules. A whole module is whole because it contains everything you need to work on it. It can be imported as per usual, yet when executed as a program it demonstrates its own correctness.

Because whole modules take responsibility for their own demonstration, tooling can be almost trivially basic. For example, running

$ watch --color deno run double.js

in a terminal window provides continual feedback as double.js is modified.

But whole modules are not limited to the command line: they can also be written for the browser. Although no browser implements import.meta.main today, it is not hard to selectively replace occurrences of import.meta.main with true in source code served up during development.

My tool of choice for working with whole modules is Replete. It lets me quickly and easily run JavaScript in the browser and Deno, directly from my text editor. In code evaluated by Replete, import.meta.main is always true. That makes running a demo is as simple as highlighting a module's code and evaluating it. You can try it for yourself here. See what happens when you change quick brown fox to slow brown fox in the test.

As I began writing whole modules for the browser, it became clear that automated tests are just one kind of demo. Whole modules in the browser have the full capabilities of the DOM at their disposal, so their demos can be visual and interactive. Whole modules are a convenient way to exercise user interface components in isolation, a practice that helps to reduce coupling in web applications.

I have been writing whole modules exclusively for about 3 years now. When I encounter a module without a demo, I feel frustrated that I can not interact with it immediately. It is like attempting to strike up a conversation, only to be stonewalled.

Further reading

• Locality of Behaviour
• Ecomcon
• Replete