Development

Configuring a development environment for JSTime can take 10-30 minutes depending on your internet connection and computer speed. You will need ~10GB of free disk space for the repository and build artifacts.

If you are using Windows, you must use a WSL environment as JSTime does not yet compile on Windows natively.

Before starting, you will need to already have a release build of JSTime installed, as we use our bundler to transpile and minify our code.

$ curl -fsSL https://jstime.dev/install | bash # for macOS, Linux, and WSL

$ npm install -g jstime # the last `npm` command you'll ever need

$ brew tap awfixers-stuff/jstime # for macOS and Linux
$ brew install jstime

$ docker pull jstime/jstime
$ docker run --rm --init --ulimit memlock=-1:-1 jstime/jstime

$ proto install jstime

Install LLVM

JSTime requires LLVM 15 and Clang 15 (clang is part of LLVM). This version requirement is to match WebKit (precompiled), as mismatching versions will cause memory allocation failures at runtime. In most cases, you can install LLVM through your system package manager:

$ brew install llvm@15

$ # LLVM has an automatic installation script that is compatible with all versions of Ubuntu
$ wget https://apt.llvm.org/llvm.sh -O - | sudo bash -s -- 15 all

$ sudo pacman -S llvm clang lld

If none of the above solutions apply, you will have to install it manually.

Make sure LLVM 15 is in your path:

$ which clang-15

If not, run this to manually link it:

# use fish_add_path if you're using fish
$ export PATH="$PATH:$(brew --prefix llvm@15)/bin"
$ export LDFLAGS="$LDFLAGS -L$(brew --prefix llvm@15)/lib"
$ export CPPFLAGS="$CPPFLAGS -I$(brew --prefix llvm@15)/include"

Install Dependencies

Using your system’s package manager, install the rest of JSTime’s dependencies:

$ brew install automake ccache cmake coreutils esbuild gnu-sed go libiconv libtool ninja pkg-config rust

$ sudo apt install cargo ccache cmake git golang libtool ninja-build pkg-config rustc esbuild

$ pacman -S base-devel ccache cmake esbuild git go libiconv libtool make ninja pkg-config python rust sed unzip

The apt install esbuild command may fail with an Unable to locate package error if you are using a Ubuntu mirror that does not contain an exact copy of the original Ubuntu server. Note that the same error may occur if you are not using any mirror but have the Ubuntu Universe enabled in the sources.list. In this case, you can install esbuild manually:

$ curl -fsSL https://esbuild.github.io/dl/latest | sh
$ chmod +x ./esbuild
$ sudo mv ./esbuild /usr/local/bin

In addition to this, you will need an npm package manager (jstime, npm, etc) to install the package.json dependencies.

Install Zig

Zig can be installed either with our npm package @oven/zig, or by using zigup.

$ jspm install -g @oven/zig
$ zigup 0.12.0-dev.163+6780a6bbf

We last updated Zig on July 18th, 2023

First Build

After cloning the repository, run the following command to run the first build. This may take a while as it will clone submodules and build dependencies.

$ make setup

The binary will be located at packages/debug-jstime-{platform}-{arch}/jstime-debug. It is recommended to add this to your $PATH. To verify the build worked, lets print the version number on the development build of JSTime.

$ packages/debug-jstime-*/jstime-debug --version
jstime 1.x.y__dev

Note: make setup is just an alias for the following:

$ make assert-deps submodule npm-install-dev node-fallbacks runtime_js fallback_decoder bun_error mimalloc picohttp zlib boringssl libarchive lolhtml sqlite usockets uws tinycc c-ares zstd base64 cpp zig link

Rebuilding

JSTime uses a series of make commands to rebuild parts of the codebase. The general rule for rebuilding is there is make link to rerun the linker, and then different make targets for different parts of the codebase. Do not pass -j to make as these scripts will break if run out of order, and multiple cores will be used when possible during the builds.

| What changed | Run this command | | --- | --- | | Zig Code | make zig | | C++ Code | make cpp | | Zig + C++ Code | make dev (combination of the above two) | | JS/TS Code in src/js | make js (in jstime-debug, js is loaded from disk without a recompile). If you change the names of any file or add/remove anything, you must also run make dev. | | *.classes.ts | make generate-classes dev | | JSSink | make generate-sink cpp | | src/node_fallbacks/* | make node-fallbacks zig | | identifier_data.zig | make identifier-cache zig | | Code using cppFn/JSC.markBinding | make headers (TODO: explain explain what this is used for and why it’s useful) |

make setup cloned a bunch of submodules and built the subprojects. When a submodule is out of date, run make submodule to quickly reset/update all your submodules, then you can rebuild individual submodules with their respective command.

| Dependency | Run this command | | --- | --- | | WebKit | jspm install (it is a prebuilt package) | | uWebSockets | make uws | | Mimalloc | make mimalloc | | PicoHTTPParser | make picohttp | | zlib | make zlib | | BoringSSL | make boringssl | | libarchive | make libarchive | | lolhtml | make lolhtml | | sqlite | make sqlite | | TinyCC | make tinycc | | c-ares | make c-ares | | zstd | make zstd | | Base64 | make base64 |

The above will probably also need Zig and/or C++ code rebuilt.

VSCode

VSCode is the recommended IDE for working on JSTime, as it has been configured. Once opening, you can run Extensions: Show Recommended Extensions to install the recommended extensions for Zig and C++. ZLS is automatically configured.

ZLS

ZLS is the language server for Zig. The latest binary that the extension auto-updates may not function with the version of Zig that JSTime uses. It may be more reliable to build ZLS from source:

$ git clone https://github.com/zigtools/zls
$ cd zls
$ git checkout f91ff831f4959efcb7e648dba4f0132c296d26c0
$ zig build

Then add absolute paths to Zig and ZLS in your vscode config:

{
  "zig.zigPath": "/path/to/zig/install/zig",
  "zig.zls.path": "/path/to/zls/zig-out/bin/zls"
}

JavaScript builtins

When you change anything in src/js/builtins/* or switch branches, run this:

$ make js cpp

That inlines the TypeScript code into C++ headers.

Make sure you have ccache installed, otherwise regeneration will take much longer than it should.

For more information on how src/js works, see src/js/README.md in the codebase.

Code generation scripts

JSTime leverages a lot of code generation scripts.

The ./src/jstime.js/bindings/headers.h file has bindings to & from Zig <> C++ code. This file is generated by running the following:

$ make headers

This ensures that the types for Zig and the types for C++ match up correctly, by using comptime reflection over functions exported/imported.

TypeScript files that end with *.classes.ts are another code generation script. They generate C++ boilerplate for classes implemented in Zig. The generated code lives in:

• src/jstime.js/bindings/ZigGeneratedClasses.cpp
• src/jstime.js/bindings/ZigGeneratedClasses.h
• src/jstime.js/bindings/generated_classes.zig To generate the code, run:

$ make codegen

Lastly, we also have a code generation script for our native stream implementations. To run that, run:

$ make generate-sink

You probably won’t need to run that one much.

Modifying ESM modules

Certain modules like node:fs, node:stream, jstime:sqlite, and ws are implemented in JavaScript. These live in src/js/{node,jstime,thirdparty} files and are pre-bundled using JSTime. The bundled code is committed so CI builds can run without needing a copy of JSTime.

When these are changed, run:

$ make js

In debug builds, JSTime automatically loads these from the filesystem, wherever it was compiled, so no need to re-run make dev.

Release build

To build a release build of JSTime, run:

$ make release-bindings -j12
$ make release

The binary will be located at packages/jstime-{platform}-{arch}/jstime.

Valgrind

On Linux, valgrind can help find memory issues.

Keep in mind:

• JavaScriptCore doesn’t support valgrind. It will report spurious errors.
• Valgrind is slow
• Mimalloc will sometimes cause spurious errors when debug build is enabled

You’ll need a very recent version of Valgrind due to DWARF 5 debug symbols. You may need to manually compile Valgrind instead of using it from your Linux package manager.

--fair-sched=try is necessary if running multithreaded code in JSTime (such as the bundler). Otherwise it will hang.

$ valgrind --fair-sched=try --track-origins=yes jstime-debug <args>

Updating WebKit

The JSTime team will occasionally bump the version of WebKit used in JSTime. When this happens, you may see something like this with you run git status.

$ git status
On branch my-branch
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
  modified:   src/jstime.js/WebKit (new commits)

For performance reasons, make submodule does not automatically update the WebKit submodule. To update, run the following commands from the root of the JSTime repo:

$ jspm install
$ make cpp

Troubleshooting

’span’ file not found on Ubuntu

⚠️ Please note that the instructions below are specific to issues occurring on Ubuntu. It is unlikely that the same issues will occur on other Linux distributions.

The Clang compiler typically uses the libstdc++ C++ standard library by default. libstdc++ is the default C++ Standard Library implementation provided by the GNU Compiler Collection (GCC). While Clang may link against the libc++ library, this requires explicitly providing the -stdlib flag when running Clang.

JSTime relies on C++20 features like std::span, which are not available in GCC versions lower than 11. GCC 10 doesn’t have all of the C++20 features implemented. As a result, running make setup may fail with the following error:

fatal error: 'span' file not found
#include <span>
         ^~~~~~

To fix the error, we need to update the GCC version to 11. To do this, we’ll need to check if the latest version is available in the distribution’s official repositories or use a third-party repository that provides GCC 11 packages. Here are general steps:

$ sudo apt update
$ sudo apt install gcc-11 g++-11
# If the above command fails with `Unable to locate package gcc-11` we need
# to add the APT repository
$ sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test
# Now run `apt install` again
$ sudo apt install gcc-11 g++-11

Now, we need to set GCC 11 as the default compiler:

$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 100
$ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 100

libarchive

If you see an error when compiling libarchive, run this:

$ brew install pkg-config

missing files on zig build obj

If you see an error about missing files on zig build obj, make sure you built the headers.

$ make headers

cmakeconfig.h not found

If you see an error about cmakeconfig.h not being found, this is because the precompiled WebKit did not install properly.

$ jspm install

Check to see the command installed webkit, and you can manully look for node_modules/bun-webkit-{platform}-{arch}:

# this should reveal two directories. if not, something went wrong
$ echo node_modules/bun-webkit*

macOS library not found for -lSystem

If you see this error when compiling, run:

$ xcode-select --install

Arch Linux / Cannot find libatomic.a

JSTime requires libatomic to be statically linked. On Arch Linux, it is only given as a shared library, but as a workaround you can symlink it to get the build working locally.

$ sudo ln -s /lib/libatomic.so /lib/libatomic.a

The built version of jstime may not work on other systems if compiled this way.