Getting Started with SOMns Development
This guide gives an overview of the development infrastructure and setup used for working on SOMns. First, the basics of obtaining the code, setting up software dependencies, the development environment, and how to contribute code are discussed. Afterwards, we briefly give an overview of the repository and code layout. And finally, look at the additional infrastructure used by the project.
1.1 Minimal Software Requirements
SOMns is implemented in Java 8, uses Ant as a build system, git as source control system, and Python for a launcher script.
On Ubuntu, the following instructions will install the necessary dependencies:
sudo add-apt-repository ppa:webupd8team/java sudo apt install oracle-java8-installer git ant
1.2 Getting the Code and Running Hello World
To checkout the code:
git clone https://github.com/smarr/SOMns.git
Then, SOMns can be build with Ant:
cd SOMns ant compile ## will also download dependencies
Afterwards, the simple Hello World program is executed with:
./som -G core-lib/Hello.som
1.3 Development Environment
To setup a complete development environment, we need also the Graal just-in-time compiler, and Node.js to work on the Kompos Web Debugger.
On Ubuntu, the necessary software can be installed with:
## First, installing Node.js and the Node Package Manager (NPM) curl -sL https://deb.nodesource.com/setup_7.x | sudo -E bash - sudo apt install npm nodejs ## Second, installing and compiling the Graal JIT Compiler cd .. ## leaving the SOMns folder git clone --recursive https://github.com/smarr/GraalBasic.git cd GraalBasic yes "n" | ./build.sh cd ../SOMns ant ## to ensure everything is compiled
At this point, it should be possible to use Graal to run SOMns by dropping the
-G option from the command line:
1.3.1 Eclipse to Develop the Interpreter
SOMns is currently developed with Eclipse. While other Java IDEs can also be used, for brevity, we’ll focus on Eclipse only.
For development, we also need to setup all Eclipse projects:
After the Eclipse projects for the Truffle library were generated by this step, we can import the existing projects into Eclipse with File -> Import… -> Existing Projects into Workspace and pointing Eclipse to the SOMns folder.
I like to import the Truffle projects into a separate Truffle working set. This makes working in Eclipse with many projects easier.
The result should be looking roughly, but not exactly like in the screenshot on the right.
For debugging the interpreter with Eclipse, create a Eclipse run configuration
for the SOMns project. The main class is
som.VM. To run for instance the
Mandelbrot benchmark, add the following as program arguments:
core-lib/Benchmarks/Harness.som Mandelbrot 2 0 500
In VM arguments, enable assertions with:
I personally start the various SOMns programs from the command line:
./som -d -G core-lib/Benchmarks/Harness.som Mandelbrot 2 0 500
For this approach, we need a Remote Java Application debug configuration
in Eclipse. After starting SOMns, it should tell you that it is waiting on port
8000, which is used as the port in the Eclipse debug configuration.
1.3.2 VS Code to Develop with the SOMns Language
To have IDE-like support for working with SOMns code, we are using VS Code, a cross-platform editor for which we have SOMns integration.
SOMns support can be installed via the VS Code marketplace.
A brief list of steps:
Install software dependencies: ant, git, Java 8, Eclipse 4.6 (or later), VS Code 1.8 (or later), Node.js, NPM, Graal JIT compiler
Create Truffle Eclipse projects:
Import Eclipse projects
antfrom the command line or Eclipse
2. Repository and Code Layout
This section gives a brief overview over the most relevant elements.
2.1 Repository Root
.gitlab-ci.yml # Configuration for the SOFT Build Server .travis.yml # Configuration for the Travis Build Service CHANGELOG.md # The changelog contains a high-level change overview build.xml # The Ant build description codespeed.conf # Benchmark configuration, based on ReBench core-lib # All SOMns code, including standard lib, tests, and benchmarks som # Launcher script src # Java sources src_gen # Java sources generated by the Truffle DSL tests # Java unit tests and tests for DynamicMetrics tool tools # Contains the Kompos Web Debugger
2.2 Code Layout: Java
som |- compiler # Parser, AST creation, and source representation |- instrumentation # AST instrumentation support, used e.g. by DynamicMetrics tool |- interop # Interoperability with Truffle languages, only minimally implemented |- interpreter # Dynamic SOMns language semantics, i.e., interpreter and AST nodes |- nodes # AST and dispatch node implementations |- objectstorage # SOMns object model implementation |- actors, processes, transactions # core elements of concurrency models |- primitives # Basic operations, exposed via vmMirror object to SOMns |- vm # Basic VM setup, startup, and object system initialization |- vmobjects # Representation for build-in object and arrays |- VM.java # Java entry point and bridge between interpreter and tools tests |- debugger.JsonTests # Tests GSON serialization of debugger messages |- som # JUnit tests for interpreter, including runner for BasicInterpreterTests tools |- actors # Tracing support for actor features |- debugger # Connection to the Kompos Web Debugger |- dym # DynamicMetrics tool
2.3 Code Layout: SOMns
core-lib |- Benchmarks # Collection of various benchmarks |- TestSuite # Collection of tests |- BasicInterpreterTests # Minimal tests only executable as JUnit tests |- Minitest.som # Newspeak's Minitest framework |- *Tests.som # Various test suites |- Actors.som # Actor and promise classes |- Collections.som # Sets, Dictionaries, etc |- Hello.som # Hello World program |- Kernel.som # Core classes: Integer, Boolean, String, Array etc |- Mirrors.som # Minimal and incomplete mirror API |- Platform.som # Application loader |- Processes.som # Communicating Sequential Processes classes |- System.som # Minimal API to access system functionality |- Threading.som # Threading and fork/join-related classes |- Transactions.som # Software transactional memory classes
2.4 Code Layout: Kompos
Kompos is a web-based debugger integrated into SOMns. It is based on Truffle’s language agnostic debugger support and extends it to provide actor-specific debugging facilities. It is implemented in TypeScript.
Note that Kompos and the VS Code debugger share the TypeScript code that is used to interface with the Java backend.
/tools/kompos/ |- index.html # The HTML elements for the debugger |- package.json # NPM package definition |- src |- breakpoints.ts # Breakpoint related code |- controller.ts # MVC controller to connect model and view |- debugger.ts # Debugger model to manager state needed in UI |- history-data.ts # Model for the actor message history |- main.js # Main JS file used from HTML frontend |- messages.ts # Definitions of messages exchanged with SOMns interpreter |- view.ts # The HTML view code |- visualizations.ts # Visualizes actor interactions as graph |- vm-connection.ts # Web socket connection to the SOMns interpreter |- tests # Tests debugger interaction with SOMns, including some SOMns integration tests |- tsconfig.json # TypeScript config |- tslint.json # TypeScript lint settings
3. Basic Design
This section gives a brief overview of some basic design aspects of SOMns and Kompos.
3.1 Communication with Kompos
Communication between Kompos and SOMns is implemented with WebSockets, the code can be found in tools.debugger.FrontendConnector and vm-connection.ts. Data is transferred on two separate sockets, one is for JSON and the other one is for binary data.
JSON is used for two-way communication in a message-oriented style. Message objects are serialized to JSON when sending, and received JSON data is used to create message objects. Kompos also (de)serializes communication data, messages.ts provides some interfaces that help accessing data of message objects.
On the SOMns side, message classes need to be “registered” in tools.debugger.WebDebugger.createJsonProcessor(), they extend either IncomingMessage or OutgoingMessage. When an IncomingMessage is received, its process method is called.
The binary socket is used for directly sending tracing data, which is pushed to Kompos whenever available. Kompos parses the tracing data according to the trace format and uses the data to generate the actor graph.
3.2 Trace Format
The trace data includes currently the following events:
- Actor creation
- Promise creation
- Promise resolution: when a Promise is resolved with a value.
- Promise chained: when a Promise is resolved with another Promise.
- Mailbox (continued): set the context for following message events (receiver, base message id), occurs when a mailbox is processed.
- Message: can only occur after a mailbox (continued) or a message event, represents messages in the mailbox.
3.3 Startup Protocol
The following diagram gives an overview of the startup protocol. For simplicity the binary WebSocket for trace data and the view object are not included.
Give a brief overview of some of the infrastructure used in SOMns.
som Launcher Script
To document, simplify, and standardize how SOMns is started or debugged, we use a Python script to execute the actual Java program.
The Python script manages various command-line parameters and selects the JVM to be used for execution.
Below, we see that
./som --help supports a large set of options, of which we
detail only a few.
The basic options include
-d to allow to attach a Java debugger, for instance
-dnu to print stack traces on a
-G to run without Graal-based JIT compilation.
Generally, the options are designed to use upper-case shorthands when they
disable a feature.
Further below, we see different categories of other options. This includes
flags to investigate and understand how Graal executes the interpreter,
options for various profiling tools, as well as tools for SOMns code coverage,
dynamic execution metrics, or interactive debugging of SOMns code
(currently called ‘web debugger’
$ ./som --help usage: som [-h] [-d] [-t THREADS] [-p SOM_PLATFORM] [-k SOM_KERNEL] [-dnu] [-i] [-if] [-io ONLY_IGV] [-l] [-ti] [-w] [-f] [-v] [-gp] [-ga] [-gi] [-gb] [-tp] [-td] [-wd] [-dm] [-at] [-atcfg ACTOR_TRACING_CFG] [-mt] [-tf TRACE_FILE] [-TF] [--coverage COVERAGE] [-o ONLY_COMPILE] [-A] [-B] [-C] [-G] [-X] [-T] [--no-graph-pe] [-vv] [--print-graal-options] [-D JAVA_PROPERTIES] ... optional arguments: -h, --help show this help message and exit -d, --debug wait for debugger to attach -dnu, --stack-trace-on-dnu print a stack trace on #doesNotUnderstand: -G, --interpreter run without Graal Investigate Execution -i, --igv dump compilation details to IGV -l, --low-level enable low-level optimization output -v, --visual-vm connect to VisualVM for profiling Profile Execution -gp, --graal-profile enable Graal-level profiling after warmup -tp, --truffle-profile enable Graal-level profiling after warmup Tools -td, --truffle-debugger start Truffle debugger -wd, --web-debugger start Web debugger -dm, --dynamic-metrics capture Dynamic Metrics --coverage COVERAGE determine code coverage and store in given file
4.2 Build System
As seen earlier, SOMns uses Ant as build system. The setup tries to minimize the external software dependencies. Currently, instead of using some automatic dependency management system for SOMns, we use an uberjar that combines all rarely changing Java dependencies.
The Truffle library is however directly used as a git submodule dependency, because it changes frequently, and we sometimes need changes in Truffle. Currently, SOMns also relies on a personal fork of Truffle to support changes in the instrumentation and debugging support.
SOMns relies on GitHub, its issue tracking, and pull request system for development.
Change Tracking with Pull Requests: The general approach is that all kind of changes are supposed to be documented at least with a pull request to simplify tracking.
When you are getting started with working on the SOMns interpreter internals, consider checking out the Getting Started label. These issues are more or less simple changes that with a bit of guidance should provide a good introduction to the SOMns code base, an basic understanding of how Truffle-based interpreters work, and a few SOMns specific insights.
4.4 Code Style
When working on SOMns code, please look at the code around you and stick to the style. It might be particular, but it is consistent in this code base.
To ensure basic compliance with the style, we use checkstyle. It is integrated into the build system and continuous integration system. Please use something like Eclipse Checkstyle to integrate it in your editor.
We are also using Codacy to monitor additional style issues or potential bugs. See the STM pull request for examples.
4.5 Development Support
Continuous Integration: To automatically run unit tests for the interpreter, SOMns, and the debugger,
we use Travis CI (see
as well as a private GitLab instance to run benchmarks (see
Performance Tracking: Since one goal of SOMns is to be a platform for research on concurrency with performance close to state-of-the-art JVMs, we continuously track benchmark performance, for startup as well as peak performance with Codespeed. It is run on every change to the master branch, and can be used to track and compare performance of experimental changes as well.
The benchmark execution is configured with
codespeed.conf and are executed
with the ReBench tool.
SOMns Code Coverage: To track SOMns code coverage, we use Coveralls.