Distributed Erlang

For a generic introduction to Distributed Erlang Systems, please refer to the dedicated section of Erlang/OTP documentation.

AtomVM provides an implementation of Erlang distribution protocol and AtomVM nodes can take part in clusters with both AtomVM and BEAM nodes.

Distribution is currently available on all platforms with TCP/IP communication, namely:

  • Generic Unix

  • ESP32

  • RP2 (Pico)

Two examples are provided:

  • disterl in examples/erlang/disterl.erl: distribution on Unix systems

  • epmd_disterl in examples/erlang/esp32/epmd_disterl.erl: distribution on ESP32 devices

Starting and stopping distribution

Distribution has to be started programmatically. Following Erlang/OTP, distribution relies on kernel which is started by init:boot/1.

The following lines will start distribution on Unix systems with long name atomvm@127.0.0.1.

{ok, _NetKernelPid} = net_kernel:start('atomvm@127.0.0.1', #{name_domain => longnames}),
ok = net_kernel:set_cookie(<<"AtomVM">>).

net_kernel:stop/0 can be used to stop distribution.

Distribution options

The options map passed to net_kernel:start/2 supports an avm_dist_opts key containing a map of options that are forwarded to the distribution module’s listen/2 function.

socket_dist options

The built-in socket_dist module supports the following avm_dist_opts:

  • listen_port_min — minimum port number to listen on

  • listen_port_max — maximum port number to listen on

Both must be specified together. socket_dist will try each port in the range until one is available. This is useful on systems where only a specific range of ports is open (e.g. firewall rules on embedded devices).

{ok, _NetKernelPid} = net_kernel:start(mynode, #{
    name_domain => shortnames,
    avm_dist_opts => #{listen_port_min => 9100, listen_port_max => 9110}
}).

When avm_dist_opts is omitted or the port keys are not set, the OS assigns an ephemeral port (the default behaviour).

epmd

AtomVM nodes can use Erlang/OTP’s epmd on Unix systems. AtomVM is also bundled with a pure Erlang implementation of epmd which can be used on all platforms. Module is called epmd, to be distinguished from erl_epmd which is the client.

AtomVM’s epmd daemon can be started with:

{ok, _EPMDPid} = epmd:start_link([]).

This has to be called before invoking net_kernel:start/2.

Erlang/OTP compatibility

AtomVM can connect to Erlang/OTP 24 and higher.

Security

AtomVM supports cookie authentication. However, distribution over TLS is not supported yet.

Alternative carrier

Following Erlang/OTP, AtomVM supports alternative carriers with distribution modules. Please refer to Erlang/OTP’s dedicated documentation.

The main difference is that packets exchanged by f_recv and f_send handlers must be binaries instead of list of integers, for memory usage reasons.

AtomVM’s f_send has the following signature:

fun (DistCtrlr, Data :: binary()) -> ok | {error, Error}

AtomVM’s f_recv has the following signature:

fun (DistCtrlr, Length :: pos_integer(), Timeout :: timeout()) -> {ok, Packet} | {error, Reason}

AtomVM’s distribution is based on socket_dist and socket_dist_controller modules which can also be used with BEAM by definining BEAM_INTERFACE to adjust for the difference.

Distribution features

Distribution implementation is (very) partial. The most basic features are available:

  • serialization of all types

  • epmd protocol (client and server)

  • message passing

  • monitoring processes

  • I/O distribution (“group leader”).

RPC (remote procedure call) from Erlang/OTP to AtomVM is also supported.

Shell requires several OTP standard library modules. See the example project.

Please do not hesitate to file issues or pull requests for additional features.