Experimental Erlang backend to replace Node-REDs existing NodeJS backend, aiming for 100% compatibility with existing Node-RED flow code.

Tooling that even HAL uses.

Bring the advantages of low-code visual flow-based programming to a programming language that is designed for message passing and concurrency from the ground up, hence Erlang. More details described in the corresponding blog post.

Node-RED is great for creating data flows that actually describe concurrent processing, it is just a shame the NodeJS is single threaded. So why not use something that is multi-process from the ground up? Concurrency is guaranteed and included.

Also Erlang isn't the most understandable of programming language - unless one has fallen into in a cauldron of Prolog, spiced with Lisp.

So won't it be great to have the simplicity of low-code visual flow based programming and the performance (and concurrency) of Erlang?

Online: Thanks to @mwmiller, Erlang-Red can be tested at ered.fly.dev.

Locally: Using docker:

docker run --interactive --tty --publish 8080:8080 gorenje/erlang-red:0.3.4

Starts Erlang-Red web listener on localhost:8080/erlang-red and drops the console into an Eshell.

There are two screencasts on using the Erlang-Red docker image:

• Starting and running tests - 5 mins
• Using Erlang-Red to breadboard Diameter protocol - 10 mins

Erlang-Red at Docker Hub.

For more details on the project, check out my interview at the TADS Blog - I go into why Erlang-Red and how it differs from Node-RED and what influence Flow Based Programming has on both.

Erlang-Red by Example screencasts

• Creating a HTTP API Backend - Flow - 8 mins

• Using message tracing to better understand flow execution - 5 mins

• Configuring a visual genserver behaviour in Erlang-Red - Flow - 9 mins

• Using the supervisor node in Erlang-Red - Flow - 8 mins

• Supervisors of Supervisors of Supervisors ... - Flow - 6 mins

• Configuring the intensity of Supervisors - Flow - 5 mins

• Binary node and interpreting binary data in Erlang-Red - Flow and Flow - 9 mins

Erlang-Red implementing Statemachines screencasts

1. Creating Mermaid (UML) Diagrams in the flow editor - Flow - 8 mins

2. Creating a state-machine definition from a flow diagram = Flow - 10 mins

3. Unittesting state-machines in Erlang-Red - Flow - 9 mins

Erlang-Red - accessing existing processes

1. Using the Process node to access existing Erlang processes - 9 mins

2. CaptureIO node for better debugging - 6 mins

Erlang-Red Walkthrough - diameter protocol

Thanks to Alan @ TADSummit, I was able to do a walkthrough of Erlang-Red and how to implement an extension to the diameter protocol in Erlang-Red - visually.

For that walkthrough, I created a collection of flows that can be imported into Erlang-Red (or Node-RED or Browser-Red).

Many thanks to @joaohf there is a Erlang-Red recipe for Yocto project.

Also I did a quick experiment with a Raspberry Pi 4 to get the I2C nodes working. That wasn't "true" embedding since Erlang-Red was running in a docker container on a Raspberry running a debian distribution!

Breadboards are prototyping devices found in electronics. Erlang-Red can be best thought of as a programming breadboard.

What are some tools for software prototyping? Besides AI and VScode. Software developers create prototypes but they don't prototype software.

A telnet session flow describes how breadboard programming can be done using Erlang-Red. That flow prototypes a possible software solution starting with a simple concurrent approach until a first final approach is found. All solutions are testable and usable - instantly and all solutions build on previous solutions - simply copy and paste the flows. That's prototyping.

Implementation of the MQTT specs to create an MQTT broker in Erlang-Red. The broker is created as a flow and at the same time, a client is created using the Erlang-Red MQTT nodes so that the broker implementation can be tested. Again a breadboard: thing of the MQTT nodes as an oscilloscope testing the voltage!

My development process is best described as flow driven development based around a set of test flows to ensure that node functionality is implemented correctly - meaning that it matches the existing Node-RED functionality.

Test flows are mirrored in a separate repository for better maintainability and also integration with existing Node-RED installations.

Erlang architecture is best described by describing various use cases:

• Deploying flows to Erlang-Red. Explains the start up process and how Erlang processes are started for nodes.
• Workings of a supervisor node supervising a function node.
• The challenges of function node which must support timeouts, sub-processes and being supervised by a supervisor.
• Inner workings of link nodes and how to deal with dynamic link calls.

This is a non-complete list of nodes that partially or completely work:

These nodes represent specific Erlang features as nodes and as such, could be implemented in NodeJS to provide Node-RED with the same functionality.

These nodes can be installed using the corresponding Node-RED node package. In Node-RED these nodes are placebos, doing nothing.

There is also a sidebar tab for listing all existing processes and importing these into Erlang-Red.

Nodes for ensuring truth in unit test flows.

These nodes can be installed using the corresponding Node-RED node package.

• Contexts are not supported, so there is no setting things on flow, node or global.
• JSONata has been partially implemented by the Erlang JSONata Parser.

Elixir helpers can be added to erlang-red-elixir-helpers repository.

There is nothing stopping anyone from creating a complete node in Elixir provided there is a Erlang "node-wrapper", i.e., a bit of Erlang code in the src/nodes directory that references the Elixir node.

The initial example markdown node is an Erlang node that references Elixir code. I also wrote an Elixir wrapper function whereby I could have just as easily referenced Earmark directly from the Erlang code. That was a stylist choice.

I intend to use Elixir code for importing Elixir libraries to the project and less coding nodes in Elixir. I simply prefer Erlang syntax. But each to their own :)

$ rebar3 get-deps && rebar3 compile

$ rebar3 eunit

rebar3 shell --apps erlang_red

Open the Node-RED visual flow editor in a browser:

open -a Firefox http://localhost:9090/node-red

I use docker to develop this so for me, the following works:

git clone [email protected]:gorenje/erlang-red.git
cd erlang-red

docker run -it -v $(pwd)/erlang-red:/code -v $(pwd)/data:/data -p 8080:8080 -w /code --rm erlang bash

## inside docker shell:
rebar3 shell --apps erlang_red

Then from the docker host machine, open a browser:

open -a Firefox http://localhost:8080/node-red

That should display the Node-RED visual editor.

A release can be bundled together:

$ rebar3 as prod release -n erlang_red

All static frontend code (for the Node-RED flow editor) and the test flow files in priv/testflows are bundled into the release.

Cowboy server will started on port 8080 unless the PORT env variable is set.

A sample Dockerfile Dockerfile.fly is provided to allow for easy launching of an instance as a fly application. The provided shell script (fly_er.sh) sets some common expected parameters for the launch. Advanced users may wish to examine the fly launch line therein and adjust for their requirements.

Using the container stack at heroku, deployment becomes a git push heroku after the usual heroku setup:

• heroku login --> heroku git:remote -a <app name> --> heroku stack:set container --> git push heroku

However the Dockerfile.heroku does not start the flow editor, the image is designed to run a set of flows, in this case (at time of writing) a simple website with a single page.

Basically this flow is the red-erik.org site.

The image does this by setting the following ENV variables:

• COMPUTEFLOW=499288ab4007ac6a - flow to be used. This can also be a comma separated list of flows that are all started.
• DISABLE_FLOWEDITOR=YES - any value will do, if set the flow editor is disabled.

Also be aware that Erlang-Red supports a PORT env variable to specifying the port upon which Cowboy will listen on for connections. The default is 8080.

Heroku uses this to specify the port to connect for a docker image so that its load balancer can get it right.

What the gif shows is executing a simple flow using Erlang as a backend. The flow demonstrates the difference in the switch node of 'check all' or 'stop at first match'.

All nodes are are processes - these are shown on the left in the terminal window.

This example is extremely trivial but it does lay the groundwork for expansion.

To create unit tests for this, Node-RED frontend has been extended with a "Create Test Case" button on the export dialog:

Test flows are stored in the testflows directory and will be picked up the next time make eunit-test is called. In this way it is possible to create unit tests visually.

Flow tests can also be tested within the flow editor, for more details see below.

The flow test suite is now maintained in a separate repository but is duplicated here.

To better support testing of flows, two new nodes have been created:

"Assert Failed" node cases unit tests to fail if a message reaches it, regardless of any message values. It's basically the same as a assert(false) call. The intention is to ensure that specific parts of a flow aren't reached.

The second node (in green) is an equivalent to a change node except it contains test on attributes of the message object. Possible tests include 'equal', 'match', 'unset' and the respective inverses. Here the intention is that a message passes through is tested for specific values else the unit test fails.

These nodes are necessary since there is no other way to test whether flow is working or not.

Also remember these flow tests are designed to ensure the Erlang backend is correctly implementing node functionality. The purpose of these nodes is not to ensure that a flow is correct, rather that the functionality of implemented nodes works and continues to work correctly.

My plan is to create test flows that represent specific NodeRED functionality that needs to be implemented by Erlang-Red. This provides regression testing and todos for the implementation.

I created a keyboard shortcut for creating and storing these test flows from the flow editor directly. However I was still use the terminal to execute tests make eunit-test - which became painful. So instead I pulled this testing into Node-RED, as the gif demonstrates:

What the gif shows is my list of unit tests, which at the press of a button, can all be tested. Notifications for each test shows the result. In addition, the tree list shows which tests failed/succeed (red 'x' or green check). Also tests can be executed individually so that failures can be checked individually.

The best bit though is that all errors are pushed to the debug panel and from there I get directly to the node causing the error. Visual unit testing is completely integrated into Erlang-Red.

My intention is to create many small flows that represent functionality that needs to be implemented by Erlang-Red. These unit tests shows the compatibility to Node-RED and less the correctness of the Erlang code.

Contributions very much welcome in the form of Erlang code or as Node-RED test-flows, ideally with the Erlang implementation. Elixir code is also welcome, only it has its own home.

Each test flow should test exactly one feature and use the assert nodes to check correctness of expected results. Tests can also be pending to indicate that the corresponding Erlang functionality is still missing.

An overview of the sibling projects for both the reader and me:

• Unit test flow suite provides visual unit tests that verify the functionality being implemented here is the same as in Node-RED. Those test flows are designed to be executed in both Node-RED and Erlang-Red. FlowHub.org maintains the repository and is used to synchronise flow tests between Erlang-Red and Node-RED. These tests can also be used for other projects that aim to replicate Node-RED functionality in an alternative programming language.
• Node-RED and Erlang-Red unit testing nodes are used to define and automatically ensure the correct functionality. These nodes are embedded in test flows and ensure that test flows are correct. This makes testing repeatable and reliable and fast! As an aside, these nodes are maintained in an Node-RED flow.
• JSONata support for Erlang-Red is implemented by an Erlang parser with a grammer that covers most of JSONata syntax, no guarantees made. Support of JSONata functionality is limited to what the test flows require. Nothing prevents others from extending the functionality themselves, it is not a priority of mine.
• Elixir helper library allows Elixir code to be also part of Erlang-Red. Erlang-Red is not intended to be a pure Erlang project, it is intended to be a pure BEAM project. Anything that compiles down to the BEAM VM, why not include it?
• Supervisor and Behaviour nodes and other Erlang behaviours as Node-RED nodes. Node package includes gen_statem and gen_event as nodes that can be used with Erlang-Red flows. These nodes can also be installed into Node-RED but there they do nothing.
• Type parsers for parsing specific Node-RED types such as Number or Buffer. Also for handling attribute access of maps and arrays.
• Erlang Processes Nodes provide structured access to existing Erlang processes allowing message interaction with processes and capturing I/O from existing processes.

Questions and Answers at either the Erlang Forum or the Node-RED Forum.

Also for more details, there was also a discussion on Hacker News.

Nick and Dave for bring Node-RED to live - amazing quality and flexibility and the entire Node-RED community.

Much thanks to

• @mwmiller for providing a fly server for running a live version of Erlang-Red,
• @joaohf many tips on coding Erlang and structuring an Erlang project, and
• @Maria-12648430 for debugging my initial attempt to create a gen_server for nodes.
• @joergen7 for the Erlang insights and explaining dialyzer to me and the importance of clean code
• @vkatsuba for the great tips on using ETS tables for buffering messages

This offers multi-licensing smorgasbord to pick the license that best meets your needs:

• if you wish to do evil and are not concerned with the impact of your behaviour (probably because you gain a financial reward from said behaviour), then you want to using the apache-2 license.

• if your concerned about the impact of closed source software and the erosion of the commons of shared knowledge, then you might consider the gpl license

• if you're planning to do good, for example, for educational purposes and provide others with the knowledge to make informed decisions, then you might want to consider the don't do evil license

No Artificial Intelligence was harmed in the creation of this codebase. This codebase is old skool search engine (ddg), stackoverflow, blog posts and RTFM technology.

AI contributions can be made according to the rules defined in .aiignore.