architecture,erlang,docker,elixir,microservices , Where does Elixir/erlang fit into the microservices approach? [closed]

Where does Elixir/erlang fit into the microservices approach? [closed]


Tag: architecture,erlang,docker,elixir,microservices

Lately I've been doing some experiments with docker compose in order to deploy multiple collaborating microservices. I can see the many benefits that microservices provide, and now that there is a good toolset for managing them, I think that it's not extremely hard to jump into the microservices wagon.

But, I have been experimenting with Elixir too, and I am quite fond of the benefits that provides by itself. Given that it encourages packing your code into multiple decoupled applications, and supports hot code upgrades, how would you mix docker with elixir (or erlang, for that matter)?

For example, if I want to use docker because it provides dev-prod parity, how does elixir fit in that? Given that docker containers are immutable, I lose the ability to do hot-code upgrades, right? What about blue/green deployments or canary releases?

I mean, I could just write microservices with Elixir and use them as if they were written in any other language, polyglotism is one of the benefits of microservices anyway, but then I'm not getting the full benefits of using the OTP platform, I guess that pure collaborative erlang applications are way more optimal that using intermediate queues to communicate between microservices written in different (or not) languages.

Thanks in advance.


This is a very open question but I will try to illustrate why Elixir/Erlang may be the best platform out there for developing distributed systems (regardless if you are working with microservices).

First, let's start with some background. The Erlang VM and its standard library were designed upfront for building distributed systems and this really shows up. As far as I know, it is the only runtime and VM used widely in production designed upfront for this use case.


For example, you have already hinted at "applications". In Erlang/Elixir, code is packaged inside applications which:

  1. are started and stopped as unit. Starting and stopping your system is a matter of starting all applications in it

  2. provide a unified directory structure and configuration API (which is not XML!). If you have already worked with and configured an OTP application, you know how to work with any other one

  3. contains your application supervision tree, with all processes (by process I mean "VM processes" which are lightweight threads of computation) and their state

The impact of this design is huge. It means that Elixir developers, when writing applications have a more explicit approach to:

  1. how their code is started and stopped

  2. what are the processes that make part of an application and therefore what is the application state

  3. how those process will react and be affected in case of crashes or when something goes wrong

Not only that, the tooling around this abstraction is great. If you have Elixir installed, open up "iex" and type: :observer.start(). Besides showing information and graphs about your live system, you can kill random processes, see their memory usage, state and more. Here is an example of running this in a Phoenix application:

Observer running with a Phoenix application

The difference here is that Applications and Processes give you an abstraction to reason about your code in production. Many languages provides packages, objects and modules mostly for code organization with no reflection on the runtime system. If you have a class attribute or a singleton object: how can you reason about the entities that can manipulate it? If you have a memory leak or a bottleneck, how can you find the entity responsible for it?

If you ask anyone running a distributed system, that's the kind of insight that they want, and with Erlang/Elixir you have that as the building block.


All of this is just the beginning really. When building a distributed system, you need to choose a communication protocol and the data serializer. A lot of people choose HTTP and JSON which, when you think about it, is a very verbose and expensive combination for performing what is really RPC calls.

With Erlang/Elixir, you already have a communication protocol and a serialization mechanism out of the box. If you want to have two machines communicating with each other, you only need to give them names, ensure they have the same secret, and you are done.

Jamie talked about this at Erlang Factory 2015 and how they were able to leverage this to build a game platform:

If you want to use HTTP and JSON, that is fine too and libraries like Plug and frameworks like Phoenix will guarantee you are productive here too.


So far I haven't talked about microservices. That's because, up to this point, they don't really matter. You are already designing your system and nodes around very tiny processes that are isolated. Call them nanoservices if you'd like to!

Not only that, they are also packaged into applications, which group them as entities that can be started and stopped as unit. If you have applications A, B and C, and then you want to deploy them as [A, B] + [C] or [A] + [B] + [C], you will have very little trouble in doing so due to their inherent design. Or, even better, if you want to avoid adding the complexity of microservices deployments into your system upfront, you can just deploy them altogether in the same node.

And, at the end of the day, if you are running all of this using the Erlang Distributed Protocol, you can run them in different nodes and they will be able to reach other as long as you refer to them by {:[email protected], :name} instead of :name.

I could go further but I hope I have convinced you at this point. :)


os kern error : “ld: symbol(s) not found for architecture x86_64”

I have looked all over Stack Overflow and other websites about this famous error, and all of them are very specific, and in my case I cannot find a solution. I am making an ncurses application and when i try to compile it, it causes the following error: Undefined symbols...

Erlang spawning large amounts of C processes

I've been looking into how I could embed languages (let's use Lua as an example) in Erlang. This of course isn't a new idea and there are many libraries out there that can do this. However I was wondering if it was possible to start a Genserver with state which...

Does Erlang has Map?

It seems that Erlang introduced maps in version R17A. But, if I go to downloads I only see version 17.5, there's no version R17A. So, is it released and stable yet? Does the latest stable Erlang has Map support?...

Micro Service cross service dependencies

Just to simplify my situation I currently have 3 micro services. Authentication Locations Inventory The authentication service authenticates the user and sends back a JWT access token and I use that across the other services. Its stateless and all works well. I setup locations among some other things in the...

Elixir exrm release crashes on eredis start_link

I'm fairly new to Elixir and this is the first app that I'm attempting to release using exrm. My app interacts with a Redis database for consuming jobs from a queue (using exq), and also stores results of processed jobs in Redis using eredis. My app works perfectly when I...

passing arrays to a function in erlang

I am new to erlang programming and as a matter of fact no very experienced programmer. I need to pass an array to a function and then manipulate it.. however I have so far not be this what I found. module(easy). export([myfunction/1]). myfunction([myarray]) -> %% mycode.. hier I will need...

How to invoke Erlang function with variable?

4> abs(1). 1 5> X = abs. abs 6> X(1). ** exception error: bad function abs 7> erlang:X(1). 1 8> Is there any particular reason why I have to use the module name when I invoke a function with a variable? This isn't going to work for me because, well,...

Getting SSL related error against my request to Ejabberd

Following is the code snippet where i open a socket to write APNS notifications on: get_socket()-> %%Options Options = [{certfile, ?Cert}, {keyfile, ?Key}, {mode, binary}], %%ssl connection ssl:connect(?Address, ?Port, Options, infinity) . close_socket(Socket)-> ssl:close(Socket). I am getting the following crash in my ejabberd.log file 2015-06-05 12:33:17.112 [error] <0.3134.0> gen_fsm <0.3134.0>...

Erlang: Returning a function from a function

I know Erlang supports anonymous functions. My question is, can I return a function from a function then call that returned function from outside? If so, how do I do it? I know this is possible in many languages such as C and Python. Here is what I tried to...

Java generic class that contains an instance of implementation of generic interface

I am developing an independent, self contained component, that needs domain specific parts to function properly. The part of the idea is to create a generic interface, that will settle the usage of interface's implementation in another part of this component (in my example in class B). I have written...

Data flow of MVC application architecture

Attempting to validate the approach for data flow in an MVC application that i am cleaning up and streamlining, (after a bit of refactoring) things currently looks like the diagram below (Data flow indicated by arrows). and some parts are written to access the the repository services jumping over layers....

Why doesn't spawn link cause the calling process to die?

From the example given here, Erlang and process_flag(trap_exit, true) -module(play). -compile(export_all). start() -> process_flag(trap_exit, true), spawn_link(?MODULE, inverse, [***0***]), loop(). loop() -> receive Msg -> io:format("~p~n", [Msg]) end, loop(). inverse(N) -> 1/N. If I run it as, A = spawn(play, start, []). The spawned process <0.40.0> dies as it is suppose...

WCF service architecture query,architecture,wcfserviceclient
I have an application that consists of a web application, and mutliple windows services, only one windows service is installed depending on what version of the backend sofware is used. Currently, Data is saved by the web app in a database, then the relevant service is installed and this picks...

erlang processes and message passing architecture

The task I have in hand is to read the lines of large file, process them, and return ordered results. My algorithm is: start with master process that will evaluate the workload (written in the first line of the file) spawn worker processes: each worker will read part of the...

Erlang syntax error unclear

I just got started with Erlang. I am trying if statement. I found out one particular behavior which I do not understand. the following statement does work perfectly. some_comp(Arg1) -> if (cal(Arg1)>50000)->'reached'; true -> 'invalid' end. cal(Arg2)-> %% some calculation. However the following shows an error syntax near if: some_comp(Arg1)...

How does ETL (database to database) fit into SOA?

Lets imagine, that our application needs ETL (extract, transform, load) data from relation database to another relation database. Most simple (and most performance, IMHO) way is to make link between databases and write simple stored procedure. In this case we use minimal technologies and components, all features are "out of...

How to asynchronously send data to a client via a different application path?

I am currently researching large scale application optimisation and scaling, and through my research I have gotten to grips with the standard ways of doing large scale with DNS Round Robin for splitting the load across load balancers, using load balancing to divide traffic across web-servers like Nginx, which again...

how to handle low_entropy exception of crypto:strong_rand_bytes(N)?

I want to generate cryptographically strong pseudorandom numbers in erlang for session IDs. There is crypto:strong_rand_bytes(N). What if it throws the low_entropy exception? From strong_rand_bytes(N) -> binary() Types: N = integer() Generates N bytes randomly uniform 0..255, and returns the result in a binary. Uses a cryptographically secure prng...

Value from binding in LFE interpreter using Erlang

I'd like to use Lisp Flavored Erlang as a scripting extension language for an Erlang application. If you want, in a similar way GNU Emacs is configured and extended via Emacs Lisp. I know that the argument is wide and structured; but in the specific case of this question I'd...

erlang os:cmd() command with UTF8 binary

I'm trying to get an Erlang function to execute a bash command containing unicode characters. For example, I want to execute the equivalent of: touch /home/jani/ჟანიweł I put that command in variable D, for example: io:fwrite("~ts", [list_to_binary(D)]). touch /home/jani/ჟანიwełok but after I execute: os:cmd(D) I get file called á??á??á??á??weÅ?. How...

Client-Server architecture: 100% Android (Android as a server) or J2EE+Android?

Context I am considering going into a client-server architecture with Java. The idea is that several Android tablets (let's say around 15) need to display a content from a server. Content can vary times to times (e.g. day display v/s night display). Furthermore, tablets will also display a Yes /...

erlang message passing architecture

In erlang if two processes A and B are sending message to a process C simultaneously. Will there be a race condition? C ! {very large message} sent by A C ! {very large message} sent by B Will C receive the complete message from A and then proceed for...

Mnesia pagination with fragmented table

I have a mnesia table configured as follow: -record(space, {id, t, s, a, l}). mnesia:create_table(space, [ {disc_only_copies, nodes()}, {frag_properties, [ {n_fragments, 400}, {n_disc_copies, 1}]}, {attributes, record_info(fields, space)}]), I have at least 4 million records for test purposes on this table. I have implemented something like this Pagination search in Erlang...

Erlang pass-by-reference nuances

9> A = lists:seq(1,10). [1,2,3,4,5,6,7,8,9,10] 13> Fn = fun (L) -> [0|L] end. #Fun<erl_eval.6.90072148> 14> Fn(A). [0,1,2,3,4,5,6,7,8,9,10] 15> A. [1,2,3,4,5,6,7,8,9,10] If erlang internally passes by reference (see this), why does the value of A not reflect the change? What fundamental am I missing about passing-by-reference or erlang?...

What does BEAM stand for in iex for the Elixir programming language?

I'm sort of curious as to what the B. E. A. and M. stand for. I recall seeing an explanation of the acronym BEAM, but I have not managed to find it again. It comes up in error codes: ➜ gentoo iex Erlang/OTP 17 [erts-6.4.1] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false]...

What's the recommended way to load an object graph from Data Access Layer?

From a relatively old software architecture book: In other situations, the same conceptual Get-Orders query might generate different data—for example, a collection of Order objects plus order items information. What should you do? Should you expose two slightly different GetOrders methods out of your DAL? And what if, depending on...

Fast mutable objects in Erlang

What is the best approach to store and manage high-performance mutable objects in Erlang? Assume that I want to write really simple online game server with realtime gameplay. Somehow I need to represent player's state in Erlang memory. For example, it could be just a simple tuple like {name, "Bob",...

What is the difference between .app and .app.src files in Erlang?

I found these description in the Erlang doc and, but not sure which one is the "Application Resource File". When should I use .app and .app.src? 7.3 Application Resource File The Application Resource File

simple model when requesting collection and extended model when requesting resource - how

I have the following URI: /articles/:id, where article is a resource on web-service and have associated model/class. Now I need to return only partial data for each resource (to save bandwidth and make for speed) when collection is requested, but when a single item is requested from collection I need...

Erlang driver erl_errno issue

I am playing with erl_driver. Start callback of my driver is below: ErlDrvData drv_start(ErlDrvPort port, char* command) { char* file_name = command + sizeof(drv_name); GenTtyData* port_data = (GenTtyData*)driver_alloc(sizeof(GenTtyData)); erl_errno = gt_open(file_name, &port_data->file); if (erl_errno != 0) { // Assertion there is just to show you my intention assert(erl_errno == ENOENT);...

Segregating the read-only and read-write in Spring/J2EE Apps

We using Spring, Spring-Data and JPA in our project. For production servers, we would like to setup database cluster such that all read queries are directed to one server and all write queries are directed to another server. This obviously will require some changes in the way the DAOs are...

Should I create another model for admins? Or what's the best way to do it in Ruby on Rails?

I'm developing an app, which requires two types of users regular user and admin. Regular users have roles in the app, and an admin doesn't have any roles, he just should have an access to an admin panel. What's the best way to implement this task? ...

Difference between switch & bus architecture?

I was just going through a operating systems textbook but I got stuck at switch architecture . Please explain what it is and how it is different from bus architecture ?...

Does the pattern [_|_] in Erlang mean anything specific?

I am learning Erlang from Learn You Some Erlang and I've seen the pattern [_|_] twice already but couldn't find any info on it. This usage seems superfluous because omitting it from (1) and subsituting it with _ in (2) yields the same result without degrading readability. It is my...

C# Code design / Seperate classes for each TabControl

My main problem is that my tool grows and grows and I start loosing the focus on the different parts of my code. The main-Form got a docked tabControl at fullsize. I got 5 different tabs with for really different functions. So I can say my tool is splitted into...

Getting a sibling process in Elixir

I have an Elixir/Erlang process tree: parent (Supervisor) ├── child1 (GenServer) └── child2 (GenServer) child1 (a DB client) has information that child2 needs to use. What's a good way to pass a reference from the Supervisor process to child2 so that child2 will always have a valid reference to child1?...

How to list all the bucket types in riak?

Using Riak Client(erlang), I can list all the buckets of the default bucket type. But how can I list all the bucket types? Or If I can't do this by the client, how can I find out about this?

Type errors in dialyzer in function with specified type

I am receiving an error in dialyzer when it analyzes the following function. -spec do_request(Method, Type, Url, Expect, Headers, Body, Client) -> Response::client_response() when Method :: method(), Type :: content_type(), Url :: url(), Expect :: status_codes(), Headers :: headers(), Body :: body(), Client :: #client{}. do_request(Method, Type, Url, Expect, Headers,...

First word of binary string erlang

I need to do something like this <<"Some text in binary">>, and it should return <<"Some">. How can i do it without split function, only by pattern matching and select/if cases with Erlang. Thank you.

How to model data for in-memory processing

I have a lot of static data (i.e. read only data, which is not transactional) which gets updated only once in few days. I have to support searches on that data (api calls, not sql). So I am thinking I will just load it in Memory, and refresh the in-memory...

Architecture for creating a JavaScript framework

Around one year ago we started a web system that over the time has grown quite a bit. From the beginning the goal was to build reusable code that would speed up the development of future projects, and it has. With every new project, the reusable code from the previous...

Riak mapReduce fails with > 15 records

Problem I've been learning Riak and ran into an issue with mapReduce. My mapReduce functions work fine when there's 15 records, but after that, it throws a stack trace error. I'm new to Riak and Erlang, so I'm unsure whether it's my code or it's Riak. Any advice on how...

Why doesn't JavaScript get its own thread in common browsers?

Not enough that JavaScript isn't multithreaded, apparently JavaScript doesn't even get its own but shares a thread with a load of other stuff. Even in most modern browsers JavaScript is typically in the same queue as painting, updating styles, and handling user actions. Why is that? From my experience an...

Libgdx: Objects creating other objects

I have kind of a general question about a simple game architecture. In my game I have these classes: Main class who's responsible for drawing and rendering. Ball object which has a few attributes and update() function that does certain things. The main class has an array of all exist...

MVC Web application architectural concern

I have inherited an MVC application designed along the above project structure. the application uses the Unity framework for dependency injection, and the user interactions go upstream to the database in the following order View -> Controller -> ViewModels -> Repository Services -> ORM -> Database The infrastructure components...

Where to format collections / objects

From front end architectural point of view, what is the most common way to store scripts that perform transformations on collections of objects/models? In what folder would you store it, and what would you name the file / function? Currently I have models, views, controllers, repositories, presenters, components and services....

ctrl+G in erl doesn't work

I'm trying to interconnect erlang nodes, but entering ctrl+G doesn not work: Eshell V6.4.1 (abort with ^G) 1> ^G Eshell V6.4.1 (abort with ^G) 1> ^G Eshell V6.4.1 (abort with ^G) 1> ^G Eshell V6.4.1 (abort with ^G) any idea why this can happen? I was thinking about locale settings,...

Subtypes of an interface only compatible with a subtype of another interface

I've been scratching my head for a long time on a problem and I still don't know what would be the best solution. Since the application domain is very technical, I'll illustrate my problem with a simple example. Suppose I have the following interfaces: public interface Animal { public void...

Some doubts related this Swing MVC implementation. Opening a database connection should be a Controller task?

I am following a Java tutorial related to the implementation of the observer pattern (using Swing) and I have some doubts. My doubts are not related to the observer pattern but about the architecture of this tutorial application (that is based on something like an MVC logic) So it contains...

Fetching and updating data in mnesia

I have multiple mnesia tuples like (GroupID is the primary key) {GroupID, GroupName, GroupType, GroupDescription, GroupTag, CreatorID, AdminID, MemberList, Counter}. MemberList = "[email protected],[email protected],[email protected]". GroupName = "Any String Value". % e.g.: "basketball" GroupTag = "comma separated values". % e.g.: "Sports,Cricket,Fifa,Ronaldo" I will pass a character or word to a function. This...