Elixir for Python Developers (Part 3)


Elixir

Now for something completely different ;)

Basic Elixir Concepts for Distributed Programming

In order to use Elixir for develop distributed applications, we still need to learn some more concepts :(. Don’t worry, we will cover it soon. By the end of this tutorial, we will have a fault-tolerant Factorial Server :D.

Actors/Processes

Actors are entities which, communicate using message passing. They use asynchronous Send/Receive messages. They form the building blocks for concurrency and distribution. All actors have a message box, to which other actors can send send messages. Actors are isolated and hence crashing of one actors doesn’t affect another actor. Each process/actor have a PID(process id), which can be used to communicate with it. Actors/processes runs inside the Erlang VM, which can handle millions of them at a time. Location/Network transparency of actors provides the added benefit of easy distribution when building distributed applications.

Erlang VM Overview

Next we will look into how Erlang VM handles many actors at the same time. As shown in the below diagram, Erlang VM, starts a scheduler thread for each processor core. Scheduling of actors on a core is performed by this thread. The scheduler is pre-emptive providing a soft real time system. Erlang processes are extremely lightweight and should not be confused with OS processes. Garbage collection is performed per actor, resulting in a responsive system unlike many Stop the world garbage collectors.

In order to build systems which can handle failure, it is required to have support for monitoring/linking processes. If we sense a failure, we can then remedy the situation by restarting/killing processes. Links and Monitors provides us with this functionality. Link can be used to link one process to another. When two processes are linked and one of them crashes, the other linked process also crashes. This can be useful for building dependent processes. In most cases what we require is, just a monitoring process, which will receive notifications when monitored processes die. This functionality can be found in Monitors. When notifications arrive, we can make decisions about killing/restarting processes. Monitors and Links form the pillars for building fault tolerant applications.

A simple ping pong server

Lets build a simple Ping Pong in Elixir to demonstrate creation of actors using spawn and sending/receiving of messages using send/receive.

defmodule PingPong do
  import :timer
  @time_interval 1000

  def ping_pong do
    receive do # <- Receive messages 
      {:ping, sender, ping_id} -> # <- Pattern matching :D
        IO.puts("Ping received #{ping_id}") # <- Notice the string interpolation
        :timer.sleep(@timer_interval)
        send(sender, {:pong, self(), ping_id + 1})

      {:pong, sender, pong_id} ->
        IO.puts("Pong received #{pong_id}")
        :timer.sleep(1000)
        send(sender, {:ping, self(), pong_id + 1})
    after # <- Timeout functionality
      @timer_interval -> "No messages received"
    end

    ping_pong() # <- Recursive call
  end
end
iex(1)> one = spawn(PingPong, :ping_pong, []) # <- Create actor
iex(2)> two = spawn(PingPong, :ping_pong, [])
iex(3)> send(one, {:ping, two, 0}) # <- Start by sending message
Ping received 0
Pong received 1

As you can see we defined a ping_pong function which runs a receive loop which receives messages and pattern matches on them. Messages are then send using send. We also sleep using :timer.sleep from the timer library in Erlang.

Since we are familiar with the basics, lets talk about some high level patterns/libraries shipped with Elixir/Erlang for building distributed applications. It is named as OTP(Open Telecom Platform).

OTP Concepts

Supervisors

Supervisor are processes which perform monitoring. They are built on top of monitors. They can supervise actors/processes as well as other supervisors. Supervisors enable us to automatically kill/restart actors based on defined rules. We can even form a tree of supervisors called supervision tree.

GenServer (Generic Server)

GenServers provide the functionality to hold state and expose supported operations/messages. They are an abstraction for generic servers and come with a lot of goodies. Lets look at an example combining GenServer and Supervisor.

Lets make a simple Factorial Server, which returns the factorial and also keeps track of the number of factorials computed.

defmodule Factorial do
  def factorial(0), do: 1
  def factorial(n), do: n * factorial(n - 1)
end

Lets make this Factorial module available as a server, where you can send numbers and get factorials back. A Genserver provides a few functions/callbacks which will be called automatically. Below we use init callback, which will be executed when a GenServer is spawned and output of this function becomes the state of the Genserver. GenServer.start_link is used to start the GenServer linked with the current process. GenServer also provides handle_call and handle_cast for handling synchronous and asynchronous messages respectively. The below code is commented to show the relevant parts.

defmodule CoolerFactorialServer do
  use GenServer

  def init(_) do
    {:ok, %{count: 0}} # <- Initial Server State. Count of factorials computed.
  end

  # Starts a GenServer process running this module
  def start_link(worker_name) do
    GenServer.start_link(__MODULE__, [], name: String.to_atom("#{worker_name}"))
  end

  def handle_call({:factorial, number}, from, state) do # <- Handle :factorial msgs
    {:reply, {Factorial.factorial(number), state[:count]}, # <- Reply with factorial
      %{state | count: state[:count] + 1}} # <- and updates genserver state.
  end

  # Helper function for use by clients
  def get_factorial(pid, number) do
    GenServer.call(pid, {:factorial, number})
  end
end

Now that we have barely working GenServer, lets make it fault tolerant by adding a Supervisor.

Supervising Factorial Server

defmodule FactorialSupervisor do
  use Supervisor # <- Brings in Supervisor functionality

  # Start a supervisor with current module
  def start_link(opts) do
    Supervisor.start_link(__MODULE__, :ok, opts)
  end

  # Initialization function, which generates list of children with names
  def init(:ok) do
    children =
      Enum.map(1..3, fn n ->
        Supervisor.child_spec(
          {CoolerFactorialServer, n},
          id: String.to_atom("#{n}")
        )
      end)

    # Starts supervisor, with one_for_one strategy. 
    # This strategy restarts processes which crashed.
    Supervisor.init(children, strategy: :one_for_one)
  end
end

Now lets test our Factorial Server.

Working of Supervisor

iex(1)> {:ok, pid} = FactorialSupervisor.start_link([])
{:ok, #PID<0.112.0>}

iex(2)> Supervisor.which_children(pid)
[
  {:"3", #PID<0.115.0>, :worker, [CoolerFactorialServer]},
  {:"2", #PID<0.114.0>, :worker, [CoolerFactorialServer]},
  {:"1", #PID<0.113.0>, :worker, [CoolerFactorialServer]}
]

iex(3)> GenServer.call(:"1", {:factorial, 5})
120

iex(4)> CoolerFactorialServer.get_factorial(:"1", "hello") # <- This will crash as we sent a string instead of an integer
  Process.whereis(:"1"), "hello") # <- Call factorial with invalid data
** (exit) exited in: GenServer.call(#PID<0.113.0>, {:factorial, "hello"}, 5000)
    ** (EXIT) an exception was raised:
      ** (ArithmeticError) bad argument in arithmetic expression # <- Process crashed
          (f) lib/gensuper.ex:2: Factorial.factorial/1 ....

iex(5)> Supervisor.which_children(pid)
[
  {:"3", #PID<0.115.0>, :worker, [CoolerFactorialServer]},
  {:"2", #PID<0.114.0>, :worker, [CoolerFactorialServer]},
  {:"1", #PID<0.125.0>, :worker, [CoolerFactorialServer]} # <- Restarted with new PID
]
iex(6)> CoolerFactorialServer.get_factorial(:"1", 5)
{120, 0}

iex(7)> CoolerFactorialServer.get_factorial(:"1", 5)
{120, 1} # <- Counter incremented

Now that we got it working, lets take a look at using multiple hosts for running Elixir.

Distributed Elixir

Lets start two nodes and send messages from one to other. We name the first node foo and second bar.

# Start the first node
iex --sname foo

Now lets try some Elixir code.

iex(foo@jvc-rider)1>  # <- You will have a different hostname here
defmodule H do
  def say, do: IO.puts "Hello World"
end
# Now start other node
iex --sname bar
# Lets spawn a process on the other node
iex(bar@jvc-rider)1> Node.spawn_link :"foo@jvc-rider", fn -> H.say end
Hello World
#PID<10257.102.0>

As you can see, we executed the say function in another node and got the console output back. It is because group leader, which handles console output redirects output to node which send the request.

Elixir provides a better option to do remote procedure calls by making use Erlang rpc module. Elixir enables seamless use of Erlang libraries. Notice the : before rpc.call, that is how we access Erlang libraries.

 iex(bar@jvc-rider)2> :rpc.call(:"foo@jvc-rider", H, :say, [])
 Hello World
 :ok

There are a lot more features that Elixir offers. If you are interested, I highly recommend checking out the official docs.

Other goodies

  • A standard code formatter just like golang (other need to install them like flake8 or autopep8)
  • doctest, where you can write code in documentation and get it executed as test. It is similar to doctest in python.

  • Better error checking than Python (even though both are dynamically typed)

    a = "hello"
b = a + 5
#Gives out warning -> this expression will fail with ArithmeticError

def fact(n), do: n*fact(n-1)
def fact(0), do: 1

#This clause cannot match because a previous clause at line 2 always matches

def printName(name), do: name

#warning: variable "nam" does not exist and is being expanded to "nam()", please use parentheses to remove the ambiguity or change the variable name.
# warning: variable "name" is unused

There also exists option to return non-zero return value when there are warnings. This can to a good extent prevent some errors.

Where to go from here

Here are a few resources worth checking out. * Official Docs * Elixir School * Elixir in Action