20 years of open source Erlang: OpenErlang Interview with Miriam Pena

7 min readNov 6, 2018

Originally posted 2018–11–06 by Erlang Solutions

We’ve been holding a year-long celebration in the honour of Erlang’s 20th anniversary as an open source language. This week, we’re excited to introduce Miriam Pena as our next #OpenErlang enthusiast.

Previously, we looked into WhatsApp with one of their talented Server Engineer’s Anton Lavrik. WhatsApp has used Erlang since its humble beginnings back in 2009, but what other giants use the language? Miriam Pena is one equally talented Staff Engineer at AdRoll, a company that uses both Erlang and Elixir!

Adroll uses both languages “to respond to each bid request in less than 80 milliseconds” and Miriam wouldn’t have it any other way. Here, she discusses when she saw the real potential of Erlang and how its concurrency methods are used to full advantage at AdRoll.

We have the transcript listed at the bottom of this blog post.

About Miriam

Miriam has been an engineer at AdRoll for nearly 5 years. During that time, she has worked with Erlang, Elixir, AWS, Python, Dynamodb, Kinesys and Memcached. Currently, AdRoll’s real-time bidding system handles a peak volume of 1.5 Million req every SECOND. Previously, Miriam has worked as an Erlang specialist software engineer, including with us at Erlang Solutions back in 2008–2011!

With over a decade of experience working with scalable systems, at AdRoll, Miriam designs critical parts of their Real-Time Bidding and Ad Server infrastructure, organizes the Erlang and Elixir meetup in San Francisco and has previously been voted one of the “women to watch” by Women 2.0.

Miriam loves working with distributed, scalable, high performance, high concurrency, and high availability systems. Bonus points if they are written in Erlang and Elixir! She has also open sourced a lightweight Memcached Erlang library called Mero that is used at a massive scale in their RTB infrastructure.

About AdRoll

AdRoll was founded in 2007 with the purpose of advertising online without the need for massive budgets only rewarded to major corporate companies, and the needless complexity that came with this.

It works by retargeting — those ads that follow you around the internet encouraging you to buy whatever it was that your willpower denied you the first time around!

Retargeting is an extremely successful advertising tool, and AdRoll has allowed businesses of nearly any size to benefit from it. The benefit for us? Much more variety! It keeps a particular brand at the front of the consumer’s mind, so when they decide to commit to a purchase, they commit to your business.

To retarget most effectively, you must segment your users so the ad they’re receiving is 100% relevant. You must also consider the type of product. For example, if a user wants to buy flights or hotels, it’s best practice to target near immediately, whilst bigger/luxury purchases require further thought and therefore you shouldn’t bombard the consumer.

Why AdRoll uses Erlang

AdRoll has said that they have generated over $7 billion in revenue for their customers, and as previously mentioned, AdRoll must deal with approximately 1.5 million actions per second and answer them in less than 80 milliseconds.

This huge amount of requests requires a highly reliable system such as Erlang to process actions very quickly and even more efficiently. Erlang is an extremely concurrent system that can handle this real-time bidding with a very minimal possibility of the systems crashing. AdRoll’s two main issues are:

Real-time bidding — there will be multiple businesses battling it out for the same ad space. Real-time bidding, like an online auction for the best spots. AdRoll participates in over 1 million programmatic auctions per second. Erlang also allows buyers to respond just as quickly. Ad targeting — AdRoll has a thorough system in place to funnel users to the correct advertisement on the correct website.

The main benefits of Erlang for AdRoll:

• Erlang/OTP offers AdRoll scalability — robust building blocks can be easily replicated. • Erlang/OTP offers robust system designs that means if something crashes, the system will continue. • Erlang’s syntax and design means there is less coding and therefore a reduced chance of bugs.

AdRoll also has an Erlang MultiLangDaemon interface launched by their Erlang node that means each stream is processed as a shard processing program. Each shard is an independent Erlang process. This has the benefits of Java without writing any Java code.

AdRoll also uses ‘profile data systems’ written in DynamoDB (Amazon’s proprietary NoSQL database service).

You can read more about AdRoll, Erlang and Elixir on AdRoll’s blog. You can visit their careers page too.

Interview Transcript

At work with the boss breathing down your neck? Or don’t want to be one of those playing videos out loud on public transport? Here’s the transcript, although not as exciting as the real thing.

Miriam Pena: I first got in touch with Erlang when writing my thesis in La Coruña University.

The first time I saw its potential in a real professional environment was this time we had to implement a software to monitor a solar-powered station. We have 15 different types of devices, each with its own protocol which was really easy to implement with binary comprehension and pattern matching. Also, the concurrency model of Erlang allows scaling the system to a thousand devices.

At the moment, I work as a Staff Engineer in AdRoll in San Francisco. AdRoll is a marketing company. We use Erlang and Elixir in most of our most complex systems.

We participate in over one million actions per second. We have to respond to each of these requests in less than 50 milliseconds. The amount of volume of data we have to process is really high and Erlang is really great for these. It’s very lightweight. The concurrency model allows us to scale thousands of requests at the same time per machine. When there’s a problem in one of the requests, the whole system doesn’t fall.

Well, these 20 years of Erlang has had many amazing contributions. One of the most important ones is the recent addition of maps in Erlang and doing the pattern matching. The system is so compact, it’s just so great. That one thing that really changed my life was when they were able to add line numbers to the error reports. It really only took them 2 hours to get that done.

[00:01:42] [END OF AUDIO]

OpenErlang; 20 Years of Open Sourced Erlang

Erlang was originally built for Ericsson and Ericsson only, as a proprietary language, to improve telephony applications. It can also be referred to as “Erlang/OTP” and was designed to be a fault-tolerant, distributed, real-time system that offered pattern matching and functional programming in one handy package.

Robert Virding, Joe Armstrong and Mike Williams were using this programming language at Ericsson for approximately 12 years before it went open source to the public in 1998. Since then, it has been responsible for a huge number of businesses big and small, offering massively reliable systems and ease of use.

OpenErlang Interview Series

As mentioned, this isn’t the first in the #OpenErlang Interview series. We have three more existing videos to enjoy.

Robert Virding and Joe Armstrong

It only seems fitting to have launched with the creators of Erlang; Robert Virding and Joe Armstrong (minus Mike Williams). Robert and Joetalk about their journey with Erlang including the early days at Ericsson and how the Erlang community has developed.

Christopher Price

Last week was the launch of our second #OpenErlang Interview from Ericsson’s Chris Price. Currently the President of Ericsson’s Software Technology, Chris has been championing open source technologies for a number of years.

Chris chats to us about how Erlang has evolved, 5G standardization technology, and his predictions for the future.

Jane Walerud

Jane is a serial entrepreneur of the tech persuasion. She was instrumental in promoting and open sourcing Erlang back in the 90s. Since then, she has continued her entrepreneurial activities, helping launch countless startups within the technology sector from 1999 to present day. Her work has spanned across many influential companies who use the language including Klarna, Tobil Technology, Teclo Networks and Bluetail, which she founded herself.

Other roles have included Member of the Board at Racefox, Creades AB and Royal Swedish Academy of Engineering Sciences, and a key role in the Swedish Government Innovation Council.

Simon Phipps

Having become an open source programming language, Erlang was allowed to flourish. It gained a passionate following which has since developed into a close community. Simon Phipps dedicates his time to open source promoting languages such as Erlang through the Open Source Initiative and other similar schemes.

Why are open source languages such as Erlang so important? Find out more!

Anton Lavrik

Anton is a Server Engineer at one of the biggest mobile applications in the world. Yep, we’re talking about WhatsApp, which runs on Erlang! WhatsApp is capable of sending billions of messages every single day, and Erlang’s stability and concurrency is a significant reason why it’s perfect for the amount of traffic WhatsApp received. Discover WhatsApp’s Erlang journey with Anton.

The #OpenErlang Parties

Catch up with our Stockholm party held in May 2018!