What serverless means for DevOps engineering

When you ask people in industry what serverless means, you’ll probably receive one of these definitions as an answer:

I’ve already defined what DevOps engineering is but serverless seems to need clarification. I feel there is a lot of confusion around serverless and the confusion is led by cloud providers making serverless sound like something magical, but the way they use the word is simply a buzzword that means nothing

Here is a more meaningful definition of serverless: the word “serverless” is used to describe the idea that the servers processing the application logic are not always present

What serverless means for DevOps engineering

Implementing serverless

The first definition of serverless I described above is a more accurate definition of serverless and there are multiple strategies of implementing a dynamically provisioned code execution environment. These strategies are internal serverless and external serverless. Both of these strategies ship with positive attributes

Look at this from the point of view of a non-cloud provider (i.e.: you are not Google or Amazon)

Internal serverless

Internal serverless may be appealing to you if you already have your own servers (on-premises, at a colocation data centre, under the desk near the secretary etc) and you would like to use the resources more effectively

External serverless

External serverless is serverless in the cloud. If the company decides to go with external serverless, this decision may result in a paradigm shift for DevOps engineering

The paradigm shift

External serverless does not deprecate DevOps engineering in any way. It simply shifts where some of the DevOps engineering (including hiring) occurs. Cloud providers have a big requirement for hiring DevOps engineers and cultivating the DevOps engineering principals

This also does not mean that DevOps engineering in its entirety is subjected to this paradigm shift. Large enterprises and well as SME’s can still very much have a requirement for DevOps engineering, particularly considering everything DevOps engineering is involved in and what DevOps engineering brings to the company

There is still plenty work to do in terms of process efficiency, building productivity tooling, automating routine tasks, designing service architecture, adopting multi or hybrid-cloud–I could keep going but I think you get the point 

A good way to understand serverless

A good way to understand serverless is to think in terms of abstraction. Serverless is just another layer in the abstraction model. Both of the following abstraction models involve servers and people/software managing them and like I explained above, it just depends where the people and software that manage the servers are placed

Internal serverless

Everyone should be aware of the appeal of operating system virtualization and the reasons why you’d want to do this but just keep in this concept in mind and and place app virtualization on top of this

Physical hardware —> OS virtualization —> App virtualization

External serverless

You’d abstract physical hardware management to Infrastructure-as-a-Service (IaaS) offered by a cloud provider. IaaS will typically still require operating system management so the next layer up is serverless

Physical hardware and OS management —> IaaS OS management —-> Serverless

The appeal with external serverless, from the point of view of a company that wants to shift this responsibility away, is very much “no operations” and this is a really good idea all round

Addressing the contradictions

How many engineers work on serverless at AWS or how many servers power serverless in GCP?

How many engineers built the software that manages the serverless servers in AWS and GCP?

These are just two of the questions you should ask people that feel serverless deprecates DevOps engineering or if they feel code genuinely runs on water vapour. You aren’t looking for a exact numerical answer but rather observe the thought process in the person's answer

Some people in industry really believe that there are no servers involved with serverless or they understand that servers do exist in serverless but somehow they are managed themselves

These beliefs are completely false. Servers do power serverless and applying the DevOps engineering principals resulted in tooling and automation that manages the serverless servers (you are playing a semantic game if you feel another type of software developer built the DevOps engineering tooling/automation, we are all software developers)

External serverless considerations

The appeal of external serverless is really good but there are some considerations that you should be aware of

Long running workloads

It stands to reason that serverless is not designed for long running processing. Technically it can, but just be careful of a surprise bill. I think it’s billed in milliseconds for a reason

I have been running a WebSocket server (persistent, long running HTTP connections that send the WS upgrade headers) in Google Cloud Run and it has worked out but I do set a maximum execution time though

Cold starts

Reserving a minimum amount of instances will—at least in my case this has been true, dramatically increase the bill so scaling to zero is the only real option

When the app has scaled to zero and an incoming request or event has been triggered, the execution environment will need to be initially prepared and this can slow down the processing

If you don’t cater for this in app logic (it’s as easy as simply trying again), you may have some failed requests/events

There are some things you can do that can improve cold starts:

If your app uses a distributed application architecture, the cost involved with reserving a minimum amount of instances will increase even more (it makes no sense to still cold start of some of the serverless instances if they have a dependency relationship)

Convoluted billing structure

If anyone ever says that it’s possible to accurately predict what the bill will be when using serverless— or the cloud for that matter, is lying

You are billed per millisecond execution time over how much memory is used during this time and network ingress/egress is billed per byte that varies to specific regions in the world. The network billing alone will make the predictions near impossible because how do you know how many users will sign up and where will they mostly use the app from?

It’s best to rather double what your best guess is if you want to play it safe in terms of cost

Persistent storage

No more quick-wins in terms of persistent storage (you can’t use the local file system like you have always been) so you’ll have to put some thought into how you’ll use an external object store. There are many object stores to choose from: AWS has S3, GCP has GCS, MinIO standardised the app integration lawyer while plugging into S3 and GCS behind the scenes

This is also dependent on the type of serverless solution you opt for because Kubernetes (GKE Autopilot, DOKS, LKE can manage the worker nodes for you) does offer persistent storage so it is possible to get locally mounted block storage on the pods

GKE and GCP filestore work well together but the pricing is just off-the-charts

Where DevOps engineering gets chopped by external serverless

If you think DevOps engineering is operational support

If you ask five different people what DevOps engineering means, you’ll probably receive five different answers. Some feel that DevOps engineering is some type of improved operational support. If you feel this is correct, DevOps engineering will completely get chopped by serverless and this is perfectly fine

You feelings and ideas are wrong and will be chopped by serverless, rightfully so

Possibly configuration management if your serverless solution involves containerisation

I used to use Puppet and a few modules I found on Puppet Forge to prepare servers both manually and in the autoscaling fleets but I’ve found myself not using Puppet or any other configuration management ever since I discovered containerisation

I would find myself pulling a pre-built container image off of a container registry, like DockerHub, and I didn't really care how it was initially prepared. I have found myself making changes to things like the entrypoint scripts but I would simply docker commit the changes (there is no need to repeatedly do this). I know there is a concept of “trusted containers” but I don’t feel I’d need to use configuration management to prepare them

So, configuration management (Puppet, SaltStack, Ansible etc) could potentially get chopped if the external serverless solution you opt for is containerised. The same logic can actually apply to internal serverless as well

If there is a need to repeatedly prepare base container images, then I’m wrong. I would not want to repeatedly perform the same task over-and-over and considering repeat work is a violation of the DevOps engineering principles I’ve laid out before

If you are at small scale

I’ve worked for a few companies that hardly operate at anything I’d call scale. They don’t like to admit to this because it seems everyone likes to feel they are going to change the world or have a deep impact on society

It’s very obvious that if your entire codebase runs on a single serverless instance with minimal resources and with everything basically left default then you are hardly at scale

In this case, hiring internal DevOps engineers is not a good idea for you or your company. This works both ways: it will cost you money and the DevOps engineers will be bored

Serverless containers

I really felt the benefits of serverless when I discovered serverless containers (I use Docker)

I was able to change very little in my development workflow in order to implement serverless containers (the exact container that was running locally is the exact container that is running in production)

There was a slight change to actually get containerised and some learning involved in how application virtualisation generally works but it was well worth the effort

The build scripts and configuration all travel with the apps repository so everything I need is all in one place. It also makes things like getting a development environment and an application stack (in my case: LAMP) up and online really quickly (simply pull the already prepared container images, mount the directories containing the source code and perform the port mapping)

Achieving serverless without serverless

There are other ways of achieving a similar feel to external serverless without actually drinking the kool aid 

I’ve configured traditional autoscaling using OS virtualisation that pretty much achieves the same feel as serverless (it’s pretty hands off in terms of manual operations and works well but it does require initial planning and configuration)

The setup was configured as following:

More can be said about this later