The Case Against Sidecars - Revisited: Part 2 - The Case for Libraries

· Series: sidecars

In part 1 I talked about why I don’t like sidecars generally. In this part 2, I want to look at why we don’t use good ole libraries instead.

I Wish Libraries Were Easier to Make and More In Vogue

Perhaps remind yourself: Why do we have sidecars in the first place again?

There are usually a couple of answers:

  1. Separation of concerns: keep the app small and externalize functionality to the sidecar
  2. Independent deployment lifecycle: upgrade sidecars independently of your main app
  3. Independent resource/request limits: Clamp your sidecar to a given amount of RAM/CPU

Let’s look at each of these reasons more closely.

1. Separation of Concerns: Are Sidecars The Solution?

Sidecars today exist for the same reason we have Microservice madness. We think that the right way to separate out concerns is at the process level.

Sidecars take this to an extreme, where the concern is separated out into an independent binary, so independent that it gets its own filesystem (container)!

This means we need to include a whole new class of complexity to make this sidecar work: We need an RPC protocol, serialization, and client libraries. Now your function calls are (localhost) network calls (like microservices)!

What if I told you there was a way for one binary to talk to another binary, in a polyglot way, but without any of that? It’s called: A Shared Object Library (.so).

Now we have two new pain points:

  1. Writing libraries sucks (Usually in C/C++)
  2. Consuming libraries sucks (If you are not using C/C++)

Why Does Writing (Shared Object) Libraries Suck?

Historically, all shared libraries had to be written in C or C++. This “makes sense” because a shared library has to be a piece of compiled binary code, and it needs to export symbols, and use the C ABI for calling. (I sure wish there was some other standard ABI)

What if I told you that you can write shared libraries in Golang! Of course there are limitations, but super interesting alternative universe if this became very popular.

And of course you can write in rust.

And other languages like Fortran, D, Swift, and Ada can also make shared libraries. Interpreted languages (Python, Ruby, Javascript, etc) and managed runtime languages (Java, C#) cannot of course.

And yes, you only get functions with C type inputs. There is no way to export structs or consts in a generic way.

And yes, memory allocation sucks.

There also is just no … LibraryHub™ where you can upload it?

Why Does Consuming Libraries Suck?

Other than traditional C/C++ & linking, using a shared library from any other language is a little bit of a pain.

Most languages have some Foreign Function Interface (FFI) that lets you call into C libraries. The ergonomics are usually somewhere between “acceptable” and “I’d rather rewrite this in my language.”

You need to:

  • Write bindings/wrappers that map C types to your language’s types
  • Deal with memory management across the boundary (who owns what?)
  • Handle the mismatch between C’s error handling and your language’s (exceptions, result types, etc)
  • Package and distribute the .so alongside your application code somehow

Go has cgo, Rust has good FFI support, Python has ctypes, Java has JNI, and so on.

None of this is that hard, but in total, the friction is too high to make it worth it.

Just functions are not enough.

But wouldn’t it be cool if, as an industry, we made FFI super great? Maybe if we did, all those sidecars and localhost curls would be just… good ole function calls?

There also is just no … LibraryHub™ where you can run libraryhub get awesomedns?

Seriously, someone needs to rebrand libraries and turn it into a product, just like Docker did with linux containers.

Polyglot Objections

Sidecars give the illusion of solving the polyglot problem.

Let’s say you have a great java library, and you want to share that functionality with other languages. Your first instinct might be to build a sidecar out of it.

Whatever the API is that you have for that library, now you must expose that over some network protocol.

Now, you still have to solve the polyglot problem, but only for that specific interface.

Generated client libraries from an IDL (protobuf) help here, and are kinda the only saving grace.

Sidenote: WebAssembly Component Model

The WebAssembly Component Model is like libraries for WASM, but with their OWN ABI.

This is kinda exciting, that we have a new ABI in the mix.

I just wish it wasn’t fundamentally JavaScript.

2: Independent Lifecycle: Is It A Good Idea?

It is true that, in theory, having sidecar containers allows you to update the sidecars without redeploying the app.

As mentioned in part 1, you need external tooling to actually update deployed sidecar containers in the wild.

In practice, this is kinda dangerous.

I think that eventually you will hit a point where you want a sidecar upgrade to be part of an apps deployment. This gives you better rollback/rollforward capabilities. It makes the change visible in CI/CD stuff, etc.

Call this a kind of “shift left” of the sidecar update mechanism, contrast to the “behind your back” style of upgrade. Again, for some sidecars.

But guess what, if you have shifted left this style of deployment, you know what you could also do? Upgrade a library!

In summary, if you already are going to build up tooling to do “shift left” style upgrades, that same tooling could also just upgrade a library that does the same functionality, and skip the whole sidecar complexity! (if we, as an industry, were good at libraries)

3: Independent Resource Requests/Limits

This is a legitimate pro for sidecar containers.

There is just no way (that I’m aware of) to control the ram/cpu of a library, function, or thread.

However, I think specifying per-container resource requests/limits is somewhat cumbersome in practice. I’m glad Kubernetes now offers pod-level resource settings, which simplifies this for many common cases.

That said, the ability to isolate and restart a misbehaving component (like a memory-leaking DNS resolver) without restarting your main application is operationally valuable.

This is probably the strongest argument for sidecars when resource isolation is a real concern.

Conclusion

Sidecars aren’t going away, and they shouldn’t. The k8s pod model has won, and sidecars are the “standard” way of composing code now. Especially code from different repos, languages, teams, or organizations.

The promise of “separation of concerns” and “independent deployments” led us to add a network hop, serialization overhead, and operational complexity where a simple library could have sufficed.

What I’d love to see is a world where:

  • FFI tooling is good enough that calling a Rust library from Python feels natural
  • Teams default to libraries first, and reach for sidecars only when they truly need process isolation
  • We have better stories for versioning and updating shared libraries in production1
  • We have LibraryHub™2, some low-friction way to publish and consume libraries, regardless of the language

Until then, we’ll keep paying the “localhost tax”.

If you’re building infrastructure tooling today, consider: could this be a library instead of a sidecar?

Addendum: Realistic Sidecar Alternatives

Envoy: Did you know that you can configure your gPRC clients to read xDS configuration directly?. This gets you… some of the benefits of a service mesh (centralized configuration). It is client-side only though. Still, wouldn’t it be cool if we could go back to 2010 when it was cool to have your service mesh as a library?

Envoy Again: What if you don’t want to configure gRPC, but what you really want is to embed envoy itself into your application, as a library? Well, it’s called Envoy Mobile. Unfortunately it is Android/iOS specific.

DNS: I’m kinda surprised that in-process DNS stub resolver libraries are not more popular. At least with rust there is this crate. Most DNS libraries just call the OS (resolv.conf) resolvers over and over, and don’t cache themselves. Then again, WTF Java, with a cache forever policy?

Observability: Of course, you don’t have to use the Otel Collector for metrics, logs, and traces. You can always use OpenTelemetry as a library (send telemetry data directly to backends). But it does seem unnatural to do that.

Netflix: Netflix used to be the king of (Java) libraries and not sidecars. Ribbon IPC, Hystrix (circuit breakers), Eureka (service discovery), Archaius (config), and so on. That too is changing.

Redis/memcached: It isn’t uncommon to see a redis or memcached sidecar running in a pod. The thing is, this cache comes up and down with the your app. Why do we need a separate process? If you only need a local cache, you could use something like caffeine (JVM) or SugarDB (golang). Or any number of the language-native libraries for that.

Cron: Have some light scheduled tasks to run in your container? Instead of running a sidecar with some complexity to share files and stuff, just pick a cron library? cron4j can even read crontab files directly.

More? I’m just going through this list and asking the question: why not a library instead?

  1. Library containers? Did you know that, in k8s, you can mount an image as part of your container’s filesystem? That is right, one can publish a .so in a SCRATCH container image, and have it mounted at /lib/foo/. You would still get the ability to bump that library independently of the main app, because it is different image. But it is not a sidecar, there is still just 1 container at play. ↩︎

  2. Given the above ^, maybe a good old container registry would work for this? It kinda has everything we need already, we just need to think of the container image as the library, and have a solid story for ensuring that it only has a single layer, and the conventions for how to mount it correctly are understood. ↩︎

This is a post in the sidecars series.
Other posts in this series:

Comment via email