Service-to-service authentication & authorisation patterns
Problem
The modern backend software architecture is a complex mix of applications and services.
Each application or service communicates with each other to fulfill the business needs. It is not a trivial task to secure these communications and identify who is communicating to who and which resources those applications can access.
An increasing number of cyber security breaches in well-known organisations put extra pressure on architects, system designers, and software engineers to take a good amount of design time to solve this problem. And place security controls in every layer (security in depth).
The variety of applications and resources in the systems increases the complexity of this problem. There are many flavours of communication patterns. Communicating with another service, a system with another system, and many others.
This article is an effort to explore the service-to-service authentication and authorisation problem space and common approaches to solving it.
Before that, let’s skim through the other flavours of this problem since it helps to understand the uniqueness of service-to-service authentication and authorisation.
User to service
Typical user-to-service authentication and authorisation is solved by abstracting the services at a layer like an API gateway backed by an identity management service.
The identity management service answer the “who are you ?” question; if answered correctly, the API gateway takes care of the “what you can access” part via an authorisation delegation framework like OAuth 2.0.
System to system
Here we consider systems that are required to integrate with several other systems for their operation and needed to secure communication in both directions between a client and server. There is no end-user involvement in this communication.
For example, API Gateway integrates with the identity management system. It is communication from one system user to another system. At the same time, the identity management system also wants to communicate to the API gateway for its operations.
A micro-service accessing a topic in a Kafka cluster is another example. The Kafka consumer or producer residing in the micro-services must communicate with the Kafka broker. At the same time, the broker needs to communicate with these consumers and producers.
These are usually solved with an mTLS certificate.
Since the above two are out of this article’s scope, We will not discuss those flavours further. But understanding these two integration patterns helps us better understand service-to-service authentication and authorisation.
Service to service
Now we have enough background knowledge to identify the uniqueness of the problem space we are exploring.
Requirements
Requirements of service-to-service authentication and authorisation.
1. Must be able to authenticate the resource access requests.
2. Must be able to enforce authorisations levels for resource access requests.
3. Good to have mechanism to transfer the identity information of the end-user if required.
Let’s try to apply the well-known existing solutions and compare the pros and cons of that.
Basic auth
Using basic auth is an easy and a bit of a naive approach.
Pros
- Easy to understand and develop, as it requires an “Authorisation” HTTP header with a “Basic” schema.
Cons
- It doesn’t address any authorisation aspects, “service A” can access all endpoints in “service B.”
- The service owner has to rotate the passwords regularly and make sure those are communicated and synchronised with other service owners. Otherwise, end up with a nasty production issue with logs 403 forbidden status code.
- Create a substantial operational overhead if the team owns multiple services. (I have seen some teams struggle with these cumbersome processes daily, the pain grows proportional to the scale of the system.)
mTLS certificates
mTLS is a well-known approach for business-to-business authentication and authorisation. In this case, the client must present X.509 certificates to verify their identity.
pros
- A well-understood approach, hence easy develop and deploy.
cons
- It requires some other mechanism to implement authorisation.
- Higher in complexity than basic auth requires to load Key-store and Trust-store and manage the certificates in a secure store like AWS secret manager or a vault.
- It requires regular rotation of the certificates.
OAuth 2.0
OAuth 2.0 is the de-facto standard framework for user-to-system, system-to-system and authorisation.
Can we use OAuth 2.0 to solve the service-to-service authentication and authorisation ?
OAuth 2.0 is designed to delegate the authorisation to use a resource on behalf of a user. The target service can interrogate the authorisation server about the validity of the delegated authorisation. However, there is no way for the service to know the identity of the token owner.
However, if you want to enforce some form of authentication in the session, you can use client_id and client_secret authentication details to identify the client. In this case, it would be the relevant service (service A, service B).
This solution does not offer such details if your application requires end-user identification. The access_token does not give any hints about who obtains the token.
The above diagram shows how this solution can be achieved by using known micro-service architecture pattern call side car. You can read more about it from reference link [1]
pros
- Solve authorisation aspects of the problem.
- Capable of enforcing the access level using OAuth 2.0 scopes.
- Easy to bake into infrastructure as code.
cons
- When the number of client services grows, the number of OAuth clients grows proportional to that.
- Require careful scope management with the increasing number of operations.
- OAuth 2.0 scopes may not be flexible enough to meet the fine-grained access requirements by teams.
- There is no information of the end-user, in case the integrated service require end-user information.
JWS (JSON Web Signature)
JWS is a lightweight yet powerful approach. Usually known as JWT, pronounced as “jot”, it is a JSON web token. A plain JWT has little use in security as it is compact serialised JSON content.
JWS, on the other hand, has a signature which is usually used to verify the identity of the signing party (authentication)
JWS can be enhanced to have an authorisation context with custom claim attributes like “scope” and validate that scope via a service-specific configuration.
pros
- It can be improved provide authorisation with the help of a rule engine or policy enforcement service which validates the JWS claims set.
- Ubiquitous technology, JWS generation, and verification are supported by most languages and are easy to implement.
cons
- Each application requires a Key store and Trust Store.
- When the number of applications grows, maintaining and rotating these large numbers of keys, key stores, and trust stores may become an operational overhead.
JWS with an Open policy agent
This approach tries to mitigate the shortcomings of the previous solutions by combining the JWS with Open Policy Agent.
The Open Policy Agent is a cloud-native general-purpose policy engine written in Go. The policies are defined in a declarative language called “Rego”
pros
- One of the critical pain points of the pure JWS-based approach was maintaining the Key store and Trust Store. This solution eliminates it by centralising the JWS provider.
- And key rotation and related maintenance tasks are also excluded from the service team’s backlog.
- Provide comprehensive authentication and authorisation capabilities.
- Higher level of flexibility when crafting the authorisation scopes via claims map.
- The claims map can contain identity or reference to the identity of the end-user.
cons
- Additional cost and effort of maintaining Open Policy Agent
- Require a build utility solution to manage the consumer policies so that the team who owns this can delegate policy management to service owners.
Conclusion
In this article, we have explored various solution patterns to solve service-service authentication and authorisation. In some solutions, we throw well-known mechanisms like basic-auth and OAuth at the problem. Some came short of fulfilling the list of requirements we specified.
Iteratively filling those gaps and introducing new tools like Open Policy Agent allows us to address those gaps and arrive at a good enough solution to tick all the boxes.
References
[1 ] — https://learn.microsoft.com/en-us/azure/architecture/patterns/sidecar
