Ever heard about the Correlation ID header ? Or even think about a way to aggregate your logs amongst multiple systems ? Here is a right place to get some explanation on what is the Correlation ID and what is it used for.
Correlation ID, what is it for and why ?
In the past, we were used to think of consuming a process as a single transaction as request / process / answer. With the advent of micro services, other internal services, third parties, etc, it becomes more difficult to follow the journey of a call to an API and even the associated logs.
The Correlation ID header is the solution to both these problems. It consists of adding a header to your request, and propagate it in other dependencies.
How it works ?
If you ever header about an HTTP header, the correlation ID is … a custom HTTP header.
When building your HTTP request, you have to add a custom HTTP header, where the key is X-Correlation-ID, and as the value, a new and unique GUID identifier.
|
1 2 3 4 5 6 7 8 9 10 |
GET /my/api/endpoint Host: myHost User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0 Accept: text/json Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate, br, zstd X-Correlation:ID: cdc83a05-037b-425a-be21-42ffa0e526ae ... |
X-Correlation-ID vs X-Request-ID
You may have heard about the request ID too and you’re asking yourself what’s the difference. Well, it’s really simple :
The request ID is used to identify a single request with a system, in other word, this is the ID of the request inside a service, and only inside it.
While the correlation ID, as already said, has to be propagated from service to service, so, in other words, it is the ID of the whole process across multiple systems.
A C# example
Let’s see how to implement that in C# using HttpClient.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
public async Task<string> CallApiWithCorrelationId(string apiUrl, string correlationId) { // Create the HttpRequestMessage var request = new HttpRequestMessage(HttpMethod.Get, apiUrl); // Add X-Correlation-ID header to the request request.Headers.Add("X-Correlation-ID", correlationId); // Send the request and get the response var response = await _httpClient.SendAsync(request); // Ensure the response is successful before reading response.EnsureSuccessStatusCode(); // Read the response content return await response.Content.ReadAsStringAsync(); } |
In order to avoid hard coded strings, I suggest to create an enum of your custom HTTP headers to use it as a constant string.
And the propagation
In fact, to propagate the correlation ID to other services, you just have to get it from the incoming request and put it in your outgoing request, just don’t forget or even better, put a factory in place to do it automatically each time you create a new HTTP request.
But wen someone call your API and didn’t put a correlation ID the his request, you’d better to create one before entering your own set of processes to have the correlation ID present in your logs.
Here is how to proceed.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
public class CorrelationIdMiddleware { private readonly RequestDelegate _next; public CorrelationIdMiddleware(RequestDelegate next) { _next = next; } public async Task Invoke(HttpContext context) { // Check if X-Correlation-ID header exists in incoming request if (!context.Request.Headers.ContainsKey("X-Correlation-ID")) { // Generate a new correlation ID var correlationId = CorrelationIdGenerator.GenerateCorrelationId(); // Add X-Correlation-ID to the outgoing request context.Request.Headers.Add("X-Correlation-ID", correlationId); } // Call the next middleware in the pipeline await _next(context); } } |
Assuming you’ve implemented a correlation ID generator, this way, a correlation ID will be always added to every incoming request if it has not been added by the caller.
Few words about aggregating logs from multiple services
As the correlation ID will now be propagated from service to service, it becomes obvious that searching for logs into each service, even with the correlation ID, will be painful and not really efficient.
I will not delve into details about data collect and search platforms here, but if you want to Google some, here is a short list of few the mostly used :
The article provides a clear and concise introduction to Correlation IDs, effectively explaining their purpose and importance in tracking requests across distributed systems. The distinction from Request IDs is helpful, and the C# example adds practical value. However, it could be improved with more detailed implementation guidance, visual aids like flowcharts, and a discussion of edge cases like missing IDs. A solid read, but a bit more depth would make it even better!