Filter Application Insights events in ASP.NET Core

Application Insights ingests lots of data: requests, traces, events, metrics, etc. If your web site has lots of users, the amount of data can be huge. This mean you pay for this data. While Application Insights is cheap, you may want to reduce the bill. One way is to sample the data. In short, you send only xx% of the events. This is the simplest solution. The other solution is to filter low value events for your usage. For instance, if you use Application Insights to diagnose errors, you may want to filter success requests.

The Application Insights SDK provides an interface to manipulate the telemetry events: ITelemetryProcessor. This interface allows to change the events, for instance to add a property or remove sensible data. If also allow to filter the event. Telemetry processors are chained. Each telemetry processor handle the event and pass it to the next processor. A telemetry processor can also choose to discard the event by not sending it to the next processor. We'll use this to filter the events.

First, create a class that implements ITelemetryProcessor. This class will filter request events with status code 200.

public class MyTelemetryProcessor : ITelemetryProcessor
{
    private ITelemetryProcessor _next;

    public MyTelemetryProcessor(ITelemetryProcessor next)
    {
        // Next TelemetryProcessor in the chain
        _next = next;
    }

    public void Process(ITelemetry item)
    {
        if (item is RequestTelemetry request)
        {
            if (request.ResponseCode == "200")
            {
                // Filter the event
                return;
            }
        }

        // Send the item to the next TelemetryProcessor
        _next.Process(item);
    }
}

Then, you have to configure Application Insights to use the MyTelemetryProcessor. In the Startup.cs file, add the following code in the Configure method:

public void Configure(IApplicationBuilder app)
{
    var configuration = app.ApplicationServices.GetService<TelemetryConfiguration>();
    configuration.TelemetryProcessorChainBuilder.Use(next => new MyTelemetryProcessor(next));
    configuration.TelemetryProcessorChainBuilder.Build();

    // ...
}

You filter is now configured and active. Now you can customize the filter to remove other kind of events. For instance, you can remove some specific traces based on their category:

public void Process(ITelemetry item)
{
    if (item is TraceTelemetry trace)
    {
        if (trace.Properties.TryGetValue("CategoryName", out var category) &&
            category == "Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware")
        {
            return;
        }
    }

    _next.Process(item);
}

If you have a public website, you may have lots of requests to the WordPress login page /wp-login.php. If you are not using a WordPress website, you can filter those events:

public void Process(ITelemetry item)
{
    if (item is RequestTelemetry request)
    {
        // The url is not handled, so 404
        if (string.Equals(request.ResponseCode, "404", StringComparison.Ordinal))
        {
            var path = request.Url?.AbsolutePath;
            if (path != null && path.EndsWith("/wp-login.php", StringComparison.OrdinalIgnoreCase))
            {
                return;
            }
        }
    }

    Next.Process(item);
}

Conclusion

The Application Insights SDK allows you to control the data sent to Azure. You can augment events with custom data, edit the events to remove sensitive data, or filter unneeded events. So, you can better detect & diagnose issues and understand usage for your web apps.

Comments

David De Sloovere -

Thanks for this post. I was finally able to ignore ApplicationInsights in an area of my application. I could not find this solution in the official documentation.

Leave a reply