.NET: Configure logs in context

The following diagram illustrates the flow of log messages through Apache log4net, highlighting specific components of the New Relic log4net extension. Many log forwarders are available. This example uses Fluentd.

Appender: The NewRelicAppender adds contextual information from the .NET agent (using the API) to the log events generated by the application. This contextual information, known as linking metadata, is used by New Relic to link log messages to the transactions and spans from which they were created. This appender will pass the enriched log events to downstream appenders for further processing.

Since the NewRelicAppender is ForwardingAppender type, it needs to be the first appender in the chain. It also requires another appender that can write to an actual output destination as its child in order to work.

Layout: The NewRelicLayout formats the enriched log events into the JSON format expected by New Relic. The appender, which this layout is assigned to, instructs log4net to output the JSON to a file in the location that the log forwarder expects.

Log Forwarder: The log forwarder monitors an output folder and incrementally sends the properly formatted and enriched log information to the New Relic logging endpoint.

Log4net uses appender and layout to store and format log messages. NewRelicAppender enriches log messages with contextual information from the New Relic .NET agent if it is attached to your application. The appender passes enriched log messages to downstream appenders to handle specific use cases for log messages.

For more information about logging with log4net, see the Apache log4net Getting started documentation.

To configure logs in context with the log4net extension:

1. Using the Visual Studio NuGet Package Manager, locate and install the NewRelic.LogEnrichers.Log4Net package.

2. In your log4net configuration file, update your logging configuration to use the NewRelicAppender as the first level appender, and reference your existing appenders as its children. Also replace the layout of the appender that writes log messages to an output destination with the NewRelicLayout.

   The following log4net configuration example enriches log events with New Relic linking metadata. In addition to the existing log files, it outputs new log files in a specific JSON format at C:\logs\log4netExample.log.json for consumption by the log forwarder:

   <log4net>
     <root>
       <level value="ALL" />
       <appender-ref ref="NewRelicAppender" />
     </root>
   
     <appender name="NewRelicAppender" type="NewRelic.LogEnrichers.Log4Net.NewRelicAppender, NewRelic.LogEnrichers.Log4Net" >
       <threshold value="ALL"/>
       <appender-ref ref="FileAppender" />
     </appender>
   
     <appender name="FileAppender" type="log4net.Appender.FileAppender">
       <file value="C:\logs\log4netExample.log.json" />
       <param name="AppendToFile" value="true" />
       <layout type="NewRelic.LogEnrichers.Log4Net.NewRelicLayout, NewRelic.LogEnrichers.Log4Net">
       </layout>
     </appender>
   </log4net>

   Copy

   After you configure the log4net extension and update your logging file, you can configure your extension to send data to New Relic. Here is an example configuration using the Fluentd plugin for New Relic Logs:

   <!--NewRelicLoggingExample.conf-->
   <source>
       @type tail 
       path C:\logs\log4netExample.log.json
       pos_file C:\logs\log4netExample.log.json.pos
       tag logfile.*
     <parse> 
       @type json
     </parse>
   </source>
     <match **> 
       @type newrelic
       license_key <YOUR NEW_RELIC_LICENSE_KEY>
       base_uri https://log-api.newrelic.com/log/v1
     </match>

   Copy

The New Relic NLog extension provides a NewRelicJsonLayout that formats a log event in the way required by the New Relic logging endpoint. Next, it adds contextual information from the .NET agent when attached to your application. Then, a target can be configured to write logging data to an output folder. The log forwarder can monitor this folder and incrementally send log information to New Relic.

The following diagram illustrates the flow of log messages through NLog, highlighting specific components of the New Relic NLog extension.

New Relic JSON Layout: The NewRelicJsonLayout adds contextual information from the .NET agent (using the API) to the log events generated by the application, and outputs log messages in the JSON format expected by New Relic. This contextual information, known as linking metadata, is used by New Relic to link log messages to the transactions and spans where they were created.

Since the NewRelicAppender is ForwardingAppender type, it needs to be the first appender in the chain. It also requires another appender that can write to an actual output destination as its child in order to work.

File Target: A FileTarget defines a file on disk where log messages are written. Adding the NewRelicJsonLayout to that target allows the output to be formatted correctly for forwarding to New Relic.

Log Forwarder: The log forwarder is configured to send the properly formatted and enriched log data from the FileTarget output to the New Relic logging endpoint.

For more information about logging with NLog, see the nlog-project.org documentation.

Use our NLog 4.5 or higher extension to link to your log data with related data across the rest of the New Relic platform.

1. Using the Visual Studio NuGet Package Manager, locate and install the NewRelic.LogEnrichers.NLog package.

2. In your application code, update your logging configuration to add the NewRelicJsonLayout and decide if you want to collect MappedDiagnosticsContext (MDC) or the MappedDiagnosticsLogicalContext (MDLC) data.

   The following configuration examples result in new JSON files that are written to disk. Some of these configuration options may be useful for managing the amount of disk space used and/or the performance of the target.

• archiveAboveSize

• maxArchiveFiles

• bufferSize

• enableArchiveFileCompression

• autoFlush

• concurrentWrites

  Although the NLog AsyncWrapper Target is not required, it may help improve performance by performing formatting and output of log files on a different thread.

  Don't collect MDC or the MDLC data (default):

  The following code example enriches log events with New Relic linking metadata, but not with MDC or the MDLC data. In addition to the existing log files, it outputs new log files in a specific JSON format at C:\logs\NLogExample.log.json for consumption by the log forwarder:

  var loggerConfig = new LoggingConfiguration();
  var newRelicFileTarget = new FileTarget("NewRelicFileTarget");
  newRelicFileTarget.Layout = new NewRelicJsonLayout();
  newRelicFileTarget.FileName = "C:\logs\NLogExample.json";
  loggerConfig.AddTarget(newRelicFileTarget);
  loggerConfig.AddRuleForAllLevels("NewRelicFileTarget");
  LogManager.Configuration = loggerConfig;
  var logger = LogManager.GetLogger("Example");

  Copy

  Collect MDC or the MDLC data:

  If your application uses the MDC or the MDLC, you can configure the NewRelicJsonLayout to include items in those collections. The following code example adds the additional configuration to enable collecting MDC and MDLC data. As in the previous example, it outputs new log files in a specific JSON format at C:\logs\NLogExample.log.json for consumption by the log forwarder:

  var loggerConfig = new LoggingConfiguration();
  var newRelicFileTarget = new FileTarget("NewRelicFileTarget");
  var newRelicLayout = new NewRelicJsonLayout
  {
    IncludeMdc = `true,`
    IncludeMdlc = `true`
  };
  
  newRelicFileTarget.Layout = newRelicLayout;
  newRelicFileTarget.FileName = "C:\logs\NLogExample.json";
  loggerConfig.AddTarget(newRelicFileTarget);
  loggerConfig.AddRuleForAllLevels("NewRelicFileTarget");
  LogManager.Configuration = loggerConfig;
  var logger = LogManager.GetLogger("Example");

  Copy

  Once you have configured the NLog extension and updated your logging file, you can configure your extension to send data to New Relic. Here is an example configuration using the Fluentd plugin to forward logs to New Relic:

  <!--NewRelicLoggingExample.conf-->
  <source>
    @type tail
    path C:\logs\NLogExample.log.json
    pos_file C:\logs\NLogExample.log.json.pos 
    tag logfile.*
    <parse>
      @type json
    </parse>
  </source>
  <match **>
    @type newrelic
    license_key <YOUR NEW_RELIC_LICENSE_KEY>
    base_uri https://log-api.newrelic.com/log/v1
  </match>

  Copy

You can also configure the New Relic NLog extension with file-based configuration providers. The folowing example code creates a logger based on settings contained in an App.config file.

Instantiating Logger using .config file

var logger = LogManager.GetLogger("NewRelicLog");
logger.Info("Hello, New Relic!");

Sample App.config file

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
    <section name="nlog" type="NLog.Config.ConfigSectionHandler, NLog"/>
  </configSections>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
  </startup>
  <nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <extensions>
      <add assembly="NewRelic.LogEnrichers.NLog" />
    </extensions>
    <targets>
      <target name="NewRelicLogFile" xsi:type="File" fileName="C:/path/to/NewRelicLog.json">
        <layout xsi:type="newrelic-jsonlayout">
        </layout>
      </target>
    </targets>
    <rules>
      <logger name="NewRelicLog" minlevel="Info" writeTo="newRelicLogFile" />
    </rules>
  </nlog>
</configuration>

Serilog is a structured logging framework that records log messages from your application and creates a LogEvent to store the message data. Using Enrichers, you can add additional information to the log events. Sinks and Formatters allow you to format and output those log events for downstream consumption and viewing.

The following diagram illustrates the flow of log messages through Serilog, highlighting specific components of the New Relic Serilog extension. Many log forwarders are available. This example uses Fluentd.

New Relic Enricher: The NewRelicEnricher adds contextual information from the .NET agent (using the API) to the log events generated by the application. This contextual information, called linking metadata, is used by New Relic to link log messages to the transactions and spans where they were created.

New Relic Formatter: The NewRelicFormatter translates enriched log events into the JSON format expected by New Relic. A sink instructs Serilog to output the JSON to a file in the location that the log forwarder expects.

New Relic Log Forwarder: The log forwarder is configured to send the properly formatted and enriched log data from the FileTarget output to the New Relic logging endpoint.

For more information about Serilog log events, see the Serilog documentation on GitHub.

To configure logs in context with the Serilog extension:

1. Use the Visual Studio NuGet Package Manager to locate and install the NewRelic.LogEnrichers.Serilog package.

2. In your application code, update your logging configuration to add the NewRelicEnricher and NewRelicFormatter.

   The following code example enriches log events with New Relic linking metadata. In addition to the existing log files, it outputs new log files in a specific JSON format at C:\logs\SerilogExample.log.json for consumption by the log forwarder:

   var loggerConfig = new LoggerConfiguration()
            
     loggerConfig
        .Enrich.WithThreadName()
        .Enrich.WithThreadId()    
        .Enrich.WithNewRelicLogsInContext()
        .WriteTo.File( path: @"C:\logs\ExistingLoggingOutput.txt")
        .WriteTo.File(
            formatter: new NewRelicFormatter(), 
            path: @"C:\logs\SerilogExample.log.json");
   
   var log = loggerConfig.CreateLogger();

   Copy

   This configuration results in new JSON files that are written to disk. Some of these configuration options may be useful for managing the amount of disk space used and/or the performance of the sink.

• restrictedToMinimumLevel

• buffered

• rollingInterval

• rollOnFileSizeLimit

• retainedFileCountLimit

  Although not required, using the Serilog Asynchronous Sink Wrapper may help improve the performance by performing formatting and output of log files on a different thread.

3. Once you have configured the Serilog extension and updated your logging file, you can configure your extension to send data to New Relic.

   Here is an example configuration using the Fluentd plugin to forward logs to New Relic:

   <!--NewRelicLoggingExample.conf-->
    <source> 
        @type tail 
        path C:\logs\SerilogExample.log.json
        pos_file C:\logs\SerilogExample.log.json.pos 
        tag logfile.*
     <parse> 
        @type json 
    </parse>
    </source>
    <match **> 
        @type newrelic 
        license_key <YOUR NEW_RELIC_LICENSE_KEY>
        base_uri https://log-api.newrelic.com/log/v1
    </match>

   Copy

You can also configure the New Relic Serilog extension with file-based configuration providers.The following additional NuGet Packages are required:

• Microsoft.Extensions.Configuration

• Serilog.Settings.Configuration

  The following example code creates a logger based on settings contained in an appSettings.json file.

  Instantiating logger using appsettings.json

  var builder = new ConfigurationBuilder()
          .AddJsonFile("appsettings.json");
      
  var configuration = builder.Build();
  
  var logger = new LoggerConfiguration()
          .ReadFrom.Configuration(configuration)
          .CreateLogger();

  Copy

  Sample appsettings.json file

  {
    "Serilog": {
      "Using": [ 
            "Serilog.Sinks.Console",
            "Serilog.Sinks.File",
            "NewRelic.LogEnrichers.Serilog" 
      ],
      "MinimumLevel": "Debug",
      "Enrich": [ "WithNewRelicLogsInContext" ],
      "WriteTo": [
        {
          "Name": "File",
          "Args": {
            "path": "C:\\Logs\\SerilogExample.log.json",
            "formatter": "NewRelic.LogEnrichers.Serilog.NewRelicFormatter, NewRelic.LogEnrichers.Serilog"
            }
        }
      ],
  
      "Properties": {
        "Application": "NewRelic Logging Serilog Example"
      }
    }
  }

  Copy

  The following example code creates a logger based on settings contained in a web.config file. The Serilog.Settings.AppSettings NuGet Package is required.

  Instantiating logger using .config file

  var logger = new LoggerConfiguration()
      .ReadFrom.AppSettings()
      .CreateLogger();

  Copy

  Sample web.config file

  <?xml version="1.0" encoding="utf-8"?>
  <configuration>
    <appSettings>
      <add key="serilog:using:NewRelic" value="NewRelic.LogEnrichers.Serilog" />
      <add key="serilog:using:File" value="Serilog.Sinks.File" />
      <!--Add other enrichers here-->
      <add key="serilog:enrich:WithNewRelicLogsInContext" />
      <add key="serilog:write-to:File.path" value="C:\logs\SerilogExample.log.json" />
      <add key="serilog:write-to:File.formatter" value="NewRelic.LogEnrichers.Serilog.NewRelicFormatter, NewRelic.LogEnrichers.Serilog" />
    </appSettings>

  Copy