Feeding logstash with log4net through SocketAppender
If you are a Java developer and for some reason have to develop a .Net application you probably heard about Log4net. Rotating log files is awesome and the framework is pretty much easier to be consumed by your application.
Recently I started a project based on ELK stack (Elasticsearch + Logstash + Kibana). Our java projects are able to send the log messaging to a logstash forwarder (or shipper) through log4j2 SocketAppender easily, having all processed messages “persisted” into an elasticsearch instance with all metrics computed and all the data ready in a “queriable” state. Goal has been achieved.
Not so far… after that, we had to support some .Net applications. The adventure starts here.
The project architecture basically follows the “standard” logstash architecture where:
there is a logstash instance acting as a forwarder.
all logging messages are stored into a redis queue.
a logstash indexer instance process all messages, ignoring debug messages and applies metrics filters over all TRACE messages.
If you pay attention about the third bullet you may notice there is an IMPORTANT project convention: “all TRACE messages are considered something to be measured”. The problem is log4net does not provide TRACE method capabilities!
The good part is this can be “workarounded” if you build a extension. Just need to implement a static class called ILogExtensions and trace method will be available for all your applications Something like…
using System;
namespace log4net {
namespace Ext {
public static class ILogExtentions
{
//private static readonly log4net.ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
public static void Trace(this ILog log, string message, Exception exception)
{
log.Logger.Log(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType,log4net.Core.Level.Trace, message, exception);
}
public static void Trace(this ILog log, string message)
{
log.Trace(message, null);
}
}
}
}
This strategy may be useful if you decide to implement other log categories like Critical, Fine, Alert… Just don’t forget to add have your extension namespace added in the classes will use your custom methods. Otherwise, they won’t will be available.
Once this little problem was solved, the second one was found: log4net does not has SocketAppender. Apparently, UdpAppender would be enough. Some testes were made, no issue found. But we can’t forget the concepts: you can’t trust on UDP because it was not supposed to be this way. Wen I decided to test sending about 2 millions of messages about 10% of the logging events didn’t reach the forwarder, even when running all on same server (127.0.0.1).
Besides that, I was not in the mood to:
implement my own logging lib.
configure logstash to monitor the log4net rotating file.
configure a different input on logstash because few .Net applications.
As an alternative, I decided to implement a SocketAppender for log4net. The best part is log4net allows you to do such kind of thing by your own. The “receipt” basically is:
create your own appender user log4net AppenderSkeleton as your base class.
override ActivateOptions method if your appender needs to “reserve”/prepare some resources or configure when log4net starts.
override Append method to determine the logic of “appending”. Don’t forget logging is about appending something to somewhere.
override onClose method to release resources. This is VERY IMPORTANT to avoid having the TCP port stuck and not having your stuff being disposed properly.
using System;
using System.Text;
using System.Net;
using System.Net.Sockets;
using log4net.Core;
using log4net.Appender;
namespace log4net
{
namespace Appender {
public class SocketAppender : AppenderSkeleton
{
// properties set by log4net xml config file
public string remoteAddress { get; set;}
public int remotePort { get; set;}
public bool debugMode { get; set;}
// this is the socket (you don’t say!)
private Socket sender = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
// connect to a remote device
public override void ActivateOptions() {
// Establish the remote endpoint for the socket.
try {
sender.Connect (remoteAddress, remotePort);
} catch (ArgumentNullException ane) {
Console.WriteLine(“ArgumentNullException : {0}”,ane.ToString());
} catch (SocketException se) {
Console.WriteLine(“SocketException : {0}”,se.ToString());
} catch (Exception e) {
Console.WriteLine(“Unexpected exception : {0}”, e.ToString());
}
}
protected override void Append(LoggingEvent loggingEvent) {
string rendered = ”“;
// check connectivity before send
if (sender.Connected) {
// Renderizing event
rendered = RenderLoggingEvent (loggingEvent);
// Encode the data string into a byte array.
byte[] msg = Encoding.UTF8.GetBytes(rendered);
// Send the data through the socket.
int bytesSent = sender.Send(msg);
if (debugMode) {
Console.WriteLine (“- Bytes sent: ” + bytesSent.ToString());
}
} else {
// accumulate?!
Console.WriteLine (“[SKIPPED]:: ” + rendered);
}
}
protected override void OnClose() {
sender.Shutdown (SocketShutdown.Both);
sender.Close ();
}
}
}
}
Things will happen “magically” and your appender will be available to be configured in your log4net.config file with no issues.
<?xml version=”1.0” encoding=”utf-8” ?>
<configuration>
<configSections>
<section name=”log4net” type=”log4net.Config.Log4NetConfigurationSectionHandler,log4net”/>
</configSections>
<log4net>
<appender name=”ConsoleAppender” type=”log4net.Appender.ConsoleAppender”>
<layout type=”log4net.Layout.PatternLayout”>
<conversionPattern value=”[%property{log4net:HostName}][%-5level] %date{ISO8601} [%thread]::%logger - %message%newline” />
</layout>
<filter type=”log4net.Filter.LevelRangeFilter”>
<levelMin value=”INFO”/>
</filter>
</appender>
<appender name=”SocketAppender” type=”log4net.Appender.SocketAppender”>
<remoteAddress value=”localhost” />
<remotePort value=”9500” />
<layout type=”log4net.Layout.PatternLayout, log4net”>
<conversionPattern value=”%property{log4net:HostName} %level %date{ISO8601} %thread %logger - %message%newline” />
</layout>
</appender>
<root>
<level value=”Trace” />
<appender-ref ref=”ConsoleAppender” />
<appender-ref ref=”SocketAppender” />
</root>
</log4net>
</configuration>
THE TRICKY PART: how to read additional attributes from xml file nad make them available to be used by my appender? As you can see, remoteAddress and remotePort are configurable and we need to set them to have our appender working properly.
Don’t worry. Your job is create properties inside your appender class with same name! Log4net will do the job and then, you can avoid logging events loss through a TCP, reusing same input configured in logstash for log4j or log4j2.
The full code of this example can be found on github (CLICK HERE). Log4net-extensions project provides TRACE method and SocketAppender. I hope it helps someone who faced the same issues I did.
