Cassandra Log4j appender solutions
DataStax Enterprise allows you to stream your web and application log information into a database cluster via Apache log4j.
DataStax Enterprise allows you to stream your web and application log information into a database cluster via Apache log4j.
About the log4j Utility
Apache log4j is a Java-based logging framework that provides runtime application feedback. It provides the ability to control the granularity of log statements using an external configuration file (log4j.properties).
With the Cassandra Appender, you can store the log4j messages in a table where they're available for in-depth analysis using the Hadoop and Solr capabilities provided by DataStax Enterprise. For information about Cassandra logging, see Logging Configuration. Additionally, DataStax provides a Log4j Search Demo.
The log4j utility has three main components: loggers, appenders, and layouts. Loggers are logical log file names. They are the names known to the Java application. Each logger is independently configurable for the level of logging. Outputs are controlled by Appenders. Numerous Appenders are available and multiple Appenders can be attached to any Logger. This makes it possible to log the same information to multiple outputs. Appenders use Layouts to format log entries. In the example below, messages show the level, the thread name, the message timestamp, the source code file, the line number, and the log message.
Log levels
- All - turn on all logging
- OFF - no logging
- FATAL - severe errors causing premature termination
- ERROR - other runtime errors or unexpected conditions
- WARN - use of deprecated APIs, poor use of API, near errors, and other undesirable or unexpected runtime situations
- DEBUG - detailed information on the flow through the system
- TRACE - more detailed than DEBUG
- INFO - highlight the progress of the application at a coarse-grained level
Datastax does not recommend using TRACE or DEBUG in production due to verbosity and performance.
Log messages
As mentioned above, the messages that appear in the log are controlled via the conf/log4j.properties file. Using this properties file, you can control the granularity to the Java package and class levels. For example, DEBUG messages from a particular class can be included in the log while messages from others remain at a higher level. This is helpful to reduce clutter and to identify messages. The log is most commonly a file and/or stdout. The format, behavior (such as file rolling), and so on is also configurable at runtime.
Below are sample log messages from a Cassandra node startup:
INFO [main ] 2012-02-10 09:15:33,112 DatabaseDescriptor.java (line 495 ) Found table data in data directories. Consider using the CLI to define your schema. INFO [main ] 2012-02-10 09:15:33,135 CommitLog.java (line 166 ) No commitlog files found; skipping replay INFO [main ] 2012-02-10 09:15:33,150 StorageService.java (line 400 ) Cassandra version: 1.0.7 INFO [main ] 2012-02-10 09:15:33,150 StorageService.java (line 401 ) Thrift API version: 19.20.0 INFO [main ] 2012-02-10 09:15:33,150 StorageService.java (line 414 ) Loading persisted ring state ...
Storing log4j messages in a table
The Cassandra Appender provides the capability to store log4j messages in a Cassandra table.