Structured Logging#

This document describes a newer “structured” logging facility which reports messages from the driver itself using a BSON format defined across driver implementations by the MongoDB Logging Specification. See Unstructured Logging for the original freeform logging facility.

These two systems are configured and used independently.

Unstructured logging is global to the entire process, but structured logging is configured separately for each mongoc_client_t or mongoc_client_pool_t. See mongoc_client_set_structured_log_opts() and mongoc_client_pool_set_structured_log_opts().

Options#

Structured log settings are tracked explicitly by a mongoc_structured_log_opts_t instance.

Like other drivers supporting structured logging, we take default settings from environment variables and offer additional optional programmatic configuration. Environment variables are captured during mongoc_structured_log_opts_new(), refer there for a full list of the supported variables.

Normally environment variables provide defaults that can be overridden programmatically. To request the opposite behavior, where your programmatic defaults can be overridden by the environment, see mongoc_structured_log_opts_set_max_levels_from_env().

Structured log messages may be filtered in arbitrary ways by the handler, but as both a performance optimization and a convenience, a built-in filter limits the maximum log level of reported messages with a per-component setting.

mongoc_structured_log_opts_t

Levels and Components#

Log levels and components are defined as mongoc_structured_log_level_t and mongoc_structured_log_component_t enumerations. Utilities are provided to convert between these values and their standard string representations. The string values are case-insensitive.

typedef enum {
  MONGOC_STRUCTURED_LOG_LEVEL_EMERGENCY = 0,  // "Emergency" ("off" also accepted)
  MONGOC_STRUCTURED_LOG_LEVEL_ALERT = 1,      // "Alert"
  MONGOC_STRUCTURED_LOG_LEVEL_CRITICAL = 2,   // "Critical"
  MONGOC_STRUCTURED_LOG_LEVEL_ERROR = 3,      // "Error"
  MONGOC_STRUCTURED_LOG_LEVEL_WARNING = 4,    // "Warning" ("warn" also accepted)
  MONGOC_STRUCTURED_LOG_LEVEL_NOTICE = 5,     // "Notice"
  MONGOC_STRUCTURED_LOG_LEVEL_INFO = 6,       // "Informational" ("info" also accepted)
  MONGOC_STRUCTURED_LOG_LEVEL_DEBUG = 7,      // "Debug"
  MONGOC_STRUCTURED_LOG_LEVEL_TRACE = 8,      // "Trace"
} mongoc_structured_log_level_t;

typedef enum {
  MONGOC_STRUCTURED_LOG_COMPONENT_COMMAND = 0,           // "command"
  MONGOC_STRUCTURED_LOG_COMPONENT_TOPOLOGY = 1,          // "topology"
  MONGOC_STRUCTURED_LOG_COMPONENT_SERVER_SELECTION = 2,  // "serverSelection"
  MONGOC_STRUCTURED_LOG_COMPONENT_CONNECTION = 3,        // "connection"
} mongoc_structured_log_component_t;

Log Handlers#

Each mongoc_client_pool_t or standalone mongoc_client_t has its own instance of the structured logging subsystem, with its own settings and handler.

When using mongoc_client_pool_t, the pooled clients all share a common logging instance. Handlers must be thread-safe.

The handler is called for each log entry with a level no greater than its component’s maximum. A mongoc_structured_log_entry_t pointer provides access to further details, during the handler only.

Handlers must take care not to re-enter libmongoc with the same mongoc_client_t or mongoc_client_pool_t that the handler has been called by.

mongoc_structured_log_func_t

Log Entries#

Each log entry is represented within the handler by a short-lived mongoc_structured_log_entry_t pointer. During the handler, this pointer can be used to access the individual properties of an entry: its level, component, and message.

The message will be assembled as a bson_t only when explicitly requested by a call to mongoc_structured_log_entry_message_as_bson(). This results in a standalone document that may be retained for any amount of time and must be explicitly destroyed.

mongoc_structured_log_entry_t

Example#

example-structured-log.c#

/* gcc example-structured-log.c -o example-structured-log \
 *     $(pkg-config --cflags --libs libmongoc-1.0) */

#include <mongoc/mongoc.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>

static pthread_mutex_t handler_mutex;

static void
example_handler (const mongoc_structured_log_entry_t *entry, void *user_data)
{
   mongoc_structured_log_component_t component = mongoc_structured_log_entry_get_component (entry);
   mongoc_structured_log_level_t level = mongoc_structured_log_entry_get_level (entry);
   const char *message_string = mongoc_structured_log_entry_get_message_string (entry);

   /*
    * With a single-threaded mongoc_client_t, handlers will always be called
    * by the thread that owns the client. On a mongoc_client_pool_t, handlers
    * are shared by multiple threads and must be reentrant.
    *
    * Note that unstructured logging includes a global mutex in the API,
    * but structured logging allows applications to avoid lock contention
    * even when multiple threads are issuing commands simultaneously.
    *
    * Simple apps like this example can achieve thread safety by adding their
    * own global mutex. For other apps, this would be a performance bottleneck
    * and it would be more appropriate for handlers to process their log
    * messages concurrently.
    *
    * In this example, our mutex protects access to a global log counter.
    * In a real application, you may need to protect access to a shared stream
    * or queue.
    */
   pthread_mutex_lock (&handler_mutex);

   static unsigned log_serial_number = 0;

   printf ("%u. Log entry with component=%s level=%s message_string='%s'\n",
           ++log_serial_number,
           mongoc_structured_log_get_component_name (component),
           mongoc_structured_log_get_level_name (level),
           message_string);

   /*
    * At this point, the handler might make additional filtering decisions
    * before asking for a bson_t. As an example, let's log the component and
    * level for all messages but only show contents for command logs.
    */
   if (component == MONGOC_STRUCTURED_LOG_COMPONENT_COMMAND) {
      bson_t *message = mongoc_structured_log_entry_message_as_bson (entry);
      char *json = bson_as_relaxed_extended_json (message, NULL);
      printf ("Full log message, as json: %s\n", json);
      bson_destroy (message);
      bson_free (json);
   }

   pthread_mutex_unlock (&handler_mutex);
}

int
main (void)
{
   const char *uri_string = "mongodb://localhost:27017";
   int result = EXIT_FAILURE;
   bson_error_t error;
   mongoc_uri_t *uri = NULL;
   mongoc_structured_log_opts_t *log_opts = NULL;
   mongoc_client_t *client = NULL;
   mongoc_client_pool_t *pool = NULL;

   /*
    * Note that structured logging only applies per-client or per-pool,
    * and it won't be used during or before mongoc_init.
    */
   mongoc_init ();

   /*
    * Logging options are represented by a mongoc_structured_log_opts_t,
    * which can be copied into a mongoc_client_t or mongoc_client_pool_t
    * using mongoc_client_set_structured_log_opts() or
    * mongoc_client_pool_set_structured_log_opts(), respectively.
    *
    * Default settings are captured from the environment into
    * this structure when it's constructed.
    */
   log_opts = mongoc_structured_log_opts_new ();

   /*
    * For demonstration purposes, set up a handler that receives all possible log messages.
    */
   pthread_mutex_init (&handler_mutex, NULL);
   mongoc_structured_log_opts_set_max_level_for_all_components (log_opts, MONGOC_STRUCTURED_LOG_LEVEL_TRACE);
   mongoc_structured_log_opts_set_handler (log_opts, example_handler, NULL);

   /*
    * By default libmongoc proceses log options from the environment first,
    * and then allows you to apply programmatic overrides. To request the
    * opposite behavior, allowing the environment to override programmatic
    * defaults, you can ask for the environment to be re-read after setting
    * your own defaults.
    */
   mongoc_structured_log_opts_set_max_levels_from_env (log_opts);

   /*
    * Create a MongoDB URI object. This example assumes a local server.
    */
   uri = mongoc_uri_new_with_error (uri_string, &error);
   if (!uri) {
      fprintf (stderr, "URI parse error: %s\n", error.message);
      goto done;
   }

   /*
    * Create a new client pool.
    */
   pool = mongoc_client_pool_new (uri);
   if (!pool) {
      goto done;
   }

   /*
    * Set the client pool's log options.
    * This must happen only once, and only before the first mongoc_client_pool_pop.
    * There's no need to keep log_opts after this point.
    */
   mongoc_client_pool_set_structured_log_opts (pool, log_opts);

   /*
    * Check out a client, and do some work that we'll see logs from.
    * This example just sends a 'ping' command.
    */
   client = mongoc_client_pool_pop (pool);
   if (!client) {
      goto done;
   }

   bson_t *command = BCON_NEW ("ping", BCON_INT32 (1));
   bson_t reply;
   bool command_ret = mongoc_client_command_simple (client, "admin", command, NULL, &reply, &error);
   bson_destroy (command);
   bson_destroy (&reply);
   mongoc_client_pool_push (pool, client);
   if (!command_ret) {
      fprintf (stderr, "Command error: %s\n", error.message);
      goto done;
   }

   result = EXIT_SUCCESS;
done:
   mongoc_uri_destroy (uri);
   mongoc_structured_log_opts_destroy (log_opts);
   mongoc_client_pool_destroy (pool);
   mongoc_cleanup ();
   return result;
}