MongoDB - Audit Log

MongoDB Audit Log

Introduction

For MongoDB, audit messages can be logged into syslog, console or file (JSON or BSON format).

Details of how to setup MongoDB Audit log can be found here

For example, to setup audit log with JSON output using configuration file:

auditLog:
   destination: file
   format: JSON
   path: data/db/auditLog.json

Note:

If format is not defined, MongoDB will default to use BSON for the log file.
Using JSON format degrades server performance more than BSON format.

What is being logged

Generally speaking, the following actions can be logged:

Authentication and authorization
Cluster operations
Read and write operations (logged under authCheck event and require auditAuthorizationSuccess parameter to be enabled)
Schema operations
Custom application messages (logged under applicationMessage event if the client/app issues a logApplicationMessage command, the user needs to have clusterAdmin role or the one that inherits from it to issue this command)

Check details of what can be logged on (here).

Logging for CRUD events

By default, MongoDB (using authCheck) only logs failed CRUD (create, read, update, and delete) events. To log all successful CRUD events, we need to enable auditAuthorizationSuccess by simply run this command with Mongo Shell:

db.adminCommand( { setParameter: 1, auditAuthorizationSuccess: true } )

Or setting the configuraton file with

auditLog:
  destination: file
  format: JSON
  path: /var/lib/mongodb/auditLog.json
setParameter: { auditAuthorizationSuccess: true }

Note: When the auditAuthorizationSuccess is set to true, there will be performance impacts as each event will have to be logged in the audit before the oplog.

Here is an example of what an authCheck event looks like in the audit log

{
   "atype": "authCheck",
   "ts": { "$date": "2016-11-14T18:36:57.806+0000" },
   "local":{ "ip": "127.0.0.1", "port": 27017 },
   "remote":{ "ip": "127.0.0.1", "port": 48472 ,
   "users": [{ "user": "accountUser", "db": "books" }],
   "roles": [
      { "role": "dbAdmin", "db": "books" },
      { "role": "readWrite", "db": "books" }
   ],
   "param": {
      "command": "find",
      "ns": "books.books",
      "args": {
         "find": "books",
         "filter": {
            "year":   { "$gt": 1800 }
         }
      }
   },
   "result": 0
}

Audit with Filter

To only log a subset of events we can apply audit filters.

See configure audit filters for turtorials.
See here for the details of auditable system events.ss
Following example changes audit filters to audit specific user

...
...
# Audit log setting
auditLog:
   destination: file
   format: JSON
   path: /var/log/mongodb/auditLog.json
   filter: '{ users: { user: "superuser", db: "admin" } }'
   
# enable DML auditing by audit atype:authCheck
setParameter: {auditAuthorizationSuccess: true}

...

Audit who had log into on a database - Authentication Operations on a Single Database

filter: '{ atype: "authenticate", "param.db": "test" }'

Audit filtered by Authorization Role - The following example audits operations by users with readWrite role on the test database, including users with roles that inherit from readWrite, by using the filter

filter: '{roles: {role: "readWrite", db: "test"}}'

Audit filtered on Read and Write Operations. - capture read and write operations in the audit log.

filter: '{ atype: "authCheck", "param.command": { $in: [ "find", "insert", "delete", "update", "findandmodify" ] } }'

Note: you must also enable the audit system to log authorization successes using the auditAuthorizationSuccess parameter.

Audit read and Write Operations for a Collection - The following example audits the find(), insert(), remove(), update(), save(), and findAndModify() operations for the collection orders in the database test

filter: '{ atype: "authCheck", "param.ns": "test.orders", "param.command": { $in: [ "find", "insert", "delete", "update", "findandmodify" ] } }'

Audit events on creating or droping collections

filter: '{atype: {$in: ["createCollection", "dropCollection"]}}'

Auidt filtered on Collection Creation and Drop Operations for a Single Database

filter: '{ atype: { $in: [ "createCollection", "dropCollection" ] }, "param.ns": /^test\\./ } }'

note: The regular expression requires two backslashes (\) to escape the dot (.).

Misc

Using Bsondump

If default is used for Audit log file, BSON format is used

To read BSON file, use the /usr/bin/bsondump command

bsondump /var/log/mongodb/auditLog.bson
{"atype":"authenticate","ts":{"$date":"2018-01-29T03:12:09.680Z"},"local":{"ip":"10.206.97.133","port":27017},"remote":{"ip":"10.206.97.133","port":45802},"users":[{"user":"monitor","db":"admin"}],"roles":[{"role":"clusterMonitor","db":"admin"}],"param":{"user":"monitor","db":"admin","mechanism":"SCRAM-SHA-1"},"result":0}
{"atype":"authenticate","ts":{"$date":"2018-01-29T03:12:29.679Z"},"local":{"ip":"10.206.97.133","port":27017},"remote":{"ip":"10.206.97.133","port":45802},"users":[{"user":"monitor","db":"admin"}],"roles":[{"role":"clusterMonitor","db":"admin"}],"param":{"user":"monitor","db":"admin","mechanism":"SCRAM-SHA-1"},"result":0}

The following is an example on how to find out who create a database called "prod"

bsondump auditLog.bson | jq -c 'select(.atype == "createDatabase") | select(.param.ns == "prod")'
{"atype":"createDatabase","ts":{"$date":"2017-02-17T12:13:48.142+0100"},"local":{"ip":"127.0.1.1","port":27017},"remote":{"ip":"127.0.0.1","port":47896},"users":[{"user":"prod_app","db":"admin"}],"roles":[{"role":"root","db":"admin"}],"param":{"ns":"prod"},"result":0}

Importing the Log into MongoDB for Seraching

> mongorestore -d auditdb -c auditcol auditLog.bson
2017-02-17T12:28:56.756+0100    checking for collection data in auditLog.bson
2017-02-17T12:28:56.797+0100    restoring auditdb.auditcol from auditLog.bson
2017-02-17T12:28:56.858+0100    no indexes to restore
2017-02-17T12:28:56.858+0100    finished restoring auditdb.auditcol (142 documents)
2017-02-17T12:28:56.858+0100    done

The import is done, and now we can query the collection for the same data from the MongoDB client:

> use auditdb
switched to db auditdb
> db.auditcol.find({atype: "createDatabase", param: {ns: "prod"}})
{ "_id" : ObjectId("58a6de78bdf080b8e8982a4f"), "atype" : "createDatabase", "ts" : { "$date" : "2017-02-17T12:13:48.142+0100" }, "local" : { "ip" : "127.0.1.1", "port" : 27017 }, "remote" : { "ip" : "127.0.0.1", "port" : 47896 }, "users" : [ { "user" : "prod_app", "db" : "admin" } ], "roles" : [ { "role" : "root", "db" : "admin" } ], "param" : { "ns" : "prod" }, "result" : 0 }

DevOps tips and tricks

Sunday, February 11, 2018