MongoDB Listener

Description

The MongoDB listener plugin polls for records from a MongoDB collection and publishes them to 1Gateway. A MongoDB query can be run to filter out Documents and avoid getting duplicate data.

Prerequisites

Network access to the MongoDB database from which data is retrieved.

Installation

Make sure you have the 1gateway zip file that includes the plugin files and valid licences information ready. This file has been provided to you with the installation files.

Login to 1gateway, and make sure you are in "Advanced mode". If this is the first time you log in, the default user and password are both 'admin' (without the quotes).

2 - System Maintenance

Click on the menu icon and choose the option "System maintenance"

3 - Upload file

In the System maintenance view you can either drag and drop the downloaded zip file or select it by clicking the "choose file" button.

4 - Restart

After the upload installation will be confirmed. Restart 1Gateway to ensure any locked files are updated.

Configuration

Open the main menu and click on "New plugin"

Select the plugin you want to configure. Use the Filter field if needed.

In the profiles tab, click on ADD PROFILE and give it a name.

Fill in the fields accordingly.

Field name	Supported values	Description	Example
Connection string	Valid connection string	Connection string to the MongoDB database	mongodb://192.168.0.40:27017/?directConnection=true
Database name	Any string	Name of the database to connect to	test
Collection name	Any string	Name of the collection to extract data from	audit_data
Database user	Any string	MongoDB user. Leave blank if the MongoDB doesn't require authentication.
Database password	Any string	MongoDB user's password. Leave blank if the MongoDB doesn't require authentication.
Query	Valid MongoDB query.	Query to be executed. The $value placeholder will get replaced with the value of a field in the last message from the last poll cycle or with a default value.	{ "timestamp": { "$gt": { "$date" : "$value" } }}
Field parameter (optional)	Any string	An existing message field name. The value of this field will replace the $value placeholder in the query.	timestamp/$date
Field default value (optional)	Any string	Starting parameter value (needs to be valid)	2022-12-15T08:20:16.702+00:00
Projection	Comma-separated list	Determines which fields from the result to select in the output. Leave blank to get all fields.	action,status,objectname,objectid
Sort	Comma-separated list	Determines the order of the results in the output. Leave blank for any sorting.	{"timestamp": 1}
Message type	Any string	1Gateway message type of the MongoDB records	MongoDBMessage
Polling interval (in seconds)	Integer	Recommended: 86,400 (once a day)	86400

Examples of MongoDB queries and their meaning:

Example 1

Getting data where the timestamp is greater than the previous value (timestamp/$date in the last message from the last poll cycle). If the last message from the last poll cycle doesn't contain a timestamp or its the first poll cycle, the value will default to the Field default value.

Field name	Example
Query	{ "timestamp": { "$gt": ISODate("$value") }}
Field parameter (optional)	timestamp/$date
Field default value (optional)	2022-12-15T08:20:16.702+00:00
Sort	{ "timestamp": 1}

Example 2

Getting data of a specific objecttype (plugin) and where the timestamp is greater than the previous value (timestamp/$date in the last message from the last poll cycle). If the last message from the last poll cycle doesn't contain a timestamp or its the first poll cycle, the value will default to the Field default value.

Field name	Example
Query	{ "objecttype": "plugin", "timestamp": { "$gt": ISODate("$value") } }
Field parameter (optional)	timestamp/$date
Field default value (optional)	2022-12-15T08:20:16.702+00:00
Sort	{ "timestamp": 1}

Example 3

Getting data of a specific objecttype (plugin or mapper) and where the timestamp is greater than the previous value (timestamp/$date in the last message from the last poll cycle). If the last message from the last poll cycle doesn't contain a timestamp or its the first poll cycle, the value will default to the Field default value.

Field name	Example
Query	{ "$and": [ { "objecttype": { "$exists": true, "$in": ["plugin", "mapper"] } }, { "timestamp": { "$gt": ISODate("2022-04-20 10:04:16.702Z") }} ] }
Field parameter (optional)	timestamp/$date
Field default value (optional)	2022-12-15T08:20:16.702+00:00
Sort	{ "timestamp": 1}

Example 4

Getting data of a specific objecttype (any existing value except for plugin) and where the timestamp is greater than the previous value (timestamp/$date in the last message from the last poll cycle). If the last message from the last poll cycle doesn't contain a timestamp or its the first poll cycle, the value will default to the Field default value.

Field name	Example
Query	{ "$and": [ { "objecttype": { "$ne": "plugin" } }, { "objecttype": { "$exists": true } }, { "objecttype": { "$ne": null } }, { "timestamp": { "$gt": ISODate("$value") }} ] }
Field parameter (optional)	timestamp/$date
Field default value (optional)	2022-12-15T08:20:16.702+00:00
Sort	{ "timestamp": 1}