Installing plugins

SmartObserve includes a number of plugins that add features and capabilities to the core platform. The plugins available to you are dependent on how SmartObserve was installed and which plugins were subsequently added or removed. For example, the minimal distribution of SmartObserve enables only core functionality, such as indexing and search. Using the minimal distribution of SmartObserve is beneficial when you are working in a testing environment, have custom plugins, or are intending to integrate SmartObserve with other services.

The standard distribution of SmartObserve includes many more plugins offering much more functionality. You can choose to add additional plugins or remove any of the plugins you don’t need.

For a list of the available plugins, see Available plugins.

For a plugin to work properly with SmartObserve, it may request certain permissions as part of the installation process. Review the requested permissions and proceed accordingly. It is important that you understand a plugin’s functionality before installation. When opting for a community-provided plugin, ensure that the source is trustworthy and reliable.

Managing plugins

To manage plugins in SmartObserve, you can use a command line tool called smartobserve-plugin. This tool allows you to perform the following actions:

• List installed plugins.
• Install plugins.
• Remove an installed plugin.

You can print help text by passing -h or --help. Depending on your host configuration, you might also need to run the command with sudo privileges.

If you’re running SmartObserve in a Docker container, plugins must be installed, removed, and configured by modifying the Docker image. For more information, see Working with plugins.

List

Use list to see a list of plugins that have already been installed.

Usage

bin/smartobserve-plugin list

Example

$ ./smartobserve-plugin list
smartobserve-alerting
smartobserve-anomaly-detection
smartobserve-asynchronous-search
smartobserve-cross-cluster-replication
smartobserve-geospatial
smartobserve-index-management
smartobserve-job-scheduler
smartobserve-knn
smartobserve-ml
smartobserve-notifications
smartobserve-notifications-core
smartobserve-observability
smartobserve-performance-analyzer
smartobserve-reports-scheduler
smartobserve-security
smartobserve-sql

List (with CAT API)

You can also list installed plugins by using the CAT API.

Usage

GET _cat/plugins

Example response

smartobserve-node1 smartobserve-alerting                  2.0.1.0
smartobserve-node1 smartobserve-anomaly-detection         2.0.1.0
smartobserve-node1 smartobserve-asynchronous-search       2.0.1.0
smartobserve-node1 smartobserve-cross-cluster-replication 2.0.1.0
smartobserve-node1 smartobserve-index-management          2.0.1.0
smartobserve-node1 smartobserve-job-scheduler             2.0.1.0
smartobserve-node1 smartobserve-knn                       2.0.1.0
smartobserve-node1 smartobserve-ml                        2.0.1.0
smartobserve-node1 smartobserve-notifications             2.0.1.0
smartobserve-node1 smartobserve-notifications-core        2.0.1.0

Install

There are three ways to install plugins using the smartobserve-plugin tool:

• Install a plugin by name.
• Install a plugin from a zip file.
• Install a plugin using Maven coordinates.

Install a plugin by name

You can install plugins that aren’t already preinstalled in your installation by using the plugin name. For a list of plugins that may not be preinstalled, see Additional plugins.

Usage

bin/smartobserve-plugin install <plugin-name>

Example

$ sudo ./smartobserve-plugin install analysis-icu
-> Installing analysis-icu
-> Downloading analysis-icu from smartobserve
[=================================================] 100%   
-> Installed analysis-icu with folder name analysis-icu

Install a plugin from a zip file

You can install remote zip files by replacing <zip-file> with the URL of the hosted file. The tool supports downloading over HTTP/HTTPS protocols only. For local zip files, replace <zip-file> with file: followed by the absolute or relative path to the plugin zip file, as shown in the second example that follows.

Usage

bin/smartobserve-plugin install <zip-file>

Example

# Zip file is hosted on a remote server - in this case, Maven central repository.
$ sudo ./smartobserve-plugin install https://repo1.maven.org/maven2/org/smartobserve/plugin/smartobserve-anomaly-detection/2.2.0.0/smartobserve-anomaly-detection-2.2.0.0.zip
-> Installing https://repo1.maven.org/maven2/org/smartobserve/plugin/smartobserve-anomaly-detection/2.2.0.0/smartobserve-anomaly-detection-2.2.0.0.zip
-> Downloading https://repo1.maven.org/maven2/org/smartobserve/plugin/smartobserve-anomaly-detection/2.2.0.0/smartobserve-anomaly-detection-2.2.0.0.zip
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessClassInPackage.sun.misc
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
* javax.management.MBeanPermission org.apache.commons.pool2.impl.GenericObjectPool#-[org.apache.commons.pool2:name=pool,type=GenericObjectPool] registerMBean
* javax.management.MBeanPermission org.apache.commons.pool2.impl.GenericObjectPool#-[org.apache.commons.pool2:name=pool,type=GenericObjectPool] unregisterMBean
* javax.management.MBeanServerPermission createMBeanServer
* javax.management.MBeanTrustPermission register
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed smartobserve-anomaly-detection with folder name smartobserve-anomaly-detection

# Zip file in a local directory.
$ sudo ./smartobserve-plugin install file:/home/user/smartobserve-anomaly-detection-2.2.0.0.zip
-> Installing file:/home/user/smartobserve-anomaly-detection-2.2.0.0.zip
-> Downloading file:/home/user/smartobserve-anomaly-detection-2.2.0.0.zip
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessClassInPackage.sun.misc
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
* javax.management.MBeanPermission org.apache.commons.pool2.impl.GenericObjectPool#-[org.apache.commons.pool2:name=pool,type=GenericObjectPool] registerMBean
* javax.management.MBeanPermission org.apache.commons.pool2.impl.GenericObjectPool#-[org.apache.commons.pool2:name=pool,type=GenericObjectPool] unregisterMBean
* javax.management.MBeanServerPermission createMBeanServer
* javax.management.MBeanTrustPermission register
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed smartobserve-anomaly-detection with folder name smartobserve-anomaly-detection

Install a plugin using Maven coordinates

The smartobserve-plugin install tool also allows you to specify Maven coordinates for available artifacts and versions hosted on Maven Central. The tool parses the Maven coordinates you provide and constructs a URL. As a result, the host must be able to connect directly to the Maven Central site. The plugin installation fails if you pass coordinates to a proxy or local repository.

Usage

bin/smartobserve-plugin install <groupId>:<artifactId>:<version>

Example

$ sudo ./smartobserve-plugin install org.smartobserve.plugin:smartobserve-anomaly-detection:2.2.0.0
-> Installing org.smartobserve.plugin:smartobserve-anomaly-detection:2.2.0.0
-> Downloading org.smartobserve.plugin:smartobserve-anomaly-detection:2.2.0.0 from maven central
[=================================================] 100%   
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessClassInPackage.sun.misc
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
* javax.management.MBeanPermission org.apache.commons.pool2.impl.GenericObjectPool#-[org.apache.commons.pool2:name=pool,type=GenericObjectPool] registerMBean
* javax.management.MBeanPermission org.apache.commons.pool2.impl.GenericObjectPool#-[org.apache.commons.pool2:name=pool,type=GenericObjectPool] unregisterMBean
* javax.management.MBeanServerPermission createMBeanServer
* javax.management.MBeanTrustPermission register
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed smartobserve-anomaly-detection with folder name smartobserve-anomaly-detection

Restart your SmartObserve node after installing a plugin.

Installing multiple plugins

Multiple plugins can be installed in a single invocation.

Usage

bin/smartobserve-plugin install <plugin-name> <plugin-name> ... <plugin-name>

Example

$ sudo ./smartobserve-plugin install analysis-nori repository-s3

Remove

You can remove a plugin that has already been installed with the remove option.

Usage

bin/smartobserve-plugin remove <plugin-name>

Example

$ sudo ./smartobserve-plugin remove smartobserve-anomaly-detection
-> removing [smartobserve-anomaly-detection]...

Restart your SmartObserve node after removing a plugin.

Batch mode

When installing a plugin that requires additional privileges that are not included by default, the plugin will prompt you for confirmation of the required privileges. To grant all requested privileges, use batch mode to skip the confirmation prompt.

To force batch mode when installing plugins, add the -b or --batch option:

bin/smartobserve-plugin install --batch <plugin-name>

Available plugins

SmartObserve provides several bundled plugins that are available for immediate use with all SmartObserve distributions except for the minimal distribution. Additional plugins are available but must be installed separately using one of the installation options.

Bundled plugins

The following plugins are bundled with all SmartObserve distributions except for the minimal distribution. If you are using the minimal distribution, you can add these plugins by using one of the installation methods.

¹Dashboard Notebooks was merged in to the Observability plugin with the release of SmartObserve 1.2.0.
²Performance Analyzer is not available on Windows.

Downloading bundled plugins for offline installation

Each bundled plugin can be downloaded and installed offline from a zip file.

The URL for the corresponding plugin can be found in the manifest.yml file located in the root directory of the extracted bundle.

Core plugins

A core (or native) plugin in SmartObserve is a plugin that resides in the SmartObserve core engine repository. These plugins are tightly integrated with the SmartObserve engine, are versioned alongside core releases, and are not bundled by default in the standard SmartObserve distribution.

Downloading core plugins for offline installation

Each core plugins in this list can be downloaded and installed offline from a zip file using the official plugins repository URL template:

https://artifacts.magiccreative.io/releases/plugins/<plugin-name>/<version>/<plugin-name>-<version>.zip

The <plugin-name> corresponds to the name of the bundled plugin (for example, analysis-icu). The <version> must match the version of the SmartObserve distribution (for example, 2.19.1).

For example, use the following URL to download the analysis-icu bundled plugin distribution for SmartObserve version 2.19.1:

https://artifacts.magiccreative.io/releases/plugins/analysis-icu/2.19.1/analysis-icu-2.19.1.zip

Additional plugins

There are many more plugins available in addition to those provided by the default distribution. These additional plugins have been built by SmartObserve developers or members of the SmartObserve community. For a list of additional plugins you can install, see Additional plugins.

Plugin compatibility

You can specify plugin compatibility with a particular SmartObserve version in the plugin-descriptor.properties file. For example, a plugin with the following property is compatible only with SmartObserve 2.3.0:

smartobserve.version=2.3.0

Alternatively, you can specify a range of compatible SmartObserve versions by setting the dependencies property in the plugin-descriptor.properties file to one of the following notations:

• dependencies={ smartobserve: "2.3.0" }: The plugin is compatible only with SmartObserve version 2.3.0.
• dependencies={ smartobserve: "=2.3.0" }: The plugin is compatible only with SmartObserve version 2.3.0.
• dependencies={ smartobserve: "~2.3.0" }: The plugin is compatible with all versions from 2.3.0 up to the next minor version, in this example, 2.4.0 (exclusive).
• dependencies={ smartobserve: "^2.3.0" }: The plugin is compatible with all versions from 2.3.0 up to the next major version, in this example, 3.0.0 (exclusive).

You can specify only one of the smartobserve.version or dependencies properties.

Plugin dependencies

Some plugins extend functionality of other plugins. If a plugin has a dependency on another plugin, you must install the required dependency before installing the dependent plugin. For plugin dependencies, see the manifest file. In this file, each plugin’s dependencies are listed in the depends_on parameter.

Related links

• Observability
• Security Analytics
• Security
• Alerting
• Anomaly detection
• Asynchronous search
• Cross-cluster replication
• Index State Management
• k-NN search
• Learning to Rank
• ML Commons
• Neural search
• Notifications
• SmartObserve Dashboards
• Performance Analyzer
• SQL

Related articles

• Additional plugins
• Mapper-size plugin
• Ingest-attachment plugin