Data sources
SmartObserve data sources are the applications that SmartObserve can connect to and ingest data from. Once your data sources have been connected and your data has been ingested, it can be indexed, searched, and analyzed using REST APIs or the SmartObserve Dashboards UI.
This documentation focuses on using the OpenSeach Dashboards interface to connect and manage your data sources. For information about using an API to connect data sources, see the developer resources linked under Next steps.
Prerequisites
The first step in connecting your data sources to SmartObserve is to install SmartObserve and SmartObserve Dashboards on your system. Refer to the installation instructions for information.
Once you have installed SmartObserve and SmartObserve Dashboards, you can use Dashboards to connect your data sources to SmartObserve and then use Dashboards to manage data sources, create index patterns based on those data sources, run queries against a specific data source, and combine visualizations in one dashboard.
Configuration of the YAML files and installation of the dashboards-observability and smartobserve-sql plugins is necessary. For more information, see SmartObserve plugins.
To securely store and encrypt data source connections in SmartObserve, you must add the following configuration to the smartobserve.yml file on all the nodes:
plugins.query.datasources.encryption.masterkey: "YOUR_GENERATED_MASTER_KEY_HERE"
The key must be 16, 24, or 32 characters. You can use the following command to generate a 24-character key:
openssl rand -hex 12
Generating 12 bytes results in a hexadecimal string that is 12 * 2 = 24 characters.
Permissions
To work with data sources in SmartObserve Dashboards, you must be assigned the correct cluster-level data source permissions.
Types of data streams
To configure data sources through SmartObserve Dashboards, go to Management > Dashboards Management > Data sources. This flow can be used for SmartObserve data stream connections. See Configuring and using multiple data sources.
Alternatively, if you are running SmartObserve Dashboards 2.16 or later, go to Management > Data sources. This flow can be used to connect Amazon Simple Storage Service (Amazon S3) and Prometheus. See Connecting Amazon S3 to SmartObserve and Connecting Prometheus to SmartObserve for more information.
Next steps
- Learn about managing index patterns through SmartObserve Dashboards.
- Learn about indexing data using Index Management through SmartObserve Dashboards.
- Learn about how to connect multiple data sources.
- Learn about how to connect SmartObserve and Amazon S3 and SmartObserve and Prometheus using the SmartObserve Dashboards interface.
- Learn about the Integrations plugin, which gives you the flexibility to use various data ingestion methods and connect data to SmartObserve Dashboards.