Creating Queries

The following sections describe how to use the Query Tool. You can then create queries to use in Dashboards, Actions, and SmartLabels.

Because queries allow you to find data in the Lucidum database, you might find it helpful to learn about and view the available date in Lucidum. For details about viewing the available data in Lucidum, see:

• Viewing Details about Individual Assets, Users, or Vulnerabilities
• Viewing Data About All Assets, All Users, and All Vulnerabilities
• Viewing Details About Data Sources for Assets and Users

Accessing the Query Tool #

To create a query, either click the button in the feature you are using or click the Query button in the upper right of any page in Lucidum.

• Charts in Dashboards. Clicking the Configure Filters button leads to the Query Tool.
• Actions. Clicking the Configure Filters button leads to the Query Tool.
• SmartLabels. Clicking the Add Rule or Edit Rule button leads to the Query Tool.
• Dashboards. Clicking the Refine Scope icon leads to the Query Tool, where you can create a global filter that applies a pre-filter to all charts in the dashboard.

1. The Query Tool appears.

   

2. The Build a Query page appears. In the Build a Query page, define the following fields:

   • Type of Data. This is the top-level category for each query and specifies the type of Lucidum object you want to get information about. Choices are:

     • Asset. Retrieve information about assets.

     • Asset-IP Mapping. Lucidum uses proprietary machine-learning algorithms to align each asset with an IP address. You can retrieve information about these asset/IP pairs.

     • User. Retrieve information about users.

     • User-IP Mapping. Lucidum uses proprietary machine-learning algorithms to align each user with an IP address. You can retrieve information about these user/IP pairs.

     • Vulnerability. Retrieve information about vulnerabilities.

   • Time Range. You must first select whether you are interested in current values or historic values. The choices are:

     • Current. The default value is from the present day to 7 days old.

     • History. Older than current.

     NOTE: You can customize or view the Time Range values in Settings > System Settings > Data Settings. Current uses the value of Data Lookback in Days.

3. In the Build a Query page, click the Next button. The Build a Current Asset Query page appears.

   

4. Follow the steps in the section Building the Query.

Building the Query #

1. In the both the Build a Query page and the Configure Filters pages, you follow the same steps to build a query.

   

2. Provide values in the following fields:

   • Field. In this field, you select a field for the query. Fields are one or more properties of an asset or user, like first time seen”, “ip address”, “county code”, or “risk score”. For details about fields, see the section on Fields. Fields map to column names in a Lucidum database table. For a list of possible fields, see the chapter on Fields and Regular Expressions

   • Operators. In this field, you select an Operator. Operators define the relationship between the fields and the values. The list of operators is dependent on the value you selected in Field. For example, if you select a numeric field, the operators include “equals”, “less than”. For details on operators for each data type, see the section on Operators. For a list of possible operators, see Data Types and Operators.

   • Values. In this field, you select a value. The value is the value stored in a field. For example, if the field is “email”, the value might be “[email protected]”. The list of available values is dependent on the value you selected in Field and the value you selected in Operators.

3. For the example query in the screen capture above, we specified:

   • Type of Data. Asset. We are interested in assets.

   • Time Range. Current. We will collect data from the database for the current .

   • Field. Risk Score. We will examine the field “Risk Score”.

   • Operator. is greater than or equal to. Because the Operator field includes operators for numeric values, we know “Risk Score” is a numeric value. We are looking for values of “Risk Score” that are greater than or equal to the value in Value.

   • Value. 15.65. Lucidum populates the list of values with all the values for “Risk Score” in the assets table for the current . We chose 15.65. So we are looking for values of “Risk Score” that are greater than or equal to 15.65.

4. To see the results of the query, click the Show Result button. The Query Result page appears.

   

Fields #

The Types of Data field specifies an object to examine. Choices are:

• Asset

• Asset-IP Mapping

• User

• User-IP Mapping

• Vulnerability

When building a query, Fields are characteristics of the Lucidum objects. For example, characteristics of a user are the user’s name and email address. A characteristic of an asset is the asset’s IP address. Usually, a field maps to a column name in a Lucidum database.

Lucidum ingests information about assets, users, and data from your environment. Lucidum correlates that information and uses machine learning to enrich that information to provide you with details about assets, users, asset-IP mapping, user-IP mapping, and vulnerabilities.

For details on the standard list of fields in Lucidum and special characters you can use in queries, see the appendix on Fields and Regular Expressions.

You might see fields in the appendix that don’t appear in your Lucidum system. This means that Lucidum has not fetched that data from your environment, either because you have not yet configured the connector or because your environment doesn’t include that type of asset.

You might see fields called “Extra Fields” in your Lucidum system that don’t appear in the list of fields in the appendix. This means that Lucidum has fetched data from your environment that is either specific to your environment or not available in all environments.

Creating Favorite Fields #

Lucidum allows you to add frequently used fields to a list of Favorites.

If you frequently use a field, you can click the star icon next to the field name. The field then appears in the Favorite category at the top of the list of fields.

Operators #

Operators define the relationship between the fields and the values.

Operators are dependent upon the data type.

For example:

• Numeric data includes operators like “is equal to” or “is greater than”.
• Date and time data includes operators like “within past”.
• Text data includes operators like “match” or “is equal to”.

For details on each data type and its operators, see the appendix on Data Types and Operators.

Values #

The list of values is dependent on the Lucidum object, the Field, and the Operator. Lucidum populates the list of Values after you select a Lucidum object, Field, and Operator.

You can also use regular expressions in the Values field. For details, see the appendix on Fields and Regular Expressions.

For example, if you selected the following:

• Build Query from (Lucidum object). Asset
• Field. Risk Score
• Operator. is greater than or equal to
• Value. 15.65

The Build Query from field indicates that we are interested in Assets.

Field indicates that we want to retrieve assets with a specific Risk Score.

Because Operators includes operators for numeric values, we know Risk Score is a numeric value.

Lucidum populates the Values with all the numeric values for all Risk Scores in the Assets database table. In our example above, we chose 15.65.

Writing Queries that Use AND and OR #

Lucidum allows you to create multi-part queries. To do this, you can use AND and OR keywords.

• The link for AND creates an AND condition.

• The link for OR creates an OR condition.

AND Condition #

An AND condition specifies that Lucidum should retrieve all records that match all conditions. You can specify as many conditions as you choose.

For example:

This query specifies that we are interested in all assets:

• where the OS is Windows Server 2019

AND

• the department is HR

Any asset that matches both criteria will be included in the Query Results page.

• On this Lucidum system, 747 assets are running Windows Server 2019.

• However, only 44 assets have both are running Windows Server 2019 and are in the HR department.

• Therefore, the Query Results page displays 44 assets.

OR Condition #

An OR condition specifies that Lucidum should retrieve all records that match at least one of multiple conditions. You can specify as many conditions as you choose.

For example:

This query specifies that we are interested in all assets that:

• have an Operation System of Windows Server 2016

OR

• have an Operation System of Windows Server 2019

Any asset that matches either of these criteria will be included in the Query Results page.

For example, using the query above, the Query Results page looks like this:

• 722 assets are running Windows Server 2016

• 747 assets are running Windows Server 2019

• The Query Results page displays 1469 assets

Viewing Query Results #

The Show Results (checklist) icon opens the Query Results page. The Query Results page displays a table populated with the results of the query for the chart.

Using an example query:

• This query specifies that we are interested in all assets that:

  • have an OS and Version that matches Windows Server 2016

OR

• • have an OS and Version that matches Windows Server 2019

• Any asset that matches the criteria will be included in the Query Results page.

• For example, using the query above, the Query Results page looks like this:

  

• The Query Results page displays a list of assets that are running either Windows 2019 or Windows 2016.

  • You can sort the page by the column headings.

  • To see additional details about an asset, user, or vulnerability, click the > (right arrow) to see the Details page.

  • Notice the Changed Fields column. If a value appears in this column, you can view details about the changed fields in the Field Change History tab in the Viewing Details page. These are the changed fields between the latest ingestion and the ingestion before the last ingestion.

In the Query Results page, you can perform the following:

For the current page of results, you can:

• Columns. Add or remove columns from the page.

• Export. Download the results as a CSV file..

For each column name, you can:

• Sort by ASC. Sort the results by this column, in ascending order.

• Sort by DESC. Sort the results by this column, in descending order.

• Pin to left. The column is pinned to the left border. When you scroll left to right to view all the columns, this column stays on the left border.

• Pin to right. The column is pinned to the right border. When you scroll left to right to view all the columns, this column stays on the right border.

• Filter. Allows you to filter the table by one or more columns in the results table.

• Hide Column. Removes the column from the page.

• Manage columns. You can include or not include one or more columns in the results table.

If you select one or more checkboxes in the Query Results page, you can also:

• Create a recurring action with this query. Use the results of the query to create an action. Upon selecting this option, Lucidum displays the Create a new action page, where you can define the action. For details on actions, see Actions.

• Create a one-time action with selected data. You can select one or more records and create an action that uses selected records and runs only once. For details on actions, see Actions.

• Assign a Tag. Apply a tag. Tags are defined in the Tag Management page (Settings > Tag Management), can be manually applied to assets and users, and can be selected as a fields in a query. For details, see the section on tags in the manual Streamlining Queries with Smart Labels and Tags.

Details page #

In Lucidum, you can view details about the data for a single asset, user, or vulnerability. When we go to the row for asset MOSGOOD-ZXBTNV5 and click on click on the > icon, we see the Details page.

There are three types of data in Lucidum, all of which you can view in Lucidum and use in dashboards:

• Data Source. Data sources map to connectors. For each asset, user, and vulnerability, you can view all the data sources from which Lucidum ingested data about the asset, user, or vulnerability. You can select a Data Source to exactly see which raw data was provided by that data source.

• Lucidum Data Group. Enriched data about assets, user, and vulnerabilities. Lucidum ingests data from multiple data sources and uses machine learning to enrich and normalize that data. The Lucidum Data Group tab allows you to view this enriched data for each asset, user, and vulnerability. Data in the Lucidum Data Group appears as fields in the Query tool.

• Field Change History.  Displays fields that have changed, each field’s current value and each field’s previous value. These are the fields that have changed in the last seven days.

For more information on the Details page and its tabs, see Viewing Details about Individual Assets, Users, or Vulnerabilities.

Exporting a Query to a .CSV File #

From the Query button, you can export queries to a .CSV file for use in reports or analysis outside of Lucidum.

To do this:

1. 1. Create a query or choose a Saved Query or choose a query from Query History.

   2. To see the results of the query, click the Show Result button. The Query Result page appears.

   3. In the Query Results page, click the Export icon in the upper right corner. Choose Export.

      

1. The query and its results are saved to a .CSV file on your local computer.

Using Saved Queries and Query History #

The Query page includes fields for re-using queries:

• Saved Query. Leads to a list of saved queries. Saved queries are associated with your login. You cannot view the saved queries from other Lucidum users.

• Query History. Leads to a list of cached queries. Query history is not associated with your login but instead includes all queries from all Lucidum users on the current Lucidum system. Query history is deleted when the Lucidum system is rebooted or restarted.

For details on creating, using, and managing saved queries and historical queries, see Saved Queries and Historical Queries.

NOTE: Saved Queries are not dynamically updated. This means that if you include a Save Query in an Action or a Chart and then later update the definition of the Saved Query, the Action or Chart will continue to use the previous definition, not the updated definition.

Using Smart Labels and Tags #

Lucidum includes two types of user-defined, query-based fields to make repetitive tasks easier and to help organize assets and users. For details on creating, using, and managing Smart Labels and tags, see Streamlining Queries with Smart Labels and Tags.

NOTE: Unliked Saved Queries, SmartLabels are dynamically updated. This means that if you include a SmartLabel in an Action or a Chart and then later update the definition of the SmartLabel, the Action or Chart will automatically use the updated definition.

Creating a Global Filter for a Dashboard #

A global filter is a parent filter that is applied to all queries and all charts in a specific Dashboard.

1. To create a global filter:

2. Open the dashboard you want to add a filter to.

   

3. Click on the Refine Scope (funnel) button.

4. In the Build a Query page, create a query as you normally would.

   

5. Click Apply to apply the query to all charts in the current dashboard.

6. The new filter appears to the left of the Refine Scope button.

7. The new filter is applied to all the charts and queries in the dashboard.

   

8. In our example, we created a query that filtered assets by the Country Code match United States. Notice how the value in the “Workstations” chart, the “Servers & VMn” chart, and the Containers & Microservices chart has changed after we applied the global filter. This is because all queries and charts include only assets with the Country Name “United States”.