Lucidum has been acquired by Cyderes → [Read the announcement]

Lucidum has been acquired by Cyderes → [Read the announcement]

View Categories

AWS

Estimated Reading Time: 7 min read

AWS (Amazon Web Services) is a comprehensive, evolving cloud computing platform provided by Amazon that includes a mixture of infrastructure as a service (IaaS), platform as a service (PaaS) and packaged software as a service (SaaS) offerings. Meridian uses the AWS connector to ingest data from the following AWS services:

  • AWS Config
  • AWS Database Services (DynamoDB)
  • AWS EC2 Instance
  • AWS ECS Container Service
  • AWS EKS Kubernetes Service
  • AWS Elastic Cache
  • AWS Elastic Network Interface
  • AWS ELB Load Balancer
  • AWS GovCloud
  • AWS IAM User/Policy
  • AWS Inspector
  • AWS Lambda Function
  • AWS Logs (CloudWatch/CloudTrail)
  • AWS Organizations
  • AWS Route53
  • AWS S3 File Storage
  • AWS Security Groups
  • AWS Workspaces

Requirements #

There are two ways to use the AWS Connector in Meridian:

  1. You can configure a single connector profile to ingest data data from multiple AWS accounts. To do this, you define cross-account access that allows Meridian to ingest information from multiple AWS accounts.

    NOTE: To maintain ingestion performance, Meridian recommends that you assign no more than five AWS accounts per Meridian connector profile.

    NOTE: To ingest data from AWS GovCloud, you cannot configure a single connector profile to ingest data from multiple AWS account. Cross-partition role-assuming is not supported for AWS GovCloud.

  2. You can configure a single connector profile to ingest data from a single AWS account you want to ingest information from. You can then create multiple connector profiles, each ingesting data from a specific AWS account.

For both options, you must then configure the AWS connector in Meridian and start ingesting data from AWS. This is described in each section.

Configure a Single Connector Profile to Ingest from Multiple AWS Accounts #

If you want to use a single Meridian connector profile to ingest data from multiple AWS accounts, perform these steps.

NOTE: To maintain ingestion performance, Meridian recommends that you assign no more than five AWS accounts per Meridian connector profile.

Video (8 minutes) #

https://lucidum.io/wp-content/uploads/configuring_aws_connector_for_multi_acounts_compressed.mp4?_=3

Required Configuration Tasks in AWS (4-5 minutes for each AWS account you want Meridian to ingest from) #

1. Copy JSON file from Meridian’s Private github.
2. Get the 12-digit AWS account ID for your Meridian instance.
  • Ask your Meridian account representative
3. Log in to the AWS console
4. Note the AWS region in the URL.
  • For example, if the URL is:

https://us-east-1.console.aws.amazon.com/billing/home#/account

  • The region is “us-east-1”
  • Write down the region. You will need it later.
5. Click IAM > Policies > Create Policy.
6. In the Specify Permissions page:
  • Click JSON
  • Delete the contents of the Policy Editor pane
  • Paste the JSON file you copied earlier
  • Click Next
7. In the Review and create page:
  • Policy Name. Enter a name for the new policy. For example, lucidum_readonly
  • Click Create policy.
8. Click IAM > Roles > Create role.
9. In the Select trusted entity page:
  • Trust entity type. AWS Account.
  • Click Another AWS Account.
  • Account ID. Enter the 12-digit AWS account ID for your Meridian instance that you retrieved in step 2.
  • Click Require external ID.
  • External ID. Enter an external name for the role. The default value in Meridian is lucidum-assume-role.
  • Note the External ID value. You will need it later.
  • Click Next.
10. In the Add Permission page:
  • In the Search bar, enter the name of the policy.
  • Click the checkbox next to the policy name.
  • Click Next.
11. In the Name, review, and Create page:
  • Role name. Enter a name for the role (for example, lucidum-assume-role).
  • Note the Role name. You will need it later.
  • Click Create role.
12. Click IAM > Roles.
13. In the Role page:
  • Click on the new role.
  • Click Edit
  • Maximum session duration. Set to 4 hours.
  • Note the value of Maximum session duration. You will need it later.
  • Click Save Changes.
14. Click the Admin name in the upper right corner.
  • Note the AWS account ID. It should be the same value as the account ID you retrieved from your Meridian account representative. You will need it later.
15. Repeat these steps for each AWS account you want to ingest data from.
  • Remember to use the same Policy Name and Role Name (as well as the same settings) for each AWS account.

Configuring the AWS Connector in Meridian (3 minutes) #

  1. Login to Meridian
  2. In the left menu bar, click the Connectors icon
  3. In the Connectors page, click on the tile for AWS.
  4. In the Settings page, go to the Configured Profiles Click the Add New Profile (plus-sign) icon.
  5. In the right pane, provide values in the following fields:
Field Description Example
Profile Name A name for the Connector profile. lucidum_cross_account
External Role ID Provide the value of External ID from step 9 above. lucidum-access
Role Duration Provide the value from Maximum session duration in step 13 above 4
Role Name Provide the value from Role name from step 11 above lucidum-assume-role
AWS Accounts One or more AWS account IDs from step 14 above.After entering an account ID, press the Return key. You can then enter another account ID.

NOTE: To maintain ingestion performance, Meridian recommends that you assign no more than five AWS accounts per Meridian connector profile.

 365329389986, 456789239998, 769943206052
Auto Scaling Regions Optional Specify the regions where you have implemented AWS Auto Scaling. us-east1
AWS Regions Enter the AWS Region codes from step 4 above.After entering an AWS region, press the Return key. You can then enter another AWS region. us-east1, us-east2
  1. Click Save.
  2. Click Test.

Configure a Single Connector Profile for a Single AWS Account #

NOTE: Use this method to ingest data from AWS GovCloud.

If you want to use Meridian to ingest data from only a single AWS account or if you want to create a connector profile for each AWS account, perform these steps.

Video (6 minutes) #

https://lucidum.io/wp-content/uploads/configuring_aws_connector_for_single_account_compressed3.mp4?_=4

Required Configuration Tasks in AWS (4-5 minutes) #

1. Copy JSON file from Meridian’s Private github.
2. Get the 12-digit AWS account ID for your Meridian instance.
  • Ask your Meridian account representative
3. Log in to the AWS console.
4. Note the AWS region in the URL.

NOTE: For AWS GovCloud, the region will look like “us-gov-west-1” or “us-gov-east-1”
5. Click IAM > Policies > Create Policy.
6. In the Specify Permissions page:
  • Click JSON
  • Delete the contents of the Policy Editor pane
  • Paste the JSON file you copied earlier
  • Click Next
7. In the Review and create page:
  • Policy Name. Enter a name for the new policy. For example, lucidum_readonly.
  • Click Create policy.
8. Click IAM > Users > Create user.
9. In the User Details page:
  • User name. Enter a name for the use (for example, lucidum_connector).
  • Click Next.
10. In the Set Permission page:
  • In Search bar, enter the name of the policy.
  • Click the checkbox next to the policy name.
  • Click Next.
11. In the Review, and Create page:
  • Click Create user.
12. Click IAM > Users.
13. In the Users page:
  • Click on the name of the new user.
14. In the IAM > Users > user name page:
  • Click the Security Credentials tab.
  • In the Access keys pane, click Create access key.
15. In the Access key best practices & alternatives page:
  • Click Other
  • Click Next.
16. In the Set description tag page:
  • Enter an optional description for the access key.
  • Click Create access key.
17. In the Retrieve access key page:
  • Click Download .csv file to download the access key and secret access key. Store the file safely. You will need these keys later.
  • Click Done.
18. Click the Admin name in the upper right corner.
  • Note the AWS Account ID. It should be the same value as the account ID you retrieved from your Meridian account representative. You will need it later.

Configuring the AWS Connector in Meridian (3 minutes) #

  1. Login to Meridian
  2. In the left menu bar, click the Connectors icon
  3. In the Connectors page, click on the tile for AWS.
  4. In the Settings page, go to the Configured Profiles Click the Add New Profile (plus-sign) icon.
  5. In the right pane, provide value in the following fields:
Field Description Example
Profile Name A name for the Connector profile AWS Single Account
Access Key ID Enter the value for Access Key ID from the .CSV file in step 17 above.For AWS GovCloud, you must provide an AWS access key ID from an IAM user in the GovCloud account. The Keys must be government issues Commercial keys cannot authenticate with government endpoints. AKIAVRUVPPLUQO4ZZ772
Access Key Secret Enter the value for Access Key Secret from the .CSV file in step 17 above.For AWS GovCloud, you must provide an AWS access key ID from an IAM user in the GovCloud account. The Keys must be government issues Commercial keys cannot authenticate with government endpoints. wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
AWS Account Enter the AWS Account ID from step 18 above 769943206052
Auto Scaling Regions Optional. Specify the regions where you have implemented AWS Auto Scaling. us-east1
AWS Regions Enter the AWS Region code from step 4 above us-east1us-gov-east-1
  1. Click Save.
  2. Click Test.

Source Documentation #

Creating a Policy and Creating Roles #

https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_create-console.html

API Documentation #

https://docs.aws.amazon.com/cloudcontrolapi/

https://docs.aws.amazon.com/AWSEC2/latest/APIReference/Welcome.html

https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html

What are your Feelings