Storing Secrets in AWS Secrets Manager

Overview

This document describes a method of storing secret information outside of Deltix Execution Server configuration.

Introduction

Storing secrets, like API access keys, or login credentials inside system configuration files can quickly become a maintenance and security issue. Even when stored in encrypted form:

• It is hard to update the secrets stored in various configuration files on different servers

• A malicious party may obtain access to server’s file system and decrypt a secret by decompiling encryption algorithm.

There is a trend of employing specialized “secret as a service” solutions to securely store and access private information.

Deltix Execution Server (Ember) offers integration with AWS Secrets Manager, an AWS service for managing secrets.

On startup Execution Server retrieves secrets from the Secrets Manager. In low-latency systems, like Execution Server, we cannot afford to keep secrets inside Secrets Manager. In order to significantly speed up time required to digitally sign and submit trade orders secrets, like exchange API keys, are kept in Execution Server memory.

To store secrets in the AWS Secrets Manager the user will need to do the following:

• Create an AWS user account for managing secrets

• Using Secrets Manager console or CLI add the secrets, create a role that allows to read secrets and assign it to your EC2 instance running Ember Server

• Modify ember.conf to include AWS configuration and replace each secret in ember.conf with AWS URI pointing to the location of the corresponding secret in the Secrets Manager.

Step-by-step Instructions

Steps to configure AWS using console:

Open AWS Secrets Manager console, select "Store new secret" and then "Other type of secrets". Add secret keys and values, then click Next:

Specify secret name, for instance, for the secret with all the kraken connector secret attributes: deltix/connectors/kraken.

Create new policy that grants GetSecretValue permission for all secrets with names that start with "deltix/":

Create an IAM role and attach this policy to the role:

Select EC2 instance with Ember Server and attach your DeltixSecretsReader role to it:

Now modify Ember Server ember.conf file as described below and start Ember.

AWS configuration in ember.conf

Specify AWS region and optionally access and secret keys of user with read access to the Deltix secrets in ember.conf:

secrets: {
   aws: {
      region="us-east-2"
      accessKey="***"
      secretKey="***"
   }
}

Access key and secret key are optional and should be omitted when access to secrets is granted to a role assigned to Ember Server EC2 instance. When Ember is expected to access secrets as AWS user, the user access key and secret key need to be specified in ember.conf.  Their values can be encrypted using Deltix mangle tool like this:

export EMBER_HOME=<ember_home>
/deltix/ember/bin/mangle <access_key>

Secrets configuration in ember.conf

Specify secret values stored in the AWS Secrets Manager in ember.conf in this AWS secret URI form:

aws:/<secret_name>/<secret-key>

When secret is in the form of URI we will look for it in configured AWS Secrets Manager and fail when it is not found. When secret is not in this form we will decrypt and use the value in the config as we do now.

The secrets' names in the Secrets Manager should be similar to their path in ember.conf. We recommend grouping secrets of the same entity under the same Secrets Manager secret, and use a dot-separated secret key string to identify each secret value. For instance, all the KRAKEN connector secrets would be stored in the AWS under the same secret name deltix/connectors/KRAKEN, under different secret keys: apiKey, apiSecret, extraApiKeys.0.apiKey, extraApiKeys.1.apiKey, etc. So the AWS URIs for these secrets in ember.conf would look like this:

...
connectors: {
    KRAKEN: {
      settings : {
        apiKey = "aws:/deltix/connectors.KRAKEN/apiKey"
        apiSecret = "aws:/deltix/connectors.KRAKEN/apiSecret"

        extraApiKeys = [
            {
                apiKey = "aws:/deltix/connectors.KRAKEN/extraApiKeys.0.apiKey"
                apiSecret = "aws:/deltix/connectors.KRAKEN/extraApiKeys.0.apiSecret"
            },
            {
                apiKey = "aws:/deltix/connectors.KRAKEN/extraApiKeys.1.apiKey"
                apiSecret = "aws:/deltix/connectors.KRAKEN/extraApiKeys.1.apiSecret"
            }
        ]
        syncApiKeys = [
            {
                apiKey = "aws:/deltix/connectors.KRAKEN/syncApiKeys.0.apiKey"
                apiSecret = "aws:/deltix/connectors.KRAKEN/syncApiKeys.0.apiSecret"
            },
            {
                apiKey = "aws:/deltix/connectors.KRAKEN/syncApiKeys.1.apiKey"
                apiSecret = "aws:/deltix/connectors.KRAKEN/syncApiKeys.1.apiSecret"
            }
        ]
...