Ember Network Configuration

Overview

This page describes networking aspect of Ember configuration.

Each customer case may require slightly different deployment model. Some clients operate on the cloud, while others run on premise or co-lo deployments. The following diagrams show typical Ember components and their connectivity. For simplicity, this diagram does not label internal ports. Depending on system load all components may co-locate on a single server or split up between multiple servers. See architecture documentation for more information.

Please note that arrows show direction of connection (from initiator to acceptor), not a data flow direction. The diagram shows default ports used by components (can be re-configured).

• Red arrows show operator access to web apps. Normally these ports should not be opened to public Internet without additional precautions (e.g. via whitelisted source IPs). We recommend protecting these connections via VPN or SSH tunnel.
• Green arrows show connectivity to market data venues. These are always outbound connections. These connections typically go over VPN or cross-connect. In rare cases (crypto exchanges) they may use TLS over public Internet.
• Blue arrows show TimeBase clients. TimeBase is primarily used for market data distribution. Depending on performance requirements TimeBase can either use TCP (most common case) or unicast/multicast UDP for data distribution.
• Orange arrows show Ember clients who talk to Ember via RPC or subscribe for Ember events.  Depending on requirements Ember can either use TCP (most common case) or unicast/multicast UDP, as well as IPC.

The following sections explore some configuration options in detail.

TimeBase API gateway

By default, TimeBase uses port 8011 (TCP). This default can be changed using TimeBase.port property in QuantServerHome/config/admin.properties file.

TimeBase Topics

Default address for MULTICAST topics: 239.0.1.37:40456

Default ports for SINGLE_PUBLISHER (multi-destination cast) topics: 40491, 40492, 40493, 40494. This is configured by the following Java System properties:

-DTimeBase.transport.aeron.topic.udp.single.publisher.default.port=40491  
-DTimeBase.transport.aeron.topic.udp.single.subscriber.default.port=40492  
-DTimeBase.transport.aeron.topic.udp.single.metadata.publisher.default.port=40493  
-DTimeBase.transport.aeron.topic.udp.single.metadata.subscriber.default.port=40494

Note: each TimeBase topic can use custom ports (see QuantServerHome/topics/<topic-key>.xml).

Ember Message Bus

TCP

TCP message bus can be configured using the following ember.conf stanza:

messageBus : ${template.messageBus.ebit} {
  settings {
    host = "0.0.0.0"
    port = 8989
  }
}

UDP

When running in multi-server mode Ember message bus uses Aeron UDP Streams. UDP ports can be configured in ember.conf:

messageBus {
  settings {
    subscription {
      channel = "aeron:udp?endpoint=10.0.0.243:40450"
      stream = 1
    }
    publication {
      channel = "aeron:udp?control=10.0.0.243:40452"
      stream = 2
    }
  }
}

Please refer to Ember Configuration Reference manual for more information.

Ember Monitor

By default, ES Monitor uses port 8988 (TCP/HTTP). ES Monitor is a Spring Boot application. This default port can be changed using Java System property

server.port=8988

System Operation and Administration

Many routine functions can be performed via web-based applications (e.g. Ember Monitor). However, advanced troubleshooting would require SSH access to servers or containers running Ember. Ember includes several CLI diagnostic tools that do not yet have web interface.  For this reason, you need to provision a secure way to SSH into ember instances. For example, using Bastion Host pattern.

TLS

We recommend using TLS encryption for all external connections. However, using TLS for internal data transit can severely impact system performance and is not recommended for high performance setups.

Other ports

This section describes port configuration that are used by Ember but not required to be available outside of Ember LAN/VPC.

Heathcheck

network.healthcheck {
  host = "HOST"
  port = PORT
}

FIX Agent

There is a little component that allows to run FIX Gateway standalone:

fix.agent {
  host = localhost
  port = 8985
  idleStrategy = ${template.idleStrategy.daemon}
}

Summary of Firewall Rules

Inbound connections

Port	Source IP	Destination	Comment
SSH	Bastion Host	All instances	Should be accessed from bastion only
8899	Operators	Ember Monitor app	Can be accessed from bastion or SSH tunnel instead.
8080	Operators	TimeBase Admin app	Can be accessed from bastion or SSH tunnel instead.
XXXX	FIX Market Data or FIX Order Entry Clients	Ember FIX Gateway	Each FIX client will require a designated port that can be whitelisted with client source IP
XXXX	FIX Drop Copy Clients	Ember FIX Drop Copy Gateway	Each FIX client will require a designated port that can be whitelisted with client source IP

Outbound connections

We typically recommend allowing all outbound connections. In special cases, whitelist of DNS names can be provided on user request.

Examples of outbound connections performed by Ember and TimeBase components:

• TimeBase license check (can be disabled for offline licenses)
• Central Security Metadata queries
• Market Data and Order Entry connectors
• Data Warehousing of Ember journal into S3 or some database
• Clock synchronization and OS updates