Aerospike
Configuration
First of all, we need to define an Aerospike connection in our dozer-config.yaml
which can then be used as a sink.
An Aerospike connection configuration has the following parameters.
Parameter Name | Type | Description |
---|---|---|
hosts | String | The host and port of the Aerospike server. |
namespace | String | The namespace for your Aerospike connection. |
sets | List | A list of sets within the namespace. |
replication | Object (Optional) | Server configuration for receiving replication messages from Aerospike during ingestion. |
Replication Parameters
server_address
:The address of the server used to receive replication messages from Aerospike. Default is
0.0.0.0.0
.server_port
:The port of the server. Default is
5929
.
Example:
connections:
- config: !Aerospike
hosts: localhost:3000
namespace: test
sets:
- customers
- transactions
name: aerospike
After an Aerospike connection has been setup,the following configuration block can be used in dozer-config.yaml
to define a new Aerospike sink.
This particular configuration is designed to replicate data from a table named transactions
from the source, to the set transactions
within the test
namespace (after denormalisation) in the Aerospike connection referenced by the name aerospike
.
Example:
sinks:
- name: transactions
config: !Aerospike
connection: aerospike
tables:
- namespace: test
set_name: transactions
source_table_name: transactions
denormalize:
- from_namespace: test
from_set: customers
key: CUSTOMER_ID
columns:
- PHONE_NUMBER
Parameters
Parameter Name | Type | Description |
---|---|---|
connection | String | The corresponding connection for Aerospike which is being used as a destination |
tables | List | List of tables to write to on destination sink. Parameters for table configuration. |
n_threads | Integer | The maximum number of threads to spawn to write to Aerospike sink. |
Table Parameters
Parameter Name | Type | Description |
---|---|---|
namespace | String | The namespace for the table. |
set_name | String | The set within the namespace for the table. |
source_table_name | String | The name of the table in the source database. |
denormalize | List | A list of denormalization rules to apply to the data before writing it to the destination. |
Denormalisation Parameters
Parameter Name | Type | Description |
---|---|---|
from_namespace | String | The namespace for the source record. |
from_set | String | The set within the namespace for the source record. |
key | String | The key to use to join the source and destination records. |
columns | List | A list of columns to copy from the source record to the destination record. |
In essence, it is like a left-join, so the example sink configuration is equivalent to the following SQL query:
SELECT t.*, c.PHONE_NUMBER
FROM transactions AS t
LEFT JOIN customers AS c ON t.CUSTOMER_ID = c.PK -- PK being the aerospike key here