Installing Canton

This guide will guide you through the process of setting up your Canton nodes to build a distributed Daml ledger. You will learn

  1. How to setup and configure a domain
  2. How to setup and configure one or more participant nodes

Note

As no topology is the same, this guide will point out different configuration options as notes wherever possible.

This guide uses the example configurations you can find in the release bundle under example/03-advanced-configuration and explains you how to leverage these examples for your purposes. Therefore, any file named in this guide will refer to subdirectories of the advanced configuration example.

Downloading Canton

The Canton Open Source code is available from Github. You can also use our Canton Docker images by following our Docker instructions.

Daml Enterprise includes an enterprise version of the Canton ledger. If you have entitlement to Daml Enterprise you can download the enterprise version of Canton by following the Installing Daml Enterprise instructions and downloading the appropriate Canton artifact.

Your Topology

The first question we need to address is what the topology is that you are going after. The Canton topology is made up of parties, participants and domains, as depicted in the following figure.

../../_images/topology.svg

The Daml code will run on the participant node and expresses smart contracts between parties. Parties are hosted on participant nodes. Participant nodes will synchronise their state with other participant nodes by exchanging messages with each other through domains. Domains are nodes that integrate with the underlying storage technology such as databases or other distributed ledgers. As the Canton protocol is written in a way that assumes that Participant nodes don’t trust each other, you would normally expect that every organisation runs only one participant node, except for scaling purposes.

If you want to build up a test-network for yourself, you need at least a participant node and a domain.

Environment Variables

For our convenience in this guide, we will use a few environment variables to refer to a set of directions. Please set the environment variable “CANTON” to point to the place where you have unpacked the canton release bundle.

cd ./canton-X.Y.Z
export CANTON=`pwd`

And then set another variable that points to the advanced example directory

export CONF="$CANTON/examples/03-advanced-configuration"

Selecting your Storage Layer

In order to run any kind of node, you need to decide how and if you want to persist the data. You currently have three choices: don’t persist and just use in-memory stores which will be deleted if you restart your node or persist using Postgres or Oracle databases.

For this purpose, there are some storage mixin configurations (storage/) defined. These storage mixins can be used with any of the node configurations. The in-memory configurations just work out of the box without further configuration. The database based persistence will be explained in a subsequent section, as you first need to initialise the database.

The mixins work by defining a shared variable which can be referenced by any node configuration

storage = ${_shared.storage}
storage.parameters.databaseName = "participant1"

If you ever see the following error: Could not resolve substitution to a value: ${_shared.storage}, then you forgot to add the persistence mixin configuration file.

Note

Please also consult the more detailed section on persistence configurations.

Persistence using Postgres

While in-memory is great for testing and demos, for more serious tasks, you need to use a database as a persistence layer. Both the community version and the enterprise version support Postgres as a persistence layer. Make sure that you have a running Postgres server and you need to create one database per node. The recommended Postgres version to use is 11, as this is tested the most thoroughly.

The Postgres storage mixin is provided by the file storage/postgres.conf.

If you just want to experiment, you can use Docker to get a Postgres database up and running quickly. Here are a few commands that come in handy.

First, pull Postgres and start it up.

docker pull postgres:11
docker run --rm --name pg-docker -e POSTGRES_PASSWORD=docker -d -p 5432:5432 postgres:11

Then, you can run psql using:

docker exec -it pg-docker psql -U postgres -d postgres

This will invoke psql interactively. You can exit the prompt with Ctrl-D. If you want to just cat commands, change -it to -i in above command.

Then, create a user for the database using the following SQL command

create user canton with encrypted password 'supersafe';

and create a new database for each node, granting the newly created user appropriate permissions

create database participant1;
grant all privileges on database participant1 to canton;

These commands create a database named participant1 and grant the user named canton access to it using the password supersafe. Needless to say, you should use your own, secure password.

In order to use the storage mixin, you need to either write these settings into the configuration file, or pass them using environment variables:

export POSTGRES_USER=canton
export POSTGRES_PASSWORD=supersafe

If you want to run also other nodes with Postgres, you need to create additional databases, one for each.

You can reset the database by dropping then re-creating it:

drop database participant1;
create database participant1;
grant all privileges on database participant1 to canton;

Note

The storage mixin provides you with an initial configuration. Please consult the more extended documentation for further options.

If you are setting up a few nodes for a test network, you can use a little helper script to create the SQL commands to setup users and databases:

python3 examples/03-advanced-configuration/storage/dbinit.py \
   --type=postgres --user=canton --password=<choose-wisely> --participants=2 --domains=1 --drop

The command will just create the SQL commands for your convenience. You can pipe the output directly into the psql command

python3 examples/03-advanced-configuration/storage/dbinit.py ... | psql -p 5432 -h localhost ...

Important

This feature is only available in Canton Enterprise

Persistence using Oracle

The enterprise version of Canton comes with default configuration mixins using Oracle as a database backend: oracle-participant.conf and oracle.conf, which can be found in ./examples/03-advanced-configuration/storage. The former is used for a participant that requires two schemas / users to store the ledger API server data and the Canton sync service data.

The files require you to provide the necessary environment variables ORACLE_USER, ORACLE_USER_LAPI, ORACLE_PASSWORD, ORACLE_DB, ORACLE_HOST, ORACLE_PORT.

Setting up a Participant

Now that you have made your persistence choice (assuming Postgres here), you could start your participant just by using one of the example files such as $CONF/nodes/participant1.conf and start the Canton process using the Postgres persistence mixin:

$CANTON/bin/canton -c $CONF/storage/postgres.conf -c $CONF/nodes/participant1.conf

While this would work, we recommend that you rename your node by changing the configuration file appropriately.

Note

By default, the node will initialise itself automatically using the identity commands Topology Administration. As a result, the node will create the necessary keys and topology transactions and will initialise itself using the name used in the configuration file. Please consult the identity management section for further information.

This was everything necessary to startup your participant node. However, there are a few steps that you want to take care of in order to secure the participant and make it usable.

Secure the APIs

  1. By default, all APIs in Canton are only accessible from localhost. If you want to connect to your node from other machines, you need to bind to 0.0.0.0 instead of localhost. You can do this by setting address = 0.0.0.0 within the respective API configuration sections or include the api/public.conf configuration mixin.
  2. The participant node is managed through the administration API. If you use the console, almost all requests will go through the administration API. We recommend that you setup mutual TLS authentication as described in the TLS documentation section.
  3. Applications and users will interact with the participant node using the ledger API. We recommend that you secure your API by using TLS. You should also authorize your clients using either JWT or TLS client certificates. The TLS configuration is the same as on the administration API.
  4. In the example set, there are a set of additional configuration options which allow you to define various JWT based authorizations checks, enforced by the ledger API server. The settings map exactly to the options documented as part of the Daml SDK. There are a few configuration mix-ins defined in api/jwt for your convenience.

Configure Applications, Users and Connection

Canton distinguishes static configuration from dynamic configuration. Static configuration are items which are not supposed to change and are therefore captured in the configuration file. An example is to which port to bind to. Dynamic configuration are items such as Daml archives (DARs), domain connections or parties. All such changes are effected through the administration API or the console.

Note

Please consult the section on the console commands and administration APIs.

If you don’t know how to connect to domains, onboard parties or provision Daml code, please read the getting started guide.

Setting up a Domain

In order to setup a domain, you need to decide what kind of domain you want to run. We provide integrations for different domain infrastructures. These integrations have different levels of maturity. Your current options are

  1. In-Process Postgres based domain (simplest choice)
  2. Hyperledger Fabric based domain
  3. Secure enclave based domain
  4. Ethereum based domain (demo)

This section will explain how to setup an in-process based domain using Postgres. All other domains are a set of microservices and part of the Enterprise edition. In any case, you will need to operate the main domain process which is the point of contact where participants connect to for the initial handshake and parameter download. The details of how to set this up for other domains than the in-process based Postgres domain are covered by the individual documentations.

Note

Please contact us at sales@digitalasset.com to get access to the Fabric, Ethereum or enclave based integration.

The domain requires independent of the underlying ledger a place to store some governance data (or also the messages in transit in the case of Postgres based domains). The configuration settings for this storage are equivalent to the settings used for the participant node.

Once you have picked the storage type, you can start the domain using

$CANTON/bin/canton -c $CONF/storage/postgres.conf -c $CONF/nodes/domain.conf

Secure the APIs

  1. As with the participant node, all APIs bind by default to localhost. You need to bind to 0.0.0.0 if you want to access the APIs from other machines. Again, you can use the appropriate mixin api/public.conf.
  2. The administration API should be secured using client certificates as described in TLS documentation section.
  3. The public API needs to be properly secured using TLS. Please follow the corresponding instructions.

Next Steps

The above configuration provides you with an initial setup. Without going into details, the next steps would be:

  1. Configure who can join the domain by setting an appropriate permissioning strategy (default is “everyone can join”).
  2. Configure domain parameters
  3. Setup a service agreements which any client connecting has to sign before using the domain.

Multi-Node Setup

If desired, you can run many nodes in the same process. This is convenient for testing and demonstration purposes. You can either do this by listing several node configurations in the same configuration file or by invoking the Canton process with several separate configuration files (which get merged together).

$CANTON/bin/canton -c $CONF/storage/postgres.conf -c $CONF/nodes/domain.conf,$CONF/nodes/participant1.conf

Setting up a Distributed Domain With a Single Console

If you’re running a domain node in its default configuration as shown previously in this current page, it will have a sequence and mediator embedded and these components will be automatically bootstrapped for you.

If your domain operates with external sequencers and mediators, you will need to configure a domain manager node (which only runs topology management) and bootstrap your domain with at least one external sequencer node and one external mediator node.

First make sure the nodes are fresh and have not yet been initialized:

@ mediator1.health.initialized()
res1: Boolean = false

@ sequencer1.health.initialized()
res2: Boolean = false

@ domainManager1.health.initialized()
res3: Boolean = false

Now you can initialize the distributed domain as follows:

@ domainManager1.setup.bootstrap_domain(Seq(sequencer1), Seq(mediator1))

At this point a participant should be able to connect to a sequencer and operate on that domain:

@ participant1.domains.connect_local(sequencer1)

@ participant1.health.ping(participant1)
res6: concurrent.duration.Duration = 590 milliseconds

Domain managers are configured as domain-managers under the canton configuration. Domain managers are configured similarly to domain nodes, except that there are no sequencer, mediator, public api or service agreement configs.

Please note that if your sequencer is database-based and you’re horizontally scaling it as described under sequencer high availability, you do not need to pass all sequencer nodes into the command above. Since they all share the same relational database, you only need to run this initialization step on one of them.

For non-database-based sequencer such as Ethereum or Fabric sequencers you need to have each node initialized individually. You can either initialize such sequencers as part of the initial domain bootstrap shown above or dynamically add a new sequencer at a later point as described in operational processes.

Setting up a Distributed Domain With Separate Consoles

The process outlined in the previous section only works if all nodes are accessible from the same console environment. If each node has its own isolated console environment, the bootstrapping process must be coordinated in steps with the exchange of data via files using any secure channel of communication between the environments.

Initially the domain manager must transmit its domain parameters from its console by saving the parameters to a file. The domain id, serialized as a string, must also be transmitted.

@ domainManager1.service.get_static_domain_parameters.writeToFile("tmp/domain-bootstrapping-files/params.proto")

@ val domainIdString = domainManager1.id.toProtoPrimitive
domainIdString : String = "domainManager1::12203ab04694f67f472f0a7abb665561778b9d2b48e6fcf6bfd2555d27fdfa7d02b7"

Then the sequencer must receive this file, deserialize it and initialize itself. As part of the initialization, the sequencer creates a signing key pair whose public key it must then transmit via file. Optionally, repeat this for any extra sequencer nodes.

@ val domainParameters = com.digitalasset.canton.protocol.StaticDomainParameters.tryReadFromFile("tmp/domain-bootstrapping-files/params.proto")
domainParameters : com.digitalasset.canton.protocol.StaticDomainParameters = StaticDomainParameters(
  reconciliationInterval = 1m,
  maxRatePerParticipant = 1000000,
  maxInboundMessageSize = 10485760,
  uniqueContractKeys = true,
  requiredSigningKeySchemes = Set(Ed25519, ECDSA-P256, ECDSA-P384),
  requiredEncryptionKeySchemes = Set(ECIES-P256_HMAC256_AES128-GCM),
  requiredSymmetricKeySchemes = Set(AES128-GCM),
  requiredHashAlgorithms = Set(Sha256),
  requiredCryptoKeyFormats = Set(Tink),
  protocolVersion = 3.0.0
)

@ val domainId = DomainId.tryFromString(domainIdString)
domainId : DomainId = domainManager1::12203ab04694...

@ val initResponse = sequencer1.initialization.initialize_from_beginning(domainId, domainParameters)
initResponse : com.digitalasset.canton.domain.sequencing.admin.protocol.InitResponse = InitResponse(
  keyId = "sequencer-id",
  publicKey = SigningPublicKey(id = 1220aef2e281..., format = Tink, scheme = Ed25519),
  replicated = false
)

@ initResponse.publicKey.writeToFile("tmp/domain-bootstrapping-files/seq1-key.proto")

The domain manager must then authorize the sequencer’s key. Optionally, repeat this for any extra sequencer keys.

@ val sequencerPublicKey = SigningPublicKey.tryReadFromFile("tmp/domain-bootstrapping-files/seq1-key.proto")
sequencerPublicKey : SigningPublicKey = SigningPublicKey(id = 1220aef2e281..., format = Tink, scheme = Ed25519)

@ domainManager1.setup.helper.authorizeKey(sequencerPublicKey, "sequencer", SequencerId(domainManager1.id))

Now the mediator also needs to create a signing key pair and transmit it. Optionally, repeat this for any extra mediator nodes.

@ mediator1.keys.secret.generate_signing_key("initial-key").writeToFile("tmp/domain-bootstrapping-files/med1-key.proto")

The domain manager must now authorize the mediator’s key and also authorize the mediator to act as part of this domain. Optionally, repeat this for any extra mediator nodes.

@ val mediatorKey = SigningPublicKey.tryReadFromFile("tmp/domain-bootstrapping-files/med1-key.proto")
mediatorKey : SigningPublicKey = SigningPublicKey(id = 12203c92d895..., format = Tink, scheme = Ed25519)

@ val domainId = DomainId.tryFromString(domainIdString)
domainId : DomainId = domainManager1::12203ab04694...

@ domainManager1.setup.helper.authorizeKey(mediatorKey, "mediator1", MediatorId(domainId))

@ domainManager1.topology.mediator_domain_states.authorize(TopologyChangeOp.Add, domainId, MediatorId(domainId), RequestSide.Both)
res13: com.google.protobuf.ByteString = <ByteString@30e8442a size=558 contents="\n\253\004\n\331\001\n\326\001\n\323\001\022 TF35doyuYtPZ5t7zaV7gpxBVrJscaOPnR...">

After that, still on the domain manager’s console, the domain manager must collect the list of topology transactions, which include all the key authorizations and a few other things it needs to broadcast to all domain members. This is now saved to a file.

@ domainManager1.topology.all.list().collectOfType[TopologyChangeOp.Positive].writeToFile("tmp/domain-bootstrapping-files/topology-transactions.proto")

The sequencer then reads this set of initial topology transactions and sequences it as the first message to be sequenced in this domain. This will allow the domain members whose keys were authorized in previous steps to connect to this sequencer and operate with it. The sequencer will then transmit its connection info.

@ val initialTopology = com.digitalasset.canton.topology.store.StoredTopologyTransactions.tryReadFromFile("tmp/domain-bootstrapping-files/topology-transactions.proto").collectOfType[TopologyChangeOp.Positive]
initialTopology : store.StoredTopologyTransactions[TopologyChangeOp.Positive] = Seq(
  StoredTopologyTransaction(
    sequenced = 2022-07-26T16:39:22.979054Z,
    validFrom = 2022-07-26T16:39:22.979054Z,
    validUntil = 2022-07-26T16:39:22.979054Z,
    op = Add,
    mapping = NamespaceDelegation(
      12203ab04694...,
      SigningPublicKey(id = 12203ab04694..., format = Tink, scheme = Ed25519),
      true
    )
  ),
  StoredTopologyTransaction(
    sequenced = 2022-07-26T16:39:22.980926Z,
    validFrom = 2022-07-26T16:39:22.980926Z,
    validUntil = 2022-07-26T16:39:22.980926Z,
    op = Replace,
    mapping = DomainParametersChange(
      domainManager1::12203ab04694...,
      DynamicDomainParameters(
        participant response timeout = 30s,
        mediator reaction timeout = 30s,
        transfer exclusivity timeout = 1m,
        topology change delay = 0.25s,
        ledger time record time tolerance = 1m
      )
    )
  ),
  StoredTopologyTransaction(
    sequenced = 2022-07-26T16:39:22.982715Z,
    validFrom = 2022-07-26T16:39:22.982715Z,
    validUntil = 2022-07-26T16:39:22.982715Z,
    op = Add,
    mapping = OwnerToKeyMapping(
      DOM::domainManager1::12203ab04694...,
      SigningPublicKey(id = 122034ea9ab8..., format = Tink, scheme = Ed25519)
    )
  ),
  StoredTopologyTransaction(
    sequenced = 2022-07-26T16:39:24.736544Z,
    validFrom = 2022-07-26T16:39:24.736544Z,
    validUntil = 2022-07-26T16:39:24.736544Z,
    op = Add,
    mapping = OwnerToKeyMapping(
      SEQ::domainManager1::12203ab04694...,
      SigningPublicKey(id = 1220aef2e281..., format = Tink, scheme = Ed25519)
    )
  ),
  StoredTopologyTransaction(
    sequenced = 2022-07-26T16:39:25.078892Z,
    validFrom = 2022-07-26T16:39:25.078892Z,
    validUntil = 2022-07-26T16:39:25.078892Z,
    op = Add,
    mapping = OwnerToKeyMapping(
      MED::domainManager1::12203ab04694...,
      SigningPublicKey(id = 12203c92d895..., format = Tink, scheme = Ed25519)
    )
  ),
  StoredTopologyTransaction(
    sequenced = 2022-07-26T16:39:25.156732Z,
    validFrom = 2022-07-26T16:39:25.156732Z,
    validUntil = 2022-07-26T16:39:25.156732Z,
    op = Add,
    mapping = MediatorDomainState(
      Both,
      domainManager1::12203ab04694...,
      MED::domainManager1::12203ab04694...
    )
  )
)

@ sequencer1.initialization.bootstrap_topology(initialTopology)

@ sequencer1.sequencerConnection.writeToFile("tmp/domain-bootstrapping-files/sequencer-connection.proto")

To initialize the mediator, it will need a connection to the sequencer and the domain parameters. Optionally, repeat this for any extra mediator nodes.

@ val sequencerConnection = com.digitalasset.canton.sequencing.SequencerConnection.tryReadFromFile("tmp/domain-bootstrapping-files/sequencer-connection.proto")
sequencerConnection : com.digitalasset.canton.sequencing.SequencerConnection = GrpcSequencerConnection(
  endpoints = http://127.0.0.1:15049,
  transportSecurity = false,
  customTrustCertificates = None()
)

@ val domainParameters = com.digitalasset.canton.protocol.StaticDomainParameters.tryReadFromFile("tmp/domain-bootstrapping-files/params.proto")
domainParameters : com.digitalasset.canton.protocol.StaticDomainParameters = StaticDomainParameters(
  reconciliationInterval = 1m,
  maxRatePerParticipant = 1000000,
  maxInboundMessageSize = 10485760,
  uniqueContractKeys = true,
  requiredSigningKeySchemes = Set(Ed25519, ECDSA-P256, ECDSA-P384),
  requiredEncryptionKeySchemes = Set(ECIES-P256_HMAC256_AES128-GCM),
  requiredSymmetricKeySchemes = Set(AES128-GCM),
  requiredHashAlgorithms = Set(Sha256),
  requiredCryptoKeyFormats = Set(Tink),
  protocolVersion = 3.0.0
)

@ mediator1.mediator.initialize(domainId, MediatorId(domainId), domainParameters, sequencerConnection, None)
res20: PublicKey = SigningPublicKey(id = 12201cfc127b..., format = Tink, scheme = Ed25519)

@ mediator1.health.wait_for_initialized()

The domain manager will also need a connection to the sequencer in order to complete its initialization .

@ val sequencerConnection = com.digitalasset.canton.sequencing.SequencerConnection.tryReadFromFile("tmp/domain-bootstrapping-files/sequencer-connection.proto")
sequencerConnection : com.digitalasset.canton.sequencing.SequencerConnection = GrpcSequencerConnection(
  endpoints = http://127.0.0.1:15049,
  transportSecurity = false,
  customTrustCertificates = None()
)

@ domainManager1.setup.init(sequencerConnection)

@ domainManager1.health.wait_for_initialized()

At this point the distributed domain should be completely initialized and a participant should be able to operate on this domain by connection to the sequencer.

@ participant1.domains.connect_local(sequencer1)

@ participant1.health.ping(participant1)
res26: concurrent.duration.Duration = 551 milliseconds

Additionally, please note that if more than one sequencers have been initialized, any mediator node and domain manager can choose to connect to just a subset of them.