Manage Domain Entities

Setting up a Distributed Domain With a Single Console

If you’re running a domain node in its default configuration as shown previously in this current page, it will have a sequence and mediator embedded and these components will be automatically bootstrapped for you.

If your domain operates with external sequencers and mediators, you will need to configure a domain manager node (which only runs topology management) and bootstrap your domain with at least one external sequencer node and one external mediator node.

First make sure the nodes are fresh and have not yet been initialized:

@ mediator1.health.initialized()
res1: Boolean = false

@ sequencer1.health.initialized()
res2: Boolean = false

@ domainManager1.health.initialized()
res3: Boolean = false

Now you can initialize the distributed domain as follows:

@ domainManager1.setup.bootstrap_domain(Seq(sequencer1), Seq(mediator1))

At this point a participant should be able to connect to a sequencer and operate on that domain:

@ participant1.domains.connect_local(sequencer1)

@ participant1.health.ping(participant1)
res6: Duration = 437 milliseconds

Domain managers are configured as domain-managers under the canton configuration. Domain managers are configured similarly to domain nodes, except that there are no sequencer, mediator, public api or service agreement configs.

Please note that if your sequencer is database-based and you’re horizontally scaling it as described under sequencer high availability, you do not need to pass all sequencer nodes into the command above. Since they all share the same relational database, you only need to run this initialization step on one of them.

For non-database-based sequencer such as Ethereum or Fabric sequencers you need to have each node initialized individually. You can either initialize such sequencers as part of the initial domain bootstrap shown above or dynamically add a new sequencer at a later point as described in operational processes.

Setting up a Distributed Domain With Separate Consoles

The process outlined in the previous section only works if all nodes are accessible from the same console environment. If each node has its own isolated console environment, the bootstrapping process must be coordinated in steps with the exchange of data via files using any secure channel of communication between the environments.

Note

Please ensure that all of the nodes in the distributed domain are started before proceeding.

Initially the domain manager must transmit its domain parameters from its console by saving the parameters to a file. The domain id, serialized as a string, must also be transmitted.

@ domainManager1.service.get_static_domain_parameters.writeToFile("tmp/domain-bootstrapping-files/params.proto")

@ val domainIdString = domainManager1.id.toProtoPrimitive
domainIdString : String = "domainManager1::122029951e6b2ed8545f74f14b4dbbab8b3486ebea3b002a915c2fc7666163eddc94"

Then the sequencer must receive this file, deserialize it and initialize itself. As part of the initialization, the sequencer creates a signing key pair whose public key it must then transmit via file. Optionally, repeat this for any extra sequencer nodes.

@ val domainParameters = com.digitalasset.canton.admin.api.client.data.StaticDomainParameters.tryReadFromFile("tmp/domain-bootstrapping-files/params.proto")
domainParameters : StaticDomainParameters = StaticDomainParametersV1(
  uniqueContractKeys = true,
  requiredSigningKeySchemes = Set(Ed25519, ECDSA-P256, ECDSA-P384),
  requiredEncryptionKeySchemes = Set(ECIES-P256_HMAC256_AES128-GCM),
  requiredSymmetricKeySchemes = Set(AES128-GCM),
  requiredHashAlgorithms = Set(Sha256),
  requiredCryptoKeyFormats = Set(Tink),
  protocolVersion = 4
)

@ val domainId = DomainId.tryFromString(domainIdString)
domainId : DomainId = domainManager1::122029951e6b...

@ val initResponse = sequencer1.initialization.initialize_from_beginning(domainId, domainParameters)
initResponse : com.digitalasset.canton.domain.sequencing.admin.protocol.InitResponse = InitResponse(
  keyId = "sequencer-id",
  publicKey = SigningPublicKey(id = 1220cf7adb54..., format = Tink, scheme = Ed25519),
  replicated = false
)

@ initResponse.publicKey.writeToFile("tmp/domain-bootstrapping-files/seq1-key.proto")

The domain manager must then authorize the sequencer’s key. Optionally, repeat this for any extra sequencer keys.

@ val sequencerPublicKey = SigningPublicKey.tryReadFromFile("tmp/domain-bootstrapping-files/seq1-key.proto")
sequencerPublicKey : SigningPublicKey = SigningPublicKey(id = 1220cf7adb54..., format = Tink, scheme = Ed25519)

@ domainManager1.setup.helper.authorizeKey(sequencerPublicKey, "sequencer", SequencerId(domainManager1.id))

Now the mediator also needs to create a signing key pair and transmit it. Optionally, repeat this for any extra mediator nodes.

@ mediator1.keys.secret.generate_signing_key("initial-key").writeToFile("tmp/domain-bootstrapping-files/med1-key.proto")

The domain manager must now authorize the mediator’s key and also authorize the mediator to act as part of this domain. Optionally, repeat this for any extra mediator nodes.

@ val mediatorKey = SigningPublicKey.tryReadFromFile("tmp/domain-bootstrapping-files/med1-key.proto")
mediatorKey : SigningPublicKey = SigningPublicKey(id = 1220094402cb..., format = Tink, scheme = Ed25519)

@ val domainId = DomainId.tryFromString(domainIdString)
domainId : DomainId = domainManager1::122029951e6b...

@ domainManager1.setup.helper.authorizeKey(mediatorKey, "mediator1", MediatorId(domainId))

@ domainManager1.topology.mediator_domain_states.authorize(TopologyChangeOp.Add, domainId, MediatorId(domainId), RequestSide.Both)
res13: com.google.protobuf.ByteString = <ByteString@410f2f66 size=560 contents="\n\255\004\n\333\001\n\326\001\n\323\001\022 vr9K96ncoqUYT1MKFPD2teZT40ZXtaIMR...">

After that, still on the domain manager’s console, the domain manager must collect the list of topology transactions, which include all the key authorizations and a few other things it needs to broadcast to all domain members. This is now saved to a file.

@ domainManager1.topology.all.list().collectOfType[TopologyChangeOp.Positive].writeToFile("tmp/domain-bootstrapping-files/topology-transactions.proto")

The sequencer then reads this set of initial topology transactions and sequences it as the first message to be sequenced in this domain. This will allow the domain members whose keys were authorized in previous steps to connect to this sequencer and operate with it. The sequencer will then transmit its connection info.

@ val initialTopology = com.digitalasset.canton.topology.store.StoredTopologyTransactions.tryReadFromFile("tmp/domain-bootstrapping-files/topology-transactions.proto").collectOfType[TopologyChangeOp.Positive]
initialTopology : store.StoredTopologyTransactions[TopologyChangeOp.Positive] = Seq(
  StoredTopologyTransaction(
    sequenced = 2023-03-09T09:50:32.977424Z,
    validFrom = 2023-03-09T09:50:32.977424Z,
    validUntil = 2023-03-09T09:50:32.977424Z,
    op = Add,
..

@ sequencer1.initialization.bootstrap_topology(initialTopology)

@ sequencer1.sequencerConnection.writeToFile("tmp/domain-bootstrapping-files/sequencer-connection.proto")

To initialize the mediator, it will need a connection to the sequencer and the domain parameters. Optionally, repeat this for any extra mediator nodes.

@ val sequencerConnection = com.digitalasset.canton.sequencing.SequencerConnection.tryReadFromFile("tmp/domain-bootstrapping-files/sequencer-connection.proto")
sequencerConnection : com.digitalasset.canton.sequencing.SequencerConnection = GrpcSequencerConnection(
  endpoints = http://127.0.0.1:15075,
  transportSecurity = false,
  customTrustCertificates = None()
)

@ val domainParameters = com.digitalasset.canton.admin.api.client.data.StaticDomainParameters.tryReadFromFile("tmp/domain-bootstrapping-files/params.proto")
domainParameters : StaticDomainParameters = StaticDomainParametersV1(
  uniqueContractKeys = true,
  requiredSigningKeySchemes = Set(Ed25519, ECDSA-P256, ECDSA-P384),
  requiredEncryptionKeySchemes = Set(ECIES-P256_HMAC256_AES128-GCM),
  requiredSymmetricKeySchemes = Set(AES128-GCM),
  requiredHashAlgorithms = Set(Sha256),
  requiredCryptoKeyFormats = Set(Tink),
  protocolVersion = 4
)

@ mediator1.mediator.initialize(domainId, MediatorId(domainId), domainParameters, sequencerConnection, None)
res20: PublicKey = SigningPublicKey(id = 12201dc4dc96..., format = Tink, scheme = Ed25519)

@ mediator1.health.wait_for_initialized()

The domain manager will also need a connection to the sequencer in order to complete its initialization .

@ val sequencerConnection = com.digitalasset.canton.sequencing.SequencerConnection.tryReadFromFile("tmp/domain-bootstrapping-files/sequencer-connection.proto")
sequencerConnection : com.digitalasset.canton.sequencing.SequencerConnection = GrpcSequencerConnection(
  endpoints = http://127.0.0.1:15075,
  transportSecurity = false,
  customTrustCertificates = None()
)

@ domainManager1.setup.init(sequencerConnection)

@ domainManager1.health.wait_for_initialized()

At this point the distributed domain should be completely initialized and a participant should be able to operate on this domain by connection to the sequencer.

@ participant1.domains.connect_local(sequencer1)

@ participant1.health.ping(participant1)
res26: Duration = 418 milliseconds

Additionally, please note that if more than one sequencers have been initialized, any mediator node and domain manager can choose to connect to just a subset of them.

Adding new sequencers to distributed domain

For non-database-based sequencers such as Ethereum or Fabric sequencers, you can either initialize them as part of the regular distributed domain bootstrapping process or dynamically add a new sequencer at a later point as follows:

domainManager1.setup.onboard_new_sequencer(
  initialSequencer = sequencer1,
  newSequencer = sequencer2,
)

Similarly to initializing a distributed domain with separate consoles, dynamically onboarding new sequencers (supported by Fabric and Ethereum sequencers) can be achieved in separate consoles as follows:

// Second sequencer's console: write signing key to file
{
  secondSequencer.keys.secret
    .generate_signing_key(s"${secondSequencer.name}-signing")
    .writeToFile(file1)
}

// Domain manager's console: write domain params and current topology
{
  domainManager1.service.get_static_domain_parameters.writeToFile(paramsFile)

  val sequencerSigningKey = SigningPublicKey.tryReadFromFile(file1)

  domainManager1.setup.helper.authorizeKey(
    sequencerSigningKey,
    s"${secondSequencer.name}-signing",
    sequencerId,
  )

  domainManager1.setup.helper.waitForKeyAuthorizationToBeSequenced(
    sequencerId,
    sequencerSigningKey,
  )

  domainManager1.topology.all
    .list(domainId.filterString)
    .collectOfType[TopologyChangeOp.Positive]
    .writeToFile(file1)
}

// Initial sequencer's console: read topology and write snapshot to file
{
  val topologySnapshotPositive =
    StoredTopologyTransactions
      .tryReadFromFile(file1)
      .collectOfType[TopologyChangeOp.Positive]

  val sequencingTimestamp = topologySnapshotPositive.lastChangeTimestamp.getOrElse(
    sys.error("topology snapshot is empty")
  )

  sequencer.sequencer.snapshot(sequencingTimestamp).writeToFile(file2)
}

// Second sequencer's console: read topology, snapshot and domain params
{
  val topologySnapshotPositive =
    StoredTopologyTransactions
      .tryReadFromFile(file1)
      .collectOfType[TopologyChangeOp.Positive]

  val state = SequencerSnapshot.tryReadFromFile(file2)

  val domainParameters = StaticDomainParameters.tryReadFromFile(paramsFile)

  secondSequencer.initialization
    .initialize_from_snapshot(
      domainId,
      topologySnapshotPositive,
      state,
      domainParameters,
    )
    .publicKey

  secondSequencer.health.initialized() shouldBe true

}