Daml Sandbox¶
The Daml Sandbox, or Sandbox for short, is a simple ledger implementation that enables rapid application prototyping by simulating a Daml Ledger.
You can start Sandbox together with Navigator using the daml start
command in a Daml project. This command will compile the Daml file and its dependencies as specified in the daml.yaml
. It will then launch Sandbox passing the just obtained DAR packages. The script specified in the init-script
field in daml.yaml
will be loaded into the ledger. Finally, it launches the navigator connecting it to the running Sandbox.
It is possible to execute the Sandbox launching step in isolation by typing daml sandbox
.
Note: Sandbox has switched to use Wall Clock Time mode by default. To use Static Time Mode you can provide the --static-time
flag to the daml sandbox
command or configure the time mode for daml start
in sandbox-options:
section of daml.yaml
. Please refer to Daml configuration files for more information.
Sandbox can also be run manually as in this example:
$ daml sandbox Main.dar --static-time
____ ____
/ __/__ ____ ___/ / / ___ __ __
_\ \/ _ `/ _ \/ _ / _ \/ _ \\ \ /
/___/\_,_/_//_/\_,_/_.__/\___/_\_\
INFO: Initialized sandbox version 1.12.0-snapshot.20210312.6498.0.707c86aa with ledger-id = fd562651-5ebb-4a45-add7-25809ca1f297, port = 6865, dar file = List(Main.dar), time mode = static time, ledger = in-memory, auth-service = AuthServiceWildcard$, contract ids seeding = strong
Contract Identifier Generation¶
Sandbox supports two contract identifier generator schemes:
- The so-called deterministic scheme that deterministically produces
contract identifiers from the state of the underlying ledger. Those
identifiers are strings starting with
#
. - The so-called random scheme that produces contract identifiers
indistinguishable from random. In practice, the schemes use a
cryptographically secure pseudorandom number generator initialized
with a truly random seed. Those identifiers are hexadecimal strings
prefixed by
00
.
The sandbox can be configured to use one or the other scheme with one of the following command line options:
--contract-id-seeding=<seeding-mode>
. The Sandbox will use the seeding mode <seeding-mode> to seed the generation of random contract identifiers. Possible seeding modes are:no
: The Sandbox uses thedeterministic
scheme. This is only supported by Sandbox classic and it prevents Sandbox from accepting packages in Daml-LF 1.11 or newer.strong
: The Sandbox uses therandom
scheme initialized with a high-entropy seed. Depending on the underlying operating system, the startup of the Sandbox may block as entropy is being gathered to generate the seed.testing-weak
: (For testing purposes only) The Sandbox uses therandom
scheme initialized with a low entropy seed. This may be used in a testing environment to avoid exhausting the system entropy pool when a large number of Sandboxes are started in a short time interval.testing-static
: (For testing purposes only) The sandbox uses therandom
scheme with a fixed seed. This may be used in testing for reproducible runs.
Running with persistence¶
Note: Running Sandbox with persistence is deprecated as of SDK 1.8.0 (16th Dec 2020). You can use the Daml Driver for PostgreSQL instead.
By default, Sandbox uses an in-memory store, which means it loses its state when stopped or restarted. If you want to keep the state, you can use a Postgres database for persistence. This allows you to shut down Sandbox and start it up later, continuing where it left off.
To set this up, you must:
create an initially empty Postgres database that the Sandbox application can access
have a database user for Sandbox that has authority to execute DDL operations
This is because Sandbox manages its own database schema, applying migrations if necessary when upgrading versions.
To start Sandbox using persistence, pass an --sql-backend-jdbcurl <value>
option, where <value>
is a valid jdbc url containing the username, password and database name to connect to.
Here is an example for such a url: jdbc:postgresql://localhost/test?user=fred&password=secret
Due to possible conflicts between the &
character and various terminal shells, we recommend quoting the jdbc url like so:
If you’re not familiar with JDBC URLs, see the JDBC docs for more information: https://jdbc.postgresql.org/documentation/head/connect.html
Running with authentication¶
By default, Sandbox does not use any authentication and accepts all valid ledger API requests.
To start Sandbox with authentication based on JWT tokens, use one of the following command line options:
--auth-jwt-rs256-crt=<filename>
. The sandbox will expect all tokens to be signed with RS256 (RSA Signature with SHA-256) with the public key loaded from the given X.509 certificate file. Both PEM-encoded certificates (text files starting with-----BEGIN CERTIFICATE-----
) and DER-encoded certificates (binary files) are supported.--auth-jwt-es256-crt=<filename>
. The sandbox will expect all tokens to be signed with ES256 (ECDSA using P-256 and SHA-256) with the public key loaded from the given X.509 certificate file. Both PEM-encoded certificates (text files starting with-----BEGIN CERTIFICATE-----
) and DER-encoded certificates (binary files) are supported.--auth-jwt-es512-crt=<filename>
. The sandbox will expect all tokens to be signed with ES512 (ECDSA using P-521 and SHA-512) with the public key loaded from the given X.509 certificate file. Both PEM-encoded certificates (text files starting with-----BEGIN CERTIFICATE-----
) and DER-encoded certificates (binary files) are supported.--auth-jwt-rs256-jwks=<url>
. The sandbox will expect all tokens to be signed with RS256 (RSA Signature with SHA-256) with the public key loaded from the given JWKS URL.
Warning
For testing purposes only, the following options may also be used. None of them is considered safe for production:
--auth-jwt-hs256-unsafe=<secret>
. The sandbox will expect all tokens to be signed with HMAC256 with the given plaintext secret.
Token payload¶
JWTs express claims which are documented in the authorization documentation.
The following is an example of a valid JWT payload:
where
ledgerId
,participantId
,applicationId
restricts the validity of the token to the given ledger, participant, or applicationexp
is the standard JWT expiration date (in seconds since EPOCH)admin
,actAs
andreadAs
bear the same meaning as in the authorization documentation
The public
claim is implicitly held by anyone bearing a valid JWT (even without being an admin or being able to act or read on behalf of any party).
Generating RSA keys¶
To generate RSA keys for testing purposes, use the following command
which generates the following files:
sandbox.key
: the private key in PEM/DER/PKCS#1 formatsandbox.crt
: a self-signed certificate containing the public key, in PEM/DER/X.509 Certificate format
Generating EC keys¶
To generate keys to be used with ES256 for testing purposes, use the following command
which generates the following files:
ecdsa256.key
: the private key in PEM/DER/PKCS#1 formatecdsa256.crt
: a self-signed certificate containing the public key, in PEM/DER/X.509 Certificate format
Similarly, you can use the following command for ES512 keys:
Running with TLS¶
To enable TLS, you need to specify the private key for your server and
the certificate chain via daml sandbox --pem server.pem --crt
server.crt
. By default, Sandbox requires client authentication as
well. You can set a custom root CA certificate used to validate client
certificates via --cacrt ca.crt
. You can change the client
authentication mode via --client-auth none
which will disable it
completely, --client-auth optional
which makes it optional or
specify the default explicitly via --client-auth require
.
Command-line reference¶
To start Sandbox, run: sandbox [options] <archive>...
.
To see all the available options, run daml sandbox --help
.
Metrics¶
Enable and configure reporting¶
To enable metrics and configure reporting, you can use the two following CLI options:
--metrics-reporter
: passing a legal value will enable reporting; the accepted values areconsole
,csv:</path/to/metrics.csv>
andgraphite:<local_server_port>
.console
: prints captured metrics on the standard outputcsv://</path/to/metrics.csv>
: saves the captured metrics in CSV format at the specified locationgraphite://<server_host>[:<server_port>]
: sends captured metrics to a Graphite server. If the port is omitted, the default value2003
will be used.
--metrics-reporting-interval
: metrics are pre-aggregated on the sandbox and sent to the reporter, this option allows the user to set the interval. The formats accepted are based on the ISO-8601 duration formatPnDTnHnMn.nS
with days considered to be exactly 24 hours. The default interval is 10 seconds.
Types of metrics¶
This is a list of type of metrics with all data points recorded for each. Use this as a reference when reading the list of metrics.
Gauge¶
An individual instantaneous measurement.
Counter¶
Number of occurrences of some event.
Meter¶
A meter tracks the number of times a given event occurred. The following data points are kept and reported by any meter.
<metric.qualified.name>.count
: number of registered data points overall<metric.qualified.name>.m1_rate
: number of registered data points per minute<metric.qualified.name>.m5_rate
: number of registered data points every 5 minutes<metric.qualified.name>.m15_rate
: number of registered data points every 15 minutes<metric.qualified.name>.mean_rate
: mean number of registered data points
Histogram¶
An histogram records aggregated statistics about collections of events. The exact meaning of the number depends on the metric (e.g. timers are histograms about the time necessary to complete an operation).
<metric.qualified.name>.mean
: arithmetic mean<metric.qualified.name>.stddev
: standard deviation<metric.qualified.name>.p50
: median<metric.qualified.name>.p75
: 75th percentile<metric.qualified.name>.p95
: 95th percentile<metric.qualified.name>.p98
: 98th percentile<metric.qualified.name>.p99
: 99th percentile<metric.qualified.name>.p999
: 99.9th percentile<metric.qualified.name>.min
: lowest registered value overall<metric.qualified.name>.max
: highest registered value overall
Histograms only keep a small reservoir of statistically relevant data points to ensure that metrics collection can be reasonably accurate without being too taxing resource-wise.
Unless mentioned otherwise all histograms (including timers, mentioned below) use exponentially decaying reservoirs (i.e. the data is roughly relevant for the last five minutes of recording) to ensure that recent and possibly operationally relevant changes are visible through the metrics reporter.
Note that min
and max
values are not affected by the reservoir sampling policy.
You can read more about reservoir sampling and possible associated policies in the Dropwizard Metrics library documentation.
Timers¶
A timer records all metrics registered by a meter and by an histogram, where the histogram records the time necessary to execute a given operation (unless otherwise specified, the precision is nanoseconds and the unit of measurement is milliseconds).
Database Metrics¶
A “database metric” is a collection of simpler metrics that keep track of relevant numbers when interacting with a persistent relational store.
These metrics are:
<metric.qualified.name>.wait
(timer): time to acquire a connection to the database<metric.qualified.name>.exec
(timer): time to run the query and read the result<metric.qualified.name>.query
(timer): time to run the query<metric.qualified.name>.commit
(timer): time to perform the commit<metric.qualified.name>.translation
(timer): if relevant, time necessary to turn serialized Daml-LF values into in-memory objects
List of metrics¶
The following is a non-exhaustive list of selected metrics that can be particularly important to track. Note that not all the following metrics are available unless you run the sandbox with a PostgreSQL backend.
daml.commands.deduplicated_commands
¶
A meter. Number of deduplicated commands.
daml.commands.delayed_submissions
¶
A meter. Number of delayed submissions (submission who have been evaluated to transaction with a ledger time farther in the future than the expected latency).
daml.commands.failed_command_interpretation
¶
A meter. Number of commands that have been deemed unacceptable by the interpreter and thus rejected (e.g. double spends)
daml.commands.submissions
¶
A timer. Time to fully process a submission (validation, deduplication and interpretation) before it’s handed over to the ledger to be finalized (either committed or rejected).
daml.commands.valid_submissions
¶
A meter. Number of submission that pass validation and are further sent to deduplication and interpretation.
daml.commands.validation
¶
A timer. Time to validate submitted commands before they are fed to the Daml interpreter.
daml.commands.input_buffer_capacity
¶
A counter. The capacity of the queue accepting submissions on the CommandService.
daml.commands.input_buffer_length
¶
A counter. The number of currently pending submissions on the CommandService.
daml.commands.input_buffer_delay
¶
A timer. Measures the queuing delay for pending submissions on the CommandService.
daml.commands.max_in_flight_capacity
¶
A counter. The capacity of the queue tracking completions on the CommandService.
daml.commands.max_in_flight_length
¶
A counter. The number of currently pending completions on the CommandService.
daml.execution.get_lf_package
¶
A timer. Time spent by the engine fetching the packages of compiled Daml code necessary for interpretation.
daml.execution.lookup_active_contract_count_per_execution
¶
A histogram. Number of active contracts fetched for each processed transaction.
daml.execution.lookup_active_contract_per_execution
¶
A timer. Time to fetch all active contracts necessary to process each transaction.
daml.execution.lookup_active_contract
¶
A timer. Time to fetch each individual active contract during interpretation.
daml.execution.lookup_contract_key_count_per_execution
¶
A histogram. Number of contract keys looked up for each processed transaction.
daml.execution.lookup_contract_key_per_execution
¶
A timer. Time to lookup all contract keys necessary to process each transaction.
daml.execution.lookup_contract_key
¶
A timer. Time to lookup each individual contract key during interpretation.
daml.execution.retry
¶
A meter. Overall number of interpretation retries attempted due to mismatching ledger effective time.
daml.execution.total
¶
A timer. Time spent interpreting a valid command into a transaction ready to be submitted to the ledger for finalization.
daml.index.db.connection.sandbox.pool
¶
This namespace holds a number of interesting metrics about the connection pool used to communicate with the persistent store that underlies the index.
These metrics include:
daml.index.db.connection.sandbox.pool.Wait
(timer): time spent waiting to acquire a connectiondaml.index.db.connection.sandbox.pool.Usage
(histogram): time spent using each acquired connectiondaml.index.db.connection.sandbox.pool.TotalConnections
(gauge): number or total connectionsdaml.index.db.connection.sandbox.pool.IdleConnections
(gauge): number of idle connectionsdaml.index.db.connection.sandbox.pool.ActiveConnections
(gauge): number of active connectionsdaml.index.db.connection.sandbox.pool.PendingConnections
(gauge): number of threads waiting for a connection
daml.index.db.deduplicate_command
¶
A timer. Time spent persisting deduplication information to ensure the continued working of the deduplication mechanism across restarts.
daml.index.db.get_active_contracts
¶
A database metric. Time spent retrieving a page of active contracts to be served from the active contract service. The page size is configurable, please look at the CLI reference.
daml.index.db.get_completions
¶
A database metric. Time spent retrieving a page of command completions to be served from the command completion service. The page size is configurable, please look at the CLI reference.
daml.index.db.get_flat_transactions
¶
A database metric. Time spent retrieving a page of flat transactions to be streamed from the transaction service. The page size is configurable, please look at the CLI reference.
daml.index.db.get_ledger_end
¶
A database metric. Time spent retrieving the current ledger end. The count for this metric is expected to be very high and always increasing as the indexed is queried for the latest updates.
daml.index.db.get_ledger_id
¶
A database metric. Time spent retrieving the ledger identifier.
daml.index.db.get_transaction_trees
¶
A database metric. Time spent retrieving a page of flat transactions to be streamed from the transaction service. The page size is configurable, please look at the CLI reference.
daml.index.db.load_all_parties
¶
A database metric. Load the currently allocated parties so that they are served via the party management service.
daml.index.db.load_archive
¶
A database metric. Time spent loading a package of compiled Daml code so that it’s given to the Daml interpreter when needed.
daml.index.db.load_configuration_entries
¶
A database metric. Time to load the current entries in the log of configuration entries. Used to verify whether a configuration has been ultimately set.
daml.index.db.load_package_entries
¶
A database metric. Time to load the current entries in the log of package uploads. Used to verify whether a package has been ultimately uploaded.
daml.index.db.load_packages
¶
A database metric. Load the currently uploaded packages so that they are served via the package management service.
daml.index.db.load_parties
¶
A database metric. Load the currently allocated parties so that they are served via the party service.
daml.index.db.load_party_entries
¶
A database metric. Time to load the current entries in the log of party allocations. Used to verify whether a party has been ultimately allocated.
daml.index.db.lookup_active_contract
¶
A database metric. Time to fetch one contract on the index to be used by the Daml interpreter to evaluate a command into a transaction.
daml.index.db.lookup_configuration
¶
A database metric. Time to fetch the configuration so that it’s served via the configuration management service.
daml.index.db.lookup_contract_by_key
¶
A database metric. Time to lookup one contract key on the index to be used by the Daml interpreter to evaluate a command into a transaction.
daml.index.db.lookup_flat_transaction_by_id
¶
A database metric. Time to lookup a single flat transaction by identifier to be served by the transaction service.
daml.index.db.lookup_maximum_ledger_time
¶
A database metric. Time spent looking up the ledger effective time of a transaction as the maximum ledger time of all active contracts involved to ensure causal monotonicity.
daml.index.db.lookup_transaction_tree_by_id
¶
A database metric. Time to lookup a single transaction tree by identifier to be served by the transaction service.
daml.index.db.remove_expired_deduplication_data
¶
A database metric. Time spent removing deduplication information after the expiration of the deduplication window. Deduplication information is persisted to ensure the continued working of the deduplication mechanism across restarts.
daml.index.db.stop_deduplicating_command
¶
A database metric. Time spent removing deduplication information after the failure of a command. Deduplication information is persisted to ensure the continued working of the deduplication mechanism across restarts.
daml.index.db.store_configuration_entry
¶
A database metric. Time spent persisting a change in the ledger configuration provided through the configuration management service.
daml.index.db.store_ledger_entry
¶
A database metric. Time spent persisting a transaction that has been successfully interpreted and is final.
daml.index.db.store_package_entry
¶
A database metric. Time spent storing a Daml package uploaded through the package management service.
daml.index.db.store_party_entry
¶
A database metric. Time spent storing party information as part of the party allocation endpoint provided by the party management service.
daml.index.db.store_rejection
¶
A database metric. Time spent persisting the information that a given command has been rejected.
daml.lapi
¶
Every metrics under this namespace is a timer, one for each service exposed by the Ledger API, in the format:
daml.lapi.service_name.service_endpoint
As in the following example:
daml.lapi.command_service.submit_and_wait
Single call services return the time to serve the request, streaming services measure the time to return the first response.
jvm
¶
Under the jvm
namespace there is a collection of metrics that
tracks important measurements about the JVM that the sandbox is
running on, including CPU usage, memory consumption and the
current state of threads.