25. Event-sourced, resource-based API

Status

Proposed

Context

• ADR-3 concluded that a full-duplex communication channels are desirable to interact with a reactive system.

• The Client API communicates several types of messages to clients. Currently this ranges from node-level PeerConnected, over head-specific HeadIsOpen to messages about transactions like TxValid. These messages are all of type StateChanged.

• Current capabilities of the API:

  • Clients can retrieve the whole history of StateChanged messages or opt-out using a query parameter - all or nothing.

  • There is a welcome message called Greetings which is always sent, that contains the last headStatus.

  • There exists a GetUTxO query-like ClientInput, which will respond with a GetUTxOResponse containing the confirmed UTxO set in an open head, or (!) the currently committed UTxO set when the head is initializing.

  • While overall json encoded, clients can choose choose between json or binary (cbor) output of transaction fields in several of these using a query parameter.

• Many of these features have been added in a "quick and dirty" way, by monkey patching the encoded JSON.

• The current capabalities even do not satisfy all user needs:

  • Need to wade through lots of events to know the latest state (except the very basic headStatus from the Greetings).

  • Need to poll GetUTxO or aggregate confirmed transactions on client side to know the latest UTxO set for constructing transactions.

  • Inclusion of the whole UTxO set in the head is not always desirable and filtering by address would be beneficial. (not addressed in this ADR though, relevant discussion #797)

  • As ADR-15 also proposes, some clients may not need (or should not have) access to administrative information.

• It is often a good idea to separate the responsibilities of Commands and Queries (CQRS), as well as the model they use.

Decision

• Drop GetUTxO and GetUTxOResponse messages as they advocate a request/response way of querying.

• Realize that ClientInput data is actually a ClientCommand (renaming them) and that ServerOutput are just projections of the internal event stream (see ADR-24) into read models on the API layer.

• Compose a versioned (/v1) API out of resource models, which compartmentalize the domain into topics on the API layer.

  • A resource has a model type and the latest value is the result of a pure projection folded over the StateChanged event stream, i.e. project :: model -> StateChanged -> model.

  • Each resource is available at some HTTP path, also called "endpoint":

    • GET requests must respond with the latest state in a single response.

    • GET requests with Upgrade: websocket headers must start a websocket connection, push the latest state as first message and any resource state updates after.

    • Other HTTP verbs may be accepted by a resource handler, i.e. to issue resource-specific commands. Any commands accepted must also be available via the corresponding websocket connection.

  • Accept request headers can be used to configure the Content-Type of the response

    • All resources must provide application/json responses

    • Some resources might support more content types (e.g. CBOR-encoded binary)

  • Query parameters may be used to further configure responses of some resources. For example, ?address=<bech32> could be used to filter UTxO by some address.

• Keep the semantics of /, which accepts websocket upgrade connections and sends direct/raw output of ServerOutput events on /, while accepting all ClientCommand messages.

  • Define ServerOutput also in terms of the StateChanged event stream

Example resources

Example resource paths + HTTP verbs mapped to existing things to demonstrate the effects of the decision points above. The mappings may change and are to be documented by an API specification instead.

Path	GET	POST	PATCH	DELETE
/v1/head/status	HeadStatus(..)	-	-	-
/v1/head/snapshot/utxo	last confirmed snapshot utxo	-	-	-
/v1/head/snapshot/transactions	confirmed snapshot txs	NewTx + responses	-	-
/v1/head/ledger/utxo	localUTxO	-	-	-
/v1/head/ledger/transactions	localTxs	NewTx + responses	-	-
/v1/head/commit	-	Chain{draftCommitTx}	-	-
/v1/head	all /v1/head/* data	Init	Close	Fanout / Abort
/v1/protocol-parameters	current protocol parameters
/v1/cardano-transaction	-	Chain{submitTx}	-	-
/v1/peers	a list of peers	-	-	-
/v1/node-version	node version as in Greetings	-	-	-
/v1/	all /v1/* data	-	-	-

Multiple heads are out of scope now and hence paths are not including a <headId> variable section.

Consequences

• Clear separation of what types are used for querying and gets subscribed to by clients and we have dedicated types for sending data to clients

• Changes on the querying side of the API are separated from the business logic.

• Clients do not need to aggregate data that is already available on the server side without coupling the API to internal state representation.

• Separation of Head operation and Head usage, e.g. some HTTP endpoints can be operated with authentication.

• Clients have a fine-grained control over what to subscribe to and what to query.

• Versioned API allows clients to detect incompatibility easily.

• Need to rewrite how the hydra-tui is implemented.