Off-chain state management in Fat Contract

Fat Contract provides high consistency on-chain storage, but it’s only writable by commands. So it has very high costs and doesn’t scale with the size of the worker network. In the contrast, queries have high scalability, low latency, and rich off-chain functions, but they are stateless – there’s no access to the on-chain storage in a query. This makes queries in Fat Contracts hard to use. After all, most of the applications require some kind of state.

The first trail: STF

How can we turn queries to stateful functions? Probably the simplest way is to make the function a STF (state transition function):

  1. Isolate the state in the service (function)
  2. Pass the state to the service alongside the input of the query
  3. Return the update to the state alongside the output of the query

For example, here’s a simple counter program:

var a = 0;
function incBy(x) {
    a += x;
    return a;

To make it a stateless function, we simply pass the state to the function:

function incBy(state, x) {
    state.a += x;
    return [state, state.a];

Then it’s the caller’s duty to maintain the state outside the function. Each time the caller send the current state and the argument to the function, and receive the new state and the return value of the function. Next time, the caller should call the function with the new state.

Theoretically the STF model can work if the caller is honest. However it brings up a few concerns.

State meability and privacy

In the naive model, the caller can feed whatever state to the function. They can also potentially manipulate the state. This is definitely not what we want. We can fix it pretty quick by adding a layer of encryption on the state.

To do that, we can seal the state and only decrypt it within the Fat Contract:

function incBy(sealedState, x) {
    let state = decrypt(sealedState);
    state.a += x;
    sealedState = encrypt(state);
    return [sealed, state.a];

Encryption is only possible if the runtime can securely keep the state encryption keys. Fortunately, Fat Contract is such a confidential preserving runtime environment.

Usually the we should use authentication encryption to ensure the integrity of the state (thus the caller cannot manipulate the state). By sealing the state, we also address the confidentiality problem at the same time.

Data availability

Since the state is encrypted and authenticated, we can save the state data wherever the app developer feel secure. For a highly decentralized and long term project, it makes sense to store the data on a permanent storage protocol like Arweave. However for the applications that only keep the state in a short period (e.g. chat, personal pasteboard, auction, etc), a cheaper centralized storage like AWS S3 may be good enough, given we are not concerning about data integrity and privacy.

In fact, S3 might be nicer than it looks like. S3 is an object storage service offering HTTP API to read and write the data. From the Fat Contract’s point of view, S3 can be used as a key-value store. We can either save the entire state as a data blob in a kv entry directly, or potentially break the state into a few segments and save them separately (with the caution about data integrity and consistency).

Since S3 offers HTTP API, and Fat Contract also allow a query to send out HTTP requests, we can load and save the state within the contract. So there’s no need to ask the caller to fetch and update the state outside the contract. The contract just need to manage the credential to access the S3 (which is easy!).

After applying the S3 trick, the code will look like:

function incBy(x) {
    let state = getAndDecrypt(s3Url);
    state.a += x;
    sealedState = encryptAndPut(s3Url);
    return state.a;

There’s another advantage. S3 has already become the de facto of distributed storage. Once there’s a S3 client SDK implemented in Fat Contract, any service compatible with S3 can be used as the storage backend. So far, S3 is supported by almost all the cloud providers and on-prem open sources projects (minio), and even some decentralized storage (storj).

Data freshness and replay attack


  • order of state version
  • S3 and ETag
  • state channel (lighting network)

In other words, the Fat-contract is yet another contract form, because of this feature, in the user side contract developer see the STF is determinstic transition on chain. But if Phala introduce the S3 API wrappers, there is a bad situation, network faild happen always. Hence, how Phala Fat-contract billing design for this situation?

Is S3 available for workers in China?

Not AWS S3, but there are plenty of S3 compatible services offered by AliCloud and Tencent Cloud. The most widely used open source project minio is also S3 compatible

We don’t account the storage. Phala remains the computation protocol. However we may need to account the network traffic.

Another very important topic is the versioning of the states. Only if this problem is solved can we build correct and secure stateful applications. Will cover this part later.

  1. Is the transaction atomicity of the query which will modify the offchain state can be equivalent to the success of the data update operation on s3?
  2. For a query that was stateless before, if the user thinks that it was unsuccessful before because of network problems, the same content would be sent repeatedly. In this case, how can we tell if the user actually needs only one query. Do I need to bring in the version number for every request? How to tell if a query modifies off-chain state?

1 is a really good point. I was about to mention it in the next update. Updating the file on S3 is still dangerous. State versioning is very tricky. It can easily cause race condition.

Imagine there are two concurrent queries. The both queries may read the same old state and produce conflicting updates. If we just read and write the state straightforwardly, the update arriving later will overwrite the earlier one. This is a typical error in state versioning.

In fact, we want the operations to be transactional. Sometimes it means it looks like the transactions are processed sequentially (the order doesn’t matter). In traditional DBMS, this property is called ACID and can be achieved by locking or MVCC. However the synchronization primitives can vary in the different S3 providers. According to the Delta Lake paper, some cloud providers allow atomic update or swap, which are the primitives to enable ACID, but some do not (e.g. AWS). In the latter case, we still need a coordination service.

As long as we can make the operations transactional, we can provides the result to the user. They may retry if they want. Versioning is one approach.