Po.et Documentation
Technical documentation for the Po.et Network

About Po.et
Use Po.et
Reference
Process
RDD

Blockchain Confirmations

Bitcoin transactions can only be considered safe after they have a certain amount of confirmations.

The node needs to know how many confirmations transactions have and trigger a retry mechanism in case one or more transactions aren’t getting confirmed as expected.

Index

Cases

There are three different cases the node needs to be aware of and know how to respond to.

Transaction Lost

Transaction disappears from mempool and never gets into a block.

Most likely discarded after being stuck in the mempool for too long, though there is no way to know for sure.

Solution

This should seldom happen. If it does, the node should discard the lost broadcasted transaction and create and broadcast a new one.

Transaction Stuck

Transaction is in mempool for too long, never gets into a block.

Probably due to low fee, though there is no way to know for sure.

Could also be a symptom of Bitcoin scaling issues.

Solution

Though there’s no guarantee of this working, the best attempt it to replace the transaction with one with higher fee, basically creating a double-spend. Miners should choose the transaction with higher fee, rendering the one with lower fee invalid.

If the root cause is more related to Bitcoin scalability than Po.et, there is nothing to be done.

We can use getrawmempool and getmempoolentry to know whether a transaction is in the mempool or not.

Issues may arise due to the mempool not being decentralized: a transaction existing in Node A’s mempool may have been discarded from Node B’s mempool.

New Fee

The fee is currently determined by Bitcoin Core. Initially, we wil want to let Bitcoin Core determine the new fee, too, and hope for it to be high enough.

In a future iteration, we can

Chain Reorganization

Transaction gets into a block and sometime after that block becomes stale, no longer being part of the active chain.

This is a completely normal event in Bitcoin, Ethereum and other blockchains. It is also the most complicated case to handle.

We can expect an average of at least 47 one-block forks per year, according to this data.

Bitcoin nodes will usually replay transactions into new blocks, so in general this shouldn’t affect the Po.et node other than it having a wrong blockHeight and blockHash for a given claim.

Solution

Miners will usually move the transactions from the staled/orphaned block back to the mempool, which should solve the issue.

If this does not happen, there are various approaches to address it manually:

A reorg may mean that data that was valid no longer is. This does not affect our current system, as updates are not supported and Work is the only claim type in use, but the introduction of Entities, Identity Claims, Licences, etc, will require more attention.

Implementation

BlockchainReader.ClaimController.scanBlock

BlockchainWriter

Router / Controller

consume(BlockDownloaded(lightBlock, matchingAnchors)) => 
  const setBlockHeightAndHash = dao.setBlockHeightAndHash(lightBlock.height, lightBlock.hash) 
  const transactionIds = matchingAnchors.map(_ => _.transactionId)
  await transactionIds.map(setBlockHeightAndHash)
  const oldBlocklessTransactions = await dao.findOldBlocklessTransactions({ currentBlockHeight: lightBlock.height, maximumBlockAge })
  logger.warning({ maximumBlockAge, oldBlocklessTransactions }, 'Discarding unconfirmed transactions')
  await dao.clearTransactions(oldBlocklessTransactions)

DAO

  findBlocklessTransactions = maximumBlockAge => currentBlockHeight => db.blockchainWriter.find({ 
    tx: { $not: null }, 
    creationBlockHeight: { $lt: currentBlockHeight - maximumBlockAge }, 
    blockHeight: null 
  })
  
  setBlockHeightAndHash = (blockHeight, blockHash) => txid => db.blockchainWriter.update({ txid }, { $set: { blockHeight, blockHash } })

Functional Testing

Testing the different scenarios can be a bit complicated and require some orchestration.

I haven’t decided or tried anything yet, but here’s some information and ideas:

Never Confirmed

Starting bitcoind with -blockmintxfee=<amt> wih a high amt allows us to generate blocks that do not include our transaction.

dc up -d # with -blockmintxfee=1
$ WORK_ID=$(npm run --silent create-claim  | jq .id)
$ bitcoin-cli getrawmempool | jq # look for WORK_ID in returned array
$ bitcoin-cli generate 1
$ bitcoin-cli getrawmempool | jq # WORK_ID should still be there

Mempool Clearing

Clearing the mempool requires restarting the node with -zapwallettxes. I haven’t found a way to do this without restarting. And I’m not 100% sure -zapwallettxes is enough.

Reorg Simulation

Connecting and disconnecting to/from the network seems to be possible with the RPC calls setnetworkactive, addnode and disconnectnode.

Higher Level Orchestration

Given that all testing scenarios require interacting with dependencies, restarting them, passing arguments to them, etc, I believe it makes sense to consider these tests to live in a higher layer, over the application itself.

A naive example of such orchestration:

$ alias dc=docker-compose
$ dc up -d
$ dc exec tests node create-10-claims.js node-a
$ dc exec bitcoind1 bitcoin-cli setnetworkactive false
$ dc exec tests bash node second-part-of-test.js
$ dc exec bitcoind1 bitcoin-cli setnetworkactive true
$ dc exec tests node third-part-of-test.js
$ dc stop bitcoind1
$ dc -f bitcoin-zap-wallet up -d bitcoind1
$ dc exec tests node fourth-part-of-test.js

We could achieve such control with Docker exposing the Docker socket to the test orchestration container.

Notes For Clients

Build Over Confirmed Data

Reattempting transactions and reorgs mean claim order or validity cannot be guaranteed for claims with no or few confirmations.

When creating new claims that depend on or refer to previous claims, clients should make sure the previous claims have enough confirmations.

Due to how Proof of Work works, there isn’t a defined line that separates a safe from an unsafe amount of confirmations. How few confirmations one is willing to accept, or how many require, is a decision that needs to be weighted for each use case.

It is generally considered that three confirmations is safe for transactions moving a low to medium amount of funds, while for larger transactions one may want to wait for six.

To stay safe from reorgs to number could be as high as five, although less should usually suffice, while more may be required on ocassion in the future.

Special attention should be given to claims that involve money, such as purchases or sales of licenses.

RPCs

  1. getrawmempool
  2. getmempoolentry
  3. getchaintips
  4. setnetworkactive
  5. addnode
  6. disconnectnode

References

  1. Chain Reorganization
  2. Orphan / Stale Block
  3. How many confirmations do I need to ensure a transaction is successful?
  4. Why is 6 the number of confirms that is considered secure?
  5. Using Docker-in-Docker for your CI or testing environment? Think twice.
  6. What is the longest blockchain fork that has been orphaned to date?
  7. How does the mempool work? What happens to the mempool when there are two equal length chains on the network?
  8. Valid fork in regtest - Change blockchain via RPC?
  9. How to detect reorganization from bitcoind via ZMQ?
  10. How to detect a fork with bitcoin-cli?
  11. How can an earlier block have a later timestamp?