Author: chirpyli
Source: Hang Seng LIGHT cloud community
Overview of mining process
Bitcoin mining is actually a bitcoin node, which packages transactions into blocks and gets bitcoin incentives while obtaining bookkeeping rights. The main mining process is as follows:
- Collect the transactions broadcast on the network, verify and join the transaction pool;
- Construct a new block - package the transactions in the transaction pool, select the longest blockchain, and calculate the latest block header hash value as the hash of the previous block of the new block (candidate block, which has not obtained the bookkeeping right);
- Workload proof, calculation difficulty and competition for bookkeeping right. If the bookkeeping right is obtained, the new block will be broadcast. If the new block broadcast by other nodes is received, the new block will be verified. If the new block passes the verification, the mining process will be restarted.
With the emergence of special mining machines, CPU mining has been basically abandoned. The CPU mining code here is only used for learning and testing. Actual miners will not use the following code to mine. If CPU mining is enabled, mining can be started to a specific address through a generatetoaddressRPC call.
A bitcoin node can choose to mine or not. If you want to mine, enter the command bitcoin cli generatetoaddress to start the mining process.
Let's analyze the relevant source code:
// After opening the mining process, you need a transaction output and Coinbase transaction. To reward miners for mining, you need to lock the script
static UniValue generatetoaddress(const JSONRPCRequest& request)
{
if (request.fHelp || request.params.size() < 2 || request.params.size() > 3)
throw std::runtime_error(
RPCHelpMan{"generatetoaddress",
"\nMine blocks immediately to a specified address (before the RPC call returns)\n",
{
{"nblocks", RPCArg::Type::NUM, RPCArg::Optional::NO, "How many blocks are generated immediately."}, //Set how many blocks to dig
{"address", RPCArg::Type::STR, RPCArg::Optional::NO, "The address to send the newly generated bitcoin to."}, //Mining reward output address
{"maxtries", RPCArg::Type::NUM, /* default */ "1000000", "How many iterations to try."},
},
RPCResult{
"[ blockhashes ] (array) hashes of blocks generated\n"
},
RPCExamples{
"\nGenerate 11 blocks to myaddress\n"
+ HelpExampleCli("generatetoaddress", "11 \"myaddress\"")
+ "If you are running the bitcoin core wallet, you can get a new address to send the newly generated bitcoin to with:\n"
+ HelpExampleCli("getnewaddress", "")
},
}.ToString());
int nGenerate = request.params[0].get_int();
uint64_t nMaxTries = 1000000;
if (!request.params[2].isNull()) {
nMaxTries = request.params[2].get_int();
}
CTxDestination destination = DecodeDestination(request.params[1].get_str());
if (!IsValidDestination(destination)) {
throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Error: Invalid address");
}
CScript coinbase_script = GetScriptForDestination(destination); //Lock script used to reward miners
return generateBlocks(coinbase_script, nGenerate, nMaxTries); //Generate new blocks to compete for bookkeeping rights
}
static UniValue generateBlocks(const CScript& coinbase_script, int nGenerate, uint64_t nMaxTries)
{
static const int nInnerLoopCount = 0x10000;
int nHeightEnd = 0;
int nHeight = 0;
{ // Don't keep cs_main locked
LOCK(cs_main);
nHeight = ::ChainActive().Height();
nHeightEnd = nHeight+nGenerate;
}
unsigned int nExtraNonce = 0;
UniValue blockHashes(UniValue::VARR);
while (nHeight < nHeightEnd && !ShutdownRequested()) //If the specified block height is not reached or the mining request is not stopped, mining will continue
{
std::unique_ptr<CBlockTemplate> pblocktemplate(BlockAssembler(Params()).CreateNewBlock(coinbase_script)); //Construct a new block
if (!pblocktemplate.get())
throw JSONRPCError(RPC_INTERNAL_ERROR, "Couldn't create new block");
CBlock *pblock = &pblocktemplate->block;
{
LOCK(cs_main);
IncrementExtraNonce(pblock, ::ChainActive().Tip(), nExtraNonce); //Merkel root hash is calculated and assigned here.
}
//Constantly changing the nNonce value, the calculation is difficult
while (nMaxTries > 0 && pblock->nNonce < nInnerLoopCount && !CheckProofOfWork(pblock->GetHash(), pblock->nBits, Params().GetConsensus())) {
++pblock->nNonce;
--nMaxTries;
}
if (nMaxTries == 0) {
break;
}
if (pblock->nNonce == nInnerLoopCount) {
continue;
}
std::shared_ptr<const CBlock> shared_pblock = std::make_shared<const CBlock>(*pblock);
if (!ProcessNewBlock(Params(), shared_pblock, true, nullptr))
throw JSONRPCError(RPC_INTERNAL_ERROR, "ProcessNewBlock, block not accepted");
++nHeight;
blockHashes.push_back(pblock->GetHash().GetHex());
}
return blockHashes;
}Later, we will expand the code and analyze the specific process in detail. Start by collecting transactions.
Verify that the transaction is added to the transaction pool
Bitcoin wallet constructs transactions and publishes them to the bitcoin blockchain network. Other nodes receive transactions received on the network and verify them first. After verifying the transactions, bitcoin nodes will add these transactions to their own memory pool and broadcast the transactions to other nodes. The memory pool, also known as the transaction pool, is used to temporarily store transactions that have not been confirmed (added to the block). The verification rules here are as follows (long and may change):
- The syntax and data structure of the transaction must be correct.
- Transaction input and output cannot be blank.
- The size of transaction bytes must be less than MAX_BLOCK_SIZE.
- The aggregate value of each transaction output must be within the allowable range (less than 21 million bitcoin, greater than 0).
- The hash value entered in the transaction cannot be zero, and the Coinbase transaction should not be relayed.
- nLockTime is less than or equal to INT_MAX.
- The number of transaction bytes must be greater than or equal to 100.
- The number of signature operations in the transaction must be less than the limit of signature operations.
- The unlocking script (scriptSig) can only push numbers onto the stack, and the locking script (scriptPubKey) must match the isStandard format (which will reject "non-standard" transactions).
- There must be a matching transaction in the transaction pool or the block of the main branch.
- For each input, if the referenced output exists in other transactions in the transaction pool, the transaction must be rejected.
- For each input, you need to find the referenced output transaction in the main branch and transaction pool. If the output transaction corresponding to any input does not exist, then this is an orphan transaction. If the corresponding transaction is not in the orphan trading pool, it will be added to the orphan trading pool.
- For each input, if the referenced output transaction is the output of a coin transaction, it must pass at least coinbase_ Quality (100) confirmation.
- Use the output transaction to calculate the input value, check each input value and summary value to see if it exceeds the allowable range (less than 21 million bitcoin, greater than 0).
- If the input value summary is less than the output value, reject the transaction.
- If the transaction cost is too small to join an empty block, reject the transaction.
- Each input unlocking script must be verified based on the corresponding output locking script.
Construct a new block
The process of collecting and verifying transactions on the network and joining the trading pool is actually the preparatory work of the mining process. Constructing new blocks and performing difficult calculation is the core step of the mining process.
Tectonic process
To construct a new block, you can actually look at the block structure and set the relevant fields.
class CBlock : public CBlockHeader
{
public:
// network and disk
std::vector<CTransactionRef> vtx;
//... Omit some code
}Select some transactions from the transaction pool according to the transaction priority, add them to the block transaction list, and start to construct the block header. The most important ones are the previous block hash and Merkel tree root hash value. Here, a Coinbase transaction needs to be constructed as the first transaction of the whole block.
class CBlockHeader
{
public:
// header
int32_t nVersion; // Indicates which set of block validation rules to follow
uint256 hashPrevBlock; // A reference to the parent / previous block in the blockchain
uint256 hashMerkleRoot; // A hash (root hash) of the Merkle tree data structure containing a block's transactions
uint32_t nTime; // Timestamps (seconds from Unix Epoch)
uint32_t nBits; // Aka the difficulty target for this block
uint32_t nNonce; // Workload certificate nonce (value used in proof of work)
// ... Some code omitted
}The hash value of the previous block is to select the highest block head of the longest chain of the current blockchain to hash and calculate the hash of the previous block. Merkel root hash value is calculated from the transaction list. Let's take a look at the source code for constructing a new block:
// This class is responsible for constructing new blocks (to be exact, it should be candidate blocks, because it has not been proved by workload)
/** Generate a new block, without valid proof-of-work */
class BlockAssembler
{
private:
// The constructed block template
std::unique_ptr<CBlockTemplate> pblocktemplaclass CBlock : public CBlockHeader
{
public:
// network and disk
std::vector<CTransactionRef> vtx;te;
// A convenience pointer that always refers to the CBlock in pblocktemplate
CBlock* pblock;
// Configuration parameters for the block size
bool fIncludeWitness;
unsigned int nBlockMaxWeight;
CFeeRate blockMinFeeRate;
// Information on the current status of the block
uint64_t nBlockWeight;
uint64_t nBlockTx;
uint64_t nBlockSigOpsCost;
CAmount nFees;
CTxMemPool::setEntries inBlock;
// Chain context for the block
int nHeight;
int64_t nLockTimeCutoff;
const CChainParams& chainparams;
public:
struct Options {
Options();
size_t nBlockMaxWeight;
CFeeRate blockMinFeeRate;
};
explicit BlockAssembler(const CChainParams& params);
BlockAssembler(const CChainParams& params, const Options& options);
/** Construct a new block template with coinbase to scriptPubKeyIn */
std::unique_ptr<CBlockTemplate> CreateNewBlock(const CScript& scriptPubKeyIn); //The most important thing is this member function to create a new block
static Optional<int64_t> m_last_block_num_txs;
static Optional<int64_t> m_last_block_weight;
private:
// utility functions
/** Clear the block's state and prepare for assembling a new block */
void resetBlock();
/** Add a tx to the block */
void AddToBlock(CTxMemPool::txiter iter);
// Methods for how to add transactions to a block.
/** Add transactions based on feerate including unconfirmed ancestors
* Increments nPackagesSelected / nDescendantsUpdated with corresponding
* statistics from the package selection (for logging statistics). */
void addPackageTxs(int &nPackagesSelected, int &nDescendantsUpdated) EXCLUSIVE_LOCKS_REQUIRED(mempool.cs);
// helper functions for addPackageTxs()
/** Remove confirmed (inBlock) entries from given set */
void onlyUnconfirmed(CTxMemPool::setEntries& testSet);
/** Test if a new package would "fit" in the block */
bool TestPackage(uint64_t packageSize, int64_t packageSigOpsCost) const;
/** Perform checks on each transaction in a package:
* locktime, premature-witness, serialized size (if necessary)
* These checks should always succeed, and they're here
* only as an extra check in case of suboptimal node configuration */
bool TestPackageTransactions(const CTxMemPool::setEntries& package);
/** Return true if given transaction from mapTx has already been evaluated,
* or if the transaction's cached data in mapTx is incorrect. */
bool SkipMapTxEntry(CTxMemPool::txiter it, indexed_modified_transaction_set &mapModifiedTx, CTxMemPool::setEntries &failedTx) EXCLUSIVE_LOCKS_REQUIRED(mempool.cs);
/** Sort the package in an order that is valid to appear in a block */
void SortForBlock(const CTxMemPool::setEntries& package, std::vector<CTxMemPool::txiter>& sortedEntries);
/** Add descendants of given transactions to mapModifiedTx with ancestor
* state updated assuming given transactions are inBlock. Returns number
* of updated descendants. */
int UpdatePackagesForAdded(const CTxMemPool::setEntries& alreadyAdded, indexed_modified_transaction_set &mapModifiedTx) EXCLUSIVE_LOCKS_REQUIRED(mempool.cs);
};Let's take a look at the source code of CreateNewBlock, the most important member function of BlockAssembler class:
//The parameter scriptPubKeyIn indicates the Coinbase transaction output locking script
std::unique_ptr<CBlockTemplate> BlockAssembler::CreateNewBlock(const CScript& scriptPubKeyIn)
{
int64_t nTimeStart = GetTimeMicros();
resetBlock();
pblocktemplate.reset(new CBlockTemplate());
if(!pblocktemplate.get())
return nullptr;
pblock = &pblocktemplate->block; // pointer for convenience
// Add dummy coinbase tx as first transaction
pblock->vtx.emplace_back();
pblocktemplate->vTxFees.push_back(-1); // updated at end
pblocktemplate->vTxSigOpsCost.push_back(-1); // updated at end
LOCK2(cs_main, mempool.cs);
CBlockIndex* pindexPrev = ::ChainActive().Tip(); //Select the current longest chain and the highest block
assert(pindexPrev != nullptr);
nHeight = pindexPrev->nHeight + 1;
pblock->nVersion = ComputeBlockVersion(pindexPrev, chainparams.GetConsensus()); //Set version number field
// -regtest only: allow overriding block.nVersion with
// -blockversion=N to test forking scenarios
if (chainparams.MineBlocksOnDemand())
pblock->nVersion = gArgs.GetArg("-blockversion", pblock->nVersion);
pblock->nTime = GetAdjustedTime(); //Set timestamp field
const int64_t nMedianTimePast = pindexPrev->GetMedianTimePast();
nLockTimeCutoff = (STANDARD_LOCKTIME_VERIFY_FLAGS & LOCKTIME_MEDIAN_TIME_PAST)
? nMedianTimePast
: pblock->GetBlockTime();
// Decide whether to include witness transactions
// This is only needed in case the witness softfork activation is reverted
// (which would require a very deep reorganization).
// Note that the mempool would accept transactions with witness data before
// IsWitnessEnabled, but we would only ever mine blocks after IsWitnessEnabled
// unless there is a massive block reorganization with the witness softfork
// not activated.
// TODO: replace this with a call to main to assess validity of a mempool
// transaction (which in most cases can be a no-op).
fIncludeWitness = IsWitnessEnabled(pindexPrev, chainparams.GetConsensus());
int nPackagesSelected = 0;
int nDescendantsUpdated = 0;
addPackageTxs(nPackagesSelected, nDescendantsUpdated); //Add transactions from the transaction pool to the block
int64_t nTime1 = GetTimeMicros();
m_last_block_num_txs = nBlockTx;
m_last_block_weight = nBlockWeight;
// Create coinbase transaction. Create coinbase transaction
CMutableTransaction coinbaseTx;
coinbaseTx.vin.resize(1);
coinbaseTx.vin[0].prevout.SetNull(); //Coinbase transactions have no input but only output
coinbaseTx.vout.resize(1);
coinbaseTx.vout[0].scriptPubKey = scriptPubKeyIn; //Lock script
coinbaseTx.vout[0].nValue = nFees + GetBlockSubsidy(nHeight, chainparams.GetConsensus()); //Output to miners, transaction fee + mining reward
coinbaseTx.vin[0].scriptSig = CScript() << nHeight << OP_0;
pblock->vtx[0] = MakeTransactionRef(std::move(coinbaseTx)); //The Coinbase transaction is placed first in the transaction list and the first transaction in the block
pblocktemplate->vchCoinbaseCommitment = GenerateCoinbaseCommitment(*pblock, pindexPrev, chainparams.GetConsensus());
pblocktemplate->vTxFees[0] = -nFees;
LogPrintf("CreateNewBlock(): block weight: %u txs: %u fees: %ld sigops %d\n", GetBlockWeight(*pblock), nBlockTx, nFees, nBlockSigOpsCost);
// Fill in header
pblock->hashPrevBlock = pindexPrev->GetBlockHash(); //Sets the hash field of the previous block in the block header
UpdateTime(pblock, chainparams.GetConsensus(), pindexPrev);
pblock->nBits = GetNextWorkRequired(pindexPrev, pblock, chainparams.GetConsensus()); //Calculate the difficulty value and set the difficulty value field. The specific calculation method will be described in the mining difficulty section below
pblock->nNonce = 0; //Random number, initially set to 0, waiting for continuous adjustment later
pblocktemplate->vTxSigOpsCost[0] = WITNESS_SCALE_FACTOR * GetLegacySigOpCount(*pblock->vtx[0]);
CValidationState state;
if (!TestBlockValidity(state, chainparams, *pblock, pindexPrev, false, false)) {
throw std::runtime_error(strprintf("%s: TestBlockValidity failed: %s", __func__, FormatStateMessage(state)));
}
int64_t nTime2 = GetTimeMicros();
LogPrint(BCLog::BENCH, "CreateNewBlock() packages: %.2fms (%d packages, %d updated descendants), validity: %.2fms (total %.2fms)\n", 0.001 * (nTime1 - nTimeStart), nPackagesSelected, nDescendantsUpdated, 0.001 * (nTime2 - nTime1), 0.001 * (nTime2 - nTimeStart));
return std::move(pblocktemplate);
}You can see that except for Merkel root hash field, other main fields have been constructed, and the construction of Merkel root hash field will be constructed in the following code.
void IncrementExtraNonce(CBlock* pblock, const CBlockIndex* pindexPrev, unsigned int& nExtraNonce)
{
// Update nExtraNonce
static uint256 hashPrevBlock;
if (hashPrevBlock != pblock->hashPrevBlock)
{
nExtraNonce = 0;
hashPrevBlock = pblock->hashPrevBlock;
}
++nExtraNonce;
unsigned int nHeight = pindexPrev->nHeight+1; // Height first in coinbase required for block.version=2
CMutableTransaction txCoinbase(*pblock->vtx[0]);
txCoinbase.vin[0].scriptSig = (CScript() << nHeight << CScriptNum(nExtraNonce)) + COINBASE_FLAGS;
assert(txCoinbase.vin[0].scriptSig.size() <= 100);
pblock->vtx[0] = MakeTransactionRef(std::move(txCoinbase));
pblock->hashMerkleRoot = BlockMerkleRoot(*pblock); // Calculate Merkel root hash
}
uint256 BlockMerkleRoot(const CBlock& block, bool* mutated)
{
// Calculate transaction hash value
std::vector<uint256> leaves;
leaves.resize(block.vtx.size());
for (size_t s = 0; s < block.vtx.size(); s++) {
leaves[s] = block.vtx[s]->GetHash();
}
return ComputeMerkleRoot(std::move(leaves), mutated);
}
uint256 ComputeMerkleRoot(std::vector<uint256> hashes, bool* mutated) {
bool mutation = false;
while (hashes.size() > 1) { // It is calculated layer by layer up to the root hash
if (mutated) {
for (size_t pos = 0; pos + 1 < hashes.size(); pos += 2) {
if (hashes[pos] == hashes[pos + 1]) mutation = true;
}
}
if (hashes.size() & 1) { // Odd number of transactions, and the last transaction is paired with yourself
hashes.push_back(hashes.back());
}
SHA256D64(hashes[0].begin(), hashes[0].begin(), hashes.size() / 2);
hashes.resize(hashes.size() / 2);
}
if (mutated) *mutated = mutation;
if (hashes.size() == 0) return uint256();
return hashes[0];
}Mining difficulty
In order to avoid the instability of bitcoin block out time caused by the fluctuation of computing power, the mining difficulty value in bitcoin is dynamically adjusted. The calculation formula of the specific difficulty value is as follows:
New Difficulty = Old Difficulty * (Actual Time of Last 2016 Blocks / 20160 minutes). Adjust every 2016 blocks, so that the block out interval of bitcoin is stabilized at about 10 minutes.
The source code of difficulty value calculation is as follows (it's very verbose, but actually it's good to know the above formula):
unsigned int GetNextWorkRequired(const CBlockIndex* pindexLast, const CBlockHeader *pblock, const Consensus::Params& params)
{
assert(pindexLast != nullptr);
unsigned int nProofOfWorkLimit = UintToArith256(params.powLimit).GetCompact();
// Only change once per difficulty adjustment interval
// Whether the block height is an integral multiple of 2016 blocks. If not, the difficulty value of the current blockchain vertex block will be used instead of adjusting the difficulty
if ((pindexLast->nHeight+1) % params.DifficultyAdjustmentInterval() != 0)
{
if (params.fPowAllowMinDifficultyBlocks)
{
// Special difficulty rule for testnet:
// If the new block's timestamp is more than 2* 10 minutes
// then allow mining of a min-difficulty block.
if (pblock->GetBlockTime() > pindexLast->GetBlockTime() + params.nPowTargetSpacing*2)
return nProofOfWorkLimit;
else
{
// Return the last non-special-min-difficulty-rules-block
const CBlockIndex* pindex = pindexLast;
while (pindex->pprev && pindex->nHeight % params.DifficultyAdjustmentInterval() != 0 && pindex->nBits == nProofOfWorkLimit)
pindex = pindex->pprev;
return pindex->nBits;
}
}
return pindexLast->nBits;
}
// Go back by what we want to be 14 days worth of blocks
int nHeightFirst = pindexLast->nHeight - (params.DifficultyAdjustmentInterval()-1);
assert(nHeightFirst >= 0);
const CBlockIndex* pindexFirst = pindexLast->GetAncestor(nHeightFirst);
assert(pindexFirst);
return CalculateNextWorkRequired(pindexLast, pindexFirst->GetBlockTime(), params);
}
// Calculate the next difficulty value
unsigned int CalculateNextWorkRequired(const CBlockIndex* pindexLast, int64_t nFirstBlockTime, const Consensus::Params& params)
{
if (params.fPowNoRetargeting)
return pindexLast->nBits;
// Limit adjustment step
int64_t nActualTimespan = pindexLast->GetBlockTime() - nFirstBlockTime;
if (nActualTimespan < params.nPowTargetTimespan/4)
nActualTimespan = params.nPowTargetTimespan/4;
if (nActualTimespan > params.nPowTargetTimespan*4)
nActualTimespan = params.nPowTargetTimespan*4;
// Retarget
const arith_uint256 bnPowLimit = UintToArith256(params.powLimit);
arith_uint256 bnNew;
bnNew.SetCompact(pindexLast->nBits);
bnNew *= nActualTimespan;
bnNew /= params.nPowTargetTimespan;
if (bnNew > bnPowLimit)
bnNew = bnPowLimit;
return bnNew.GetCompact();
}Mining reward
Miners compete for bookkeeping rights and can obtain transaction fees and mining rewards. The mining income will be exported to miners in Coinbase transactions. So how is the mining reward calculated? The source code is as follows:
//The reward value is calculated according to the block height. The first reward is 50 bitcoin, and then it will be reduced by half every 210000 blocks
CAmount GetBlockSubsidy(int nHeight, const Consensus::Params& consensusParams)
{
int halvings = nHeight / consensusParams.nSubsidyHalvingInterval;
// Force block reward to zero when right shift is undefined.
if (halvings >= 64)
return 0;
CAmount nSubsidy = 50 * COIN;
// Subsidy is cut in half every 210,000 blocks which will occur approximately every 4 years.
nSubsidy >>= halvings;
return nSubsidy;
}This can be found in the bitcoin blockchain browser. Before the 210000 block, the mining reward value of Coinbase (without transaction fee) is 50 bitcoin, while the reward value of the 210000 block is 25 bitcoin, and so on.
Next, we will carry out difficult calculation and compete for bookkeeping rights.
proof of work
The process of workload proof is as follows, that is, constantly changing the nNonce value until it succeeds or receives a new block generated by other nodes, so as to start the process of workload proof of the next new block.
static UniValue generateBlocks(const CScript& coinbase_script, int nGenerate, uint64_t nMaxTries)
{
static const int nInnerLoopCount = 0x10000;
int nHeightEnd = 0;
int nHeight = 0;
{ // Don't keep cs_main locked
LOCK(cs_main);
nHeight = ::ChainActive().Height();
nHeightEnd = nHeight+nGenerate;
}
unsigned int nExtraNonce = 0;
UniValue blockHashes(UniValue::VARR);
while (nHeight < nHeightEnd && !ShutdownRequested()) //If the specified block height is not reached or the mining request is not stopped, mining will continue
{
std::unique_ptr<CBlockTemplate> pblocktemplate(BlockAssembler(Params()).CreateNewBlock(coinbase_script)); //Construct a new block
if (!pblocktemplate.get())
throw JSONRPCError(RPC_INTERNAL_ERROR, "Couldn't create new block");
CBlock *pblock = &pblocktemplate->block;
{
LOCK(cs_main);
IncrementExtraNonce(pblock, ::ChainActive().Tip(), nExtraNonce); //Merkel root hash is calculated and assigned here.
}
//Constantly changing the nNonce value, the calculation is difficult
while (nMaxTries > 0 && pblock->nNonce < nInnerLoopCount && !CheckProofOfWork(pblock->GetHash(), pblock->nBits, Params().GetConsensus())) {
++pblock->nNonce;
--nMaxTries;
}
if (nMaxTries == 0) {
break;
}
if (pblock->nNonce == nInnerLoopCount) {
continue;
}
std::shared_ptr<const CBlock> shared_pblock = std::make_shared<const CBlock>(*pblock);
if (!ProcessNewBlock(Params(), shared_pblock, true, nullptr))
throw JSONRPCError(RPC_INTERNAL_ERROR, "ProcessNewBlock, block not accepted");
++nHeight;
blockHashes.push_back(pblock->GetHash().GetHex());
}
return blockHashes;
}It is conceivable that this CPU mining code has no possibility of successful mining in front of GPU or special mining machine, and some bitcoin implementations have removed the mining code. The purpose of analyzing this code here is to prove a clear understanding of the mining process and deepen the understanding of the bitcoin workflow.
In addition, we need to expand here. At the beginning of mining, the difficulty value is very low. We only need to constantly modify the random number nNonce value. Now, with the improvement of computing power, the difficulty is becoming greater and greater. In fact, miners can modify not only nNonce, but also the value of nTime timestamp, because as long as nTime meets the requirements, A certain error is allowed (and the time cannot be completely synchronized). You can also modify the script in the transaction input in the Coinbase transaction to indirectly modify the Merkel root hash value hashMerkleRoot, so as to expand the random source. Therefore, the actual mining program is different from the CPU mining program above. And the most time-consuming operation is the hash operation process, and the mining machine will generally focus on the hash The algorithm is optimized.
Next, let's continue the analysis:
// Check whether the difficulty conditions are met
bool CheckProofOfWork(uint256 hash, unsigned int nBits, const Consensus::Params& params)
{
bool fNegative;
bool fOverflow;
arith_uint256 bnTarget;
bnTarget.SetCompact(nBits, &fNegative, &fOverflow);
// Check range
if (fNegative || bnTarget == 0 || fOverflow || bnTarget > UintToArith256(params.powLimit))
return false;
// Check proof of work matches claimed amount
if (UintToArith256(hash) > bnTarget)
return false;
return true;
}If the above difficulty conditions are met and the accounting right of the new block is successfully competed, the following process will be entered: the new block processing process, which mainly includes verifying the new block. If it passes the verification, it will be stored in the disk and broadcast through the network, and select to join the longest chain.
// New block processing flow
bool ProcessNewBlock(const CChainParams& chainparams, const std::shared_ptr<const CBlock> pblock, bool fForceProcessing, bool *fNewBlock)
{
AssertLockNotHeld(cs_main);
{
CBlockIndex *pindex = nullptr;
if (fNewBlock) *fNewBlock = false;
CValidationState state;
// CheckBlock() does not support multi-threaded block validation because CBlock::fChecked can cause data race.
// Therefore, the following critical section must include the CheckBlock() call as well.
LOCK(cs_main);
// Ensure that CheckBlock() passes before calling AcceptBlock, as
// belt-and-suspenders.
bool ret = CheckBlock(*pblock, state, chainparams.GetConsensus()); //Check the block. If it passes the check, it will be stored in the disk
if (ret) {
// Store to disk
ret = ::ChainstateActive().AcceptBlock(pblock, state, chainparams, &pindex, fForceProcessing, nullptr, fNewBlock);
}
if (!ret) {
GetMainSignals().BlockChecked(*pblock, state);
return error("%s: AcceptBlock FAILED (%s)", __func__, FormatStateMessage(state));
}
}
NotifyHeaderTip();
CValidationState state; // Only used to report errors, not invalidity - ignore it
if (!::ChainstateActive().ActivateBestChain(state, chainparams, pblock)) //Select to join the longest chain
return error("%s: ActivateBestChain failed (%s)", __func__, FormatStateMessage(state));
return true;
}First, verify the new block
// Verify new block
bool CheckBlock(const CBlock& block, CValidationState& state, const Consensus::Params& consensusParams, bool fCheckPOW, bool fCheckMerkleRoot)
{
// These are checks that are independent of context.
if (block.fChecked)
return true;
// Check that the header is valid (particularly PoW). This is mostly
// redundant with the call in AcceptBlockHeader.
if (!CheckBlockHeader(block, state, consensusParams, fCheckPOW))
return false;
// Check the merkle root.
if (fCheckMerkleRoot) {
bool mutated;
uint256 hashMerkleRoot2 = BlockMerkleRoot(block, &mutated); //Check whether Merkel root hash is consistent
if (block.hashMerkleRoot != hashMerkleRoot2)
return state.Invalid(ValidationInvalidReason::BLOCK_MUTATED, false, REJECT_INVALID, "bad-txnmrklroot", "hashMerkleRoot mismatch");
// Check for merkle tree malleability (CVE-2012-2459): repeating sequences
// of transactions in a block without affecting the merkle root of a block,
// while still invalidating it.
if (mutated)
return state.Invalid(ValidationInvalidReason::BLOCK_MUTATED, false, REJECT_INVALID, "bad-txns-duplicate", "duplicate transaction");
}
// All potential-corruption validation must be done before we do any
// transaction validation, as otherwise we may mark the header as invalid
// because we receive the wrong transactions for it.
// Note that witness malleability is checked in ContextualCheckBlock, so no
// checks that use witness data may be performed here.
// Size limits the block size is within the length limit
if (block.vtx.empty() || block.vtx.size() * WITNESS_SCALE_FACTOR > MAX_BLOCK_WEIGHT || ::GetSerializeSize(block, PROTOCOL_VERSION | SERIALIZE_TRANSACTION_NO_WITNESS) * WITNESS_SCALE_FACTOR > MAX_BLOCK_WEIGHT)
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-blk-length", "size limits failed");
// First transaction must be coinbase, the rest must not be
if (block.vtx.empty() || !block.vtx[0]->IsCoinBase())
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-cb-missing", "first tx is not coinbase");
for (unsigned int i = 1; i < block.vtx.size(); i++)
if (block.vtx[i]->IsCoinBase())
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-cb-multiple", "more than one coinbase");
// Check transactions uses checklists to validate transactions within blocks and ensure their validity
for (const auto& tx : block.vtx)
if (!CheckTransaction(*tx, state, true))
return state.Invalid(state.GetReason(), false, state.GetRejectCode(), state.GetRejectReason(),
strprintf("Transaction check failed (tx hash %s) %s", tx->GetHash().ToString(), state.GetDebugMessage()));
unsigned int nSigOps = 0;
for (const auto& tx : block.vtx)
{
nSigOps += GetLegacySigOpCount(*tx);
}
if (nSigOps * WITNESS_SCALE_FACTOR > MAX_BLOCK_SIGOPS_COST)
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-blk-sigops", "out-of-bounds SigOpCount");
if (fCheckPOW && fCheckMerkleRoot)
block.fChecked = true;
return true;
}
// Verify whether the workload certificate is met
static bool CheckBlockHeader(const CBlockHeader& block, CValidationState& state, const Consensus::Params& consensusParams, bool fCheckPOW = true)
{
// Check proof of work matches claimed amount
if (fCheckPOW && !CheckProofOfWork(block.GetHash(), block.nBits, consensusParams))
return state.Invalid(ValidationInvalidReason::BLOCK_INVALID_HEADER, false, REJECT_INVALID, "high-hash", "proof of work failed");
return true;
}
// Verification transaction
bool CheckTransaction(const CTransaction& tx, CValidationState &state, bool fCheckDuplicateInputs)
{
// Basic checks that don't depend on any context
if (tx.vin.empty())
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-txns-vin-empty");
if (tx.vout.empty())
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-txns-vout-empty");
// Size limits (this doesn't take the witness into account, as that hasn't been checked for malleability)
if (::GetSerializeSize(tx, PROTOCOL_VERSION | SERIALIZE_TRANSACTION_NO_WITNESS) * WITNESS_SCALE_FACTOR > MAX_BLOCK_WEIGHT)
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-txns-oversize");
// Check for negative or overflow output values
CAmount nValueOut = 0;
for (const auto& txout : tx.vout)
{
if (txout.nValue < 0)
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-txns-vout-negative");
if (txout.nValue > MAX_MONEY)
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-txns-vout-toolarge");
nValueOut += txout.nValue;
if (!MoneyRange(nValueOut))
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-txns-txouttotal-toolarge");
}
// Check for duplicate inputs - note that this check is slow so we skip it in CheckBlock
if (fCheckDuplicateInputs) {
std::set<COutPoint> vInOutPoints;
for (const auto& txin : tx.vin)
{
if (!vInOutPoints.insert(txin.prevout).second)
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-txns-inputs-duplicate");
}
}
if (tx.IsCoinBase())
{
if (tx.vin[0].scriptSig.size() < 2 || tx.vin[0].scriptSig.size() > 100)
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-cb-length");
}
else
{
for (const auto& txin : tx.vin)
if (txin.prevout.IsNull())
return state.Invalid(ValidationInvalidReason::CONSENSUS, false, REJECT_INVALID, "bad-txns-prevout-null");
}
return true;
}After the above new block is verified, save the new block to disk:
/** Store block on disk. If dbp is non-nullptr, the file is known to already reside on disk */
bool CChainState::AcceptBlock(const std::shared_ptr<const CBlock>& pblock, CValidationState& state, const CChainParams& chainparams, CBlockIndex** ppindex, bool fRequested, const FlatFilePos* dbp, bool* fNewBlock)
{
const CBlock& block = *pblock;
if (fNewBlock) *fNewBlock = false;
AssertLockHeld(cs_main);
CBlockIndex *pindexDummy = nullptr;
CBlockIndex *&pindex = ppindex ? *ppindex : pindexDummy;
if (!AcceptBlockHeader(block, state, chainparams, &pindex))
return false;
// Try to process all requested blocks that we don't have, but only
// process an unrequested block if it's new and has enough work to
// advance our tip, and isn't too many blocks ahead.
bool fAlreadyHave = pindex->nStatus & BLOCK_HAVE_DATA;
bool fHasMoreOrSameWork = (m_chain.Tip() ? pindex->nChainWork >= m_chain.Tip()->nChainWork : true);
// Blocks that are too out-of-order needlessly limit the effectiveness of
// pruning, because pruning will not delete block files that contain any
// blocks which are too close in height to the tip. Apply this test
// regardless of whether pruning is enabled; it should generally be safe to
// not process unrequested blocks.
bool fTooFarAhead = (pindex->nHeight > int(m_chain.Height() + MIN_BLOCKS_TO_KEEP));
// TODO: Decouple this function from the block download logic by removing fRequested
// This requires some new chain data structure to efficiently look up if a
// block is in a chain leading to a candidate for best tip, despite not
// being such a candidate itself.
// TODO: deal better with return value and error conditions for duplicate
// and unrequested blocks.
if (fAlreadyHave) return true;
if (!fRequested) { // If we didn't ask for it:
if (pindex->nTx != 0) return true; // This is a previously-processed block that was pruned
if (!fHasMoreOrSameWork) return true; // Don't process less-work chains
if (fTooFarAhead) return true; // Block height is too high
// Protect against DoS attacks from low-work chains.
// If our tip is behind, a peer could try to send us
// low-work blocks on a fake chain that we would never
// request; don't process these.
if (pindex->nChainWork < nMinimumChainWork) return true;
}
if (!CheckBlock(block, state, chainparams.GetConsensus()) ||
!ContextualCheckBlock(block, state, chainparams.GetConsensus(), pindex->pprev)) {
assert(IsBlockReason(state.GetReason()));
if (state.IsInvalid() && state.GetReason() != ValidationInvalidReason::BLOCK_MUTATED) {
pindex->nStatus |= BLOCK_FAILED_VALID;
setDirtyBlockIndex.insert(pindex);
}
return error("%s: %s", __func__, FormatStateMessage(state));
}
// Header is valid/has work, merkle tree and segwit merkle tree are good...RELAY NOW
// (but if it does not build on our best tip, let the SendMessages loop relay it)
if (!IsInitialBlockDownload() && m_chain.Tip() == pindex->pprev)
GetMainSignals().NewPoWValidBlock(pindex, pblock); // If it's the latest block, broadcast it
// Write block to history file
if (fNewBlock) *fNewBlock = true;
try {
FlatFilePos blockPos = SaveBlockToDisk(block, pindex->nHeight, chainparams, dbp);
if (blockPos.IsNull()) {
state.Error(strprintf("%s: Failed to find position to write new block to disk", __func__));
return false;
}
ReceivedBlockTransactions(block, pindex, blockPos, chainparams.GetConsensus());
} catch (const std::runtime_error& e) {
return AbortNode(state, std::string("System error: ") + e.what());
}
FlushStateToDisk(chainparams, state, FlushStateMode::NONE);
CheckBlockIndex(chainparams.GetConsensus());
return true;
}In fact, the broadcast is the block header, and the source code is as follows:
/**
* Maintain state about the best-seen block and fast-announce a compact block
* to compatible peers.
*/
void PeerLogicValidation::NewPoWValidBlock(const CBlockIndex *pindex, const std::shared_ptr<const CBlock>& pblock) {
std::shared_ptr<const CBlockHeaderAndShortTxIDs> pcmpctblock = std::make_shared<const CBlockHeaderAndShortTxIDs> (*pblock, true);
const CNetMsgMaker msgMaker(PROTOCOL_VERSION);
LOCK(cs_main);
static int nHighestFastAnnounce = 0;
if (pindex->nHeight <= nHighestFastAnnounce)
return;
nHighestFastAnnounce = pindex->nHeight;
bool fWitnessEnabled = IsWitnessEnabled(pindex->pprev, Params().GetConsensus());
uint256 hashBlock(pblock->GetHash());
{
LOCK(cs_most_recent_block);
most_recent_block_hash = hashBlock;
most_recent_block = pblock;
most_recent_compact_block = pcmpctblock;
fWitnessesPresentInMostRecentCompactBlock = fWitnessEnabled;
}
// Traverse the connection node and send it out
connman->ForEachNode([this, &pcmpctblock, pindex, &msgMaker, fWitnessEnabled, &hashBlock](CNode* pnode) {
AssertLockHeld(cs_main);
// TODO: Avoid the repeated-serialization here
if (pnode->nVersion < INVALID_CB_NO_BAN_VERSION || pnode->fDisconnect)
return;
ProcessBlockAvailability(pnode->GetId());
CNodeState &state = *State(pnode->GetId());
// If the peer has, or we announced to them the previous block already,
// but we don't think they have this one, go ahead and announce it
if (state.fPreferHeaderAndIDs && (!fWitnessEnabled || state.fWantsCmpctWitness) &&
!PeerHasHeader(&state, pindex) && PeerHasHeader(&state, pindex->pprev)) {
LogPrint(BCLog::NET, "%s sending header-and-ids %s to peer=%d\n", "PeerLogicValidation::NewPoWValidBlock",
hashBlock.ToString(), pnode->GetId());
connman->PushMessage(pnode, msgMaker.Make(NetMsgType::CMPCTBLOCK, *pcmpctblock));
state.pindexBestHeaderSent = pindex;
}
});
}Next, choose to join the longest chain, which involves the bifurcation of bitcoin blockchain and related processing logic. It will not be described in detail here. Those interested can read the bitcoin source code themselves. Moreover, not all blockchains have the concept of bifurcation. In blockchains such as license chains that adopt PBFT consensus algorithm, once a consensus is formed, each block is deterministic, while bitcoin is not deterministic after the generation of new blocks due to Pow consensus algorithm, so it has the concept of bifurcation.
So far, these are basically the processes of bitcoin mining. It can be seen that Pow consensus algorithm is very simple and efficient in the public chain. Compared with traditional consensus algorithms PBFT and Raft, its biggest feature is that the consensus process does not need to interact with other nodes, Whether there is only one node or thousands of nodes in the network (at present, the scale of consensus (mining) nodes of bitcoin is about 10000), so it has high adaptability to the dynamic changes of the network. However, the cost is that it needs the support of huge computing power, which is not a problem in the traditional consensus algorithm.