The headless daemon bitcoind has the JSON-RPC API enabled by default, the GUI bitcoin-qt has it disabled by default. This can be changed with the -server option. In the GUI it is possible to execute RPC methods in the Debug Console Dialog.
The RPC interface might change from one major version of Bitcoin Core to the next. This makes the RPC interface implicitly versioned on the major version. The version tuple can be retrieved by e.g. the getnetworkinfo RPC in version.
Usually deprecated features can be re-enabled during the grace-period of one major version via the -deprecatedrpc= command line option. The release notes of a new major release come with detailed instructions on what RPC features were deprecated and how to re-enable them temporarily.
The RPC interface allows other programs to control Bitcoin Core, including the ability to spend funds from your wallets, affect consensus verification, read private data, and otherwise perform operations that can cause loss of money, data, or privacy. This section suggests how you should use and configure Bitcoin Core to reduce the risk that its RPC interface will be abused.
Securing the executable: Anyone with physical or remote access to the computer, container, or virtual machine running Bitcoin Core can compromise either the whole program or just the RPC interface. This includes being able to record any passphrases you enter for unlocking your encrypted wallets or changing settings so that your Bitcoin Core program tells you that certain transactions have multiple confirmations even when they aren't part of the best block chain. For this reason, you should not use Bitcoin Core for security sensitive operations on systems you do not exclusively control, such as shared computers or virtual private servers.
Securing local network access: By default, the RPC interface can only be accessed by a client running on the same computer and only after the client provides a valid authentication credential (username and passphrase). Any program on your computer with access to the file system and local network can obtain this level of access. Additionally, other programs on your computer can attempt to provide an RPC interface on the same port as used by Bitcoin Core in order to trick you into revealing your authentication credentials. For this reason, it is important to only use Bitcoin Core for security-sensitive operations on a computer whose other programs you trust.
Securing remote network access: You may optionally allow other computers to remotely control Bitcoin Core by setting the rpcallowip and rpcbind configuration parameters. These settings are only meant for enabling connections over secure private networks or connections that have been otherwise secured (e.g. using a VPN or port forwarding with SSH or stunnel). Do not enable RPC connections over the public Internet. Although Bitcoin Core's RPC interface does use authentication, it does not use encryption, so your login credentials are sent as clear text that can be read by anyone on your network path. Additionally, the RPC interface has not been hardened to withstand arbitrary Internet traffic, so changing the above settings to expose it to the Internet (even using something like a Tor onion service) could expose you to unconsidered vulnerabilities. See bitcoind -help for more information about these settings and other settings described in this document.
Related, if you use Bitcoin Core inside a Docker container, you may need to expose the RPC port to the host system. The default way to do this in Docker also exposes the port to the public Internet. Instead, expose it only on the host system's localhost, for example: -p 127.0.0.1:8332:8332
Secure authentication: By default, Bitcoin Core generates unique login credentials each time it restarts and puts them into a file readable only by the user that started Bitcoin Core, allowing any of that user's RPC clients with read access to the file to login automatically. The file is .cookie in the Bitcoin Core configuration directory, and using these credentials is the preferred RPC authentication method. If you need to generate static login credentials for your programs, you can use the script in the share/rpcauth directory in the Bitcoin Core source tree. As a final fallback, you can directly use manually-chosen rpcuser and rpcpassword configuration parameters---but you must ensure that you choose a strong and unique passphrase (and still don't use insecure networks, as mentioned above).
Secure string handling: The RPC interface does not guarantee any escaping of data beyond what's necessary to encode it as JSON, although it does usually provide serialized data using a hex representation of the bytes. If you use RPC data in your programs or provide its data to other programs, you must ensure any problem strings are properly escaped. For example, multiple websites have been manipulated because they displayed decoded hex strings that included HTML <script> tags. For this reason, and other non-security reasons, it is recommended to display all serialized data in hex form only.
State that can be queried via RPCs is guaranteed to be at least up-to-date with the chain state immediately prior to the call's execution. However, the state returned by RPCs that reflect the mempool may not be up-to-date with the current mempool state.
The mempool state returned via an RPC is consistent with itself and with the chain state at the time of the call. Thus, the mempool state only encompasses transactions that are considered mine-able by the node at the time of the RPC.
The mempool state returned via an RPC reflects all effects of mempool and chain state related RPCs that returned prior to this call.
The wallet state returned via an RPC is consistent with itself and with the chain state at the time of the call.
Wallet RPCs will return the latest chain state consistent with prior non-wallet RPCs. The effects of all blocks (and transactions in blocks) at the time of the call is reflected in the state of all wallet transactions. For example, if a block contains transactions that conflicted with mempool transactions, the wallet would reflect the removal of these mempool transactions in the state.
However, the wallet may not be up-to-date with the current state of the mempool or the state of the mempool by an RPC that returned before this RPC. For example, a wallet transaction that was BIP-125-replaced in the mempool prior to this RPC may not yet be reflected as such in this RPC response.
Bitcoin Core is the original Bitcoin client and it builds the backbone of the network. It downloads and, by default, stores the entire history of Bitcoin transactions, which requires a few hundred gigabytes of disk space. Depending on the speed of your computer and network connection, the synchronization process can take anywhere from a few hours to a day or more.
To download Bitcoin Core, visit bitcoincore.org.
The following are some helpful notes on how to run Bitcoin Core on your native platform.
Unpack the files into a directory and run:
bin/bitcoin-qt (GUI) orbin/bitcoind (headless)Unpack the files into a directory, and then run bitcoin-qt.exe.
Drag Bitcoin Core to your applications folder, and then run Bitcoin Core.
The following are developer notes on how to build Bitcoin Core on your native platform. They are not complete guides, but include notes on the necessary libraries, compile flags, etc.
The Bitcoin repo's root README contains relevant information on the development process and automated testing.
Distributed under the MIT software license. \mainpage notitle
\section intro_sec Introduction
This is the developer documentation of the reference client for an experimental new digital currency called Bitcoin, which enables instant payments to anyone, anywhere in the world. Bitcoin uses peer-to-peer technology to operate with no central authority: managing transactions and issuing money are carried out collectively by the network.
The software is a community-driven open source project, released under the MIT license.
See https://github.com/bitcoin/bitcoin and https://bitcoincore.org/ for further information about the project.
\section Navigation Use Modules, Namespaces, Classes, or Files at the top of the page to start navigating the code.
The REST API can be enabled with the -rest option.
The interface runs on the same port as the JSON-RPC interface, by default port 8332 for mainnet, port 18332 for testnet, and port 18443 for regtest.
The same guarantees as for the RPC Interface apply.
There is a known issue in the REST interface that can cause a node to crash if too many http connections are being opened at the same time because the system runs out of available file descriptors. To prevent this from happening you might want to increase the number of maximum allowed file descriptors in your system and try to prevent opening too many connections to your rest interface at the same time if this is under your control. It is hard to give general advice since this depends on your system but if you make several hundred requests at once you are definitely at risk of encountering this issue.
GET /rest/tx/<TX-HASH>.<bin|hex|json>
Given a transaction hash: returns a transaction in binary, hex-encoded binary, or JSON formats.
By default, this endpoint will only search the mempool. To query for a confirmed transaction, enable the transaction index via "txindex=1" command line / configuration option.
GET /rest/block/<BLOCK-HASH>.<bin|hex|json> GET /rest/block/notxdetails/<BLOCK-HASH>.<bin|hex|json>
Given a block hash: returns a block, in binary, hex-encoded binary or JSON formats. Responds with 404 if the block doesn't exist.
The HTTP request and response are both handled entirely in-memory.
With the /notxdetails/ option JSON response will only contain the transaction hash instead of the complete transaction details. The option only affects the JSON response.
GET /rest/headers/<COUNT>/<BLOCK-HASH>.<bin|hex|json>
Given a block hash: returns
GET /rest/blockhashbyheight/<HEIGHT>.<bin|hex|json>
Given a height: returns hash of block in best-block-chain at height provided.
GET /rest/chaininfo.json
Returns various state info regarding block chain processing. Only supports JSON as output format.
GET /rest/getutxos/<checkmempool>/<txid>-<n>/<txid>-<n>/.../<txid>-<n>.<bin|hex|json>
The getutxo command allows querying of the UTXO set given a set of outpoints. See BIP64 for input and output serialisation: https://github.com/bitcoin/bips/blob/master/bip-0064.mediawiki
Example:
$ curl localhost:18332/rest/getutxos/checkmempool/b2cdfd7b89def827ff8af7cd9bff7627ff72e5e8b0f71210f92ea7a4000c5d75-0.json 2>/dev/null | json_pp
{
"chainHeight" : 325347,
"chaintipHash" : "00000000fb01a7f3745a717f8caebee056c484e6e0bfe4a9591c235bb70506fb",
"bitmap": "1",
"utxos" : [
{
"height" : 2147483647,
"value" : 8.8687,
"scriptPubKey" : {
"asm" : "OP_DUP OP_HASH160 1c7cebb529b86a04c683dfa87be49de35bcf589e OP_EQUALVERIFY OP_CHECKSIG",
"hex" : "76a9141c7cebb529b86a04c683dfa87be49de35bcf589e88ac",
"reqSigs" : 1,
"type" : "pubkeyhash",
"addresses" : [
"mi7as51dvLJsizWnTMurtRmrP8hG2m1XvD"
]
}
}
]
}
GET /rest/mempool/info.json
Returns various information about the TX mempool. Only supports JSON as output format.
GET /rest/mempool/contents.json
Returns transactions in the TX mempool. Only supports JSON as output format.
<script src="http://127.0.0.1:8332/rest/tx/1234567890.json"> which might break the nodes privacy. The list of assets used in the bitcoin source and their attribution can now be found in contrib/debian/copyright. BenchmarkingBitcoin Core has an internal benchmarking framework, with benchmarks for cryptographic algorithms (e.g. SHA1, SHA256, SHA512, RIPEMD160, Poly1305, ChaCha20), rolling bloom filter, coins selection, thread queue, wallet balance.
For benchmarks purposes you only need to compile bitcoin_bench. Beware of configuring without --enable-debug as this would impact benchmarking by unlatching log printers and lock analysis.
make -C src bitcoin_bench
After compiling bitcoin-core, the benchmarks can be run with:
src/bench/bench_bitcoin
The output will look similar to:
| ns/byte | byte/s | error % | benchmark
|--------------------:|--------------------:|--------:|:----------------------------------------------
| 64.13 | 15,592,356.01 | 0.1% | `Base58CheckEncode`
| 24.56 | 40,722,672.68 | 0.2% | `Base58Decode`
...
src/bench/bench_bitcoin --help
To print options like scaling factor or per-benchmark filter.
More benchmarks are needed for, in no particular order:
To monitor Bitcoin Core performance more in depth (like reindex or IBD): https://github.com/chaincodelabs/bitcoinperf
To generate Flame Graphs for Bitcoin Core: https://github.com/eklitzke/bitcoin/blob/flamegraphs/doc/flamegraphs.html BIPs that are implemented by Bitcoin Core (up-to-date up to v0.21.0):
BIP 9: The changes allowing multiple soft-forks to be deployed in parallel have been implemented since v0.12.1 (PR #7575)BIP 11: Multisig outputs are standard since v0.6.0 (PR #669).BIP 13: The address format for P2SH addresses has been implemented since v0.6.0 (PR #669).BIP 14: The subversion string is being used as User Agent since v0.6.0 (PR #669).BIP 16: The pay-to-script-hash evaluation rules have been implemented since v0.6.0, and took effect on April 1st 2012 (PR #748).BIP 21: The URI format for Bitcoin payments has been implemented since v0.6.0 (PR #176).BIP 22: The 'getblocktemplate' (GBT) RPC protocol for mining has been implemented since v0.7.0 (PR #936).BIP 23: Some extensions to GBT have been implemented since v0.10.0rc1, including longpolling and block proposals (PR #1816).BIP 30: The evaluation rules to forbid creating new transactions with the same txid as previous not-fully-spent transactions were implemented since v0.6.0, and the rule took effect on March 15th 2012 (PR #915).BIP 31: The 'pong' protocol message (and the protocol version bump to 60001) has been implemented since v0.6.1 (PR #1081).BIP 32: Hierarchical Deterministic Wallets has been implemented since v0.13.0 (PR #8035).BIP 34: The rule that requires blocks to contain their height (number) in the coinbase input, and the introduction of version 2 blocks has been implemented since v0.7.0. The rule took effect for version 2 blocks as of block 224413 (March 5th 2013), and version 1 blocks are no longer allowed since block 227931 (March 25th 2013) (PR #1526).BIP 35: The 'mempool' protocol message (and the protocol version bump to 60002) has been implemented since v0.7.0 (PR #1641). As of v0.13.0, this is only available for NODE_BLOOM (BIP 111) peers.BIP 37: The bloom filtering for transaction relaying, partial Merkle trees for blocks, and the protocol version bump to 70001 (enabling low-bandwidth SPV clients) has been implemented since v0.8.0 (PR #1795). Disabled by default since v0.19.0, can be enabled by the -peerbloomfilters option.BIP 42: The bug that would have caused the subsidy schedule to resume after block 13440000 was fixed in v0.9.2 (PR #3842).BIP 61: The 'reject' protocol message (and the protocol version bump to 70002) was added in v0.9.0 (PR #3185). Starting v0.17.0, whether to send reject messages can be configured with the -enablebip61 option, and support is deprecated (disabled by default) as of v0.18.0. Support was removed in v0.20.0 (PR #15437).BIP 65: The CHECKLOCKTIMEVERIFY softfork was merged in v0.12.0 (PR #6351), and backported to v0.11.2 and v0.10.4. Mempool-only CLTV was added in PR #6124.BIP 66: The strict DER rules and associated version 3 blocks have been implemented since v0.10.0 (PR #5713).BIP 68: Sequence locks have been implemented as of v0.12.1 (PR #7184), and have been buried since v0.19.0 (PR #16060).BIP 70 71 72: Payment Protocol support has been available in Bitcoin Core GUI since v0.9.0 (PR #5216). Support can be optionally disabled at build time since v0.18.0 (PR 14451), and it is disabled by default at build time since v0.19.0 (PR #15584). It has been removed as of v0.20.0 (PR 17165).BIP 90: Trigger mechanism for activation of BIPs 34, 65, and 66 has been simplified to block height checks since v0.14.0 (PR #8391).BIP 111: NODE_BLOOM service bit added, and enforced for all peer versions as of v0.13.0 (PR #6579 and PR #6641).BIP 112: The CHECKSEQUENCEVERIFY opcode has been implemented since v0.12.1 (PR #7524), and has been buried since v0.19.0 (PR #16060).BIP 113: Median time past lock-time calculations have been implemented since v0.12.1 (PR #6566), and has been buried since v0.19.0 (PR #16060).BIP 125: Opt-in full replace-by-fee signaling honoured in mempool and mining as of v0.12.0 (PR 6871). Enabled by default in the wallet GUI as of v0.18.1 (PR #11605)BIP 130: direct headers announcement is negotiated with peer versions >=70012 as of v0.12.0 (PR 6494).BIP 133: feefilter messages are respected and sent for peer versions >=70013 as of v0.13.0 (PR 7542).BIP 141: Segregated Witness (Consensus Layer) as of v0.13.0 (PR 8149), defined for mainnet as of v0.13.1 (PR 8937), and buried since v0.19.0 (PR #16060).BIP 143: Transaction Signature Verification for Version 0 Witness Program as of v0.13.0 (PR 8149), defined for mainnet as of v0.13.1 (PR 8937), and buried since v0.19.0 (PR #16060).BIP 144: Segregated Witness as of 0.13.0 (PR 8149).BIP 145: getblocktemplate updates for Segregated Witness as of v0.13.0 (PR 8149).BIP 147: NULLDUMMY softfork as of v0.13.1 (PR 8636 and PR 8937), buried since v0.19.0 (PR #16060).BIP 152: Compact block transfer and related optimizations are used as of v0.13.0 (PR 8068).BIP 155: The 'addrv2' and 'sendaddrv2' messages which enable relay of Tor V3 addresses (and other networks) are supported as of v0.21.0 (PR 19954).BIP 158: Compact Block Filters for Light Clients can be indexed as of v0.19.0 (PR #14121).BIP 159: The NODE_NETWORK_LIMITED service bit is signalled as of v0.16.0 (PR 11740), and such nodes are connected to as of v0.17.0 (PR 10387).BIP 173: Bech32 addresses for native Segregated Witness outputs are supported as of v0.16.0 (PR 11167). Bech32 addresses are generated by default as of v0.20.0 (PR 16884).BIP 174: RPCs to operate on Partially Signed Bitcoin Transactions (PSBT) are present as of v0.17.0 (PR 13557).BIP 176: Bits Denomination [QT only] is supported as of v0.16.0 (PR 12035).BIP 325: Signet test network is supported as of v0.21.0 (PR 18267).BIP 339: Relay of transactions by wtxid is supported as of v0.21.0 (PR 18044).BIP 340 341 342: Validation rules for Taproot (including Schnorr signatures and Tapscript leaves) are implemented as of v0.21.0 (PR 19953), without mainnet activation.bitcoin.conf Configuration FileThe configuration file is used by bitcoind, bitcoin-qt and bitcoin-cli.
All command-line options (except for -?, -help, -version and -conf) may be specified in a configuration file, and all configuration file options (except for includeconf) may also be specified on the command line. Command-line options override values set in the configuration file and configuration file options override values set in the GUI.
The configuration file is a plain text file and consists of option=value entries, one per line. Leading and trailing whitespaces are removed.
In contrast to the command-line usage:
-;testnet=1 (for chain selection options), noconnect=1 (for negated options).Blank lines are allowed and ignored by the parser.
A comment starts with a number sign (#) and extends to the end of the line. All comments are ignored by the parser.
Comments may appear in two ways:
option=value entry.Network specific options can be:
[main] (not [mainnet]), [test] (not [testnet]) or [regtest];regtest.maxmempool=100.Network specific options take precedence over non-network specific options. If multiple values for the same option are found with the same precedence, the first one is generally chosen.
This means that given the following configuration, regtest.rpcport is set to 3000:
regtest=1
rpcport=2000
regtest.rpcport=3000
[regtest]
rpcport=4000
The configuration file is not automatically created; you can create it using your favorite text editor. By default, the configuration file name is bitcoin.conf and it is located in the Bitcoin data directory, but both the Bitcoin data directory and the configuration file path may be changed using the -datadir and -conf command-line options.
The includeconf=<file> option in the bitcoin.conf file can be used to include additional configuration files.
| Operating System | Data Directory | Example Path |
|---|---|---|
| Windows | %APPDATA%\Bitcoin\ |
C:\Users\username\AppData\Roaming\Bitcoin\bitcoin.conf |
| Linux | $HOME/.bitcoin/ |
/home/username/.bitcoin/bitcoin.conf |
| macOS | $HOME/Library/Application Support/Bitcoin/ |
/Users/username/Library/Application Support/Bitcoin/bitcoin.conf |
You can find an example bitcoin.conf file in share/examples/bitcoin.conf.
Updated for FreeBSD 12.2
This guide describes how to build bitcoind, command-line utilities, and GUI on FreeBSD.
The following dependencies are required:
| Library | Purpose | Description |
|---|---|---|
| autoconf | Build | Automatically configure software source code |
| automake | Build | Generate makefile (requires autoconf) |
| libtool | Build | Shared library support |
| pkgconf | Build | Configure compiler and linker flags |
| git | Clone | Version control system |
| gmake | Compile | Generate executables |
| boost-libs | Utility | Library for threading, data structures, etc |
| libevent | Networking | OS independent asynchronous networking |
The following dependencies are optional:
| Library | Purpose | Description |
|---|---|---|
| db5 | Berkeley DB | Wallet storage (only needed when wallet enabled) |
| qt5 | GUI | GUI toolkit (only needed when GUI enabled) |
| libqrencode | QR codes in GUI | Generating QR codes (only needed when GUI enabled) |
| libzmq4 | ZMQ notification | Allows generating ZMQ notifications (requires ZMQ version >= 4.0.0) |
| sqlite3 | SQLite DB | Wallet storage (only needed when wallet enabled) |
| python3 | Testing | Python Interpreter (only needed when running the test suite) |
See dependencies.html for a complete overview.
Install the required dependencies the usual way you install software on FreeBSD - either with pkg or via the Ports collection. The example commands below use pkg which is usually run as root or via sudo. If you want to use sudo, and you haven't set it up: use this guide to setup sudo access on FreeBSD.
Now that git and all the required dependencies are installed, let's clone the Bitcoin Core repository to a directory. All build scripts and commands will run from this directory.
It is not necessary to build wallet functionality to run bitcoind or the GUI. To enable legacy wallets, you must install db5. To enable descriptor wallets, sqlite3 is required. Skip db5 if you intend to exclusively use descriptor wallets
db5 is required to enable support for legacy wallets. Skip if you don't intend to use legacy wallets
sqlite3 is required to enable support for descriptor wallets. Skip if you don't intend to use descriptor wallets.
Bitcoin Core includes a GUI built with the cross-platform Qt Framework. To compile the GUI, we need to install qt5. Skip if you don't intend to use the GUI.
The GUI can encode addresses in a QR Code. To build in QR support for the GUI, install libqrencode. Skip if not using the GUI or don't want QR code functionality.
There is an included test suite that is useful for testing code changes when developing. To run the test suite (recommended), you will need to have Python 3 installed:
There are many ways to configure Bitcoin Core, here are a few common examples:
This explicitly enables legacy wallet support and disables the GUI. If sqlite3 is installed, then descriptor wallet support will be built.
./autogen.sh
./configure --with-gui=no --with-incompatible-bdb \
BDB_LIBS="-ldb_cxx-5" \
BDB_CFLAGS="-I/usr/local/include/db5" \
MAKE=gmakeThis explicitly enables the GUI and disables legacy wallet support. If qt5 is not installed, this will throw an error. If sqlite3 is installed then descriptor wallet functionality will be built. If sqlite3 is not installed, then wallet functionality will be disabled.
Important: Use gmake (the non-GNU make will exit with an error).
(updated for NetBSD 8.0)
This guide describes how to build bitcoind and command-line utilities on NetBSD.
This guide does not contain instructions for building the GUI.
You will need the following modules, which can be installed via pkgsrc or pkgin:
autoconf
automake
boost
git
gmake
libevent
libtool
pkg-config
python37
git clone https://github.com/bitcoin/bitcoin.git
See dependencies.html for a complete overview.
BerkeleyDB is only necessary for the wallet functionality. To skip this, pass --disable-wallet to ./configure and skip to the next section.
It is recommended to use Berkeley DB 4.8. You cannot use the BerkeleyDB library from ports, for the same reason as boost above (g++/libstd++ incompatibility). If you have to build it yourself, you can use the installation script included in contrib/ like so:
from the root of the repository. Then set BDB_PREFIX for the next section:
Important: Use gmake (the non-GNU make will exit with an error).
With wallet:
./autogen.sh
./configure --with-gui=no CPPFLAGS="-I/usr/pkg/include" \
LDFLAGS="-L/usr/pkg/lib" \
BOOST_CPPFLAGS="-I/usr/pkg/include" \
BOOST_LDFLAGS="-L/usr/pkg/lib" \
BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \
BDB_CFLAGS="-I${BDB_PREFIX}/include" \
MAKE=gmakeWithout wallet:
./autogen.sh
./configure --with-gui=no --disable-wallet \
CPPFLAGS="-I/usr/pkg/include" \
LDFLAGS="-L/usr/pkg/lib" \
BOOST_CPPFLAGS="-I/usr/pkg/include" \
BOOST_LDFLAGS="-L/usr/pkg/lib" \
MAKE=gmakeBuild and run the tests:
(updated for OpenBSD 6.7)
This guide describes how to build bitcoind, bitcoin-qt, and command-line utilities on OpenBSD.
Run the following as root to install the base dependencies for building:
pkg_add git gmake libevent libtool boost
pkg_add qt5 # (optional for enabling the GUI)
pkg_add autoconf # (select highest version, e.g. 2.69)
pkg_add automake # (select highest version, e.g. 1.16)
pkg_add python # (select highest version, e.g. 3.8)
pkg_add bash
git clone https://github.com/bitcoin/bitcoin.gitSee dependencies.html for a complete overview.
Important: From OpenBSD 6.2 onwards a C++11-supporting clang compiler is part of the base image, and while building it is necessary to make sure that this compiler is used and not ancient g++ 4.2.1. This is done by appending CC=cc CC_FOR_BUILD=cc CXX=c++ to configuration commands. Mixing different compilers within the same executable will result in errors.
BerkeleyDB is only necessary for the wallet functionality. To skip this, pass --disable-wallet to ./configure and skip to the next section.
It is recommended to use Berkeley DB 4.8. You cannot use the BerkeleyDB library from ports, for the same reason as boost above (g++/libstd++ incompatibility). If you have to build it yourself, you can use the installation script included in contrib/ like so:
from the root of the repository. Then set BDB_PREFIX for the next section:
Important: Use gmake (the non-GNU make will exit with an error).
Preparation:
# Replace this with the autoconf version that you installed. Include only
# the major and minor parts of the version: use "2.69" for "autoconf-2.69p2".
export AUTOCONF_VERSION=2.69
# Replace this with the automake version that you installed. Include only
# the major and minor parts of the version: use "1.16" for "automake-1.16.1".
export AUTOMAKE_VERSION=1.16
./autogen.shMake sure BDB_PREFIX is set to the appropriate path from the above steps.
To configure with wallet:
./configure --with-gui=no CC=cc CXX=c++ \
BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \
BDB_CFLAGS="-I${BDB_PREFIX}/include" \
MAKE=gmakeTo configure without wallet:
To configure with GUI:
./configure --with-gui=yes CC=cc CXX=c++ \
BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \
BDB_CFLAGS="-I${BDB_PREFIX}/include" \
MAKE=gmakeBuild and run the tests:
If the build runs into out-of-memory errors, the instructions in this section might help.
The standard ulimit restrictions in OpenBSD are very strict:
data(kbytes) 1572864
This is, unfortunately, in some cases not enough to compile some .cpp files in the project, (see issue #6658). If your user is in the staff group the limit can be raised with:
ulimit -d 3000000
The change will only affect the current shell and processes spawned by it. To make the change system-wide, change datasize-cur and datasize-max in /etc/login.conf, and reboot.
The commands in this guide should be executed in a Terminal application. The built-in one is located in
/Applications/Utilities/Terminal.app
Install the macOS command line tools:
xcode-select --install
When the popup appears, click Install.
Then install Homebrew.
brew install automake libtool boost miniupnpc pkg-config python qt libevent qrencode
If you run into issues, check Homebrew's troubleshooting page. See dependencies.html for a complete overview.
If you want to build the disk image with make deploy (.dmg / optional), you need RSVG:
brew install librsvg
The wallet support requires one or both of the dependencies (SQLite and Berkeley DB) in the sections below. To build Bitcoin Core without wallet, see Disable-wallet mode.
Usually, macOS installation already has a suitable SQLite installation. Also, the Homebrew package could be installed:
brew install sqlite
In that case the Homebrew package will prevail.
It is recommended to use Berkeley DB 4.8. If you have to build it yourself, you can use this script to install it like so:
./contrib/install_db4.sh .
from the root of the repository.
Also, the Homebrew package could be installed:
brew install berkeley-db4
Clone the Bitcoin Core source code:
git clone https://github.com/bitcoin/bitcoin
cd bitcoin
Build Bitcoin Core:
Configure and build the headless Bitcoin Core binaries as well as the GUI (if Qt is found).
You can disable the GUI build by passing --without-gui to configure.
./autogen.sh
./configure
make
It is recommended to build and run the unit tests:
make check
You can also create a .dmg that contains the .app bundle (optional):
make deploy
When the intention is to run only a P2P node without a wallet, Bitcoin Core may be compiled in disable-wallet mode with:
./configure --disable-wallet
In this case there is no dependency on Berkeley DB and SQLite.
Mining is also possible in disable-wallet mode using the getblocktemplate RPC call.
Bitcoin Core is now available at ./src/bitcoind
Before running, you may create an empty configuration file:
mkdir -p "/Users/${USER}/Library/Application Support/Bitcoin"
touch "/Users/${USER}/Library/Application Support/Bitcoin/bitcoin.conf"
chmod 600 "/Users/${USER}/Library/Application Support/Bitcoin/bitcoin.conf"
The first time you run bitcoind, it will start downloading the blockchain. This process could take many hours, or even days on slower than average systems.
You can monitor the download process by looking at the debug.log file:
tail -f $HOME/Library/Application\ Support/Bitcoin/debug.log
./src/bitcoind -daemon # Starts the bitcoin daemon.
./src/bitcoin-cli --help # Outputs a list of command-line options.
./src/bitcoin-cli help # Outputs a list of RPC commands when the daemon is running.
(For BSD specific instructions, see build-*bsd.html in this directory.)
Always use absolute paths to configure and compile Bitcoin Core and the dependencies. For example, when specifying the path of the dependency:
../dist/configure --enable-cxx --disable-shared --with-pic --prefix=$BDB_PREFIX
Here BDB_PREFIX must be an absolute path - it is defined using $(pwd) which ensures the usage of the absolute path.
This will build bitcoin-qt as well, if the dependencies are met.
These dependencies are required:
| Library | Purpose | Description |
|---|---|---|
| libboost | Utility | Library for threading, data structures, etc |
| libevent | Networking | OS independent asynchronous networking |
Optional dependencies:
| Library | Purpose | Description |
|---|---|---|
| miniupnpc | UPnP Support | Firewall-jumping support |
| libdb4.8 | Berkeley DB | Optional, wallet storage (only needed when wallet enabled) |
| qt | GUI | GUI toolkit (only needed when GUI enabled) |
| libqrencode | QR codes in GUI | Optional for generating QR codes (only needed when GUI enabled) |
| univalue | Utility | JSON parsing and encoding (bundled version will be used unless --with-system-univalue passed to configure) |
| libzmq3 | ZMQ notification | Optional, allows generating ZMQ notifications (requires ZMQ version >= 4.0.0) |
| sqlite3 | SQLite DB | Optional, wallet storage (only needed when wallet enabled) |
For the versions used, see dependencies.html
C++ compilers are memory-hungry. It is recommended to have at least 1.5 GB of memory available when compiling Bitcoin Core. On systems with less, gcc can be tuned to conserve memory with additional CXXFLAGS:
./configure CXXFLAGS="--param ggc-min-expand=1 --param ggc-min-heapsize=32768"
Alternatively, or in addition, debugging information can be skipped for compilation. The default compile flags are -g -O2, and can be changed with:
./configure CXXFLAGS="-O2"
Finally, clang (often less resource hungry) can be used instead of gcc, which is used by default:
./configure CXX=clang++ CC=clang
Build requirements:
sudo apt-get install build-essential libtool autotools-dev automake pkg-config bsdmainutils python3
Now, you can either build from self-compiled depends or install the required dependencies:
sudo apt-get install libevent-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev libboost-thread-dev
BerkeleyDB is required for the wallet.
Ubuntu and Debian have their own libdb-dev and libdb++-dev packages, but these will install BerkeleyDB 5.1 or later. This will break binary wallet compatibility with the distributed executables, which are based on BerkeleyDB 4.8. If you do not care about wallet compatibility, pass --with-incompatible-bdb to configure.
Otherwise, you can build from self-compiled depends (see above).
SQLite is required for the wallet:
sudo apt install libsqlite3-dev
To build Bitcoin Core without wallet, see Disable-wallet mode
Optional (see --with-miniupnpc and --enable-upnp-default):
sudo apt-get install libminiupnpc-dev
ZMQ dependencies (provides ZMQ API):
sudo apt-get install libzmq3-dev
GUI dependencies:
If you want to build bitcoin-qt, make sure that the required packages for Qt development are installed. Qt 5 is necessary to build the GUI. To build without GUI pass --without-gui.
To build with Qt 5 you need the following:
sudo apt-get install libqt5gui5 libqt5core5a libqt5dbus5 qttools5-dev qttools5-dev-tools
libqrencode (optional) can be installed with:
sudo apt-get install libqrencode-dev
Once these are installed, they will be found by configure and a bitcoin-qt executable will be built by default.
Build requirements:
sudo dnf install gcc-c++ libtool make autoconf automake libevent-devel boost-devel libdb4-devel libdb4-cxx-devel python3
Optional (see --with-miniupnpc and --enable-upnp-default):
sudo dnf install miniupnpc-devel
ZMQ dependencies (provides ZMQ API):
sudo dnf install zeromq-devel
To build with Qt 5 you need the following:
sudo dnf install qt5-qttools-devel qt5-qtbase-devel
libqrencode (optional) can be installed with:
sudo dnf install qrencode-devel
SQLite can be installed with:
sudo dnf install sqlite-devel
The release is built with GCC and then "strip bitcoind" to strip the debug symbols, which reduces the executable size by about 90%.
miniupnpc may be used for UPnP port mapping. It can be downloaded from here. UPnP support is compiled in and turned off by default. See the configure options for upnp behavior desired:
--without-miniupnpc No UPnP support miniupnp not required
--disable-upnp-default (the default) UPnP support turned off by default at runtime
--enable-upnp-default UPnP support turned on by default at runtime
It is recommended to use Berkeley DB 4.8. If you have to build it yourself, you can use the installation script included in contrib/ like so:
./contrib/install_db4.sh `pwd`
from the root of the repository.
Note: You only need Berkeley DB if the wallet is enabled (see Disable-wallet mode).
If you need to build Boost yourself:
sudo su
./bootstrap.sh
./bjam install
To help make your Bitcoin Core installation more secure by making certain attacks impossible to exploit even if a vulnerability is found, binaries are hardened by default. This can be disabled with:
Hardening Flags:
./configure --enable-hardening
./configure --disable-hardening
Hardening enables the following features:
Position Independent Executable: Build position independent code to take advantage of Address Space Layout Randomization offered by some kernels. Attackers who can cause execution of code at an arbitrary memory location are thwarted if they don't know where anything useful is located. The stack and heap are randomly located by default, but this allows the code section to be randomly located as well.
On an AMD64 processor where a library was not compiled with -fPIC, this will cause an error such as: "relocation R_X86_64_32 against `......' can not be used when making a shared object;"
To test that you have built PIE executable, install scanelf, part of paxutils, and use:
scanelf -e ./bitcoin
The output should contain:
TYPE ET_DYN
Non-executable Stack: If the stack is executable then trivial stack-based buffer overflow exploits are possible if vulnerable buffers are found. By default, Bitcoin Core should be built with a non-executable stack, but if one of the libraries it uses asks for an executable stack or someone makes a mistake and uses a compiler extension which requires an executable stack, it will silently build an executable without the non-executable stack protection.
To verify that the stack is non-executable after compiling use: scanelf -e ./bitcoin
The output should contain: STK/REL/PTL RW- R-- RW-
The STK RW- means that the stack is readable and writeable but not executable.
When the intention is to run only a P2P node without a wallet, Bitcoin Core may be compiled in disable-wallet mode with:
./configure --disable-wallet
In this case there is no dependency on Berkeley DB 4.8 and SQLite.
Mining is also possible in disable-wallet mode using the getblocktemplate RPC call.
A list of additional configure flags can be displayed with:
./configure --help
This example lists the steps necessary to setup and build a command line only, non-wallet distribution of the latest changes on Arch Linux:
pacman -S git base-devel boost libevent python
git clone https://github.com/bitcoin/bitcoin.git
cd bitcoin/
./autogen.sh
./configure --disable-wallet --without-gui --without-miniupnpc
make check
Note: Enabling wallet support requires either compiling against a Berkeley DB newer than 4.8 (package db) using --with-incompatible-bdb, or building and depending on a local version of Berkeley DB 4.8. The readily available Arch Linux packages are currently built using --with-incompatible-bdb according to the PKGBUILD. As mentioned above, when maintaining portability of the wallet between the standard Bitcoin Core distributions and independently built node software is desired, Berkeley DB 4.8 must be used.
These steps can be performed on, for example, an Ubuntu VM. The depends system will also work on other Linux distributions, however the commands for installing the toolchain will be different.
Make sure you install the build requirements mentioned above. Then, install the toolchain and curl:
sudo apt-get install g++-arm-linux-gnueabihf curl
To build executables for ARM:
cd depends
make HOST=arm-linux-gnueabihf NO_QT=1
cd ..
./autogen.sh
./configure --prefix=$PWD/depends/arm-linux-gnueabihf --enable-glibc-back-compat --enable-reduce-exports LDFLAGS=-static-libstdc++
make
Below are some notes on how to build Bitcoin Core for Windows.
The options known to work for building Bitcoin Core on Windows are:
Other options which may work, but which have not been extensively tested are (please contribute instructions):
With Windows 10, Microsoft has released a new feature named the Windows Subsystem for Linux (WSL). This feature allows you to run a bash shell directly on Windows in an Ubuntu-based environment. Within this environment you can cross compile for Windows without the need for a separate Linux VM or server. Note that while WSL can be installed with other Linux variants, such as OpenSUSE, the following instructions have only been tested with Ubuntu.
This feature is not supported in versions of Windows prior to Windows 10 or on Windows Server SKUs. In addition, it is available only for 64-bit versions of Windows.
Full instructions to install WSL are available on the above link. To install WSL on Windows 10 with Fall Creators Update installed (version >= 16215.0) do the following:
OptionalFeatures.exe)After the bash shell is active, you can follow the instructions below, starting with the "Cross-compilation" section. Compiling the 64-bit version is recommended, but it is possible to compile the 32-bit version.
The steps below can be performed on Ubuntu (including in a VM) or WSL. The depends system will also work on other Linux distributions, however the commands for installing the toolchain will be different.
First, install the general dependencies:
sudo apt update
sudo apt upgrade
sudo apt install build-essential libtool autotools-dev automake pkg-config bsdmainutils curl git
A host toolchain (build-essential) is necessary because some dependency packages need to build host utilities that are used in the build process.
See dependencies.html for a complete overview.
If you want to build the windows installer with make deploy you need NSIS:
sudo apt install nsis
Acquire the source in the usual way:
git clone https://github.com/bitcoin/bitcoin.git
cd bitcoin
The first step is to install the mingw-w64 cross-compilation tool chain:
sudo apt install g++-mingw-w64-x86-64
Ubuntu Bionic 18.04 1:
sudo update-alternatives --config x86_64-w64-mingw32-g++ # Set the default mingw32 g++ compiler option to posix.
Once the toolchain is installed the build steps are common:
Note that for WSL the Bitcoin Core source path MUST be somewhere in the default mount file system, for example /usr/src/bitcoin, AND not under /mnt/d/. If this is not the case the dependency autoconf scripts will fail. This means you cannot use a directory that is located directly on the host Windows file system to perform the build.
Additional WSL Note: WSL support for launching Win32 applications results in Autoconf configure scripts being able to execute Windows Portable Executable files. This can cause unexpected behaviour during the build, such as Win32 error dialogs for missing libraries. The recommended approach is to temporarily disable WSL support for Win32 applications.
Build using:
PATH=$(echo "$PATH" | sed -e 's/:\/mnt.*//g') # strip out problematic Windows %PATH% imported var
sudo bash -c "echo 0 > /proc/sys/fs/binfmt_misc/status" # Disable WSL support for Win32 applications.
cd depends
make HOST=x86_64-w64-mingw32
cd ..
./autogen.sh # not required when building from tarball
CONFIG_SITE=$PWD/depends/x86_64-w64-mingw32/share/config.site ./configure --prefix=/
make
sudo bash -c "echo 1 > /proc/sys/fs/binfmt_misc/status" # Enable WSL support for Win32 applications.
For further documentation on the depends system see README.html in the depends directory.
After building using the Windows subsystem it can be useful to copy the compiled executables to a directory on the Windows drive in the same directory structure as they appear in the release .zip archive. This can be done in the following way. This will install to c:\workspace\bitcoin, for example:
make install DESTDIR=/mnt/c/workspace/bitcoin
You can also create an installer using:
make deploy
These are the dependencies currently used by Bitcoin Core. You can find instructions for installing them in the build-*.html file for your platform.
| Dependency | Version used | Minimum required | CVEs | Shared | Bundled Qt library |
|---|---|---|---|---|---|
| Berkeley DB | 4.8.30 | 4.8.x | No | ||
| Boost | 1.71.0 | 1.58.0 | No | ||
| Clang | 5.0+ (C++17 support) | ||||
| Expat | 2.2.7 | No | Yes | ||
| fontconfig | 2.12.1 | No | Yes | ||
| FreeType | 2.7.1 | No | Yes (Android only) | ||
| GCC | 7+ (C++17 support) | ||||
| HarfBuzz-NG | Yes | ||||
| libevent | 2.1.11-stable | 2.0.21 | No | ||
| libpng | Yes | ||||
| librsvg | |||||
| MiniUPnPc | 2.0.20180203 | No | |||
| PCRE | Yes | ||||
| Python (tests) | 3.6 | ||||
| qrencode | 3.4.4 | No | |||
| Qt | 5.9.8 | 5.5.1 | No | ||
| SQLite | 3.32.1 | 3.7.17 | |||
| XCB | Yes (Linux only) | ||||
| xkbcommon | Yes (Linux only) | ||||
| ZeroMQ | 4.3.1 | 4.0.0 | No | ||
| zlib | 1.2.11 | No |
Some dependencies are not needed in all configurations. The following are some factors that affect the dependency list.
./configure--with-miniupnpc=no.--disable-wallet or --without-bdb.--disable-wallet or --without-sqlite.--without-gui.--with-qrencode.--with-zmq option.make deploy on (cross-compilation to) macOS.Since Bitcoin Core v0.17, there is support for Output Descriptors. This is a simple language which can be used to describe collections of output scripts. Supporting RPCs are:
scantxoutset takes as input descriptors to scan for, and also reports specialized descriptors for the matching UTXOs.getdescriptorinfo analyzes a descriptor, and reports a canonicalized version with checksum added.deriveaddresses takes as input a descriptor and computes the corresponding addresses.listunspent outputs a specialized descriptor for the reported unspent outputs.getaddressinfo outputs a descriptor for solvable addresses (since v0.18).importmulti takes as input descriptors to import into the wallet (since v0.18).generatetodescriptor takes as input a descriptor and generates coins to it (regtest only, since v0.19).utxoupdatepsbt takes as input descriptors to add information to the psbt (since v0.19).createmultisig and addmultisigaddress return descriptors as well (since v0.20)This document describes the language. For the specifics on usage, see the RPC documentation for the functions mentioned above.
Output descriptors currently support:
pk function.pkh function.wpkh function.sh function.wsh function.multi function.sortedmulti function.addr function.raw function.pk(0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798) describes a P2PK output with the specified public key.pkh(02c6047f9441ed7d6d3045406e95c07cd85c778e4b8cef3ca7abac09b95c709ee5) describes a P2PKH output with the specified public key.wpkh(02f9308a019258c31049344f85f89d5229b531c845836f99b08601f113bce036f9) describes a P2WPKH output with the specified public key.sh(wpkh(03fff97bd5755eeea420453a14355235d382f6472f8568a18b2f057a1460297556)) describes a P2SH-P2WPKH output with the specified public key.combo(0279be667ef9dcbbac55a06295ce870b07029bfcdb2dce28d959f2815b16f81798) describes any P2PK, P2PKH, P2WPKH, or P2SH-P2WPKH output with the specified public key.sh(wsh(pkh(02e493dbf1c10d80f3581e4904930b1404cc6c13900ee0758474fa94abe8c4cd13))) describes an (overly complicated) P2SH-P2WSH-P2PKH output with the specified public key.multi(1,022f8bde4d1a07209355b4a7250a5c5128e88b84bddc619ab7cba8d569b240efe4,025cbdf0646e5db4eaa398f365f2ea7a0e3d419b7e0330e39ce92bddedcac4f9bc) describes a bare 1-of-2 multisig output with keys in the specified order.sh(multi(2,022f01e5e15cca351daff3843fb70f3c2f0a1bdd05e5af888a67784ef3e10a2a01,03acd484e2f0c7f65309ad178a9f559abde09796974c57e714c35f110dfc27ccbe)) describes a P2SH 2-of-2 multisig output with keys in the specified order.sh(sortedmulti(2,03acd484e2f0c7f65309ad178a9f559abde09796974c57e714c35f110dfc27ccbe,022f01e5e15cca351daff3843fb70f3c2f0a1bdd05e5af888a67784ef3e10a2a01)) describes a P2SH 2-of-2 multisig output with keys sorted lexicographically in the resulting redeemScript.wsh(multi(2,03a0434d9e47f3c86235477c7b1ae6ae5d3442d49b1943c2b752a68e2a47e247c7,03774ae7f858a9411e5ef4246b70c65aac5649980be5c17891bbec17895da008cb,03d01115d548e7561b15c38f004d734633687cf4419620095bc5b0f47070afe85a)) describes a P2WSH 2-of-3 multisig output with keys in the specified order.sh(wsh(multi(1,03f28773c2d975288bc7d1d205c3748651b075fbc6610e58cddeeddf8f19405aa8,03499fdf9e895e719cfd64e67f07d38e3226aa7b63678949e6e49b241a60e823e4,02d7924d4f7d43ea965a465ae3095ff41131e5946f3c85f79e44adbcf8e27e080e))) describes a P2SH-P2WSH 1-of-3 multisig output with keys in the specified order.pk(xpub661MyMwAqRbcFtXgS5sYJABqqG9YLmC4Q1Rdap9gSE8NqtwybGhePY2gZ29ESFjqJoCu1Rupje8YtGqsefD265TMg7usUDFdp6W1EGMcet8) describes a P2PK output with the public key of the specified xpub.pkh(xpub68Gmy5EdvgibQVfPdqkBBCHxA5htiqg55crXYuXoQRKfDBFA1WEjWgP6LHhwBZeNK1VTsfTFUHCdrfp1bgwQ9xv5ski8PX9rL2dZXvgGDnw/1/2) describes a P2PKH output with child key 1/2 of the specified xpub.pkh([d34db33f/44'/0'/0']xpub6ERApfZwUNrhLCkDtcHTcxd75RbzS1ed54G1LkBUHQVHQKqhMkhgbmJbZRkrgZw4koxb5JaHWkY4ALHY2grBGRjaDMzQLcgJvLJuZZvRcEL/1/*) describes a set of P2PKH outputs, but additionally specifies that the specified xpub is a child of a master with fingerprint d34db33f, and derived using path 44'/0'/0'.wsh(multi(1,xpub661MyMwAqRbcFW31YEwpkMuc5THy2PSt5bDMsktWQcFF8syAmRUapSCGu8ED9W6oDMSgv6Zz8idoc4a6mr8BDzTJY47LJhkJ8UB7WEGuduB/1/0/*,xpub69H7F5d8KSRg.htmlJg2KhpAK8SR3DjMwAdkxj3ZuxV27CprR9LgpeyGmXUbC6wb7ERfvrnKZjXoUmmDznezpbZb7ap6r1D3tgFxHmwMkQTPH/0/0/*)) describes a set of 1-of-2 P2WSH multisig outputs where the first multisig key is the 1/0/i child of the first specified xpub and the second multisig key is the 0/0/i child of the second specified xpub, and i is any number in a configurable range (0-1000 by default).wsh(sortedmulti(1,xpub661MyMwAqRbcFW31YEwpkMuc5THy2PSt5bDMsktWQcFF8syAmRUapSCGu8ED9W6oDMSgv6Zz8idoc4a6mr8BDzTJY47LJhkJ8UB7WEGuduB/1/0/*,xpub69H7F5d8KSRg.htmlJg2KhpAK8SR3DjMwAdkxj3ZuxV27CprR9LgpeyGmXUbC6wb7ERfvrnKZjXoUmmDznezpbZb7ap6r1D3tgFxHmwMkQTPH/0/0/*)) describes a set of 1-of-2 P2WSH multisig outputs where one multisig key is the 1/0/i child of the first specified xpub and the other multisig key is the 0/0/i child of the second specified xpub, and i is any number in a configurable range (0-1000 by default). The order of public keys in the resulting witnessScripts is determined by the lexicographic order of the public keys at that index.Descriptors consist of several types of expressions. The top level expression is either a SCRIPT, or SCRIPT#CHECKSUM where CHECKSUM is an 8-character alphanumeric descriptor checksum.
SCRIPT expressions:
sh(SCRIPT) (top level only): P2SH embed the argument.wsh(SCRIPT) (not inside another 'wsh'): P2WSH embed the argument.pk(KEY) (anywhere): P2PK output for the given public key.pkh(KEY) (anywhere): P2PKH output for the given public key (use addr if you only know the pubkey hash).wpkh(KEY) (not inside wsh): P2WPKH output for the given compressed pubkey.combo(KEY) (top level only): an alias for the collection of pk(KEY) and pkh(KEY). If the key is compressed, it also includes wpkh(KEY) and sh(wpkh(KEY)).multi(k,KEY_1,KEY_2,...,KEY_n) (anywhere): k-of-n multisig script.sortedmulti(k,KEY_1,KEY_2,...,KEY_n) (anywhere): k-of-n multisig script with keys sorted lexicographically in the resulting script.addr(ADDR) (top level only): the script which ADDR expands to.raw(HEX) (top level only): the script whose hex encoding is HEX.KEY expressions:
[/NUM or /NUM' path elements to indicate unhardened or hardened derivation steps between the fingerprint and the key or xpub/xprv root that follows]02 or 03 for a compressed pubkey, or 130 characters starting with 04 for an uncompressed pubkey).
wpkh and wsh, only compressed public keys are permitted.xpub encoded extended public key or xprv encoded extended private key (as defined in BIP 32).
/NUM unhardened and /NUM' hardened BIP32 derivation steps./* or /*' final step to denote all (direct) unhardened or hardened children.(Anywhere a ' suffix is permitted to denote hardened derivation, the suffix h can be used instead.)
ADDR expressions are any type of supported address:
1... for mainnet or [nm]... for testnet). Note that P2PKH addresses in descriptors cannot be used for P2PK outputs (use the pk function instead).3... for mainnet or 2... for testnet, defined in BIP 13).bc1... for mainnet or tb1... for testnet, defined in BIP 173).Many single-key constructions are used in practice, generally including P2PK, P2PKH, P2WPKH, and P2SH-P2WPKH. Many more combinations are imaginable, though they may not be optimal: P2SH-P2PK, P2SH-P2PKH, P2WSH-P2PK, P2WSH-P2PKH, P2SH-P2WSH-P2PK, P2SH-P2WSH-P2PKH.
To describe these, we model these as functions. The functions pk (P2PK), pkh (P2PKH) and wpkh (P2WPKH) take as input a KEY expression, and return the corresponding scriptPubKey. The functions sh (P2SH) and wsh (P2WSH) take as input a SCRIPT expression, and return the script describing P2SH and P2WSH outputs with the input as embedded script. The names of the functions do not contain "p2" for brevity.
Several pieces of software use multi-signature (multisig) scripts based on Bitcoin's OP_CHECKMULTISIG opcode. To support these, we introduce the multi(k,key_1,key_2,...,key_n) and sortedmulti(k,key_1,key_2,...,key_n) functions. They represent a k-of-n multisig policy, where any k out of the n provided KEY expressions must sign.
Key order is significant for multi(). A multi() expression describes a multisig script with keys in the specified order, and in a search for TXOs, it will not match outputs with multisig scriptPubKeys that have the same keys in a different order. Also, to prevent a combinatorial explosion of the search space, if more than one of the multi() key arguments is a BIP32 wildcard path ending in /* or *', the multi() expression only matches multisig scripts with the ith child key from each wildcard path in lockstep, rather than scripts with any combination of child keys from each wildcard path.
Key order does not matter for sortedmulti(). sortedmulti() behaves in the same way as multi() does but the keys are reordered in the resulting script such that they are lexicographically ordered as described in BIP67.
Most modern wallet software and hardware uses keys that are derived using BIP32 ("HD keys"). We support these directly by permitting strings consisting of an extended public key (commonly referred to as an xpub) plus derivation path anywhere a public key is expected. The derivation path consists of a sequence of 0 or more integers (in the range 0..231-1) each optionally followed by ' or h, and separated by / characters. The string may optionally end with the literal /* or /*' (or /*h) to refer to all unhardened or hardened child keys in a configurable range (by default 0-1000, inclusive).
Whenever a public key is described using a hardened derivation step, the script cannot be computed without access to the corresponding private key.
In order to describe scripts whose signing keys reside on another device, it may be necessary to identify the master key and derivation path an xpub was derived with.
For example, when following BIP44, it would be useful to describe a change chain directly as xpub.../44'/0'/0'/1/* where xpub... corresponds with the master key m. Unfortunately, since there are hardened derivation steps that follow the xpub, this descriptor does not let you compute scripts without access to the corresponding private keys. Instead, it should be written as xpub.../1/*, where xpub corresponds to m/44'/0'/0'.
When interacting with a hardware device, it may be necessary to include the entire path from the master down. BIP174 standardizes this by providing the master key fingerprint (first 32 bit of the Hash160 of the master pubkey), plus all derivation steps. To support constructing these, we permit providing this key origin information inside the descriptor language, even though it does not affect the actual scriptPubKeys it refers to.
Every public key can be prefixed by an 8-character hexadecimal fingerprint plus optional derivation steps (hardened and unhardened) surrounded by brackets, identifying the master and derivation path the key or xpub that follows was derived with.
Note that the fingerprint of the parent only serves as a fast way to detect parent and child nodes in software, and software must be willing to deal with collisions.
Often it is useful to communicate a description of scripts along with the necessary private keys. For this reason, anywhere a public key or xpub is supported, a private key in WIF format or xprv may be provided instead. This is useful when private keys are necessary for hardened derivation steps, or for dumping wallet descriptors including private key material.
In order to easily represent the sets of scripts currently supported by existing Bitcoin Core wallets, a convenience function combo is provided, which takes as input a public key, and describes a set of P2PK, P2PKH, P2WPKH, and P2SH-P2WPH scripts for that key. In case the key is uncompressed, the set only includes P2PK and P2PKH scripts.
Descriptors can optionally be suffixed with a checksum to protect against typos or copy-paste errors.
These checksums consist of 8 alphanumeric characters. As long as errors are restricted to substituting characters in 0123456789()[],'/*abcdefgh@:$%{} for others in that set and changes in letter case, up to 4 errors will always be detected in descriptors up to 501 characters, and up to 3 errors in longer ones. For larger numbers of errors, or other types of errors, there is a roughly 1 in a trillion chance of not detecting the errors.
deriveaddress and importmulti. The checksum for a descriptor without one can be computed using the getdescriptorinfo RPC. Developer NotesTable of Contents
Various coding styles have been used during the history of the codebase, and the result is not very consistent. However, we're now trying to converge to a single style, which is specified below. When writing patches, favor the new style over attempting to mimic the surrounding style, except for move-only commits.
Do not submit patches solely to modify the style of existing code.
Indentation and whitespace rules as specified in src/.clang-format. You can use the provided clang-format-diff script tool to clean up patches automatically before submission.
public/protected/private or for namespace.( this ).if, for and while.if only has a single-statement then-clause, it can appear on the same line as the if, without braces. In every other case, braces are required, and the then and else clauses must appear correctly indented on a new line.Symbol naming conventions. These are preferred in new code, but are not required when doing so would need changes to significant pieces of existing code.
_ to separate words (snake_case).
m_ prefix.g_ prefix._ to separate words.C.src/test/foo_tests.cpp should be named foo_tests. Test suite names must be unique.Miscellaneous
++i is preferred over i++.nullptr is preferred over NULL or (void*)0.static_assert is preferred over assert where possible. Generally; compile-time checking is preferred over run-time checking.Block style example:
int g_count = 0;
namespace foo {
class Class
{
std::string m_name;
public:
bool Function(const std::string& s, int n)
{
// Comment summarising what this section of code does
for (int i = 0; i < n; ++i) {
int total_sum = 0;
// When something fails, return early
if (!Something()) return false;
...
if (SomethingElse(i)) {
total_sum += ComputeSomething(g_count);
} else {
DoSomething(m_name, total_sum);
}
}
// Success return is usually at the end
return true;
}
}
} // namespace fooRefer to /test/functional/README.html#style-guidelines.
Bitcoin Core uses Doxygen to generate its official documentation.
Use Doxygen-compatible comment blocks for functions, methods, and fields.
For example, to describe a function use:
/**
* ... Description ...
*
* @param[in] arg1 input description...
* @param[in] arg2 input description...
* @param[out] arg3 output description...
* @return Return cases...
* @throws Error type and cases...
* @pre Pre-condition for function...
* @post Post-condition for function...
*/
bool function(int arg1, const char *arg2, std::string& arg3)A complete list of @xxx commands can be found at http://www.doxygen.nl/manual/commands.html. As Doxygen recognizes the comments by the delimiters (/** and */ in this case), you don't need to provide any commands for a comment to be valid; just a description text is fine.
To describe a class, use the same construct above the class definition:
/**
* Alerts are for notifying old versions if they become too obsolete and
* need to upgrade. The message is displayed in the status bar.
* @see GetWarnings()
*/
class CAlertTo describe a member or variable use:
or
Also OK:
Not picked up by Doxygen:
Also not picked up by Doxygen:
A full list of comment syntaxes picked up by Doxygen can be found at http://www.doxygen.nl/manual/docblocks.html, but the above styles are favored.
Recommendations:
Avoiding duplicating type and input/output information in function descriptions.
Use backticks (``) to refer to argument names in function and parameter descriptions.
Backticks aren't required when referring to functions Doxygen already knows about; it will build hyperlinks for these automatically. See http://www.doxygen.nl/manual/autolink.html for complete info.
Avoid linking to external documentation; links can break.
Javadoc and all valid Doxygen comments are stripped from Doxygen source code previews (STRIP_CODE_COMMENTS = YES in Doxyfile.in). If you want a comment to be preserved, it must instead use // or /* */.
The documentation can be generated with make docs and cleaned up with make clean-docs. The resulting files are located in doc/doxygen/html; open index.html in that directory to view the homepage.
Before running make docs, you'll need to install these dependencies:
Linux: sudo apt install doxygen graphviz
MacOS: brew install doxygen graphviz
Run configure with --enable-debug to add additional compiler flags that produce better debugging builds.
Run configure with the --enable-gprof option, then make.
debug.logIf the code is behaving strangely, take a look in the debug.log file in the data directory; error and debugging messages are written there.
The -debug=... command-line option controls debugging; running with just -debug or -debug=1 will turn on all categories (and give you a very large debug.log file).
The Qt code routes qDebug() output to debug.log under category "qt": run with -debug=qt to see it.
Run with the -testnet option to run with "play bitcoins" on the test network, if you are testing multi-machine code that needs to operate across the internet.
If you are testing something that can run on one machine, run with the -regtest option. In regression test mode, blocks can be created on-demand; see test/functional/ for tests that run in -regtest mode.
Bitcoin Core is a multi-threaded application, and deadlocks or other multi-threading bugs can be very difficult to track down. The --enable-debug configure option adds -DDEBUG_LOCKORDER to the compiler flags. This inserts run-time checks to keep track of which locks are held and adds warnings to the debug.log file if inconsistencies are detected.
The util file src/util/check.h offers helpers to protect against coding and internal logic bugs. They must never be used to validate user, network or any other input.
assert or Assert should be used to document assumptions when any violation would mean that it is not safe to continue program execution. The code is always compiled with assertions enabled.
CHECK_NONFATAL should be used for recoverable internal logic bugs. On failure, it will throw an exception, which can be caught to recover from the error.
Assume should be used to document assumptions when program execution can safely continue even if the assumption is violated. In debug builds it behaves like Assert/assert to notify developers and testers about nonfatal errors. In production it doesn't warn or log anything, though the expression is always evaluated.
Valgrind is a programming tool for memory debugging, memory leak detection, and profiling. The repo contains a Valgrind suppressions file (valgrind.supp) which includes known Valgrind warnings in our dependencies that cannot be fixed in-tree. Example use:
$ valgrind --suppressions=contrib/valgrind.supp src/test/test_bitcoin
$ valgrind --suppressions=contrib/valgrind.supp --leak-check=full \
--show-leak-kinds=all src/test/test_bitcoin --log_level=test_suite
$ valgrind -v --leak-check=full src/bitcoind -printtoconsole
$ ./test/functional/test_runner.py --valgrind
LCOV can be used to generate a test coverage report based upon make check execution. LCOV must be installed on your system (e.g. the lcov package on Debian/Ubuntu).
To enable LCOV report generation during test runs:
./configure --enable-lcov
make
make cov
# A coverage report will now be accessible at `./test_bitcoin.coverage/index.html`.
Profiling is a good way to get a precise idea of where time is being spent in code. One tool for doing profiling on Linux platforms is called perf, and has been integrated into the functional test framework. Perf can observe a running process and sample (at some frequency) where its execution is.
Perf installation is contingent on which kernel version you're running; see this thread for specific instructions.
Certain kernel parameters may need to be set for perf to be able to inspect the running process's stack.
Make sure you understand the security trade-offs of setting these kernel parameters.
To profile a running bitcoind process for 60 seconds, you could use an invocation of perf record like this:
You could then analyze the results by running:
or using a graphical tool like Hotspot.
See the functional test documentation for how to invoke perf within tests.
Bitcoin Core can be compiled with various "sanitizers" enabled, which add instrumentation for issues regarding things like memory safety, thread race conditions, or undefined behavior. This is controlled with the --with-sanitizers configure flag, which should be a comma separated list of sanitizers to enable. The sanitizer list should correspond to supported -fsanitize= options in your compiler. These sanitizers have runtime overhead, so they are most useful when testing changes or producing debugging builds.
Some examples:
# Enable both the address sanitizer and the undefined behavior sanitizer
./configure --with-sanitizers=address,undefined
# Enable the thread sanitizer
./configure --with-sanitizers=threadIf you are compiling with GCC you will typically need to install corresponding "san" libraries to actually compile with these flags, e.g. libasan for the address sanitizer, libtsan for the thread sanitizer, and libubsan for the undefined sanitizer. If you are missing required libraries, the configure script will fail with a linker error when testing the sanitizer flags.
The test suite should pass cleanly with the thread and undefined sanitizers, but there are a number of known problems when using the address sanitizer. The address sanitizer is known to fail in sha256_sse4::Transform which makes it unusable unless you also use --disable-asm when running configure. We would like to fix sanitizer issues, so please send pull requests if you can fix any errors found by the address sanitizer (or any other sanitizer).
Not all sanitizer options can be enabled at the same time, e.g. trying to build with --with-sanitizers=address,thread will fail in the configure script as these sanitizers are mutually incompatible. Refer to your compiler manual to learn more about these options and which sanitizers are supported by your compiler.
Additional resources:
The code is multi-threaded and uses mutexes and the LOCK and TRY_LOCK macros to protect data structures.
Deadlocks due to inconsistent lock ordering (thread 1 locks cs_main and then cs_wallet, while thread 2 locks them in the opposite order: result, deadlock as each waits for the other to release its lock) are a problem. Compile with -DDEBUG_LOCKORDER (or use --enable-debug) to get lock order inconsistencies reported in the debug.log file.
Re-architecting the core code so there are better-defined interfaces between the various components is a goal, with any necessary locking done by the components (e.g. see the self-contained FillableSigningProvider class and its cs_KeyStore lock for example).
Main thread (bitcoind) : Started from main() in bitcoind.cpp. Responsible for starting up and shutting down the application.
ThreadImport (b-loadblk) : Loads blocks from blk*.dat files or -loadblock=<file> on startup.
ThreadScriptCheck (b-scriptch.x) : Parallel script validation threads for transactions in blocks.
ThreadHTTP (b-http) : Libevent thread to listen for RPC and REST connections.
HTTP worker threads(b-httpworker.x) : Threads to service RPC and REST requests.
Indexer threads (b-txindex, etc) : One thread per indexer.
SchedulerThread (b-scheduler) : Does asynchronous background tasks like dumping wallet contents, dumping addrman and running asynchronous validationinterface callbacks.
TorControlThread (b-torcontrol) : Libevent thread for tor connections.
Net threads:
ThreadMessageHandler (b-msghand) : Application level message handling (sending and receiving). Almost all net_processing and validation logic runs on this thread.
ThreadDNSAddressSeed (b-dnsseed) : Loads addresses of peers from the DNS.
ThreadMapPort (b-upnp) : Universal plug-and-play startup/shutdown.
ThreadSocketHandler (b-net) : Sends/Receives data from peers on port 8333.
ThreadOpenAddedConnections (b-addcon) : Opens network connections to added nodes.
ThreadOpenConnections (b-opencon) : Initiates new connections to peers.
In closed-source environments in which everyone uses the same IDE, it is common to add temporary files it produces to the project-wide .gitignore file.
However, in open source software such as Bitcoin Core, where everyone uses their own editors/IDE/tools, it is less common. Only you know what files your editor produces and this may change from version to version. The canonical way to do this is thus to create your local gitignore. Add this to ~/.gitconfig:
[core]
excludesfile = /home/.../.gitignore_global
(alternatively, type the command git config --global core.excludesfile ~/.gitignore_global on a terminal)
Then put your favourite tool's temporary filenames in that file, e.g.
# NetBeans
nbproject/
Another option is to create a per-repository excludes file .git/info/exclude. These are not committed but apply only to one repository.
If a set of tools is used by the build system or scripts the repository (for example, lcov) it is perfectly acceptable to add its files to .gitignore and commit them.
A few non-style-related recommendations for developers, as well as points to pay attention to for reviewers of Bitcoin Core code.
New features should be exposed on RPC first, then can be made available in the GUI.
Make sure pull requests pass CI before merging.
Rationale: Makes sure that they pass thorough testing, and that the tester will keep passing on the master branch. Otherwise, all new pull requests will start failing the tests, resulting in confusion and mayhem.
Explanation: If the test suite is to be updated for a change, this has to be done first.
Make sure that no crashes happen with run-time option -disablewallet.
Include db_cxx.h (BerkeleyDB header) only when ENABLE_WALLET is set.
For general C++ guidelines, you may refer to the C++ Core Guidelines.
Common misconceptions are clarified in those sections:
Passing (non-)fundamental types in the C++ Core Guideline.
Assertions should not have side-effects.
If you use the .h, you must link the .cpp.
.h to the .cpp should not result in build errors.Use the RAII (Resource Acquisition Is Initialization) paradigm where possible. For example, by using unique_ptr for allocations in a function.
Use MakeUnique() to construct objects owned by unique_ptrs.
MakeUnique is concise and ensures exception safety in complex expressions. MakeUnique is a temporary project local implementation of std::make_unique (C++14).Never use the std::map [] syntax when reading from a map, but instead use .find().
[] does an insert (of the default element) if the item doesn't exist in the map yet. This has resulted in memory leaks in the past, as well as race conditions (expecting read-read behavior). Using [] is fine for writing to a map.Do not compare an iterator from one data structure with an iterator of another data structure (even if of the same type).
Watch out for out-of-bounds vector access. &vch[vch.size()] is illegal, including &vch[0] for an empty vector. Use vch.data() and vch.data() + vch.size() instead.
Vector bounds checking is only enabled in debug mode. Do not rely on it.
Initialize all non-static class members where they are defined. If this is skipped for a good reason (i.e., optimization on the critical path), add an explicit comment about this.
By default, declare constructors explicit.
Use explicitly signed or unsigned chars, or even better uint8_t and int8_t. Do not use bare char unless it is to pass to a third-party API. This type can be signed or unsigned depending on the architecture, which can lead to interoperability problems or dangerous conditions such as out-of-bounds array accesses.
Prefer explicit constructions over implicit ones that rely on 'magical' C++ behavior.
Use Span as function argument when it can operate on any range-like container.
Foo(const vector<int>&) this avoids the need for a (potentially expensive) conversion to vector if the caller happens to have the input stored in another type of container. However, be aware of the pitfalls documented in span.h.Prefer enum class (scoped enumerations) over enum (traditional enumerations) where possible.
int, and name clashes due to enumerators being exported to the surrounding scope.switch statement on an enumeration example:
enum class Tabs {
INFO,
CONSOLE,
GRAPH,
PEERS
};
int GetInt(Tabs tab)
{
switch (tab) {
case Tabs::INFO: return 0;
case Tabs::CONSOLE: return 1;
case Tabs::GRAPH: return 2;
case Tabs::PEERS: return 3;
} // no default case, so the compiler can warn about missing cases
assert(false);
}Rationale: The comment documents skipping default: label, and it complies with clang-format rules. The assertion prevents firing of -Wreturn-type warning on some compilers.
Be careful of LogPrint versus LogPrintf. LogPrint takes a category argument, LogPrintf does not.
Use std::string, avoid C string manipulation functions.
\0 characters. Also, some C string manipulations tend to act differently depending on platform, or even the user locale.Use ParseInt32, ParseInt64, ParseUInt32, ParseUInt64, ParseDouble from utilstrencodings.h for number parsing.
Avoid using locale dependent functions if possible. You can use the provided lint-locale-dependence.sh to check for accidental use of locale dependent functions.
Rationale: Unnecessary locale dependence can cause bugs that are very tricky to isolate and fix.
These functions are known to be locale dependent: alphasort, asctime, asprintf, atof, atoi, atol, atoll, atoq, btowc, ctime, dprintf, fgetwc, fgetws, fprintf, fputwc, fputws, fscanf, fwprintf, getdate, getwc, getwchar, isalnum, isalpha, isblank, iscntrl, isdigit, isgraph, islower, isprint, ispunct, isspace, isupper, iswalnum, iswalpha, iswblank, iswcntrl, iswctype, iswdigit, iswgraph, iswlower, iswprint, iswpunct, iswspace, iswupper, iswxdigit, isxdigit, mblen, mbrlen, mbrtowc, mbsinit, mbsnrtowcs, mbsrtowcs, mbstowcs, mbtowc, mktime, putwc, putwchar, scanf, snprintf, sprintf, sscanf, stoi, stol, stoll, strcasecmp, strcasestr, strcoll, strfmon, strftime, strncasecmp, strptime, strtod, strtof, strtoimax, strtol, strtold, strtoll, strtoq, strtoul, strtoull, strtoumax, strtouq, strxfrm, swprintf, tolower, toupper, towctrans, towlower, towupper, ungetwc, vasprintf, vdprintf, versionsort, vfprintf, vfscanf, vfwprintf, vprintf, vscanf, vsnprintf, vsprintf, vsscanf, vswprintf, vwprintf, wcrtomb, wcscasecmp, wcscoll, wcsftime, wcsncasecmp, wcsnrtombs, wcsrtombs, wcstod, wcstof, wcstoimax, wcstol, wcstold, wcstoll, wcstombs, wcstoul, wcstoull, wcstoumax, wcswidth, wcsxfrm, wctob, wctomb, wctrans, wctype, wcwidth, wprintf
For strprintf, LogPrint, LogPrintf formatting characters don't need size specifiers.
Use .c_str() sparingly. Its only valid use is to pass C++ strings to C functions that take NULL-terminated strings.
Do not use it when passing a sized array (so along with .size()). Use .data() instead to get a pointer to the raw data.
.data() communicates the intent better.Do not use it when passing strings to tfm::format, strprintf, LogPrint[f].
Do not use it to convert to QString. Use QString::fromStdString().
In cases where do you call .c_str(), you might want to additionally check that the string does not contain embedded '\0' characters, because it will (necessarily) truncate the string. This might be used to hide parts of the string from logging or to circumvent checks. If a use of strings is sensitive to this, take care to check the string for embedded NULL characters first and reject it if there are any (see ParsePrechecks in strencodings.cpp for an example).
Although the shadowing warning (-Wshadow) is not enabled by default (it prevents issues arising from using a different variable with the same name), please name variables so that their names do not shadow variables defined in the source code.
When using nested cycles, do not name the inner cycle variable the same as in the upper cycle, etc.
Prefer Mutex type to RecursiveMutex one
Consistently use Clang Thread Safety Analysis annotations to get compile-time warnings about potential race conditions in code. Combine annotations in function declarations with run-time asserts in function definitions:
// txmempool.h
class CTxMemPool
{
public:
...
mutable RecursiveMutex cs;
...
void UpdateTransactionsFromBlock(...) EXCLUSIVE_LOCKS_REQUIRED(::cs_main, cs);
...
}
// txmempool.cpp
void CTxMemPool::UpdateTransactionsFromBlock(...)
{
AssertLockHeld(::cs_main);
AssertLockHeld(cs);
...
}// validation.h
class ChainstateManager
{
public:
...
bool ProcessNewBlock(...) EXCLUSIVE_LOCKS_REQUIRED(!::cs_main);
...
}
// validation.cpp
bool ChainstateManager::ProcessNewBlock(...)
{
AssertLockNotHeld(::cs_main);
...
LOCK(::cs_main);
...
}Build and run tests with -DDEBUG_LOCKORDER to verify that no potential deadlocks are introduced. As of 0.12, this is defined by default when configuring with --enable-debug.
When using LOCK/TRY_LOCK be aware that the lock exists in the context of the current scope, so surround the statement and the code that needs the lock with braces.
OK:
Wrong:
Use #!/usr/bin/env bash instead of obsolete #!/bin/bash.
#!/bin/bash assumes it is always installed to /bin/ which can cause issues;
#!/usr/bin/env bash searches the user's PATH to find the bash binary.
OK:
Wrong:
Implementation code should go into the .cpp file and not the .h, unless necessary due to template usage or when performance due to inlining is critical.
Use only the lowercase alphanumerics (a-z0-9), underscore (_) and hyphen (-) in source code filenames.
grep:ing and auto-completing filenames is easier when using a consistent naming pattern. Potential problems when building on case-insensitive filesystems are avoided when using only lowercase characters in source code filenames.Every .cpp and .h file should #include every header file it directly uses classes, functions or other definitions from, even if those headers are already included indirectly through other headers.
Don't import anything into the global namespace (using namespace ...). Use fully specified types such as std::string.
Terminate namespaces with a comment (// namespace mynamespace). The comment should be placed on the same line as the brace closing the namespace, e.g.
Rationale: Avoids confusion about the namespace context.
Use #include <primitives/transaction.h> bracket syntax instead of #include "primitives/transactions.h" quote syntax.
Use include guards to avoid the problem of double inclusion. The header file foo/bar.h should use the include guard identifier BITCOIN_FOO_BAR_H, e.g.
Do not display or manipulate dialogs in model code (classes *Model).
Avoid adding slow or blocking code in the GUI thread. In particular, do not add new interfaces::Node and interfaces::Wallet method calls, even if they may be fast now, in case they are changed to lock or communicate across processes in the future.
Prefer to offload work from the GUI thread to worker threads (see RPCExecutor in console code as an example) or take other steps (see https://doc.qt.io/archives/qq/qq27-responsive-guis.html) to keep the GUI responsive.
Several parts of the repository are subtrees of software maintained elsewhere.
Some of these are maintained by active developers of Bitcoin Core, in which case changes should probably go directly upstream without being PRed directly against the project. They will be merged back in the next subtree merge.
Others are external projects without a tight relationship with our project. Changes to these should also be sent upstream, but bugfixes may also be prudent to PR against Bitcoin Core so that they can be integrated quickly. Cosmetic changes should be purely taken upstream.
There is a tool in test/lint/git-subtree-check.sh (instructions) to check a subtree directory for consistency with its upstream repository.
Current subtrees include:
src/leveldb
src/crc32c
src/secp256k1
src/crypto/ctaes
src/univalue
Extra care must be taken when upgrading LevelDB. This section explains issues you must be aware of.
In most configurations, we use the default LevelDB value for max_open_files, which is 1000 at the time of this writing. If LevelDB actually uses this many file descriptors, it will cause problems with Bitcoin's select() loop, because it may cause new sockets to be created where the fd value is >= 1024. For this reason, on 64-bit Unix systems, we rely on an internal LevelDB optimization that uses mmap() + close() to open table files without actually retaining references to the table file descriptors. If you are upgrading LevelDB, you must sanity check the changes to make sure that this assumption remains valid.
In addition to reviewing the upstream changes in env_posix.cc, you can use lsof to check this. For example, on Linux this command will show open .ldb file counts:
$ lsof -p $(pidof bitcoind) |\
awk 'BEGIN { fd=0; mem=0; } /ldb$/ { if ($4 == "mem") mem++; else fd++ } END { printf "mem = %s, fd = %s\n", mem, fd}'
mem = 119, fd = 0The mem value shows how many files are mmap'ed, and the fd value shows you many file descriptors these files are using. You should check that fd is a small number (usually 0 on 64-bit hosts).
See the notes in the SetMaxOpenFiles() function in dbwrapper.cc for more details.
It is possible for LevelDB changes to inadvertently change consensus compatibility between nodes. This happened in Bitcoin 0.8 (when LevelDB was first introduced). When upgrading LevelDB, you should review the upstream changes to check for issues affecting consensus compatibility.
For example, if LevelDB had a bug that accidentally prevented a key from being returned in an edge case, and that bug was fixed upstream, the bug "fix" would be an incompatible consensus change. In this situation, the correct behavior would be to revert the upstream fix before applying the updates to Bitcoin's copy of LevelDB. In general, you should be wary of any upstream changes affecting what data is returned from LevelDB queries.
For reformatting and refactoring commits where the changes can be easily automated using a bash script, we use scripted-diff commits. The bash script is included in the commit message and our CI job checks that the result of the script is identical to the commit. This aids reviewers since they can verify that the script does exactly what it is supposed to do. It is also helpful for rebasing (since the same script can just be re-run on the new master commit).
To create a scripted-diff:
scripted-diff: (and then a description of the diff on the same line)-BEGIN VERIFY SCRIPT--END VERIFY SCRIPT-The scripted-diff is verified by the tool test/lint/commit-script-check.sh. The tool's default behavior, when supplied with a commit is to verify all scripted-diffs from the beginning of time up to said commit. Internally, the tool passes the first supplied argument to git rev-list --reverse to determine which commits to verify script-diffs for, ignoring commits that don't conform to the commit message format described above.
For development, it might be more convenient to verify all scripted-diffs in a range A..B, for example:
If you need to replace in multiple files, prefer git ls-files to find or globbing, and git grep to grep, to avoid changing files that are not under version control.
For efficient replacement scripts, reduce the selection to the files that potentially need to be modified, so for example, instead of a blanket git ls-files src | xargs sed -i s/apple/orange/, use git grep -l apple src | xargs sed -i s/apple/orange/.
Also, it is good to keep the selection of files as specific as possible — for example, replace only in directories where you expect replacements — because it reduces the risk that a rebase of your commit by re-running the script will introduce accidental changes.
Some good examples of scripted-diff:
scripted-diff: Rename InitInterfaces to NodeContext uses an elegant script to replace occurrences of multiple terms in all source files.
scripted-diff: Remove g_connman, g_banman globals replaces specific terms in a list of specific source files.
scripted-diff: Replace fprintf with tfm::format does a global replacement but excludes certain directories.
To find all previous uses of scripted diffs in the repository, do:
git log --grep="-BEGIN VERIFY SCRIPT-"
Release notes should be written for any PR that:
Release notes should be added to a PR-specific release note file at /doc/release-notes-<PR number>.html to avoid conflicts between multiple PRs. All release-notes* files are merged into a single /doc/release-notes.html file prior to the release.
A few guidelines for introducing and reviewing new RPC interfaces:
Method naming: use consecutive lower-case names such as getrawtransaction and submitblock.
Argument naming: use snake case fee_delta (and not, e.g. camel case feeDelta)
Use the JSON parser for parsing, don't manually parse integers or strings from arguments unless absolutely necessary.
Rationale: Introduces hand-rolled string manipulation code at both the caller and callee sites, which is error-prone, and it is easy to get things such as escaping wrong. JSON already supports nested data structures, no need to re-invent the wheel.
Exception: AmountFromValue can parse amounts as string. This was introduced because many JSON parsers and formatters hard-code handling decimal numbers as floating-point values, resulting in potential loss of precision. This is unacceptable for monetary values. Always use AmountFromValue and ValueFromAmount when inputting or outputting monetary values. The only exceptions to this are prioritisetransaction and getblocktemplate because their interface is specified as-is in BIP22.
Missing arguments and 'null' should be treated the same: as default values. If there is no default value, both cases should fail in the same way. The easiest way to follow this guideline is to detect unspecified arguments with params[x].isNull() instead of params.size() <= x. The former returns true if the argument is either null or missing, while the latter returns true if is missing, and false if it is null.
Try not to overload methods on argument type. E.g. don't make getblock(true) and getblock("hash") do different things.
Rationale: This is impossible to use with bitcoin-cli, and can be surprising to users.
Exception: Some RPC calls can take both an int and bool, most notably when a bool was switched to a multi-value, or due to other historical reasons. Always have false map to 0 and true to 1 in this case.
Don't forget to fill in the argument names correctly in the RPC command table.
Set okSafeMode in the RPC command table to a sensible value: safe mode is when the blockchain is regarded to be in a confused state, and the client deems it unsafe to do anything irreversible such as send. Anything that just queries should be permitted.
Add every non-string RPC argument (method, idx, name) to the table vRPCConvertParams in rpc/client.cpp.
bitcoin-cli and the GUI debug console use this table to determine how to convert a plaintext command line to JSON. If the types don't match, the method can be unusable from there.A RPC method must either be a wallet method or a non-wallet method. Do not introduce new methods that differ in behavior based on the presence of a wallet.
Try to make the RPC response a JSON object.
Wallet RPCs call BlockUntilSyncedToCurrentChain to maintain consistency with getblockchaininfo's state immediately prior to the call's execution. Wallet RPCs whose behavior does not depend on the current chainstate may omit this call.
Be aware of RPC method aliases and generally avoid registering the same callback function pointer for different RPCs.
Rationale: RPC methods registered with the same function pointer will be considered aliases and only the first method name will show up in the help RPC command list.
Exception: Using RPC method aliases may be appropriate in cases where a new RPC is replacing a deprecated RPC, to avoid both RPCs confusingly showing up in the command list.
Use invalid bech32 addresses (e.g. in the constant array EXAMPLE_ADDRESS) for RPCExamples help documentation.
Use the UNIX_EPOCH_TIME constant when describing UNIX epoch time or timestamps in the documentation.
Internal interfaces between parts of the codebase that are meant to be independent (node, wallet, GUI), are defined in src/interfaces/. The main interface classes defined there are interfaces::Chain, used by wallet to access the node's latest chain state, interfaces::Node, used by the GUI to control the node, and interfaces::Wallet, used by the GUI to control an individual wallet. There are also more specialized interface types like interfaces::Handler interfaces::ChainClient passed to and from various interface methods.
Interface classes are written in a particular style so node, wallet, and GUI code doesn't need to run in the same process, and so the class declarations work more easily with tools and libraries supporting interprocess communication:
Interface classes should be abstract and have methods that are pure virtual. This allows multiple implementations to inherit from the same interface class, particularly so one implementation can execute functionality in the local process, and other implementations can forward calls to remote processes.
Interface method definitions should wrap existing functionality instead of implementing new functionality. Any substantial new node or wallet functionality should be implemented in src/node/ or src/wallet/ and just exposed in src/interfaces/ instead of being implemented there, so it can be more modular and accessible to unit tests.
Interface method parameter and return types should either be serializable or be other interface classes. Interface methods shouldn't pass references to objects that can't be serialized or accessed from another process.
Examples:
// Good: takes string argument and returns interface class pointer
virtual unique_ptr<interfaces::Wallet> loadWallet(std::string filename) = 0;
// Bad: returns CWallet reference that can't be used from another process
virtual CWallet& loadWallet(std::string filename) = 0;// Good: accepts and returns primitive types
virtual bool findBlock(const uint256& hash, int& out_height, int64_t& out_time) = 0;
// Bad: returns pointer to internal node in a linked list inaccessible to
// other processes
virtual const CBlockIndex* findBlock(const uint256& hash) = 0;// Good: takes plain callback type and returns interface pointer
using TipChangedFn = std::function<void(int block_height, int64_t block_time)>;
virtual std::unique_ptr<interfaces::Handler> handleTipChanged(TipChangedFn fn) = 0;
// Bad: returns boost connection specific to local process
using TipChangedFn = std::function<void(int block_height, int64_t block_time)>;
virtual boost::signals2::scoped_connection connectTipChanged(TipChangedFn fn) = 0;For consistency and friendliness to code generation tools, interface method input and inout parameters should be ordered first and output parameters should come last.
Example:
For friendliness to code generation tools, interface methods should not be overloaded:
Example:
For consistency and friendliness to code generation tools, interface method names should be lowerCamelCase and standalone function names should be UpperCamelCase.
Examples:
// Good: lowerCamelCase method name
virtual void blockConnected(const CBlock& block, int height) = 0;
// Bad: uppercase class method
virtual void BlockConnected(const CBlock& block, int height) = 0;// Good: UpperCamelCase standalone function name
std::unique_ptr<Node> MakeNode(LocalInit& init);
// Bad: lowercase standalone function
std::unique_ptr<Node> makeNode(LocalInit& init);Note: This last convention isn't generally followed outside of src/interfaces/, though it did come up for discussion before in #14635. Expectations for DNS Seed operators ====================================
Bitcoin Core attempts to minimize the level of trust in DNS seeds, but DNS seeds still pose a small amount of risk for the network. As such, DNS seeds must be run by entities which have some minimum level of trust within the Bitcoin community.
Other implementations of Bitcoin software may also use the same seeds and may be more exposed. In light of this exposure, this document establishes some basic expectations for operating dnsseeds.
A DNS seed operating organization or person is expected to follow good host security practices, maintain control of applicable infrastructure, and not sell or transfer control of the DNS seed. Any hosting services contracted by the operator are equally expected to uphold these expectations.
The DNS seed results must consist exclusively of fairly selected and functioning Bitcoin nodes from the public network to the best of the operator's understanding and capability.
For the avoidance of doubt, the results may be randomized but must not single-out any group of hosts to receive different results unless due to an urgent technical necessity and disclosed.
The results may not be served with a DNS TTL of less than one minute.
Any logging of DNS queries should be only that which is necessary for the operation of the service or urgent health of the Bitcoin network and must not be retained longer than necessary nor disclosed to any third party.
Information gathered as a result of the operators node-spidering (not from DNS queries) may be freely published or retained, but only if this data was not made more complete by biasing node connectivity (a violation of expectation (1)).
Operators are encouraged, but not required, to publicly document the details of their operating practices.
A reachable email contact address must be published for inquiries related to the DNS seed operation.
If these expectations cannot be satisfied the operator should discontinue providing services and contact the active Bitcoin Core development team as well as posting on bitcoin-dev.
Behavior outside of these expectations may be reasonable in some situations but should be discussed in public in advance.
Contents
The data directory is the default location where the Bitcoin Core files are stored.
| Platform | Data directory path |
|---|---|
| Linux | $HOME/.bitcoin/ |
| macOS | $HOME/Library/Application Support/Bitcoin/ |
| Windows | %APPDATA%\Bitcoin\ [1] |
A custom data directory path can be specified with the -datadir option.
All content of the data directory, except for bitcoin.conf file, is chain-specific. This means the actual data directory paths for non-mainnet cases differ:
| Chain option | Data directory path |
|---|---|
-chain=main (default) |
path_to_datadir/ |
-chain=test or -testnet |
path_to_datadir/testnet3/ |
-chain=signet or -signet |
path_to_datadir/signet/ |
-chain=regtest or -regtest |
path_to_datadir/regtest/ |
| Subdirectory | File(s) | Description |
|---|---|---|
blocks/ |
Blocks directory; can be specified by -blocksdir option (except for blocks/index/) |
|
blocks/index/ |
LevelDB database | Block index; -blocksdir option does not affect this path |
blocks/ |
blkNNNNN.dat[2] |
Actual Bitcoin blocks (in network format, dumped in raw on disk, 128 MiB per file) |
blocks/ |
revNNNNN.dat[2] |
Block undo data (custom format) |
chainstate/ |
LevelDB database | Blockchain state (a compact representation of all currently unspent transaction outputs (UTXOs) and metadata about the transactions they are from) |
indexes/txindex/ |
LevelDB database | Transaction index; optional, used if -txindex=1 |
indexes/blockfilter/basic/db/ |
LevelDB database | Blockfilter index LevelDB database for the basic filtertype; optional, used if -blockfilterindex=basic |
indexes/blockfilter/basic/ |
fltrNNNNN.dat[2] |
Blockfilter index filters for the basic filtertype; optional, used if -blockfilterindex=basic |
wallets/ |
Contains wallets; can be specified by -walletdir option; if wallets/ subdirectory does not exist, wallets reside in the data directory |
|
./ |
anchors.dat |
Anchor IP address database, created on shutdown and deleted at startup. Anchors are last known outgoing block-relay-only peers that are tried to re-connect to on startup |
./ |
banlist.dat |
Stores the IPs/subnets of banned nodes |
./ |
bitcoin.conf |
User-defined configuration settings for bitcoind or bitcoin-qt. File is not written to by the software and must be created manually. Path can be specified by -conf option |
./ |
bitcoind.pid |
Stores the process ID (PID) of bitcoind or bitcoin-qt while running; created at start and deleted on shutdown; can be specified by -pid option |
./ |
debug.log |
Contains debug information and general logging generated by bitcoind or bitcoin-qt; can be specified by -debuglogfile option |
./ |
fee_estimates.dat |
Stores statistics used to estimate minimum transaction fees and priorities required for confirmation |
./ |
guisettings.ini.bak |
Backup of former GUI settings after -resetguisettings option is used |
./ |
ip_asn.map |
IP addresses to Autonomous System Numbers (ASNs) mapping used for bucketing of the peers; path can be specified with the -asmap option |
./ |
mempool.dat |
Dump of the mempool's transactions |
./ |
onion_v3_private_key |
Cached Tor onion service private key for -listenonion option |
./ |
peers.dat |
Peer IP address database (custom format) |
./ |
settings.json |
Read-write settings set through GUI or RPC interfaces, augmenting manual settings from bitcoin.conf. File is created automatically if read-write settings storage is not disabled with -nosettings option. Path can be specified with -settings option |
./ |
.cookie |
Session RPC authentication cookie; if used, created at start and deleted on shutdown; can be specified by -rpccookiefile option |
./ |
.lock |
Data directory lock file |
Wallets are Berkeley DB (BDB) or SQLite databases.
Each user-defined wallet named "wallet_name" resides in the wallets/wallet_name/ subdirectory.
The default (unnamed) wallet resides in wallets/ subdirectory; if the latter does not exist, the wallet resides in the data directory.
A wallet database path can be specified with the -wallet option.
wallet.dat files must not be shared across different node instances, as that can result in key-reuse and double-spends due the lack of synchronization between instances.
Any copy or backup of the wallet should be done through a backupwallet call in order to update and lock the wallet, preventing any file corruption caused by updates during the copy.
| Subdirectory | File(s) | Description |
|---|---|---|
database/ |
BDB logging files | Part of BDB environment; created at start and deleted on shutdown; a user must keep it as safe as personal wallet wallet.dat |
./ |
db.log |
BDB error file |
./ |
wallet.dat |
Personal wallet (a BDB database) with keys and transactions |
./ |
.walletlock |
BDB wallet lock file |
| Subdirectory | File | Description |
|---|---|---|
./ |
wallet.dat |
Personal wallet (a SQLite database) with keys and transactions |
./ |
wallet.dat-journal |
SQLite Rollback Journal file for wallet.dat. Usually created at start and deleted on shutdown. A user must keep it as safe as the wallet.dat file. |
bitcoin-qt uses QSettings class; this implies platform-specific locations where application settings are stored.
These subdirectories and files are no longer used by the Bitcoin Core:
| Path | Description | Repository notes |
|---|---|---|
blktree/ |
Blockchain index; replaced by blocks/index/ in 0.8.0 |
PR #2231, 8fdc94cc |
coins/ |
Unspent transaction output database; replaced by chainstate/ in 0.8.0 |
PR #2231, 8fdc94cc |
blkindex.dat |
Blockchain index BDB database; replaced by {chainstate/, blocks/index/, blocks/revNNNNN.dat[2]} in 0.8.0 |
PR #1677 |
blk000?.dat |
Block data (custom format, 2 GiB per file); replaced by blocks/blkNNNNN.dat[2] in 0.8.0 |
PR #1677 |
addr.dat |
Peer IP address BDB database; replaced by peers.dat in 0.7.0 |
PR #1198, 928d3a01 |
onion_private_key |
Cached Tor onion service private key for -listenonion option. Was used for Tor v2 services; replaced by onion_v3_private_key in 0.21.0 |
PR #19954 |
1. The / (slash, U+002F) is used as the platform-independent path component separator in this document.
2. NNNNN matches [0-9]{5} regex.
To quickly get started fuzzing Bitcoin Core using libFuzzer:
$ git clone https://github.com/bitcoin/bitcoin
$ cd bitcoin/
$ ./autogen.sh
$ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
# macOS users: If you have problem with this step then make sure to read "macOS hints for
# libFuzzer" on https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.html#macos-hints-for-libfuzzer
$ make
$ FUZZ=process_message src/test/fuzz/fuzz
# abort fuzzing using ctrl-cprocess_message is a fuzzing harness for the ProcessMessage(...) function (net_processing). The available fuzzing harnesses are found in src/test/fuzz/.
The fuzzer will output NEW every time it has created a test input that covers new areas of the code under test. For more information on how to interpret the fuzzer output, see the libFuzzer documentation.
If you specify a corpus directory then any new coverage increasing inputs will be saved there:
$ mkdir -p process_message-seeded-from-thin-air/
$ FUZZ=process_message src/test/fuzz/fuzz process_message-seeded-from-thin-air/
INFO: Seed: 840522292
INFO: Loaded 1 modules (424174 inline 8-bit counters): 424174 [0x55e121ef9ab8, 0x55e121f613a6),
INFO: Loaded 1 PC tables (424174 PCs): 424174 [0x55e121f613a8,0x55e1225da288),
INFO: 0 files found in process_message-seeded-from-thin-air/
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: A corpus is not provided, starting from an empty corpus
#2 INITED cov: 94 ft: 95 corp: 1/1b exec/s: 0 rss: 150Mb
#3 NEW cov: 95 ft: 96 corp: 2/3b lim: 4 exec/s: 0 rss: 150Mb L: 2/2 MS: 1 InsertByte-
#4 NEW cov: 96 ft: 98 corp: 3/7b lim: 4 exec/s: 0 rss: 150Mb L: 4/4 MS: 1 CrossOver-
#21 NEW cov: 96 ft: 100 corp: 4/11b lim: 4 exec/s: 0 rss: 150Mb L: 4/4 MS: 2 ChangeBit-CrossOver-
#324 NEW cov: 101 ft: 105 corp: 5/12b lim: 6 exec/s: 0 rss: 150Mb L: 6/6 MS: 5 CrossOver-ChangeBit-CopyPart-ChangeBit-ChangeBinInt-
#1239 REDUCE cov: 102 ft: 106 corp: 6/24b lim: 14 exec/s: 0 rss: 150Mb L: 13/13 MS: 5 ChangeBit-CrossOver-EraseBytes-ChangeBit-InsertRepeatedBytes-
#1272 REDUCE cov: 102 ft: 106 corp: 6/23b lim: 14 exec/s: 0 rss: 150Mb L: 12/12 MS: 3 ChangeBinInt-ChangeBit-EraseBytes-
NEW_FUNC[1/677]: 0x55e11f456690 in std::_Function_base::~_Function_base() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/std_function.h:255
NEW_FUNC[2/677]: 0x55e11f465800 in CDataStream::CDataStream(std::vector<unsigned char, std::allocator<unsigned char> > const&, int, int) src/./streams.h:248
#2125 REDUCE cov: 4820 ft: 4867 corp: 7/29b lim: 21 exec/s: 0 rss: 155Mb L: 6/12 MS: 2 CopyPart-CMP- DE: "block"-
NEW_FUNC[1/9]: 0x55e11f64d790 in std::_Rb_tree<uint256, std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > >, std::_Select1st<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >, std::less<uint256>, std::allocator<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > > >::~_Rb_tree() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_tree.h:972
NEW_FUNC[2/9]: 0x55e11f64d870 in std::_Rb_tree<uint256, std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > >, std::_Select1st<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >, std::less<uint256>, std::allocator<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > > >::_M_erase(std::_Rb_tree_node<std::pair<uint256 const, std::chrono::duration<long, std::ratio<1l, 1000000l> > > >*) /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_tree.h:1875
#2228 NEW cov: 4898 ft: 4971 corp: 8/35b lim: 21 exec/s: 0 rss: 156Mb L: 6/12 MS: 3 EraseBytes-CopyPart-PersAutoDict- DE: "block"-
NEW_FUNC[1/5]: 0x55e11f46df70 in std::enable_if<__and_<std::allocator_traits<zero_after_free_allocator<char> >::__construct_helper<char, unsigned char const&>::type>::value, void>::type std::allocator_traits<zero_after_free_allocator<char> >::_S_construct<char, unsigned char const&>(zero_after_free_allocator<char>&, char*, unsigned char const&) /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/alloc_traits.h:243
NEW_FUNC[2/5]: 0x55e11f477390 in std::vector<unsigned char, std::allocator<unsigned char> >::data() /usr/lib/gcc/x86_64-linux-gnu/8/../../../../include/c++/8/bits/stl_vector.h:1056
#2456 NEW cov: 4933 ft: 5042 corp: 9/55b lim: 21 exec/s: 0 rss: 160Mb L: 20/20 MS: 3 ChangeByte-InsertRepeatedBytes-PersAutoDict- DE: "block"-
#2467 NEW cov: 4933 ft: 5043 corp: 10/76b lim: 21 exec/s: 0 rss: 161Mb L: 21/21 MS: 1 InsertByte-
#4215 NEW cov: 4941 ft: 5129 corp: 17/205b lim: 29 exec/s: 4215 rss: 350Mb L: 29/29 MS: 5 InsertByte-ChangeBit-CopyPart-InsertRepeatedBytes-CrossOver-
#4567 REDUCE cov: 4941 ft: 5129 corp: 17/204b lim: 29 exec/s: 4567 rss: 404Mb L: 24/29 MS: 2 ChangeByte-EraseBytes-
#6642 NEW cov: 4941 ft: 5138 corp: 18/244b lim: 43 exec/s: 2214 rss: 450Mb L: 43/43 MS: 3 CopyPart-CMP-CrossOver- DE: "verack"-
# abort fuzzing using ctrl-c
$ ls process_message-seeded-from-thin-air/
349ac589fc66a09abc0b72bb4ae445a7a19e2cd8 4df479f1f421f2ea64b383cd4919a272604087a7
a640312c98dcc55d6744730c33e41c5168c55f09 b135de16e4709558c0797c15f86046d31c5d86d7
c000f7b41b05139de8b63f4cbf7d1ad4c6e2aa7f fc52cc00ec1eb1c08470e69f809ae4993fa70082
$ cat --show-nonprinting process_message-seeded-from-thin-air/349ac589fc66a09abc0b72bb4ae445a7a19e2cd8
block^@M-^?M-^?M-^?M-^?M-^?nM-^?M-^?In this case the fuzzer managed to create a block message which when passed to ProcessMessage(...) increased coverage.
The project's collection of seed corpora is found in the bitcoin-core/qa-assets repo.
To fuzz process_message using the bitcoin-core/qa-assets seed corpus:
$ git clone https://github.com/bitcoin-core/qa-assets
$ FUZZ=process_message src/test/fuzz/fuzz qa-assets/fuzz_seed_corpus/process_message/
INFO: Seed: 1346407872
INFO: Loaded 1 modules (424174 inline 8-bit counters): 424174 [0x55d8a9004ab8, 0x55d8a906c3a6),
INFO: Loaded 1 PC tables (424174 PCs): 424174 [0x55d8a906c3a8,0x55d8a96e5288),
INFO: 991 files found in qa-assets/fuzz_seed_corpus/process_message/
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: seed corpus: files: 991 min: 1b max: 1858b total: 288291b rss: 150Mb
#993 INITED cov: 7063 ft: 8236 corp: 25/3821b exec/s: 0 rss: 181Mb
…If you find coverage increasing inputs when fuzzing you are highly encouraged to submit them for inclusion in the bitcoin-core/qa-assets repo.
Every single pull request submitted against the Bitcoin Core repo is automatically tested against all inputs in the bitcoin-core/qa-assets repo. Contributing new coverage increasing inputs is an easy way to help make Bitcoin Core more robust.
The default Clang/LLVM version supplied by Apple on macOS does not include fuzzing libraries, so macOS users will need to install a full version, for example using brew install llvm.
Should you run into problems with the address sanitizer, it is possible you may need to run ./configure with --disable-asm to avoid errors with certain assembly code from Bitcoin Core's code. See developer notes on sanitizers for more information.
You may also need to take care of giving the correct path for clang and clang++, like CC=/path/to/clang CXX=/path/to/clang++ if the non-systems clang does not come first in your path.
Full configure that was tested on macOS Catalina with brew installed llvm:
Read the libFuzzer documentation for more information. This libFuzzer tutorial might also be of interest.
afl-fuzz)To quickly get started fuzzing Bitcoin Core using afl-fuzz:
$ git clone https://github.com/bitcoin/bitcoin
$ cd bitcoin/
$ git clone https://github.com/google/afl
$ make -C afl/
$ make -C afl/llvm_mode/
$ ./autogen.sh
# It is possible to compile with afl-gcc and afl-g++ instead of afl-clang. However, running afl-fuzz
# may require more memory via the -m flag.
$ CC=$(pwd)/afl/afl-clang-fast CXX=$(pwd)/afl/afl-clang-fast++ ./configure --enable-fuzz
$ make
# For macOS you may need to ignore x86 compilation checks when running "make". If so,
# try compiling using: AFL_NO_X86=1 make
$ mkdir -p inputs/ outputs/
$ echo A > inputs/thin-air-input
$ FUZZ=bech32 afl/afl-fuzz -i inputs/ -o outputs/ -- src/test/fuzz/fuzz
# You may have to change a few kernel parameters to test optimally - afl-fuzz
# will print an error and suggestion if so.Read the afl-fuzz documentation for more information.
To quickly get started fuzzing Bitcoin Core using Honggfuzz:
$ git clone https://github.com/bitcoin/bitcoin
$ cd bitcoin/
$ ./autogen.sh
$ git clone https://github.com/google/honggfuzz
$ cd honggfuzz/
$ make
$ cd ..
$ CC=$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang CXX=$(pwd)/honggfuzz/hfuzz_cc/hfuzz-clang++ ./configure --enable-fuzz --with-sanitizers=address,undefined
$ make
$ mkdir -p inputs/
$ FUZZ=process_message honggfuzz/honggfuzz -i inputs/ -- src/test/fuzz/fuzzSample scripts and configuration files for syst.html, Upstart and OpenRC can be found in the contrib/init folder.
contrib/init/bitcoind.service: syst.html service unit configuration
contrib/init/bitcoind.openrc: OpenRC compatible SysV style init script
contrib/init/bitcoind.openrcconf: OpenRC conf.d file
contrib/init/bitcoind.conf: Upstart service configuration file
contrib/init/bitcoind.init: CentOS compatible SysV style init script
All three Linux startup configurations assume the existence of a "bitcoin" user and group. They must be created before attempting to use these scripts. The macOS configuration assumes bitcoind will be set up for the current user.
Running bitcoind as a daemon does not require any manual configuration. You may set the rpcauth setting in the bitcoin.conf configuration file to override the default behaviour of using a special cookie for authentication.
This password does not have to be remembered or typed as it is mostly used as a fixed token that bitcoind and client programs read from the configuration file, however it is recommended that a strong and secure password be used as this password is security critical to securing the wallet should the wallet be enabled.
If bitcoind is run with the "-server" flag (set by default), and no rpcpassword is set, it will use a special cookie file for authentication. The cookie is generated with random content when the daemon starts, and deleted when it exits. Read access to this file controls who can access it through RPC.
By default the cookie is stored in the data directory, but it's location can be overridden with the option '-rpccookiefile'.
This allows for running bitcoind without having to do any manual configuration.
conf, pid, and wallet accept relative paths which are interpreted as relative to the data directory. wallet only supports relative paths.
For an example configuration file that describes the configuration settings, see share/examples/bitcoin.conf.
All three configurations assume several paths that might need to be adjusted.
Binary: /usr/bin/bitcoind Configuration file: /etc/bitcoin/bitcoin.conf Data directory: /var/lib/bitcoind PID file: /var/run/bitcoind/bitcoind.pid (OpenRC and Upstart) or /run/bitcoind/bitcoind.pid (syst.html) Lock file: /var/lock/subsys/bitcoind (CentOS)
The PID directory (if applicable) and data directory should both be owned by the bitcoin user and group. It is advised for security reasons to make the configuration file and data directory only readable by the bitcoin user and group. Access to bitcoin-cli and other bitcoind rpc clients can then be controlled by group membership.
NOTE: When using the syst.html .service file, the creation of the aforementioned directories and the setting of their permissions is automatically handled by syst.html. Directories are given a permission of 710, giving the bitcoin group access to files under it if the files themselves give permission to the bitcoin group to do so (e.g. when -sysperms is specified). This does not allow for the listing of files under the directory.
NOTE: It is not currently possible to override datadir in /etc/bitcoin/bitcoin.conf with the current syst.html, OpenRC, and Upstart init files out-of-the-box. This is because the command line options specified in the init files take precedence over the configurations in /etc/bitcoin/bitcoin.conf. However, some init systems have their own configuration mechanisms that would allow for overriding the command line options specified in the init files (e.g. setting BITCOIND_DATADIR for OpenRC).
Binary: /usr/local/bin/bitcoind Configuration file: ~/Library/Application Support/Bitcoin/bitcoin.conf Data directory: ~/Library/Application Support/Bitcoin Lock file: ~/Library/Application Support/Bitcoin/.lock
Installing this .service file consists of just copying it to /usr/lib/syst.html/system directory, followed by the command systemctl daemon-reload in order to update running syst.html configuration.
To test, run systemctl start bitcoind and to enable for system startup run systemctl enable bitcoind
NOTE: When installing for syst.html in Debian/Ubuntu the .service file needs to be copied to the /lib/syst.html/system directory instead.
Rename bitcoind.openrc to bitcoind and drop it in /etc/init.d. Double check ownership and permissions and make it executable. Test it with /etc/init.d/bitcoind start and configure it to run on startup with rc-update add bitcoind
Upstart is the default init system for Debian/Ubuntu versions older than 15.04. If you are using version 15.04 or newer and haven't manually configured upstart you should follow the syst.html instructions instead.
Drop bitcoind.conf in /etc/init. Test by running service bitcoind start it will automatically start on reboot.
NOTE: This script is incompatible with CentOS 5 and Amazon Linux 2014 as they use old versions of Upstart and do not supply the start-stop-daemon utility.
Copy bitcoind.init to /etc/init.d/bitcoind. Test by running service bitcoind start.
Using this script, you can adjust the path and flags to the bitcoind program by setting the BITCOIND and FLAGS environment variables in the file /etc/sysconfig/bitcoind. You can also use the DAEMONOPTS environment variable here.
Copy org.bitcoin.bitcoind.plist into ~/Library/LaunchAgents. Load the launch agent by running launchctl load ~/Library/LaunchAgents/org.bitcoin.bitcoind.plist.
This Launch Agent will cause bitcoind to start whenever the user logs in.
NOTE: This approach is intended for those wanting to run bitcoind as the current user. You will need to modify org.bitcoin.bitcoind.plist if you intend to use it as a Launch Daemon with a dedicated bitcoin user.
Auto respawning is currently only configured for Upstart and syst.html. Reasonable defaults have been chosen but YMMV.
On unix systems, the --enable-multiprocess build option can be passed to ./configure to build new bitcoin-node, bitcoin-wallet, and bitcoin-gui executables alongside existing bitcoind and bitcoin-qt executables.
bitcoin-node is a drop-in replacement for bitcoind, and bitcoin-gui is a drop-in replacement for bitcoin-qt, and there are no differences in use or external behavior between the new and old executables. But internally (after #10102), bitcoin-gui will spawn a bitcoin-node process to run P2P and RPC code, communicating with it across a socket pair, and bitcoin-node will spawn bitcoin-wallet to run wallet code, also communicating over a socket pair. This will let node, wallet, and GUI code run in separate address spaces for better isolation, and allow future improvements like being able to start and stop components independently on different machines and environments.
Specific next steps after #10102 will be:
-ipcbind and -ipcconnect options to bitcoin-node, bitcoin-wallet, and bitcoin-gui executables so they can listen and connect to TCP ports and unix socket paths. This will allow separate processes to be started and stopped any time and connect to each other.-server and -rpcbind options to the bitcoin-wallet executable so wallet processes can handle RPC requests directly without going through the node.CreateProcess and WSADuplicateSocket. Example: https://memset.wordpress.com/2010/10/13/win32-api-passing-socket-with-ipc-method/.After #10102, the -debug=ipc command line option can be used to see requests and responses between processes.
The multiprocess feature requires Cap'n Proto and libmultiprocess as dependencies. A simple way to get starting using it without installing these dependencies manually is to use the depends system with the MULTIPROCESS=1 dependency option passed to make:
cd <BITCOIN_SOURCE_DIRECTORY>
make -C depends NO_QT=1 MULTIPROCESS=1
./configure --prefix=$PWD/depends/x86_64-pc-linux-gnu
make
src/bitcoin-node -regtest -printtoconsole -debug=ipc
BITCOIND=bitcoin-node test/functional/test_runner.py
The configure script will pick up settings and library locations from the depends directory, so there is no need to pass --enable-multiprocess as a separate flag when using the depends system (it's controlled by the MULTIPROCESS=1 option).
./configure --enable-multiprocess without using the depends system. The configure script will be able to locate the installed packages via pkg-config. See Installation section of the libmultiprocess readme for install steps. See build-unix.html and build-osx.html for information about installing dependencies in general. Productivity NotesccacheThe easiest way to faster compile times is to cache compiles. ccache is a way to do so, from its description at the time of writing:
ccache is a compiler cache. It speeds up recompilation by caching the result of previous compilations and detecting when the same compilation is being done again. Supported languages are C, C++, Objective-C and Objective-C++.
Install ccache through your distribution's package manager, and run ./configure with your normal flags to pick it up.
To use ccache for all your C/C++ projects, follow the symlinks method here to set it up.
To get the most out of ccache, put something like this in ~/.ccache/ccache.conf:
max_size = 50.0G # or whatever cache size you prefer; default is 5G; 0 means unlimited
base_dir = /home/yourname # or wherever you keep your source files
Note: base_dir is required for ccache to share cached compiles of the same file across different repositories / paths; it will only do this for paths under base_dir. So this option is required for effective use of ccache with git worktrees (described below).
You must not set base_dir to "/", or anywhere that contains system headers (according to the ccache docs).
./configureAfter running ./autogen.sh, which generates the ./configure file, use ./configure --help to identify features that you can disable to save on compilation time. A few common flags:
If you do need the wallet enabled, it is common for devs to add --with-incompatible-bdb. This uses your system bdb version for the wallet, so you don't have to find a copy of bdb 4.8. Wallets from such a build will be incompatible with any release binary (and vice versa), so use with caution on mainnet.
make -jIf you have multiple threads on your machine, you can tell make to utilize all of them with:
When rebuilding during development, note that running make, without giving a target, will do a lot of work you probably don't need. It will build the GUI (unless you've disabled it) and all the tests (which take much longer to build than the app does).
Obviously, it is important to build and run the tests at appropriate times -- but when you just want a quick compile to check your work, consider picking one or a set of build targets relevant to what you're working on, e.g.:
(You can and should combine this with -j, as above, for a parallel build.)
git worktreesIf you work with multiple branches or multiple copies of the repository, you should try git worktrees.
To create a new branch that lives under a new working directory without disrupting your current working directory (useful for creating pull requests):
To simply check out a commit-ish under a new working directory without disrupting your current working directory (useful for reviewing pull requests):
git merge-baseWhen rebasing, we often want to do a "dummy rebase," whereby we are not rebasing over an updated master but rather over the last common commit with master. This might be useful for rearranging commits, rebase --autosquashing, or rebase --execing without introducing conflicts that arise from an updated master. In these situations, we can use git merge-base to identify the last common commit with master, and rebase off of that.
To squash in git commit --fixup commits without rebasing over an updated master, we can do the following:
To execute make check on every commit since last diverged from master, but without rebasing over an updated master, we can do the following:
This synergizes well with ccache as objects resulting from unchanged code will most likely hit the cache and won't need to be recompiled.
You can also set up upstream refspecs to refer to pull requests easier in the above git worktree commands.
clang-format-diff.pySee contrib/devtools/README.html.
yapf-diff.pyUsage is exactly the same as clang-format-diff.py. You can get it here.
merge.conflictstyle diff3For resolving merge/rebase conflicts, it can be useful to enable diff3 style using git config merge.conflictstyle diff3. Instead of
you will see
This may make it much clearer what caused the conflict. In this style, you can often just look at what changed between original and theirs, and mechanically apply that to yours (or the other way around).
git diff optionsWhen reviewing patches which change indentation in C++ files, use git diff -w and git show -w. This makes the diff algorithm ignore whitespace changes. This feature is also available on github.com, by adding ?w=1 at the end of any URL which shows a diff.
When reviewing patches that change symbol names in many places, use git diff --word-diff. This will instead of showing the patch as deleted/added lines, show deleted/added words.
When reviewing patches that move code around, try using git diff --patience commit~:old/file.cpp commit:new/file/name.cpp, and ignoring everything except the moved body of code which should show up as neither + or - lines. In case it was not a pure move, this may even work when combined with the -w or --word-diff options described above. --color-moved=dimmed-zebra will also dim the coloring of moved hunks in the diff on compatible terminals.
refspecsWhen looking at other's pull requests, it may make sense to add the following section to your .git/config file:
[remote "upstream-pull"]
fetch = +refs/pull/*/head:refs/remotes/upstream-pull/*
url = git@github.com:bitcoin/bitcoin.git
This will add an upstream-pull remote to your git repository, which can be fetched using git fetch --all or git fetch upstream-pull. It will download and store on disk quite a lot of data (all PRs, including merged and closed ones). Afterwards, you can use upstream-pull/NUMBER/head in arguments to git show, git checkout and anywhere a commit id would be acceptable to see the changes from pull request NUMBER.
git range-diffIt is very common for contributors to rebase their pull requests, or make changes to commits (perhaps in response to review) that are not at the head of their branch. This poses a problem for reviewers as when the contributor force pushes, the reviewer is no longer sure that his previous reviews of commits are still valid (as the commit hashes can now be different even though the diff is semantically the same). git range-diff (Git >= 2.19) can help solve this problem by diffing the diffs.
For example, to identify the differences between your previously reviewed diffs P1-5, and the new diffs P1-2,N3-4 as illustrated below:
P1--P2--P3--P4--P5 <-- previously-reviewed-head
/
...--m <-- master
\
P1--P2--N3--N4--N5 <-- new-head (with P3 slightly modified)
You can do:
Note that git range-diff also work for rebases:
P1--P2--P3--P4--P5 <-- previously-reviewed-head
/
...--m--m1--m2--m3 <-- master
\
P1--P2--N3--N4 <-- new-head (with P3 modified, P4 & P5 squashed)
PREV=P5 N=4 && git range-diff `git merge-base --all HEAD $PREV`...$PREV HEAD~$N...HEAD
Where P5 is the commit you last reviewed and 4 is the number of commits in the new version.
git range-diff also accepts normal git diff options, see Reduce mental load with git diff options for useful git diff options.
You can also set up upstream refspecs to refer to pull requests easier in the above git range-diff commands.
Since Bitcoin Core 0.17, an RPC interface exists for Partially Signed Bitcoin Transactions (PSBTs, as specified in BIP 174).
This document describes the overall workflow for producing signed transactions through the use of PSBT, and the specific RPC commands used in typical scenarios.
PSBT is an interchange format for Bitcoin transactions that are not fully signed yet, together with relevant metadata to help entities work towards signing it. It is intended to simplify workflows where multiple parties need to cooperate to produce a transaction. Examples include hardware wallets, multisig setups, and CoinJoin transactions.
Overall, the construction of a fully signed Bitcoin transaction goes through the following steps:
scriptSig and/or scriptWitness.Generally, each of the above (excluding Creator and Extractor) will simply add more and more data to a particular PSBT, until all inputs are fully signed. In a naive workflow, they all have to operate sequentially, passing the PSBT from one to the next, until the Extractor can convert it to a real transaction. In order to permit parallel operation, Combiners can be employed which merge metadata from different PSBTs for the same unsigned transaction.
The names above in bold are the names of the roles defined in BIP174. They're useful in understanding the underlying steps, but in practice, software and hardware implementations will typically implement multiple roles simultaneously.
converttopsbt (Creator) is a utility RPC that converts an unsigned raw transaction to PSBT format. It ignores existing signatures.createpsbt (Creator) is a utility RPC that takes a list of inputs and outputs and converts them to a PSBT with no additional information. It is equivalent to calling createrawtransaction followed by converttopsbt.walletcreatefundedpsbt (Creator, Updater) is a wallet RPC that creates a PSBT with the specified inputs and outputs, adds additional inputs and change to it to balance it out, and adds relevant metadata. In particular, for inputs that the wallet knows about (counting towards its normal or watch-only balance), UTXO information will be added. For outputs and inputs with UTXO information present, key and script information will be added which the wallet knows about. It is equivalent to running createrawtransaction, followed by fundrawtransaction, and converttopsbt.walletprocesspsbt (Updater, Signer, Finalizer) is a wallet RPC that takes as input a PSBT, adds UTXO, key, and script data to inputs and outputs that miss it, and optionally signs inputs. Where possible it also finalizes the partial signatures.utxoupdatepsbt (Updater) is a node RPC that takes a PSBT and updates it to include information available from the UTXO set (works only for SegWit inputs).finalizepsbt (Finalizer, Extractor) is a utility RPC that finalizes any partial signatures, and if all inputs are finalized, converts the result to a fully signed transaction which can be broadcast with sendrawtransaction.combinepsbt (Combiner) is a utility RPC that implements a Combiner. It can be used at any point in the workflow to merge information added to different versions of the same PSBT. In particular it is useful to combine the output of multiple Updaters or Signers.joinpsbts (Creator) is a utility RPC that joins multiple PSBTs together, concatenating the inputs and outputs. This can be used to construct CoinJoin transactions.decodepsbt is a diagnostic utility RPC which will show all information in a PSBT in human-readable form, as well as compute its eventual fee if known.analyzepsbt is a utility RPC that examines a PSBT and reports the current status of its inputs, the next step in the workflow if known, and if possible, computes the fee of the resulting transaction and estimates the final weight and feerate.Alice, Bob, and Carol want to create a 2-of-3 multisig address. They're all using Bitcoin Core. We assume their wallets only contain the multisig funds. In case they also have a personal wallet, this can be accomplished through the multiwallet feature - possibly resulting in a need to add -rpcwallet=name to the command line in case bitcoin-cli is used.
Setup:
getnewaddress to create a new address; call these addresses Aalice, Abob, and Acarol.getaddressinfo "X", with X their respective address, and remember the corresponding public keys. Call these public keys Kalice, Kbob, and Kcarol.addmultisigaddress 2 ["Kalice","Kbob","Kcarol"] to teach their wallet about the multisig script. Call the address produced by this command Amulti. They may be required to explicitly specify the same addresstype option each, to avoid constructing different versions due to differences in configuration.importaddress "Amulti" "" false to make their wallets treat payments to Amulti as contributing to the watch-only balance.createmultisig 2 ["Kalice","Kbob","Kcarol"], and expecting Amulti as output. Again, it may be necessary to explicitly specify the addresstype in order to get a result that matches. This command won't enable them to initiate transactions later, however.Later, when V BTC has been received on Amulti, and Bob and Carol want to move the coins in their entirety to address Asend, with no change. Alice does not need to be involved.
walletcreatefundedpsbt [] {"Asend":V} 0 {"subtractFeeFromOutputs":[0], "includeWatching":true}. We call the resulting PSBT P. P does not contain any signatures.walletprocesspsbt "P", and gives the resulting PSBT P2 to Bob.decodepsbt "P2" to determine if the transaction has indeed just the expected input, and an output to Asend, and the fee is reasonable. If he agrees, he calls walletprocesspsbt "P2" to sign. The resulting PSBT P3 contains both Carol's and Bob's signature.finalizepsbt "P3" to extract a fully signed transaction T.sendrawtransaction "T".In case there are more signers, it may be advantageous to let them all sign in parallel, rather than passing the PSBT from one signer to the next one. In the above example this would translate to Carol handing a copy of P to each signer separately. They can then all invoke walletprocesspsbt "P", and end up with their individually-signed PSBT structures. They then all send those back to Carol (or anyone) who can combine them using combinepsbt. The last two steps (finalizepsbt and sendrawtransaction) remain unchanged.
There are a few parameters that can be dialed down to reduce the memory usage of bitcoind. This can be useful on embedded systems or small VPSes.
The size of some in-memory caches can be reduced. As caches trade off memory usage for performance, reducing these will usually have a negative effect on performance.
-dbcache=<n> - the UTXO database cache size, this defaults to 450. The unit is MiB (1024).
-dbcache is 4.-dbcache makes initial sync time much longer. After the initial sync, the effect is less pronounced for most use-cases, unless fast validation of blocks is important, such as for mining.In Bitcoin Core there is a memory pool limiter which can be configured with -maxmempool=<n>, where <n> is the size in MB (1000). The default value is 300.
-maxmempool is 5.bitcoind that process unconfirmed transactions.To completely disable mempool functionality there is the option -blocksonly. This will make the client opt out of receiving (and thus relaying) transactions completely, except as part of blocks.
-walletbroadcast=0 and -spendzeroconfchange=0. Another mechanism for broadcasting outgoing transactions (if any) should be used.Since 0.14.0, unused memory allocated to the mempool (default: 300MB) is shared with the UTXO cache, so when trying to reduce memory usage you should limit the mempool, with the -maxmempool command line argument.
-maxconnections=<n> - the maximum number of connections, this defaults to 125. Each active connection takes up some memory. This option applies only if incoming connections are enabled, otherwise the number of connections will never be more than 10. Of the 10 outbound peers, there can be 8 full-relay connections and 2 block-relay-only ones.For each thread a thread stack needs to be allocated. By default on Linux, threads take up 8MiB for the thread stack on a 64-bit system, and 4MiB in a 32-bit system.
-par=<n> - the number of script verification threads, defaults to the number of cores in the system minus one.-rpcthreads=<n> - the number of threads used for processing RPC requests, defaults to 4.By default, since glibc 2.10, the C library will create up to two heap arenas per core. This is known to cause excessive memory usage in some scenarios. To avoid this make a script that sets MALLOC_ARENA_MAX before starting bitcoind:
Some node operators need to deal with bandwidth caps imposed by their ISPs.
By default, Bitcoin Core allows up to 125 connections to different peers, 10 of which are outbound. You can therefore, have at most 115 inbound connections. Of the 10 outbound peers, there can be 8 full-relay connections and 2 block-relay-only ones.
The default settings can result in relatively significant traffic consumption.
Ways to reduce traffic:
-maxuploadtarget=<MiB per day>A major component of the traffic is caused by serving historic blocks to other nodes during the initial blocks download phase (syncing up a new node). This option can be specified in MiB per day and is turned off by default. This is not a hard limit; only a threshold to minimize the outbound traffic. When the limit is about to be reached, the uploaded data is cut by no longer serving historic blocks (blocks older than one week). Keep in mind that new nodes require other nodes that are willing to serve historic blocks.
Peers with the download permission will never be disconnected, although their traffic counts for calculating the target.
-listen=0)Disabling listening will result in fewer nodes connected (remember the maximum of 10 outbound peers). Fewer nodes will result in less traffic usage as you are relaying blocks and transactions to fewer nodes.
-maxconnections=<num>)Reducing the maximum connected nodes to a minimum could be desirable if traffic limits are tiny. Keep in mind that bitcoin's trustless model works best if you are connected to a handful of nodes.
-blocksonly)Forwarding transactions to peers increases the P2P traffic. To only sync blocks with other peers, you can disable transaction relay.
Be reminded of the effects of this setting.
configure.ac (CLIENT_VERSION_RC).configure.ac (don't forget to set CLIENT_VERSION_RC to 0).CLIENT_VERSION_MAJOR in configure.acCLIENT_VERSION_MAJOR, PACKAGE_VERSION, and PACKAGE_STRING in build_msvc/bitcoin_config.hconfigure.ac and build_msvc/bitcoin_config.h (see this commit):
CLIENT_VERSION_MINOR to 0CLIENT_VERSION_BUILD to 0CLIENT_VERSION_IS_RELEASE to truesrc/chainparams.cpp m_assumed_blockchain_size and m_assumed_chain_state_size with the current size plus some overhead (see this for information on how to calculate them).src/chainparams.cpp chainTxData with statistics about the transaction count and rate. Use the output of the getchaintxstats RPC, see this pull request for an example. Reviewers can verify the results by running getchaintxstats <window_block_count> <window_final_block_hash> with the window_block_count and window_final_block_hash from your output.src/chainparams.cpp nMinimumChainWork and defaultAssumeValid (and the block height comment) with information from the getblockheader (and getblockhash) RPCs.
contrib/gitian-descriptors/*.yml.If you're using the automated script (found in contrib/gitian-build.py), then at this point you should run it with the "--setup" command. Otherwise ignore this.
Check out the source code in the following directory hierarchy.
cd /path/to/your/toplevel/build
git clone https://github.com/bitcoin-core/gitian.sigs.git
git clone https://github.com/bitcoin-core/bitcoin-detached-sigs.git
git clone https://github.com/devrandom/gitian-builder.git
git clone https://github.com/bitcoin/bitcoin.git
Open a draft of the release notes for collaborative editing at https://github.com/bitcoin-core/bitcoin-devwiki/wiki.
For the period during which the notes are being edited on the wiki, the version on the branch should be wiped and replaced with a link to the wiki which should be used for all announcements until -final.
Write the release notes. git shortlog helps a lot, for example:
git shortlog --no-merges v(current version, e.g. 0.19.2)..v(new version, e.g. 0.20.0)
(or ping @wumpus on IRC, he has specific tooling to generate the list of merged pulls and sort them into categories based on labels).
Generate list of authors:
git log --format='- %aN' v(current version, e.g. 0.20.0)..v(new version, e.g. 0.20.1) | sort -fiu
Tag the version (or release candidate) in git:
git tag -s v(new version, e.g. 0.20.0)
If you're using the automated script (found in contrib/gitian-build.py), then at this point you should run it with the "--build" command. Otherwise ignore this.
Setup Gitian descriptors:
pushd ./bitcoin
export SIGNER="(your Gitian key, ie bluematt, sipa, etc)"
export VERSION=(new version, e.g. 0.20.0)
git fetch
git checkout v${VERSION}
popd
Ensure your gitian.sigs are up-to-date if you wish to gverify your builds against other Gitian signatures.
pushd ./gitian.sigs
git pull
popd
Ensure gitian-builder is up-to-date:
pushd ./gitian-builder
git pull
popd
pushd ./gitian-builder
mkdir -p inputs
wget -O inputs/osslsigncode-2.0.tar.gz https://github.com/mtrojnar/osslsigncode/archive/2.0.tar.gz
echo '5a60e0a4b3e0b4d655317b2f12a810211c50242138322b16e7e01c6fbb89d92f inputs/osslsigncode-2.0.tar.gz' | sha256sum -c
popd
Create the macOS SDK tarball, see the macdeploy instructions for details, and copy it into the inputs directory.
NOTE: Gitian is sometimes unable to download files. If you have errors, try the step below.
By default, Gitian will fetch source files as needed. To cache them ahead of time, make sure you have checked out the tag you want to build in bitcoin, then:
pushd ./gitian-builder
make -C ../bitcoin/depends download SOURCES_PATH=`pwd`/cache/common
popd
Only missing files will be fetched, so this is safe to re-run for each build.
NOTE: Offline builds must use the --url flag to ensure Gitian fetches only from local URLs. For example:
pushd ./gitian-builder
./bin/gbuild --url bitcoin=/path/to/bitcoin,signature=/path/to/sigs {rest of arguments}
popd
The gbuild invocations below DO NOT DO THIS by default.
pushd ./gitian-builder
./bin/gbuild --num-make 2 --memory 3000 --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-linux --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
mv build/out/bitcoin-*.tar.gz build/out/src/bitcoin-*.tar.gz ../
./bin/gbuild --num-make 2 --memory 3000 --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-win.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-win-unsigned --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-win.yml
mv build/out/bitcoin-*-win-unsigned.tar.gz inputs/bitcoin-win-unsigned.tar.gz
mv build/out/bitcoin-*.zip build/out/bitcoin-*.exe ../
./bin/gbuild --num-make 2 --memory 3000 --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-osx-unsigned --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml
mv build/out/bitcoin-*-osx-unsigned.tar.gz inputs/bitcoin-osx-unsigned.tar.gz
mv build/out/bitcoin-*.tar.gz build/out/bitcoin-*.dmg ../
popd
Build output expected:
bitcoin-${VERSION}.tar.gz)bitcoin-${VERSION}-linux[32|64].tar.gz)bitcoin-${VERSION}-win[32|64]-setup-unsigned.exe, bitcoin-${VERSION}-win[32|64].zip)bitcoin-${VERSION}-osx-unsigned.dmg, bitcoin-${VERSION}-osx64.tar.gz)gitian.sigs/${VERSION}-<linux|{win,osx}-unsigned>/(your Gitian key)/)Add other gitian builders keys to your gpg keyring, and/or refresh keys: See ../bitcoin/contrib/gitian-keys/README.html.
Verify the signatures
pushd ./gitian-builder
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-linux ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-unsigned ../bitcoin/contrib/gitian-descriptors/gitian-win.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-unsigned ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml
popd
Commit your signature to gitian.sigs:
pushd gitian.sigs
git add ${VERSION}-linux/"${SIGNER}"
git add ${VERSION}-win-unsigned/"${SIGNER}"
git add ${VERSION}-osx-unsigned/"${SIGNER}"
git commit -m "Add ${VERSION} unsigned sigs for ${SIGNER}"
git push # Assuming you can push to the gitian.sigs tree
popd
Codesigner only: Create Windows/macOS detached signatures:
Codesigner only: Sign the macOS binary:
transfer bitcoin-osx-unsigned.tar.gz to macOS for signing
tar xf bitcoin-osx-unsigned.tar.gz
./detached-sig-create.sh -s "Key ID"
Enter the keychain password and authorize the signature
Move signature-osx.tar.gz back to the gitian host
Codesigner only: Sign the windows binaries:
tar xf bitcoin-win-unsigned.tar.gz
./detached-sig-create.sh -key /path/to/codesign.key
Enter the passphrase for the key when prompted
signature-win.tar.gz will be created
Codesigner only: Commit the detached codesign payloads:
cd ~/bitcoin-detached-sigs
checkout the appropriate branch for this release series
rm -rf *
tar xf signature-osx.tar.gz
tar xf signature-win.tar.gz
git add -A
git commit -m "point to ${VERSION}"
git tag -s v${VERSION} HEAD
git push the current branch and new tag
Non-codesigners: wait for Windows/macOS detached signatures:
Create (and optionally verify) the signed macOS binary:
pushd ./gitian-builder
./bin/gbuild -i --commit signature=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-osx-signed --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-signed ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml
mv build/out/bitcoin-osx-signed.dmg ../bitcoin-${VERSION}-osx.dmg
popd
Create (and optionally verify) the signed Windows binaries:
pushd ./gitian-builder
./bin/gbuild -i --commit signature=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-win-signed --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-signed ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml
mv build/out/bitcoin-*win64-setup.exe ../bitcoin-${VERSION}-win64-setup.exe
popd
Commit your signature for the signed macOS/Windows binaries:
pushd gitian.sigs
git add ${VERSION}-osx-signed/"${SIGNER}"
git add ${VERSION}-win-signed/"${SIGNER}"
git commit -m "Add ${SIGNER} ${VERSION} signed binaries signatures"
git push # Assuming you can push to the gitian.sigs tree
popd
SHA256SUMS.asc for the builds, and GPG-sign it:The list of files should be:
bitcoin-${VERSION}-aarch64-linux-gnu.tar.gz
bitcoin-${VERSION}-arm-linux-gnueabihf.tar.gz
bitcoin-${VERSION}-riscv64-linux-gnu.tar.gz
bitcoin-${VERSION}-x86_64-linux-gnu.tar.gz
bitcoin-${VERSION}-osx64.tar.gz
bitcoin-${VERSION}-osx.dmg
bitcoin-${VERSION}.tar.gz
bitcoin-${VERSION}-win64-setup.exe
bitcoin-${VERSION}-win64.zip
The *-debug* files generated by the gitian build contain debug symbols for troubleshooting by developers. It is assumed that anyone that is interested in debugging can run gitian to generate the files for themselves. To avoid end-user confusion about which file to pick, as well as save storage space do not upload these to the bitcoin.org server, nor put them in the torrent.
gpg --digest-algo sha256 --clearsign SHA256SUMS # outputs SHA256SUMS.asc
rm SHA256SUMS
(the digest algorithm is forced to sha256 to avoid confusion of the Hash: header that GPG adds with the SHA256 used for the files) Note: check that SHA256SUMS itself doesn't end up in SHA256SUMS, which is a spurious/nonsensical entry.
Upload zips and installers, as well as SHA256SUMS.asc from last step, to the bitcoin.org server into /var/www/bin/bitcoin-core-${VERSION}
A .torrent will appear in the directory after a few minutes. Optionally help seed this torrent. To get the magnet: URI use:
Insert the magnet URI into the announcement sent to mailing lists. This permits people without access to bitcoin.org to download the binary distribution. Also put it into the optional_magnetlink: slot in the YAML file for bitcoin.org (see below for bitcoin.org update instructions).
Update bitcoin.org version
First, check to see if the Bitcoin.org maintainers have prepared a release: https://github.com/bitcoin-dot-org/bitcoin.org/labels/Core
If they have not prepared a release, follow the Bitcoin.org release instructions: https://github.com/bitcoin-dot-org/bitcoin.org/blob/master/docs/adding-events-release-notes-and-alerts.html#release-notes
After the pull request is merged, the website will automatically show the newest version within 15 minutes, as well as update the OS download links.
Update other repositories and websites for new version
bitcoincore.org blog post
bitcoincore.org maintained versions update: table
bitcoincore.org RPC documentation update
Update packaging repo
Push the flatpak to flathub, e.g. https://github.com/flathub/org.bitcoincore.bitcoin-qt/pull/2
Push the latest version to master (if applicable), e.g. https://github.com/bitcoin-core/packaging/pull/32
Create a new branch for the major release "0.xx" from master (used to build the snap package) and request the track (if applicable), e.g. https://forum.snapcraft.io/t/track-request-for-bitcoin-core-snap/10112/7
Notify MarcoFalke so that he can start building the snap package
This repo
Archive the release notes for the new version to doc/release-notes/ (branch master and branch of the release)
Create a new GitHub release with a link to the archived release notes
Announce the release:
bitcoin-dev and bitcoin-core-dev mailing list
Bitcoin Core announcements list https://bitcoincore.org/en/list/announcements/join/
Update title of #bitcoin on Freenode IRC
Optionally twitter, reddit /r/Bitcoin, ... but this will usually sort out itself
Celebrate
m_assumed_blockchain_size and m_assumed_chain_state_sizeBoth variables are used as a guideline for how much space the user needs on their drive in total, not just strictly for the blockchain. Note that all values should be taken from a fully synced node and have an overhead of 5-10% added on top of its base value.
To calculate m_assumed_blockchain_size:
mainnet -> Take the size of the data directory, excluding /regtest and /testnet3 directories.testnet -> Take the size of the /testnet3 directory.To calculate m_assumed_chain_state_size:
mainnet -> Take the size of the /chainstate directory.testnet -> Take the size of the /testnet3/chainstate directory.Notes:
m_assumed_blockchain_size, there's no need to exclude the /chainstate directory since it's a guideline value and an overhead will be added anyway.The purpose of this library is to make the verification functionality that is critical to Bitcoin's consensus available to other applications, e.g. to language bindings.
The interface is defined in the C header bitcoinconsensus.h located in src/script/bitcoinconsensus.h.
bitcoinconsensus_version returns an unsigned int with the API version (currently 1).
bitcoinconsensus_verify_script returns an int with the status of the verification. It will be 1 if the input script correctly spends the previous output scriptPubKey.
const unsigned char *scriptPubKey - The previous output script that encumbers spending.unsigned int scriptPubKeyLen - The number of bytes for the scriptPubKey.const unsigned char *txTo - The transaction with the input that is spending the previous output.unsigned int txToLen - The number of bytes for the txTo.unsigned int nIn - The index of the input in txTo that spends the scriptPubKey.unsigned int flags - The script validation flags (see below).bitcoinconsensus_error* err - Will have the error/success code for the operation (see below).bitcoinconsensus_SCRIPT_FLAGS_VERIFY_NONEbitcoinconsensus_SCRIPT_FLAGS_VERIFY_P2SH - Evaluate P2SH (BIP16) subscriptsbitcoinconsensus_SCRIPT_FLAGS_VERIFY_DERSIG - Enforce strict DER (BIP66) compliancebitcoinconsensus_SCRIPT_FLAGS_VERIFY_NULLDUMMY - Enforce NULLDUMMY (BIP147)bitcoinconsensus_SCRIPT_FLAGS_VERIFY_CHECKLOCKTIMEVERIFY - Enable CHECKLOCKTIMEVERIFY (BIP65)bitcoinconsensus_SCRIPT_FLAGS_VERIFY_CHECKSEQUENCEVERIFY - Enable CHECKSEQUENCEVERIFY (BIP112)bitcoinconsensus_SCRIPT_FLAGS_VERIFY_WITNESS - Enable WITNESS (BIP141)bitcoinconsensus_ERR_OK - No errors with input parameters (see the return value of bitcoinconsensus_verify_script for the verification status)bitcoinconsensus_ERR_TX_INDEX - An invalid index for txTobitcoinconsensus_ERR_TX_SIZE_MISMATCH - txToLen did not match with the size of txTobitcoinconsensus_ERR_DESERIALIZE - An error deserializing txTobitcoinconsensus_ERR_AMOUNT_REQUIRED - Input amount is required if WITNESS is usedbitcoinconsensus_ERR_INVALID_FLAGS - Script verification flags are invalid (i.e. not part of the libconsensus interface)It is possible to run Bitcoin Core as a Tor onion service, and connect to such services.
The following directions assume you have a Tor proxy running on port 9050. Many distributions default to having a SOCKS proxy listening on port 9050, but others may not. In particular, the Tor Browser Bundle defaults to listening on port 9150. See Tor Project FAQ:TBBSocksPort for how to properly configure Tor.
There are several ways to see your local onion address in Bitcoin Core:
getnetworkinfo in the "localaddresses" section-netinfo peer connections dashboardYou may set the -debug=tor config logging option to have additional information in the debug log about your Tor configuration.
The first step is running Bitcoin Core behind a Tor proxy. This will already anonymize all outgoing connections, but more is possible.
-proxy=ip:port Set the proxy server. If SOCKS5 is selected (default), this proxy
server will be used to try to reach .onion addresses as well.
-onion=ip:port Set the proxy server to use for Tor onion services. You do not
need to set this if it's the same as -proxy. You can use -noonion
to explicitly disable access to onion services.
-listen When using -proxy, listening is disabled by default. If you want
to run an onion service (see next section), you'll need to enable
it explicitly.
-connect=X When behind a Tor proxy, you can specify .onion addresses instead
-addnode=X of IP addresses or hostnames in these parameters. It requires
-seednode=X SOCKS5. In Tor mode, such addresses can also be exchanged with
other P2P nodes.
-onlynet=onion Make outgoing connections only to .onion addresses. Incoming
connections are not affected by this option. This option can be
specified multiple times to allow multiple network types, e.g.
ipv4, ipv6, or onion.
In a typical situation, this suffices to run behind a Tor proxy:
./bitcoind -proxy=127.0.0.1:9050
If you configure your Tor system accordingly, it is possible to make your node also reachable from the Tor network. Add these lines to your /etc/tor/torrc (or equivalent config file): Needed for Tor version 0.2.7.0 and older versions of Tor only. For newer versions of Tor see Section 3.
HiddenServiceDir /var/lib/tor/bitcoin-service/
HiddenServicePort 8333 127.0.0.1:8334
The directory can be different of course, but virtual port numbers should be equal to your bitcoind's P2P listen port (8333 by default), and target addresses and ports should be equal to binding address and port for inbound Tor connections (127.0.0.1:8334 by default).
-externalip=X You can tell bitcoin about its publicly reachable addresses using
this option, and this can be an onion address. Given the above
configuration, you can find your onion address in
/var/lib/tor/bitcoin-service/hostname. For connections
coming from unroutable addresses (such as 127.0.0.1, where the
Tor proxy typically runs), onion addresses are given
preference for your node to advertise itself with.
You can set multiple local addresses with -externalip. The
one that will be rumoured to a particular peer is the most
compatible one and also using heuristics, e.g. the address
with the most incoming connections, etc.
-listen You'll need to enable listening for incoming connections, as this
is off by default behind a proxy.
-discover When -externalip is specified, no attempt is made to discover local
IPv4 or IPv6 addresses. If you want to run a dual stack, reachable
from both Tor and IPv4 (or IPv6), you'll need to either pass your
other addresses using -externalip, or explicitly enable -discover.
Note that both addresses of a dual-stack system may be easily
linkable using traffic analysis.
In a typical situation, where you're only reachable via Tor, this should suffice:
./bitcoind -proxy=127.0.0.1:9050 -externalip=7zvj7a2.htmlgkdbg4f2dryd5rgtrn7upivr5eeij4cicjh65pooxeshid.onion -listen
(obviously, replace the .onion address with your own). It should be noted that you still listen on all devices and another node could establish a clearnet connection, when knowing your address. To mitigate this, additionally bind the address of your Tor proxy:
./bitcoind ... -bind=127.0.0.1
If you don't care too much about hiding your node, and want to be reachable on IPv4 as well, use discover instead:
./bitcoind ... -discover
and open port 8333 on your firewall (or use -upnp).
If you only want to use Tor to reach .onion addresses, but not use it as a proxy for normal IPv4/IPv6 communication, use:
./bitcoind -onion=127.0.0.1:9050 -externalip=7zvj7a2.htmlgkdbg4f2dryd5rgtrn7upivr5eeij4cicjh65pooxeshid.onion -discover
Starting with Tor version 0.2.7.1 it is possible, through Tor's control socket API, to create and destroy 'ephemeral' onion services programmatically. Bitcoin Core has been updated to make use of this.
This means that if Tor is running (and proper authentication has been configured), Bitcoin Core automatically creates an onion service to listen on. This will positively affect the number of available .onion nodes.
This new feature is enabled by default if Bitcoin Core is listening (-listen), and requires a Tor connection to work. It can be explicitly disabled with -listenonion=0 and, if not disabled, configured using the -torcontrol and -torpassword settings. To show verbose debugging information, pass -debug=tor.
Connecting to Tor's control socket API requires one of two authentication methods to be configured. It also requires the control socket to be enabled, e.g. put ControlPort 9051 in torrc config file. For cookie authentication the user running bitcoind must have read access to the CookieAuthFile specified in Tor configuration. In some cases this is preconfigured and the creation of an onion service is automatic. If permission problems are seen with -debug=tor they can be resolved by adding both the user running Tor and the user running bitcoind to the same group and setting permissions appropriately. On Debian-based systems the user running bitcoind can be added to the debian-tor group, which has the appropriate permissions. Before starting bitcoind you will need to re-login to allow debian-tor group to be applied. Otherwise you will see the following notice: "tor: Authentication cookie /run/tor/control.authcookie could not be opened (check permissions)" on debug.log.
An alternative authentication method is the use of the -torpassword=password option. The password is the clear text form that was used when generating the hashed password for the HashedControlPassword option in the tor configuration file. The hashed password can be obtained with the command tor --hash-password password (read the tor manual for more details).
The Bitcoin-Core project has been designed to support multiple localisations. This makes adding new phrases, and completely new languages easily achievable. For managing all application translations, Bitcoin-Core makes use of the Transifex online translation management tool.
Transifex is setup to monitor the GitHub repo for updates, and when code containing new translations is found, Transifex will process any changes. It may take several hours after a pull-request has been merged, to appear in the Transifex web interface.
Multiple language support is critical in assisting Bitcoin’s global adoption, and growth. One of Bitcoin’s greatest strengths is cross-border money transfers, any help making that easier is greatly appreciated.
See the Transifex Bitcoin project to assist in translations. You should also join the translation mailing list for announcements - see details below.
We use automated scripts to help extract translations in both Qt, and non-Qt source files. It is rarely necessary to manually edit the files in src/qt/locale/. The translation source files must adhere to the following format: bitcoin_xx_YY.ts or bitcoin_xx.ts
src/qt/locale/bitcoin_en.ts is treated in a special way. It is used as the source for all other translations. Whenever a string in the source code is changed, this file must be updated to reflect those changes. A custom script is used to extract strings from the non-Qt parts. This script makes use of gettext, so make sure that utility is installed (ie, apt-get install gettext on Ubuntu/Debian). Once this has been updated, lupdate (included in the Qt SDK) is used to update bitcoin_en.ts.
To automatically regenerate the bitcoin_en.ts file, run the following commands:
contrib/bitcoin-qt.pro takes care of generating .qm (binary compiled) files from .ts (source files) files. It’s mostly automated, and you shouldn’t need to worry about it.
Example Qt translation
For general PRs, you shouldn’t include any updates to the translation source files. They will be updated periodically, primarily around pre-releases, allowing time for any new phrases to be translated before public releases. This is also important in avoiding translation related merge conflicts.
When an updated source file is merged into the GitHub repo, Transifex will automatically detect it (although it can take several hours). Once processed, the new strings will show up as "Remaining" in the Transifex web interface and are ready for translators.
To create the pull-request, use the following commands:
git add src/qt/bitcoinstrings.cpp src/qt/locale/bitcoin_en.ts
git commit
Visit the Transifex Signup page to create an account. Take note of your username and password, as they will be required to configure the command-line tool.
You can find the Bitcoin translation project at https://www.transifex.com/bitcoin/bitcoin/.
The client is used to fetch updated translations. If you are having problems, or need more details, see https://docs.transifex.com/client/installing-the-client
pip install transifex-client
Setup your Transifex client config as follows. Please ignore the token field.
nano ~/.transifexrc
[https://www.transifex.com]
hostname = https://www.transifex.com
password = PASSWORD
token =
username = USERNAMEThe Transifex Bitcoin project config file is included as part of the repo. It can be found at .tx/config, however you shouldn’t need to change anything.
To assist in updating translations, a helper script is available in the maintainer-tools repo.
python3 ../bitcoin-maintainer-tools/update-translations.pygit add new translations from src/qt/locale/src/qt/bitcoin_locale.qrc manually or viasrc/Makefile.qt_locale.include manually or viaDo not directly download translations one by one from the Transifex website, as we do a few post-processing steps before committing the translations.
When new plurals are added to the source file, it's important to do the following steps:
bitcoin_en.ts in Qt Linguist (included in the Qt SDK)%n, which will take you to the parts in the translation that use pluralsEnglish Translation (Singular) and English Translation (Plural) fieldsTo create a new language template, you will need to edit the languages manifest file src/qt/bitcoin_locale.qrc and add a new entry. Below is an example of the English language entry.
Note: that the language translation file must end in .qm (the compiled extension), and not .ts.
The Bitcoin-Core translation maintainers include tcatm, seone, Diapolo, wumpus and luke-jr. You can find them, and others, in the Freenode IRC chatroom - irc.freenode.net #bitcoin-core-dev.
This document provides guidelines for internationalization of the Bitcoin Core software.
To mark a message as translatable
In GUI source code (under src/qt): use tr("...")
In non-GUI source code (under src): use _("...")
No internationalization is used for e.g. developer scripts outside src.
On a high level, these strings are to be translated:
Do not translate technical or extremely rare errors. Anything else that appears to the user in the GUI is to be translated. This includes labels, menu items, button texts, tooltips and window titles. This includes messages passed to the GUI through the UI interface through InitMessage, ThreadSafeMessageBox or ShowProgress.
Try not to burden translators with translating messages that are e.g. slight variations of other messages. In the GUI, avoid the use of text where an icon or symbol will do. Make sure that placeholder texts in forms do not end up in the list of strings to be translated (use <string notr="true">).
Try to write translation strings in an understandable way, for both the user and the translator. Avoid overly technical or detailed messages.
Do not translate internal errors, log messages, or messages that appear on the RPC interface. If an error is to be shown to the user, use a translatable generic message, then log the detailed message to the log. E.g., "A fatal internal error occurred, see debug.log for details". This helps troubleshooting; if the error is the same for everyone, the likelihood is increased that it can be found using a search engine.
Avoid dividing up a message into fragments. Translators see every string separately, so they may misunderstand the context if the messages are not self-contained.
There have been difficulties with the use of HTML in translation strings; translators should not be able to accidentally affect the formatting of messages. This may sometimes be at conflict with the recommendation in the previous section.
Plurals can be complex in some languages. A quote from the gettext documentation:
In Polish we use e.g. plik (file) this way:
1 plik,
2,3,4 pliki,
5-21 pliko'w,
22-24 pliki,
25-31 pliko'w
and so on
In Qt code, use tr's third argument for optional plurality. For example:
tr("%n hour(s)","",secs/HOUR_IN_SECONDS);
tr("%n day(s)","",secs/DAY_IN_SECONDS);
tr("%n week(s)","",secs/WEEK_IN_SECONDS);
This adds <numerusform>s to the respective .ts file, which can be translated separately depending on the language. In English, this is simply:
<message numerus="yes">
<source>%n active connection(s) to Bitcoin network</source>
<translation>
<numerusform>%n active connection to Bitcoin network</numerusform>
<numerusform>%n active connections to Bitcoin network</numerusform>
</translation>
</message>
Where possible, try to avoid embedding numbers into the flow of the string at all. E.g.,
WARNING: check your network connection, %d blocks received in the last %d hours (%d expected)
versus
WARNING: check your network connection, less blocks (%d) were received in the last %n hours than expected (%d).
The second example reduces the number of pluralized words that translators have to handle from three to one, at no cost to comprehensibility of the sentence.
During a string freeze (often before a major release), no translation strings are to be added, modified or removed.
This can be checked by executing make translate in the src directory, then verifying that bitcoin_en.ts remains unchanged.
ZeroMQ is a lightweight wrapper around TCP connections, inter-process communication, and shared-memory, providing various message-oriented semantics such as publish/subscribe, request/reply, and push/pull.
The Bitcoin Core daemon can be configured to act as a trusted "border router", implementing the bitcoin wire protocol and relay, making consensus decisions, maintaining the local blockchain database, broadcasting locally generated transactions into the network, and providing a queryable RPC interface to interact on a polled basis for requesting blockchain related data. However, there exists only a limited service to notify external software of events like the arrival of new blocks or transactions.
The ZeroMQ facility implements a notification interface through a set of specific notifiers. Currently there are notifiers that publish blocks and transactions. This read-only facility requires only the connection of a corresponding ZeroMQ subscriber port in receiving software; it is not authenticated nor is there any two-way protocol involvement. Therefore, subscribers should validate the received data since it may be out of date, incomplete or even invalid.
ZeroMQ sockets are self-connecting and self-healing; that is, connections made between two endpoints will be automatically restored after an outage, and either end may be freely started or stopped in any order.
Because ZeroMQ is message oriented, subscribers receive transactions and blocks all-at-once and do not need to implement any sort of buffering or reassembly.
The ZeroMQ feature in Bitcoin Core requires the ZeroMQ API >= 4.0.0 libzmq. For version information, see dependencies.html. Typically, it is packaged by distributions as something like libzmq3-dev. The C++ wrapper for ZeroMQ is not needed.
In order to run the example Python client scripts in the contrib/zmq/ directory, one must also install PyZMQ (generally with pip install pyzmq), though this is not necessary for daemon operation.
By default, the ZeroMQ feature is automatically compiled in if the necessary prerequisites are found. To disable, use --disable-zmq during the configure step of building bitcoind:
$ ./configure --disable-zmq (other options)
To actually enable operation, one must set the appropriate options on the command line or in the configuration file.
Currently, the following notifications are supported:
-zmqpubhashtx=address
-zmqpubhashblock=address
-zmqpubrawblock=address
-zmqpubrawtx=address
-zmqpubsequence=address
The socket type is PUB and the address must be a valid ZeroMQ socket address. The same address can be used in more than one notification. The same notification can be specified more than once.
The option to set the PUB socket's outbound message high water mark (SNDHWM) may be set individually for each notification:
-zmqpubhashtxhwm=n
-zmqpubhashblockhwm=n
-zmqpubrawblockhwm=n
-zmqpubrawtxhwm=n
-zmqpubsequencehwm=address
The high water mark value must be an integer greater than or equal to 0.
For instance:
$ bitcoind -zmqpubhashtx=tcp://127.0.0.1:28332 \
-zmqpubhashtx=tcp://192.168.1.2:28332 \
-zmqpubrawtx=ipc:///tmp/bitcoind.tx.raw \
-zmqpubhashtxhwm=10000
Each PUB notification has a topic and body, where the header corresponds to the notification type. For instance, for the notification -zmqpubhashtx the topic is hashtx (no null terminator) and the body is the transaction hash (32 bytes) for all but sequence topic. For sequence, the body is structured as the following based on the type of message:
<32-byte hash>C : Blockhash connected
<32-byte hash>D : Blockhash disconnected
<32-byte hash>R<8-byte LE uint> : Transactionhash removed from mempool for non-block inclusion reason
<32-byte hash>A<8-byte LE uint> : Transactionhash added mempool
Where the 8-byte uints correspond to the mempool sequence number.
These options can also be provided in bitcoin.conf.
ZeroMQ endpoint specifiers for TCP (and others) are documented in the ZeroMQ API.
Client side, then, the ZeroMQ subscriber socket must have the ZMQ_SUBSCRIBE option set to one or either of these prefixes (for instance, just hash); without doing so will result in no messages arriving. Please see contrib/zmq/zmq_sub.py for a working example.
The ZMQ_PUB socket's ZMQ_TCP_KEEPALIVE option is enabled. This means that the underlying SO_KEEPALIVE option is enabled when using a TCP transport. The effective TCP keepalive values are managed through the underlying operating system configuration and must be configured prior to connection establishment.
For example, when running on GNU/Linux, one might use the following to lower the keepalive setting to 10 minutes:
sudo sysctl -w net.ipv4.tcp_keepalive_time=600
Setting the keepalive values appropriately for your operating environment may improve connectivity in situations where long-lived connections are silently dropped by network middle boxes.
From the perspective of bitcoind, the ZeroMQ socket is write-only; PUB sockets don't even have a read function. Thus, there is no state introduced into bitcoind directly. Furthermore, no information is broadcast that wasn't already received from the public P2P network.
No authentication or authorization is done on connecting clients; it is assumed that the ZeroMQ port is exposed only to trusted entities, using other means such as firewalling.
Note that for *block topics, when the block chain tip changes, a reorganisation may occur and just the tip will be notified. It is up to the subscriber to retrieve the chain from the last known block to the new tip. Also note that no notification will occur if the tip was in the active chain--as would be the case after calling invalidateblock RPC. In contrast, the sequence topic publishes all block connections and disconnections.
There are several possibilities that ZMQ notification can get lost during transmission depending on the communication type you are using. Bitcoind appends an up-counting sequence number to each notification which allows listeners to detect lost notifications.
The sequence topic refers specifically to the mempool sequence number, which is also published along with all mempool events. This is a different sequence value than in ZMQ itself in order to allow a total ordering of mempool events to be constructed.