Electrum server performance report (2022)
In order for a bitcoin wallet to perform basic functions such as receiving and sending transactions, the wallet needs to communicate with the bitcoin network. This communication requires connecting to one or more nodes on the peer-to-peer network. There are a variety of methods to choose from such as Simplified Payment Verification (SPV), Bitcore, Electrum, and so on. I discussed the options in detail in this article:
However, since writing that article SPV has been deprecated for several reasons such as its poor privacy properties and DoS vulnerabilities. As such, the next most mature protocol still standing is Electrum. Casa uses Electrum servers within our own infrastructure; to ensure we employ the best possible solution we regularly test different server implementations and compare them in terms of performance. The following report will walk you through the entire process we undertook.
The contenders
ElectrumX - https://github.com/spesmilo/electrumx
The second iteration of Electrum server implementations, this project was started in late 2016. It was the only enterprise-grade option available for the next 2 years.
Esplora Electrs - https://github.com/Blockstream/electrs
Blockstream was looking to run an Electrum server to power their public Esplora instance at blockstream.info, but Electrs was not indexing data well enough to serve enterprise level queries. Blockstream launched this fork in late 2018 that builds extended indexes and database storage for improved performance under high load.
Fulcrum - https://github.com/cculianu/Fulcrum
Fulcrum is the new kid on the block — it was first released in 2020. Written in modern C++17 using multi-threaded and asynchronous programming techniques, it has been engineered with performance in mind.
Untested implementations
There are other Electrum servers such as Electrum Personal Server and electrs. These are designed to be personal use Electrum servers rather than enterprise / publicly available servers. They tend to be an order of magnitude slower and as such, their performance results are not included.
Test environment
To ensure consistency of results, we ran all of our tests on the same AWS instance image with the following properties:
AWS EC2 instance type: c5.xlarge (4 vCPU, 8GB RAM)
AWS EBS volumes:
- 8 GB gp2 (100 IOPS)
/(Operating System) - 600 GB gp2 (1800 IOPS)
/mnt/bitcoin(used for Bitcoin Core data) - 1000 GB gp2 (3000 IOPS)
/mnt/data(used for Electrum server data)
The Electrum server and Bitcoin Core were fully synced at the start of each test. During each set of tests, the only processes running in addition to normal OS processes were:
- bitcoind
- electrum daemon
- nodejs test script
It's worth noting that the entire test environment was running on a single machine, thus no data was sent over a network when making the queries we were timing. In real-world configurations your perceived performance may be lower due to the effects of network latency.
Test methodology
We're interested in understanding the performance characteristics of each implementation with regard to common queries that would be made for a wallet. Thus, for a wide variety of bitcoin addresses (script hashes) with different characteristics, we chose to benchmark the following queries:
- Get address balance
- Get entire list of transactions
- Get entire list of UTXOs
We noticed that for each implementation the first test run would be substantially slower than subsequent test runs, likely due to caching at various layers of the application stack. Thus, in order to improve the precision of our results we decided to throw out the data from run 1, then we ran 9 more passes and took the median timing result for each address.
Each test run was performed sequentially; we did not test support for handling multithreaded request loads. We expect that the results of the single-threaded tests make it quite clear which implementations would be best suited for multithreaded applications.
Test input data
All of the bitcoin addresses that received funds between block heights 599900 and 600100 were dumped to a list. They were then filtered to obtain the set of addresses that had their balances affected by between 10 and 999 transactions — initial experiments showed this range to be most interesting. This provided us with a list of 103,000 addresses that was used as the main data set for our tests.
Implementation notes
ElectrumX
- Data directory size: 70 GB
- Performance was I/O bound, constrained by data volume read speed
- Initial index build took several days
- Requires txindex enabled in Bitcoin Core
Config adjustments:
- COST_SOFT_LIMIT = 0
- COST_HARD_LIMIT = 0
- CACHE_MB = 2048
Esplora
- Data directory size: 820 GB
- I/O bound, constrained by data volume read speed
- Initial index build took several days
- Doesn't require txindex enabled in Bitcoin Core
Config adjustments:
- --electrum-txs-limit 1000
Fulcrum
- Data directory size: 110 GB
- I/O bound, constrained by data volume read speed
- Initial index build took 2 days
- Does require txindex enabled in Bitcoin Core
Config adjustments:
- txhash_cache=2000
Performance test results
Our first set of results is pretty straightforward: how long does it take to get the balance of a given bitcoin address?
We can see that ElectrumX is scaling very well within a predictable best and worst case range while Fulcrum's performance is a bit more erratic, while remaining quite fast. But what's going on with Esplora?
The answer lies in the schema.rs file:
Basically, Esplora will not bother to cache data unless an address has a certain number of transactions affecting its balance. This means it just queries bitcoind on the fly to get data for addresses with low activity. Thus we end up with an interesting trade-off in performance characteristics! ElectrumX performs better for addresses with < 100 transactions while Esplora performs better for addresses with > 100 transactions. And while ElectrumX's worst-case performance appears to slow down at a linear rate, Esplora appears to have a more performant constant-time caching system.
These are certainly considerations worth taking into account based upon the common characteristics of addresses in your system. It's also worth noting that you could always change the MIN_HISTORY_ITEMS_TO_CACHE setting for your Esplora instance.
Analysis of the performance of balance lookups against total transaction history showed explainable trend characteristics. But what about comparing balance lookups against total current UTXOs?
If you're familiar with how bitcoin transaction data is structured, then you're aware that there is no "address balance" data — rather, an address balance is calculated by summing up the values of all the UTXOs that are encumbered by the locking script that is represented by that address. In terms of query performance, we should expect to see a well-defined relationship between current number of UTXOs and time to calculate their balance, while the total number of transactions affecting an address should be more weakly correlated to balance query performance.
We can once again see the effects of MIN_HISTORY_ITEMS_TO_CACHE on Esplora. ElectrumX again vastly outperforms both Esplora and Fulcrum for addresses with fewer than 100 UTXOs while the opposite is true for addresses with greater than 100 UTXOs. Esplora scales amazingly well for addresses with high UTXO counts, but its competitors aren't doing poorly either.
Now we move on to simpler queries — how long does it take us to retrieve the entire transaction history for a given address? Naturally, this will result in more data being returned, so we expect to see a linear relationship between the amount of data and amount of time to return it.
The most interesting point to note here is that ElectrumX appears to scale significantly better than both Esplora and Fulcrum when it comes to returning transaction history. ElectrumX never takes more than 20 milliseconds to return the transaction history, while Esplora can take several hundred milliseconds and Fulcrum sometimes exceeds a full second. Perhaps Esplora is not using the same constant-time caching scheme for transaction history that it's using for UTXOs and balances?
Similarly, how long does it take us to retrieve the entire set of UTXOs for a given address?
Here we can see that Fulcrum seems to scale linearly while ElectrumX and Esplora have far better performance.
If we zoom in significantly to the previous data, we can see Esplora really shine. Near constant time scaling of UTXO lookups make it blazing fast, in some cases returning the entire result set in under a millisecond! ElectrumX returns UTXO history at a rate that scales more on the order of log(n), but its performance is no joke — 20 milliseconds for 900 UTXOs is impressive.
Conclusions
We can see that different Electrum servers have different strengths and weaknesses, likely due to how they are storing and retrieving data.
| Function | Best Performance |
|---|---|
| Get Balance | Esplora |
| List Transactions | ElectrumX |
| List UTXOs | Esplora |
While Fulcrum does not take the lead in any of the given performance metrics, it's not bad — certainly in the competitive ballpark. If Fulcrum improves their UTXO List performance, then it will be on par with both of the other leading implementations. It's also worth nothing that Fulcrum may, in fact, outperform its competitors when running on less powerful hardware. You can check out these performance tests that were run on a Raspberry Pi to compare and contrast.
In general, it appears that ElectrumX provides the best "bang for your bitcoin" with regard to performance vs. resource requirements. But if you are willing to devote 10 times as much disk space in order to achieve maximum performance, Esplora is the way to go.