A performance benchmark on mempool services

A performance benchmark on mempool services
Traders analyze the Ethereum mempool to dramatically increase the odds of capturing profit

This comparison was done in December 2020. The blockchain space moves very quickly, mileage may vary.

What is the mempool

The mempool is a group Ethereum transactions that have been broadcasted to the network and are waiting to be mined into a block. A mempool service provides direct insight into a transaction’s life cycle. A transaction waits in mempool after being broadcasted by a user to a node. From there, the transaction will either be mined into a block, replaced by another transaction, or dropped by the network. A good mempool service will monitor this process for you for every single transaction in existence. You can learn more about the mempool here.

How is the mempool used for DeFi trading

Traders analyze the Ethereum mempool to dramatically increase the odds of capturing MEV (miner extractable value). There are a variety of different types of trading strategies where mempool can be applied. Some of these strategies involve finding trading opportunities that otherwise do not exist. As such, the DeFi trading ecosystem has changed dramatically since these mempool services have come onto the market.

  • Front running — determine what gas price is required to win a gas auction.
  • Tailgating (or back running) — predict trade opportunities that will mine into a future block based on the outcome of another user’s trade.
  • Sandwiching — predict trade opportunities that will mine into a future block by pushing another user’s trade opportunity to its max slippage limit, which will result in an arbitrage opportunity at a later time (typically within the same block).
  • Generic predator — hijacking a transaction call in hopes of front running an auction that leads to capturing MEV. This is typically done in a generic fashion by replacing the original transaction caller’s address with the predator’s address and submitting a new transaction at a higher gas price.

The Contestants

Though the companies & products we are discussing below offer a large variety of products and services, we are focusing this discussion on their Ethereum mempool feature.

bloXroute

bloXroute is a Blockchain Distribution Network (BDN) that utilizes a global network of servers optimized for network performance. They have committed a great deal of resources to building network infrastructure for both Ethereum DeFi traders and builders. You can read about these developments on their medium. Their mempool feature is called “Subscribe to NewTX Stream” and can be found on their product pricing page. There you will find a variety of other useful features as well. Many of which will help you build a more robust DeFi product and gain a trading edge in the markets.

bloXroute sends you mempool events via a websocket connection. They are incredibly fast and all encompassing, meaning you do the transaction filtering on your end. So if you’re watching for trades on Uniswap, you can simply filter based on the Uniswap address of interest.

Blocknative

Blocknative operates a global mempool data platform across both Bitcoin and Ethereum. They specialize in mempool services for both Ethereum DeFi traders and builders. You can read about their latest developments on their blog. Their mempool feature is called “Mempool explorer” and can be found on their products.

Blocknative sends you mempool events via webhooks specifically based on your watch list, which is hosted on Blocknative’s server. So if you’re watching for trades on Uniswap, you can simply add that Uniswap address to your watch list and then you’ll begin receiving webhook events.

Alchemy

Alchemy provides fundamental Bitcoin and Ethereum node infrastructure services. They provide an incredibly convenient API endpoint for both Ethereum DeFi traders and builders. Their mempool feature is called “alchemy_newFullPendingTransactions” and can be found in their documentation.

Alchemy sends you mempool events via a websocket connection. The events are all encompassing, meaning you do the transaction filtering on your end. So if you’re watching for trades on Uniswap, you can simply filter based on the Uniswap address of interest.

Performance Benchmark

We’ve used all three of these services extensively for DeFi trading and have seen great differences in performance. Outlined below, you’ll find performance test results and all data obtained. Performance was measured in terms of latency in milliseconds. We connected all three services to our keeper bot and monitored Uniswap’s router contract, and documented when each service reported incoming transactions. Once recorded, the information was plotted and made available for anyone to see.

The baseline for each transaction is assumed to be when the first mempool service discovers that transaction. So the first service to discover a transaction will get the latency of zero. And each service after that will have the latency equal to the time difference of when it discovered it vs the first service.

The data in this chart paints a very clear picture. bloXroute’s latency rides the zero line consistently. Blocknative’s latency is consistently slightly worse than bloXroute’s. Alchemy is all over the place and sometimes multiple seconds behind the other services.

bloXroute was first to detect the transaction 228 times out of 235 transactions.

Data from test results

Data formatted as JSON

Data formatted as CSV

Why having the fastest possible mempool service matters

The experiment results above indicate that a trader’s mempool service choice directly affects their profitability. Right now the Ethereum block time is averaging just over 13 seconds, according to ethstats.net. Being behind by 180ms, or worse yet 1,000+ ms, will greatly cut into your trading profits by reducing the likelihood that you get your transaction mined into the very next block. Being behind also reduces your reaction time during a gas auction.

Websockets vs Webhooks

It’s worth comparing the two methods of event delivery used by these mempool services. Websockets should technically be faster, however it could potentially be slightly less reliable. With a websocket connection, it’s possible that the client could miss an event during a disconnect. Whereas with a webhooks connection, each mempool event you receive has its own network connection that is created and torn down for each mempool event. This potentially adds a small amount of latency overhead, but it also adds a bit more reliability. This is worth mentioning because a trader may prefer the lower latency whereas a DeFi product builder may prefer the increased reliability.

Best performance & redundancy solution

If you’re a trader and want the best of both worlds in terms of performance and reliability, here are some redundancy solutions that may work for you.

Solution A) Use the fastest service, bloXroute, and connect to multiple gateways. You can either run your own BDN gateways on your own servers, or you can connect to multiple hosted gateways. You could also specify that these gateways live in different regions that way if there’s a network issue in one region, you don’t completely lose mempool access for some period of time.

Solution B) Use multiple mempool services. This way you’re getting data as fast as possible, and if one service goes down for any reason you have another one to lean on. The only downside here is that it multiplies your mempool service expense.

Pricing comparison

All price comparisons are based on 2020–12–4. Be aware that each service’s feature list and price structure is subject to change at any moment.

bloXroute’s pricing is generous considering what they offer compared to the other services out there. The professional tier for $240 is great value considering you’re getting the best performing mempool service with zero rate limit, along with tons of other features.

Blocknative’s pricing is hard to compare with bloXroute’s for several reasons. First, they only offer mempool services so you’re not getting anything extra as you are with bloXroute. And second, they impose an API event per day limit which at a glance may not seem like an issue, but in reality is actually a huge limitation. Let’s see how this plays out given the information in this benchmark test above.

Let’s say you’re a DeFi trader and you’re tailgating (aka back running). You will want to monitor all incoming mempool transactions to Uniswap. You’ll actually want to do this on a large variety of protocols, way more so than just Uniswap, but let’s keep this simple and just focus on one single Uniswap router used above. In the data from the benchmark test, let’s focus on the first and last data entry so we can get an approximate transaction per day to compare with Blocknative’s API event per day limit.

The test began with this first transaction and ended with this final transaction. That’s going from block 11381311 to 11381328 in a span of about 4 minutes. During this time we evaluated 235 transactions. That’s approximately 1 transaction per second. And since there’s 86,400 seconds per day, that means approximately 86,400 transactions per day. That’s about 8x the rate limit on Blocknative’s Production plan. Which pushes you to the Growth plan for $833 per month. This is problematic though because if Uniswap becomes more popular, you’ll get rate limited and be forced to jump to the Leader plan which is $8,300 per month. This gets pricey very quickly.

The alchemy_newFullPendingTransactions subscription type is super costly to maintain and requires a large number of compute units since it emits full transaction information instead of just transaction hashes. We do not recommend keeping this subscription open for long periods of time for non-enterprise tier users.

Using these services vs operating your own nodes

Generally speaking, running and maintaining your own node is not difficult, especially if you’re using it simply for making eth_calls. The mempool, however, is a different beast. In the early days of mempool trading, traders would get mempool information from their own nodes. That’s where it got competitive. Whoever had the better mempool infrastructure, had a slight advantage over others for each and every trading opportunity. And when you multiply that slight advantage over hundreds or thousands of daily trading opportunities, it results in a massive advantage.

Say you don’t want to use one of these services, and you want to operate this network yourself. You will need a network of nodes scattered around various regions in the world. Ideally, you’ll also want both GETH and Open Ethereum (Parity) nodes since they behave a bit differently. You may even want to configure various nodes differently to mirror that of how you predict the miners may be configuring their nodes. Most miners today run GETH, so you can probably get away with only running GETH. But when you have a big trading operation, you want to cover your bases as much as possible.

Cost is a big factor here as well. You’ll have to factor in the cost of running your own nodes on various servers throughout the world. And also the opportunity cost of your time to develop a system which is capable of routing all these transactions from your network of nodes to your trading bot.

Verdict

bloXroute is hands down the best Ethereum mempool service for automated DeFi traders with a large scale trading operation. Not only do they provide the lowest latencies, but they also have the friendliest rate limit (none), and a reasonable price. And though we believe in running your own nodes, it’s highly unlikely that most traders are capable of operating their own mempool infrastructure as cost effectively and as performant as bloXroute.