Introduction
Welcome, fellow travelers of the Cosmos! While we may not be traversing the stars on a spaceship, we are all interconnected through the powerful network of blockchains. Unfortunately, just like any technology, vulnerabilities can be discovered and exploited.
In this post, we’ll present a critical vulnerability in a Cosmos-SDK blockchain that is explicitly related to the Inter-Blockchain Communication Protocol (IBC). We’ll delve into the intricacies of this vulnerability, its impact on the blockchain ecosystem and how it can be mitigated. So buckle up! Let’s take a deep dive into the Cosmos of blockchain security.
TL;DR
This blog post will cover a critical vulnerability in the Comdex blockchain that allows an attacker to forge a Band Oracle IBC packet and potentially steal millions in assets. To do that, we’ll cover the basics of Cosmos-SDK chains and the IBC protocol fundamentals.
The Cosmos Ecosystem
Franklin D. Roosevelt once said, “There are as many opinions as there are experts.” There are also as many blockchains as builders – or, at least, there might be soon. We’ve come a long way from Satoshi Nakamoto’s original whitepaper. Today’s blockchain space has more money, use cases and users than ever. With this rise in popularity and functionality, we see more and more blockchains pop up every day, and finding a way to make connections between them is becoming increasingly important. But wait, don’t we have bridges for that?
A blockchain bridge is a piece of code connecting two blockchains and allowing them to transfer information (or assets) from one to the other. Bridges are notoriously bad at security. They’re known for being vulnerable to massive hacks. In 2022 alone, they were the root cause of most hack-related decentralized finance (DeFi) losses – over $1.8 billion worth of assets (see Figure 1 for all-time numbers). The overall loss in bridge hacks since 2016 adds up to over $2.83 billion. To suggest that the security of blockchain bridges requires improvement would be an understatement.
Figure 1: Funds lost in hacks throughout the years, DeFi Llama
Enter the Cosmos.
Cosmos was built by the Interchain Foundation, founded on its vision to create the Internet of Blockchains. They’re working on a handful of projects to facilitate that vision, including the Cosmos-SDK and IBC, which we’ll delve into in just a bit.
“On Ethereum, all applications run on a shared state machine. In Cosmos, many application-specific blockchains pass assets and other messages between one another. If Ethereum is a mainframe computer, Cosmos is a protocol for networking independent servers.”
— Charlie Noyes and Dan Robinson (Paradigm), A Cosmos Thesis
Cosmos has only been around since 2019 but already has an impressive number of blockchains (also known as app-chains/zones) built on it, locking in over 65 billion dollars in assets combined. From Chihuahua, the Cosmos meme coin, to the self-sovereign identity (SSI) cheqd and the popular Osmosis decentralized exchange (DEX) – Cosmos has it all. You can find all the chains and their connections in Map of zones – Cosmos network explorer (Figure 2).
Figure 2: The Cosmos – https://mapofzones.com/
To allow cross-chain communication between the current 58 chains in Cosmos, the Interchain team developed the innovative IBC protocol. IBC is a general-purpose communication protocol that can enable simple, secure cross-chain token transfers (and eliminate the need for bridges) and data transfers, allowing for many more complex applications to be built on top of it.
Cosmos-SDK
The Cosmos-SDK is a framework for building blockchain applications. Any Cosmos-SDK chain is built of two layers:
- A base layer that is standard across all chains. It is responsible for networking and the consensus algorithm
- An application layer built using the SDK
The base layer is implemented in CometBFT, a consensus implementation based on the Byzantine-Fault Tolerant protocol (forked from Tendermint) developed in Go. The application layer is comprised of SDK modules, which are available to anyone creating an SDK chain. These modules implement standard blockchain functionality such as staking, slashing and governance.
Other than basic blockchain functionalities, SDK modules can enable a wide set of capabilities, from NFT support to smart contracts execution (using CosmWasm / EVM) and even inter-blockchain communication (using IBC).
In addition to those SDK modules, custom modules can be implemented specifically for a particular app chain by its developers. Anyone wanting to create an app chain can pick and choose from the SDK modules and implement some specific modules of their own to build a blockchain tailored to their needs (see Figure 3 for illustration).
Figure 3: Application layer plus base layer https://v1.cosmos.network/intro
IBC
IBC is a general communication standard for blockchains. It means that pretty much any blockchain that uses a consensus mechanism with finality will be able to connect to the Cosmos, not only Cosmos-SDK-based chains!
IBC is to the internet of blockchains, what TCP/IP is to the internet.
In the same way that a TCP/IP packet is unaware of whether it carries HTTPS/FTP or any other protocol data, IBC is application-agnostic and you can transfer all types of data using it. IBC enables the transport layer between two chains in a way that maintains data authenticity and packet ordering. This is why the base layer of IBC is known as TAO – transport, authentication and ordering. On top of that, we can have different modules that use IBC as their communication layer; those are referred to as IBC Applications.
For an active connection between chain A and chain B, we must have the following:
- Chain A will run a light client of chain B and vice versa. A light client maintains only the necessary information to verify transactions on the blockchain. This is done so that each chain can validate commitments from its counterparty without having to trust any third party in the middle or store the entire blockchain.
- IBC software. This is comprised of three layers:
- IBC Core layer for transport, authentication and ordering of packets. This is consistent across all chains.
- Specific IBC applications that operate on top of IBC. This is similar to HTTP running on TCP/IP. An example is ICS-20, used for fungible token transfers across different chains.
- Application Modules. These are the Cosmos-SDK modules we mentioned in the previous section. They can talk to the IBC application and use its data.
- Relayers. While the light clients in the connection are the base of trust between the two chains, they can only authenticate packet commitment on the counterparty chain. This essentially means that chain A can commit to some packet data by writing the packet hash to chain A’s ledger so that chain B can ensure this commitment has passed the consensus phase on chain A. The relayer is the code responsible for transferring the packet data (not only the hash). The relayer shouldn’t be trusted by any chain. It’s simply there to transfer the information from one chain to another, and each chain is responsible for authenticating this data using the commitment (i.e., the packet hash) in the other chain. It’s an elegant solution so that we don’t have to write all of the data in IBC packets to the blockchain itself for the sake of scalability. Figure 4 illustrates how relayers integrate into IBC. You can find more information on relayers can be found in ICS-18.
Figure 4: IBC architecture https://tutorials.cosmos.network/academy/3-ibc/1-what-is-ibc.html
A connection between two chains is usually set up once and will probably not be terminated while the two chains are operating. The connection itself is the abstraction layer for the IBC protocol. On top of a connection, channels can be opened and a unique channel ID identifies a channel. On top of those channels, application modules open a port, which they use to communicate to an application in the counterparty chain. Every packet that goes through IBC has a source channel and port in the sending chain, as well as a destination channel and port in the receiving chain.
Channel management and packet handling are implemented using callback functions that the IBC core invokes. Different modules can register callback functions to implement various functionalities, and these functions are called one after the other.
This is all the essential information we need on IBC for this blog post. If you want to learn more about IBC, there’s great documentation from the Interchain team. I especially recommend the Cosmos Academy and this series of presentations.
Comdex
Comdex is a blockchain that serves a DeFi ecosystem powered by Cosmos-SDK and Interoperable CosmWasm Smart Contracts. It’s basically an IBC-enabled Cosmos-SDK chain for building DeFi applications. You can find the Automated Market Maker (AMM) cSwap, the Harbor Protocol, Commodo lending and more among the applications built on top of Comdex.
Our focus in this blog is interchain communication, so let’s go ahead and examine how Comdex incorporates IBC components.
One of the indications that a Cosmos-SDK chain implements an IBC interface is the use of the aforementioned IBC callback functions. The function OnRecvPacket() is called whenever an IBC packet is received. Looking for an implementation of this function can quickly give us a sense of what IBC interfaces a chain has.
Comdex implements one such function in its Band Oracle module (x/bandoracle/module_ibc.go@OnRecvPacket()). But wait a minute, what’s Band anyway? Band Oracle is a Cosmos cross-chain data oracle platform that aims to connect the Internet of Blockchains to real-world information. As far as we’re concerned, it allows the Comdex chain to get various price oracle information using IBC, and we’re interested in seeing whether this connection is secure.
Now let’s understand how Comdex uses Band by having a look at the documentation:
“Bandoracle module fetches the prices of assets. A band packet is created containing the list of assets symbols for which price is to be fetched. Then the packet is relayed to the band chain (through a relayer), where the request is acknowledged, and the price is validated. Following this, the price is being sent to our chain through the packets using the same channel. After receiving the packets, the prices are mapped to the corresponding assets. The packets are relayed after every 20 blocks; hence prices are updated after every 20 blocks.”
Quite straightforward. The following illustration (Figure 5) shows the Band Oracle price fetching flow.
Figure 5: Comdex->Band price Oracle flow
The packets in this flow have specific structure; they’re defined as OracleRequestPacketData, OracleRequestPacketAcknowledgement and OracleResponsePacketData in the Band documentation. We’ll have a look at how these steps are implemented in the code and try to identify if there are any security issues.
1. The first step is to request price oracle information from Band. This request is initiated every 20 Comdex blocks. In the Cosmos-SDK, a callback function that runs at the beginning of every block is called BeginBlocker(), and we can find one in the Band module in x/bandoracle/abci.go:12. To make sure we request for oracle information every 20th block only, we have line 18:
if ctx.BlockHeight()%types.Int64Twenty == types.Int64Zero {
The actual sending of the packet is in line 21:
_, err := k.FetchPrice(ctx, msg)
In the FetchPrice() function, an OracleRequestPacketData is constructed and ultimately sent using theSendPacket() function.
packetData := packet.NewOracleRequestPacketData( msg.ClientID, msg.OracleScriptID, encodedCallData, msg.AskCount, msg.MinCount, msg.FeeLimit, msg.PrepareGas, msg.ExecuteGas, )
Most of the data in the packet can be equal across different requests and is mainly affected by Comdex’s parameters and configuration.
2. The second step is simple. As with every IBC packet, chain A would like confirmation that the packet has been received and processed on the other end – chain B. The code responsible for handling this acknowledgment is in the callback function OnAcknowledgementPacket(), which then calls handle handleOracleAcknowledgment(). There, we see that it processes the only parameter in the acknowledgment packet – RequestID.
requestID := types.OracleRequestID(oracleAck.RequestID)
This ID is being stored locally as the LastFetchPriceID Key in x/bandoracle/oracle.go:79. This is done so Comdex can verify that the response it gets is indeed the most updated one during the next step.
3. Next, Comdex needs to receive the actual information from Band in the form of an OracleResponsePacketData message. The relevant IBC callback function for that is OnRecvPacket(), which handles incoming IBC packets. The actual functionality is also in the handleOraclePacket(), which is being called from the callback. This function decodes the incoming packet data and ultimately stores the result in the local storage based on the requestID field.
4. Finally, Comdex sends an acknowledgment to Band, indicating that the OracleResponsePacketData has been received and processed with no errors.
Can you identify the vulnerable part of this flow? Let’s explore potential issues in this flow.
In the first step, the Comdex chain initiates the IBC communication by calling the GetChannel() function of the channelKeeper (keepers in the Cosmos-SDK are an abstraction for the local storage) with the designated and hardcoded bandoracleV1 port name. This will always get the trusted connection between Comdex and Band. Comdex can rest assured that all the data coming from this channel is trusted and can verify it using its light client of the Band chain. There are no apparent issues here. The next relevant step is step 3, where Comdex gets the actual price oracles from Band in an OracleResponsePacketData. The logic in handleOraclePacket() is straightforward; it unmarshals the packet data and stores it locally based on the RequestID from step 2 using SetFetchPriceResult(). Is there an actual issue here? In the context of describing the full flow like we’re doing here, it doesn’t look like it. Having said that, is anything stopping an attacker from connecting a malicious chain to Comdex and simply sending an OracleResponsePacketData that will be trusted immediately? Nope. We’ll examine what it may look like in the next section.
Exploitation / Impact
Comdex does not verify the source of the OracleResponsePacketData it gets via IBC. This is the root cause of the vulnerability. Any connected chain can send this type of packet and set prices for Comdex. An attacker can take advantage of this vulnerability, forge a price oracle response packet and set extremely low prices for assets they can buy later.
An attacker would need two components to exploit this vulnerability successfully:
- A malicious Cosmos-SDK chain, connected to Comdex. It will be used to send the forged price oracle packet.
- A relayer, or a piece of code to monitor the Band↔Comdex relayer, that will be able to identify a Comdex request for price oracles and, more importantly, the Band acknowledgment packet that contains the current RequestID. After getting the current request ID, that code can then send it to the attacker’s malicious chain, which will forge an OracleResponsePacketData with it.
An attacker would be able to guarantee this works every time since it takes the Band chain a few seconds to come to a consensus on price oracles before responding to the oracle request. In contrast, the attacker’s chain can do that immediately.
Disclosure
This issue was identified as a critical vulnerability and was awarded the maximum critical bounty for Comdex on Immunefi.
- April 18, 2023 – Vulnerability reported to Comdex via Immunefi
- April 24, 2023 – Comdex confirmed the existence of the vulnerability
- April 25, 2023 – Comdex issued a patch
- May 11, 2023 – Comdex notified CyberArk Labs that the issue has been fixed and that the critical bounty has been paid
Conclusion
IBC is a robust, battle-tested, and secure communication protocol. We can trust it to transfer information in a secure and authenticated way. The question then becomes, can we trust developers to understand its security model and be able to write secure applications that use it? We will likely see more and more interchain vulnerabilities with the rise of this communication standard, and they will probably be in the applications using IBC rather than in the protocol itself. There are currently more than 50 blockchains in the Cosmos, and it is our responsibility as the security community to ensure their interfaces are secure. Hopefully, you can join us on that expedition after learning more about the Cosmos and IBC.
