<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://s-tikhomirov.github.io//feed.xml" rel="self" type="application/atom+xml" /><link href="https://s-tikhomirov.github.io//" rel="alternate" type="text/html" /><updated>2025-11-14T21:34:44+00:00</updated><id>https://s-tikhomirov.github.io//feed.xml</id><title type="html">Sergei Tikhomirov</title><subtitle>Blockchain Researcher</subtitle><entry><title type="html"></title><link href="https://s-tikhomirov.github.io//2021-03-27-lightning-probing/" rel="alternate" type="text/html" title="" /><published>2025-11-14T21:34:44+00:00</published><updated>2025-11-14T21:34:44+00:00</updated><id>https://s-tikhomirov.github.io//2021-03-27-lightning-probing</id><content type="html" xml:base="https://s-tikhomirov.github.io//2021-03-27-lightning-probing/"><![CDATA[<p><strong>This post is kept here for archival purposes. It is superseded by <a href="/lightning-probing-2/">this later post</a>.</strong></p>

<p>The Lightning Network (LN) is a prominent Bitcoin scaling solution that uses payment channels. Channel probing is inference of supposedly non-public balances of honest users in the LN. Channel probing is a big threat to user privacy in the LN. An attacker might want to infer channel balances for several reasons: to spy on payments, to learn business operation details, to optimize other attacks (e.g., network split via jamming), etc.</p>

<p>Even though probing has received some attention over the last couple of years, the understanding of this attack is still quite limited.
Prior work makes assumptions that don’t hold well in practice, thus it is hard to understand from the existing literature whether probing is really feasible (what’s the cost of the attack) and what are the best ways to combat it.</p>

<p>We believe that ignoring <strong>parallel channels</strong> is one of the biggest shortcomings of the prior art. In our work, we expand the existing model of the LN with parallel channels, and advance the understanding of attack optimizations.
We suggest a wide range of countermeasures (both aware and unaware of parallel channels) and evaluate the feasibility of the attack.
Using our novel LN simulator, we demonstrate that our countermeasures bound the attacker’s information gain at 30% while making the attack 2-4 times longer.</p>

<p>The paper is available here: <a href="https://eprint.iacr.org/2021/384">https://eprint.iacr.org/2021/384</a></p>

<ol>
  <li><a href="#basics-of-channel-probing">Basics of channel probing</a></li>
  <li><a href="#probing-applications-and-why-parallel-channels-matter">Probing applications and why parallel channels matter</a></li>
  <li><a href="#challenges-in-spying-on-parallel-channels">Challenges in spying on parallel channels</a></li>
  <li><a href="#attack-model">Attack model</a></li>
  <li><a href="#information-metrics">Information metrics</a></li>
  <li><a href="#simulator">Simulator</a></li>
  <li><a href="#countermeasures-and-evaluation">Countermeasures and evaluation</a></li>
  <li><a href="#attack-optimizations">Attack optimizations</a></li>
  <li><a href="#future-work">Future work</a></li>
  <li><a href="#conclusions">Conclusions</a></li>
</ol>

<h1 id="basics-of-channel-probing">Basics of channel probing</h1>

<p>We assume that our readers have a basic understanding of how the Lightning Network operates.
Channel probing involves sending fake payments (with random hashes) via a target channel. 
We refer to such payments as <em>probes</em>.
Probes might result in the following outcomes:
1) “Invalid hash” error signed by the probe target, when a probe successfully reached the target;
2) “Insufficient amount” error signed by the second-last node in the route;
3) A failure somewhere along the route (e.g., “insufficient amount” on the non-last hop).</p>

<p>For a balance of a given target channel, the attacker maintains a lower and an upper bound.
The attacker picks a probing amount between the current bounds and sends a probe. If (1) is returned, then the lower bound of the target channel is updated with the chosen amount. If (2) is returned, then the upper bound is updated. In case of (3), an attacker has to find another route.
Note that even in case (3) the attacker may update the bounds for channels before the erring one, and use this information to optimize the attack.
The attacker repeats probes with new balance estimates until the target precision is reached.</p>

<p>In practice, “sending payments via a target channel” is not possible.
The sender can only specify an ordered list of nodes that form a payment path.
Routing nodes are free to choose any available channel to reach the next node in the route (this is called non-strict forwarding). Thus, if the target channel is not the only one between nodes A and B (in other words, if <em>parallel channels</em> exist), the attacker can’t guarantee that the payment went through the target channel. The attacker can’t even tell through which channel the payment actually went. We will talk about these challenges in more detail in Section <a href="#challenges">Challenges</a>.</p>

<p>Completely mitigating channel probing is probably impossible due to the nature of the LN.
However, it’s possible to make it very inefficient. The relevant metrics are:</p>
<ul>
  <li>the attacker’s information gain bounds;</li>
  <li>the attacker’s information gain speed;</li>
  <li>the attacker’s required capital allocation.</li>
</ul>

<h1 id="probing-applications-and-why-parallel-channels-matter">Probing applications and why parallel channels matter</h1>

<p>Even though inference of balance of a particular channel might itself be interesting, an attacker might also want to infer payments of a particular node (or between two nodes), or infer the full balance of a victim, or do something else. For these scenarios, probing is just a building block.</p>

<p>In our work, we focus on channel probing in general, without going into much detail about particular attack aims or scenarios.  We achieve generality by a) having a general attack success metric (see <a href="#attack-model">Attack model</a>), and b) evaluating it against 100 random targets nodes in the simulated network (see <a href="#evaluation">Evaluation</a>).</p>

<p>While being generic, we understand that an attacker might be interested in two observations:
1) the maximum amount that can be forwarded between two nodes;
2) the balances of individual channels (of a victim, or between two nodes).</p>

<p>For (1), parallel channels are irrelevant, and an attacker can be unaware of them. However, (1) is often insufficient for an attacker, and (2) is required, for example, to detect payments flowing between two nodes. But (2) can be properly inferred only if the attacker is aware of parallel channels.</p>

<p>Parallel channels are allowed by the specification (although one of the implementations, c-lightning, running on approximately 11% of nodes, is not planning to support them). As of December 2020, the LN contains 21% parallel channels that hold 45% of the network’s capacity.</p>

<h1 id="challenges-in-spying-on-parallel-channels">Challenges in spying on parallel channels</h1>

<p>Prior channel probing research ignored parallel channels. So, what’s the challenge with inferring balances of parallel channels?</p>

<p>Consider the following example where an attacker Eve is trying to infer balances of channels C1, C2 between Alice and Bob.</p>

<p><img src="../images/parallel_channels.png" alt="Parallel channels" /></p>

<p>Due to non-strict forwarding it’s up to the routing nodes to choose which channel will be used for routing. By default, routing nodes will make best effort to route a payment to earn fees.</p>

<p>In this case, it’s impossible for an attacker to infer balances of channel C1 which has both smaller balance and smaller capacity than C2. An attacker would only be able to infer that channel C2 has 3 coins on both sides, and that’s it. Even though the attacker knows it can’t possibly reflect C1 (because the capacity there is public and equals to 2 coins), an attacker can’t do much without advanced techniques like those we suggest in <a href="#attack-optimizations">Attack optimizations</a>.</p>

<p>Prior works would either get meaningless results (by claiming that channel A has more balance than its capacity), or only achieve very limited information gain (by ignoring many parallel channels).</p>

<p>We now demonstrate how the inference strategy should be adjusted in the light of parallel channels.</p>

<h1 id="attack-model">Attack model</h1>

<p>In our work, we introduce a notion of a “hop”, which is a set of all channels between two nodes, with estimates of their bounds.</p>

<p>We propose an attack strategy that keeps track of both hop bounds and channel bounds. At every probe, the attacker applies the new estimate to the hop as a whole.
Additionally, if the attacker can deduct that the probe went through a particular channel, channel-level bounds are also updated. (Currently, we ignore cases where an estimate applies for a subset of channels.)
Then, every new probe makes the best effort to infer as much knowledge about target channels as possible.</p>

<p>While this strategy is aware of parallel channels, it sometimes can’t determine which channel-level bound should be updated. While hop-level bounds (that match channel-level bounds for single-channel hops) may be sufficient in some scenarios, the attacker can also be interested in channel-level bounds in the general case. Advanced attack techniques (jamming and policies), which we describe in Section <a href="#adv-attacks">Advanced attacks</a>, may help. Our current attack strategy allows an attacker to be more efficient (e.g., make fewer probes) by taking any extra knowledge into account.</p>

<h1 id="information-metrics">Information metrics</h1>

<p>To measure the gained knowledge, we introduce a new metric that is aware of parallel channels.
Intuitively, it reflects the attacker’s <em>information gain</em>.
As soon as the desired information gain is reached for the current target hop, the attacker stops probing. We define the <em>information gain</em> as the change in <em>uncertainty</em> before and after the probing.
But what exactly is uncertainty?</p>

<p>Consider a channel between Alice and Bob with capacity 1023.
Ignoring fees, channel reserve, and in-flight payments for simplicity, Alice’s balance can take any value between 0 and 1023 - a total of 1024 possible values, which would require 10 bits to encode (the <em>initial uncertainty</em>).
After one probe, the attacker learns that Alice’s balance can only take values from 0 to 511, gaining 1 bit of information and decreasing the uncertainty to 9 bits.</p>

<style type="text/css">
.tg  {border-collapse:collapse;border-spacing:0;}
.tg td{border-color:black;border-style:solid;border-width:1px;
  overflow:hidden;padding:10px 5px;word-break:normal;}
.tg th{border-color:black;border-style:solid;border-width:1px;
  font-weight:normal;overflow:hidden;padding:10px 5px;word-break:normal;}
.tg .tg-0pky{border-color:inherit;text-align:left;vertical-align:top}
.tg .tg-dvpl{border-color:inherit;text-align:right;vertical-align:top}
</style>

<table class="tg">
<thead>
  <tr>
    <th class="tg-0pky">Step</th>
    <th class="tg-0pky">Bounds</th>
    <th class="tg-0pky">Uncertainty (granularity = 1)</th>
    <th class="tg-0pky">Uncertainty (granularity = 256)</th>
  </tr>
</thead>
<tbody>
  <tr>
    <td class="tg-0pky">0</td>
    <td class="tg-dvpl">0-1023</td>
    <td class="tg-dvpl">10</td>
    <td class="tg-dvpl">2</td>
  </tr>
  <tr>
    <td class="tg-0pky">1</td>
    <td class="tg-dvpl">0-511</td>
    <td class="tg-dvpl">9</td>
    <td class="tg-dvpl">1</td>
  </tr>
  <tr>
    <td class="tg-0pky">2</td>
    <td class="tg-dvpl">256-511</td>
    <td class="tg-dvpl">8</td>
    <td class="tg-dvpl">0</td>
  </tr>
</tbody>
</table>

<p>The attacker may only wish to learn the balance up to a certain <em>granularity</em>.
In the above example, the granularity is 1 satoshi.
For the granularity of 256 satoshis, the initial bounds (0 - 1023) only correspond to 2 bits of uncertainty (the interval 0-1023 contains 2^2=4 “buckets” of 256 values each).
We refer the reader to the paper for the exact (rather simple) formulas for uncertainty.</p>

<!---
The exact formulas are as follows.

Uncertainty before probing (c is capacity, u is granularity):
U_{before}= log_2(c_i+1) - n*log_2(u)

Uncertainty after probing:
U_{after}= max(0, log_2(b_i^h - b_i^l + 1) - \log_2(u))

Information gain:
I = U_{before} - U_{after}
--->

<p>We define the information gain for the attack as a whole as the sum of information gains (assuming independence for simplicity) for all channels in all target hops. We may also express the information gain as a percentage of the total initial uncertainty.
This allows us to say things like “the attacker is able to retrieve 80% of balance information from target channels” (see evaluation below).</p>

<p>Having defined the attack and the metric to evaluate its success, we implement it in a simulator which models the probing process using the algorithm described above.</p>

<h1 id="simulator">Simulator</h1>

<p>Multiple simulators have been developed for the LN.
We believe that our simulator improves upon the state of the art in the following ways:</p>
<ul>
  <li>it accurately models parallel channels;</li>
  <li>it reflects the fact that some channels only allow forwarding in one direction;</li>
  <li>it simulates the time a real probing would take by generating randomized networking delays based on prior real-world measurements.</li>
</ul>

<p>Another advantage of our approach is that we model the network itself and the user (in our case, the prober) separately. The simulator takes as input a network snapshot obtained from an LN node. Based on the snapshot, the simulator creates different graphs to represent the real network and the attacker’s view of the network (including the information obtained from probing). Such modularity allows for implementing other, more complex attack scenarios, for example, those with multiple probing entities, or with honest users’ payments.</p>

<p>The simulator is extendable, one can change the probing parameters and plug in other path-finding algorithms, amount selection algorithms (besides binary search), etc. We plan to release the simulator under an open-source license.</p>

<h1 id="countermeasures-and-evaluation">Countermeasures and evaluation</h1>

<p>Now, the goal of honest users, routing nodes, and stakeholders of the LN is to reduce the information gain of an attacker.</p>

<p>Since probing is based on sending fake payments and observing the resulting errors, we came up with three error-related ideas:</p>
<ol>
  <li>random failures: a node fails a payment even if the balance at the next hop is sufficient;</li>
  <li>error spoofing: a node pretends that an error happened in its next channel and not later;</li>
  <li>error delays: a node delays sending the error message to the previous hop.</li>
</ol>

<p>Each countermeasure may be applied with some probability or under some conditions.
We assess how these countermeasures (separately and in combinations) make attacks longer and limit the attacker’s information gain. Our experiments are summarized in the following graph, where we plot the attacker’s information gain (as percentage of the initial uncertainty) as it accumulated throughout the simulated time of the attack.</p>

<p><img src="../images/combined-plot.png" alt="The effectiveness of countermeasures" /></p>

<p>Each line represents one attack (the results are averaged across 10 simulations).
All lines start from 0, indicating that at the start the attacker only knows public channel capacities.
For the attacker, steeper lines are better (information is collected faster).
As expected, the line without countermeasures is the steepest.</p>

<p>If no countermeasures are deployed, an attacker can achieve 80% information gain in 20 seconds per channel (full information extraction is not always possible due to parallel channels, non-forwarding hops, leafs, etc). The most effective countermeasure is a combination of <em>error delays</em> and <em>random failure</em>, which bounds information gain at 25% while requiring 80 seconds per channel to reach it.
<em>Random failure</em> alone also provides a good result of 30% bound on information gain with 40 seconds per channel.</p>

<p>Each countermeasure alone is less promising: <em>error delays</em> and <em>error spoofing</em> can only make the attack about 2-3x longer than the baseline, and other combinations don’t yield better results than those mentioned above. For more detailed analysis, please refer to the paper.</p>

<p>Countermeasures come at a cost though. To understand the cost, we first note that honest LN users score routing nodes based on their historic reliability. Reliable nodes are prioritized for route construction. The drawbacks of the countermeasures described above are:</p>
<ol>
  <li>they damage the user experience for honest users: genuine error messages take longer to propagate to the sender and are unreliable (making channel scoring less useful);</li>
  <li>they might be not incentive-compatible: routing nodes are supposed to sacrifice their channel score to improve other users’ privacy.</li>
</ol>

<p>In the paper, we briefly discuss the tension between usability, incentives, and privacy. Some trade-offs might be acceptable.</p>

<p>We also suggest other potential countermeasures without an in-depth analysis of their performance:</p>
<ol>
  <li>intra-hop payment split: allow routing nodes to split a payment among their channels without coordination with the payer;</li>
  <li>on-demand channel rebalancing (e.g., JIT routing);</li>
  <li>adversarial strategies for multi-channel hops: a combination of special channel structure and batching to minimize information leak;</li>
  <li>using “gates” of channels / links to prevent an external observer from inferring our internal structure beyond the gate;</li>
  <li>rate-limiting via linking payments to <a href="https://thelab31.xyz/blog/stake-certificates">Stake Certificates</a> (blinded proofs of UTXO ownership).</li>
</ol>

<h1 id="attack-optimizations">Attack optimizations</h1>

<p>Even if a hypothetical attacker is aware of parallel channels and tunes the inference engine accordingly, it turns out that in many cases it’s still impossible to infer all channel balances within a hop (e.g., the example in <a href="#countermeasures">Countermeasures</a>), because an attacker can’t easily choose through which channels the payment goes.</p>

<p>In our work, we also explored how an attacker might use channel <a href="https://github.com/t-bast/lightning-docs/blob/master/spam-prevention.md">jamming</a> and channel <a href="https://github.com/lightningnetwork/lightning-rfc/pull/765#pullrequestreview-511147029">policy discrepancies</a> to overcome the issues with probing parallel channels. These are currently important unsolved issues of the Lightning Network, so it’s fair to assume they will be relevant for a while.</p>

<p>It turns out that, in many cases, using either one of those techniques indeed allows an attacker to <em>choose</em> through which channel the payment will go, and thus, reliably probe specific channels. We verified this against our own real LN nodes.</p>

<p>With jamming, an attacker has to jam certain high balance channels (probably after probing them), to reach the previously inaccessible channels. With attack enhancements based on fees and other policies, an attacker can craft probes so that a victim forwards them only over a channel from a specific subset.</p>

<p>These optimizations also come at a cost:</p>
<ul>
  <li>channel jamming assumes an attacker locks liquidity (for “capacity-based jamming”) or opens many channels and pay on-chain fees (for “slot-based jamming”);</li>
  <li>exploiting policy discrepancies requires victim channels to be heterogeneous.</li>
</ul>

<p>To reduce the costs, an attacker might combine these attack vectors: for example, tune the fees of jamming payments so that they jam a specific channel.</p>

<h1 id="future-work">Future work</h1>

<p>The following topics look to us as a natural next step towards better understanding and combating channel probing:</p>
<ol>
  <li>the impact of routing fees on the attacks;</li>
  <li>the noise that honest payment flows introduce;</li>
  <li>capital costs of probing;</li>
  <li>advancing attacks with behavior patterns of the victims and bulk probing.</li>
</ol>

<p>We would be excited to see explorations of specific applications of probing, e.g. payment flow inference, built on top of this work.</p>

<h1 id="conclusions">Conclusions</h1>

<p>It’s important to better understand channel probing because it’s a real privacy threat in the Lightning Network. In this work, we advance this understanding by considering parallel channels. We describe how an attacker may extract information from them using various probing techniques.
We also suggest ideas for combating probing attacks and demonstrate their efficiency, while also noting the shortcomings they introduce. We hope to see community discussion around these trade-offs.</p>]]></content><author><name></name></author></entry><entry><title type="html">Unjamming Lightning</title><link href="https://s-tikhomirov.github.io//unjamming-lightning/" rel="alternate" type="text/html" title="Unjamming Lightning" /><published>2022-11-18T00:00:00+00:00</published><updated>2022-11-18T00:00:00+00:00</updated><id>https://s-tikhomirov.github.io//unjamming-lightning</id><content type="html" xml:base="https://s-tikhomirov.github.io//unjamming-lightning/"><![CDATA[<p>For the past few months, I’ve been working with <a href="https://clarashk.github.io/">Clara Shikhelman</a> at <a href="https://chaincode.com/">Chaincode Labs</a> on the issue of <a href="https://bitcoinproblems.org/problems/channel-jamming.html">jamming attacks</a> in the Lightning Network.</p>

<p>Our efforts have resulted in a <a href="https://eprint.iacr.org/2022/1454">paper</a> where we propose a solution that combines small unconditional fees and local reputation.</p>

<p>I’ve written a post summarizing our findings for the Chaincode Research blog:</p>

<p>https://research.chaincode.com/2022/11/15/unjamming-lightning/</p>]]></content><author><name></name></author><summary type="html"><![CDATA[For the past few months, I’ve been working with Clara Shikhelman at Chaincode Labs on the issue of jamming attacks in the Lightning Network.]]></summary></entry><entry><title type="html">LightPIR. Privacy-Preserving Route Discovery for Lightning (paper summary and analysis)</title><link href="https://s-tikhomirov.github.io//lightpir/" rel="alternate" type="text/html" title="LightPIR. Privacy-Preserving Route Discovery for Lightning (paper summary and analysis)" /><published>2022-02-16T00:00:00+00:00</published><updated>2022-02-16T00:00:00+00:00</updated><id>https://s-tikhomirov.github.io//lightpir</id><content type="html" xml:base="https://s-tikhomirov.github.io//lightpir/"><![CDATA[<p>Lightning is currently source-routed.
This means that each sender does a local route search on the full network graph.
This may become unsustainable as Lightning grows grows.
Naively outsourcing route discovery to dedicated servers harms privacy: the servers know who is paying whom.
The <a href="https://arxiv.org/abs/2104.04293">LightPIR paper</a> proposes a solution.
The authors combine private information retrieval with all-pairs-shortest-path pre-computation with hub labeling, optimized for real LN topology.
In this post, I summarize the LightPIR protocol and outline the potential first steps to turn it from a research prototype to a real-world implementation.</p>

<h1 id="lightning-routing">Lightning routing</h1>

<p>Multi-hop routing is a key feature of Lightning.
Payments between users with no shared channels allows Lightning to truly perform as a network: every (public) channel provides liquidity not only its counterparts but also to everyone else.
Currently, Alice (the sender) defines the entire route to Bob (the receiver).
To do so, Alice stores a snapshot of the public network based on P2P gossip.</p>

<p>As LN grows, it may become infeasible for each node to store a full network snapshot.
Although the current LN size (18410 nodes and 82128 channels at the time of this writing, as per <a href="https://explorer.acinq.co/">ACINQ</a>) doesn’t sound like a lot by modern hardware standards, if we expect Lightning to grow by orders of magnitude, source routing may become challenging.
Indeed, we want Lightning to work even on an IoT device with a tiny battery, little storage, and sporadic Internet connection.
This justifies the need for alternative efficient route discovery algorithms.</p>

<h1 id="a-naive-trust-based-solution">A naive trust-based solution</h1>

<p>Let’s start with a naive trust-based solution and then improve it in both efficiency and privacy.
Imagine we have specialized nodes (<em>servers</em>) to do route discovery.
Generally, we can evaluate a client-server protocol using many metrics:</p>

<ul>
  <li>server-side computation;</li>
  <li>server-side storage;</li>
  <li>client-side computation;</li>
  <li>client-side storage;</li>
  <li>communication.</li>
</ul>

<p>There are two approaches to server-based route discovery:</p>
<ol>
  <li>a server calculates a route from Alice to Bob when Alice asks for it (just-in-time);</li>
  <li>a server <em>pre-calculates</em> shortest routes for <em>each</em> pair of nodes and looks up the appropriate route when requested. This approach is called “<strong>all pairs shortest paths</strong>”, or APSP.</li>
</ol>

<p>The just-in-time approach is less demanding but may introduce higher latency for the client.
APSP requires more computation and storage from the server.</p>

<p>The protocols we’ve considered so far are not private.
The server knows what route a client requests (that is, who is paying whom).
In contrast, the current source-based route discovery is relatively private[<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>].
Our goal is to come up with a protocol that would combine the efficiency of APSP approaches with the privacy of client-side route discovery.</p>

<h1 id="private-information-retrieval-addressing-privacy">Private information retrieval (addressing privacy)</h1>

<p>Let’s put LN specifics aside for a moment.
Consider a server that stores a database <code class="language-plaintext highlighter-rouge">DB</code>, which is an array of boolean values.
The client wants to obtain the <code class="language-plaintext highlighter-rouge">i</code>-th element.
A straightforward protocol would be: the client sends <code class="language-plaintext highlighter-rouge">i</code> to the server, and the server replies with <code class="language-plaintext highlighter-rouge">DB[i]</code>.
Clearly, the server knows which element the client has requested.</p>

<p>Can a client obtain <code class="language-plaintext highlighter-rouge">DB[i]</code> <em>without the server knowing</em> <code class="language-plaintext highlighter-rouge">i</code>?
Theoretically, the only solution is to send the whole database to the client.
This is hardly practical.
Instead of the problem we know is practically unsolvable, let’s bend the rules a bit and consider a related problem under additional security assumptions.</p>

<p>Here is where <strong>private information retrieval</strong> (PIR) comes in.
There are two branches of PIR: computational (CPIR) and information-theoretical (IT-PIR).
CPIR allows for PIR with a single server but is slower and involves advanced cryptography (keyword: fully homomorphic encryption).
IT-PIR, on the other hand, is fast, uses cheap cryptography, but only works with <em>multiple</em> servers.
LightPIR is based on IT-PIR, so we’ll use the term PIR to refer to IT-PIR unless stated otherwise.</p>

<p>Consider <em>two</em> servers holding identical database copies.
A simple PIR protocol works under an additional security assumption: <strong>the servers don’t collude</strong>.</p>

<ol>
  <li>the client generates a random string <code class="language-plaintext highlighter-rouge">r</code> and another string <code class="language-plaintext highlighter-rouge">r'</code> that only differs from <code class="language-plaintext highlighter-rouge">r</code> in the <code class="language-plaintext highlighter-rouge">i</code>-th bit;</li>
  <li>the client sends <code class="language-plaintext highlighter-rouge">r</code> to the first server;</li>
  <li>the first server selects elements at indices <code class="language-plaintext highlighter-rouge">j</code> where <code class="language-plaintext highlighter-rouge">r[j]=1</code> and responds with a single bit <code class="language-plaintext highlighter-rouge">d</code> that is a XOR of those elements;</li>
  <li>the client sends <code class="language-plaintext highlighter-rouge">r'</code> to the second server;</li>
  <li>the second server responds analogously with a single bit <code class="language-plaintext highlighter-rouge">d'</code>;</li>
  <li>the client locally computes <code class="language-plaintext highlighter-rouge">d XOR d' = D[i]</code>.</li>
</ol>

<p>In other words, the client obtains two bits that are XOR’s of <em>nearly identical</em> subsets of the database elements.
The only difference between those subsets is that one includes <code class="language-plaintext highlighter-rouge">D[i]</code> and the other doesn’t.
XOR’ing them together, the client discovers <code class="language-plaintext highlighter-rouge">D[i]</code> locally.</p>

<p>This simple PIR algorithm is not very efficient.
Server-side computation cost and the communication cost are linear in the database size.
Even though each response is a single bit long, the <em>request</em> <code class="language-plaintext highlighter-rouge">r</code> is as long as the database itself.</p>

<p>Advanced PIR protocols are more efficient.
One idea is to consider the database as a matrix instead of a linear array.
This allows to improve communication complexity by increasing the number of servers.
For instance, with four non-colluding servers, the client only sends request of length of <em>square root</em> of the number of elements.
For more details, see <a href="https://www.youtube.com/watch?v=HFFPeYrz3ak">this introductory lecture</a>.</p>

<h1 id="hub-labeling-addressing-efficiency">Hub labeling (addressing efficiency)</h1>

<p>Any digital map service finds a route across a continent in seconds.
Isn’t it amazing, considering that a road graph may contain millions of points?</p>

<p>Turns out, there is a lot of space for optimizations in route discovery algorithms that exploit the properties of particular graphs.
In particular, roads are not equally likely to occur on a random route.
Most routes (except very short ones) follow this pattern:</p>
<ol>
  <li>drive to the nearest highway entrance;</li>
  <li>drive along the highway;</li>
  <li>exit the highway and drive to your destination.</li>
</ol>

<p>Route discovery algorithms have been heavily optimized based on the fact that highways are involved in a disproportionately large share of long routes.
A key optimization is called <strong>hub labeling</strong>.
Informally, a hub is a node that is part of <em>many</em> <em>sufficiently long</em> <em>shortest</em> paths.
(Each highlighted word in the previous sentence is of course formally defined.)
Remember an earlier APSP method, where the server pre-computes routes for each pair of nodes?
Let’s now modify APSP with hubs in mind.</p>

<p>Let each server store, for each node, a list of <em>its</em> hubs and the shortest route to each of them.
Finding a route from Alice to Bob involves these steps:</p>

<ol>
  <li>find a node that both Alice and Bob consider their hub;</li>
  <li>look up the shortest route from Alice to the hub;</li>
  <li>look up the shortest route from the hub to Bob[<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>];</li>
  <li>concatenate the two partial routes to get the full route.</li>
</ol>

<p>An attentive reader might spot an issue here.
What happens if Alice and Bob have no hubs in common?
To avoid this scenario, hub sets are chosen to <em>intersect all shortest paths</em> of non-trivial length.
This task is equivalent to the <a href="https://en.wikipedia.org/wiki/Set_cover_problem">set cover problem</a>, which is NP-complete.
Heuristic algorithms do the job well enough for practical applications.</p>

<p>A (heuristic) hub set labeling algorithm computes shortest paths between any pair of nodes and assign hub labels.
The hub sets generated by a heuristic are valid (i.e., they allow for lookup-and-concatenate route discovery) but may be suboptimal (for example, include more hubs than necessary).
Still, the result is good enough for practical applications.</p>

<h1 id="lightpir-putting-this-all-together">LightPIR: putting this all together</h1>

<p>At long last, let’s discuss the contributions of the LightPIR paper!
The authors combine PIR and hub labeling (HL) for route discovery in payment channel networks.
Assuming three types of nodes (a content provider, servers, and clients), the protocol goes as follows:</p>

<ol>
  <li>the content provider (CP) creates a network snapshot;</li>
  <li>CP populates the graph with hub sets and shortest routes to hubs[<sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>];</li>
  <li>CP and sends its copies to the servers;</li>
  <li>clients query multiple servers using a PIR protocol.</li>
</ol>

<p>The novelty of the paper is an improved HL algorithm tailored to the LN topology.
The LN-optimized HL algorithm goes as follows:</p>

<ol>
  <li>select <code class="language-plaintext highlighter-rouge">K</code> nodes with the highest degree and put them in the hub set for all nodes;</li>
  <li>exclude those nodes from the graph;</li>
  <li>find the remaining hubs based on the pruned graph.</li>
</ol>

<p>Intuitively, the LightPIR exploits the fact that the LN is <em>even more</em> centralized than highway networks.
Instead of repeatedly “discovering”, for instance, that the ACINQ node (the most connected public node at the time of this writing) belongs to the hub set for Alice, and for Bob, and for Carol, etc, the proposed algorithm simply adds ACINQ to all hub sets.
The same applies to <code class="language-plaintext highlighter-rouge">K</code> most connected nodes, which are then temporarily excluded from the graph.
Additional “node-specific” hubs are then discovered on the pruned graph.
How many most-connected nodes shall we pre-select into all hub sets?
The authors run experiments on historic LN snapshots and conclude that <code class="language-plaintext highlighter-rouge">K</code> should be around 100.</p>

<h2 id="a-note-on-dataset-validity">A note on dataset validity</h2>

<p>Initially, I had doubts about the dataset the paper is based on.
The data source is Christian Decker’s <a href="https://github.com/lnresearch/topology">Lightning Network Gossip</a> repository, which is trustworthy.
However, the following phrase raised suspicion:</p>

<blockquote>
  <p>although the network size almost doubles from March 2019 to January 2021, the number of nodes in the largest SCC remains pretty consistent across all snapshots (≈ 2500 nodes)</p>
</blockquote>

<p>This seemingly contradicted other sources (like <a href="https://bitcoinvisuals.com/lightning">Bitcoin Visuals</a>), which report a consistent growth in the number of Lightning nodes during that period.
Moreover, in the snapshots I took for <a href="https://s-tikhomirov.github.io/lightning-probing-2/">my own research</a> in 2021, 99% of nodes belonged to the main connected component.</p>

<p>In fact, there is no contradiction.
The LightPIR paper uses a directed graph model and only considers the main <em>strongly</em> connected component (SCC).
The main SCC is the largest set of nodes where each pair of nodes has a <em>directed</em> route.
The (non-stront) connected component, in contrast, doesn’t take directionality into account, and hence may be much larger than the main SCC[<sup id="fnref:4" role="doc-noteref"><a href="#fn:4" class="footnote" rel="footnote">4</a></sup>].</p>

<p>Even if there is no contradiction, it’s unclear why the authors chose to only consider the main SCC, which contains only between 39% and 72% of public nodes[<sup id="fnref:5" role="doc-noteref"><a href="#fn:5" class="footnote" rel="footnote">5</a></sup>].
More thought is required to more accurately reflect the real-world network structure.
Would the results change if we consider the main <em>non-strongly</em> connected component?</p>

<h1 id="implementation-prospects">Implementation prospects</h1>

<p>Assuming LightPIR is a good idea, how should it be converted into a practically implementable proposal for Lightning?
I see a few issues that should be addressed.</p>

<h2 id="non-collusion-assumption">Non-collusion assumption</h2>

<p>PIR protocols are based on the assumption that servers don’t collude.
The paper that introduced PIR (<a href="https://madhu.seas.harvard.edu/papers/1995/pir-journ.pdf">Chor et al., 1995</a>) justifies this assumption from the reputational standpoint:</p>

<blockquote>
  <p>We assume that the servers do not collude in trying to violate the user’s privacy. &lt;…&gt; A detected violation of the privacy guarantees will result in severe damage to the server. It is as if a bank were caught in fraud. &lt;…&gt; In the rare case where the user values its potential loss as more substantial than the server’s risk, the user should not use a PIR scheme in which privacy depends on a noncollusion assumption.</p>
</blockquote>

<p>This seemingly contradicts the permissionless nature of Lightning.
However, Lightning nodes already have a somewhat persistent identity and reputation.
This is especially relevant for Lightning Service Providers, or LSPs (a vague term for large nodes  providing LN services professionally).
If LSPs put their reputation on the line and are punished for colluding, the scheme might work in practice.
The key question then is: how to provably detect collusion?</p>

<h2 id="it-pir-vs-cpir">IT-PIR vs CPIR</h2>

<p>As mentioned earlier, there are two kinds of PIR: information-theoretical (IT-PIR) and computational (CPIR).
The IT-PIR approach, which LightPIR is based on, achieves exactly zero information leakage even if the attacker has infinite computing power.
CPIR, on the other hand, allows for a single-server PIR, assuming the attacker’s resources are bounded.
The paper doesn’t justify the choice of IT-PIR as opposed to CPIR.</p>

<p>The aforementioned 1995 paper writes on the topic of non-collusion assumption:</p>

<blockquote>
  <p>The single-server computational PIR scheme of <a href="https://web.cs.ucla.edu/~rafail/PUBLIC/34.pdf">Kushilevitz and Ostrovsky (1997)</a> addresses this concern.</p>
</blockquote>

<p>Wouldn’t CPIR approach be more suitable for Lightning?
A bound on attacker’s resources is the assumption under which most real-world systems, including Bitcoin and Lightning, operate anyway.
What are the downsides of CPIR in this context?</p>

<h2 id="a-single-source-of-network-graph-data">A single source of network graph data</h2>

<p>The LightPIR model assumes a single source of truth about network topology, copiled by a dedicated entity (the content provider).
Lightning, in contrast, has no canonical network view: nodes compose their own graphs based on gossip.
(Yet, everyone is very likely to be aware of the most active and connected nodes.)</p>

<p>There might be two ways to reconcile theory and practice.
First, we could amend the theory to allow database copies to be slightly different.
The servers then could independently compile their graphs based on gossip.
Second, LN nodes could opt-in for a centralized data source.
For instance, a well-known LSP (think 1ML, LNBIG, or ACINQ) would periodically publish fresh network snapshots.
Servers (maintained by, for instance, by wallet providers) would download the snapshot and announce to their clients that they provide PIR-based route discovery based on the graph from a certain date.
Clients would then query a random subset of wallet providers to privately retrieve routes.</p>

<h2 id="a-common-route-quality-metric">A common route quality metric</h2>

<p>LightPIR assumes that all clients use the same metric to evaluate route quality.
In the simplest case (ignoring fees), the model provides <em>shortest</em> routes.
If we use fees as edge costs, we’d be talking about <em>cheapest</em> routes.
On the one hand, the cheapest routes model may capture the desires of many clients.
On the other hand, this assumption may open clients to attacks where the adversary <a href="https://arxiv.org/abs/1909.06890">attracts payments by advertising low fees</a>.
On top of that, total fees may be just one of the components of route quality function.
Some clients may want to also optimize for:</p>

<ul>
  <li>route length (longer routes increase the chance of payment failure);</li>
  <li>success probability (<a href="https://arxiv.org/abs/2107.05322">related to</a> but not fully defined by route length);</li>
  <li>avoiding specific nodes (known adversarial nodes, nodes located in a certain region, etc).</li>
</ul>

<p>We may bridge this gap between theory and practice in one of two ways (or both):</p>

<ol>
  <li>amend the theory to allow for user-defined route quality metrics;</li>
  <li>use the protocol as is for users who aim for a simple metric (such as minimal fees).</li>
</ol>

<h2 id="the-model-doesnt-account-for-amounts-and-fees">The model doesn’t account for amounts (and fees?)</h2>

<p>The graph model as presented in the paper is quite simplified.
First, it doesn’t account for payment amounts and channel capacities.
Second, it doesn’t account for fees (though I’m not exactly sure[<sup id="fnref:6" role="doc-noteref"><a href="#fn:6" class="footnote" rel="footnote">6</a></sup>]).</p>

<p>Also, the <a href="https://rusty-lightning.medium.com/lightning-routing-rough-background-dbac930abbad">key challenge</a> in scaling LN routing is accommodating <em>dynamic</em> parts of the graph.
The nodes and channels are somewhat static, whereas fee and policy updates may be frequent.
How should LightPIR be modified to not only account for fees but also to reflect ongoing fee updates?</p>

<h1 id="conclusion">Conclusion</h1>

<p>LightPIR is a route discovery protocol for payment channel networks that combines private information retrieval and optimized hub labeling.
It builds on prior work in route discovery for road networks and achieves higher efficiency by exploiting LN’s centralized topology.
Simulations based on historical LN snapshots indicate how to best parameterize the algorithm.</p>

<p>LightPIR is an example of research that applies prior scientific findings in new contexts.
In this case, results on private database lookup and routing for road networks have been applied to payment channel networks.
This approach is valuable and underappreciated.
Most likely, there are lots of valuable ideas in scientific literature from long before Bitcoin came along, waiting to be applied in modern development.
However, at least in the case of LightPIR, more effort is required to turn this protocol into an implementation-ready proposal.</p>

<h1 id="further-reading">Further reading</h1>

<p>Other papers on routing in payment channel networks:</p>

<ul>
  <li><a href="https://eprint.iacr.org/2016/1054">SilentWhispers</a> (<a href="https://s-tikhomirov.github.io/silentwhispers/">my summary</a>)</li>
  <li><a href="https://arxiv.org/abs/1709.05748">SpeedyMurmurs</a> (<a href="https://s-tikhomirov.github.io/speedymurmurs/">my summary</a>)</li>
  <li><a href="https://arxiv.org/abs/1809.05088">Spider</a> (<a href="https://s-tikhomirov.github.io/spider-network/">my summary</a>)</li>
  <li><a href="https://bitfury.com/content/downloads/whitepaper_flare_an_approach_to_routing_in_lightning_network_7_7_2016.pdf">Flare</a></li>
</ul>

<p>Some more practice-oriented proposals:</p>

<ul>
  <li><a href="https://bitcoinops.org/en/topics/rendez-vous-routing/">Rendez-vous routing</a></li>
  <li><a href="https://bitcoinops.org/en/topics/trampoline-payments/">Trampoline payments</a></li>
</ul>

<p>Unifying these and other related ideas into a practical protocol for Lightning route discovery remains a direction for future work.</p>
<div class="footnotes" role="doc-endnotes">
  <ol>
    <li id="fn:1" role="doc-endnote">
      <p>Privacy attacks exist with source-based routing too, i.e. using <a href="https://arxiv.org/abs/2006.12143">timing</a>, <a href="https://arxiv.org/abs/2007.00764">cross-layer deanonymization</a>, or <a href="https://s-tikhomirov.github.io/lightning-probing-2/">balance probing</a>, but at least no server learns right away which route the sender has chosen. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:2" role="doc-endnote">
      <p>Notice a subtlety here: we consider <em>Bob’s</em> hub set but we need a (partial) route <em>to</em> Bob, not <em>from</em> Bob. This doesn’t matter if all routes are bidirectional. Does this explain why the authors only consider the main <em>strongly</em> connected component (see below)? <a href="#fnref:2" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:3" role="doc-endnote">
      <p>It’s not exactly clear from the paper whether the CP or the servers do the hub labeling, but I would imagine it makes little sense to repeat the work on each server, as the result will be the same. <a href="#fnref:3" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:4" role="doc-endnote">
      <p>It’s not at all surprising that the optimal number of most-connected nodes <code class="language-plaintext highlighter-rouge">K</code> is nearly the same across all snapshots (Figure 3): running the same algorithm on nearly identical inputs can be expected to produce nearly identical results. <a href="#fnref:4" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:5" role="doc-endnote">
      <p>The smallest and largest snapshots contained 3480 and 6376 nodes, respectively. <a href="#fnref:5" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:6" role="doc-endnote">
      <p>Section 2 introduces a directed graph with <em>weighted</em> edges, but Section 7 states that the authors “do not yet take into account fees”, and that their work is optimized for “sending a fixed amount of Bitcoin”. Also, Section 4.2.2 says that the labeling algorithm “requires that the path weights are unique integers, but this is easily achieved with insignificant perturbations of edge weights (in our case we initially set the weights to be the channel base fees and then perturbed them)”. It’s unclear to me whether edges have weights after all, and if so, whether weights equal fees. Even if <em>base</em> fees are considered as edge weights, an accurate model of LN fees must also account for their <em>proportional</em> component, which means accounting for amounts, capacities, and balances. <a href="#fnref:6" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
  </ol>
</div>]]></content><author><name></name></author><summary type="html"><![CDATA[Lightning is currently source-routed. This means that each sender does a local route search on the full network graph. This may become unsustainable as Lightning grows grows. Naively outsourcing route discovery to dedicated servers harms privacy: the servers know who is paying whom. The LightPIR paper proposes a solution. The authors combine private information retrieval with all-pairs-shortest-path pre-computation with hub labeling, optimized for real LN topology. In this post, I summarize the LightPIR protocol and outline the potential first steps to turn it from a research prototype to a real-world implementation.]]></summary></entry><entry><title type="html">Probing parallel channels in the Lightning network</title><link href="https://s-tikhomirov.github.io//lightning-probing-2/" rel="alternate" type="text/html" title="Probing parallel channels in the Lightning network" /><published>2022-01-03T00:00:00+00:00</published><updated>2022-01-03T00:00:00+00:00</updated><id>https://s-tikhomirov.github.io//lightning-probing-2</id><content type="html" xml:base="https://s-tikhomirov.github.io//lightning-probing-2/"><![CDATA[<p>In this post, we summarize <a href="https://eprint.iacr.org/2021/384">our paper</a> on channel balance probing in the Lightning network. It supersedes our <a href="/lightning-probing/">earlier work</a> on this topic.
A video presentation based on this post (a <a href="https://youtu.be/ZiD7NqQ1YZc">longer version</a> is also available):</p>
<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/1dDC2VYTVdw" frameborder="0" allow="encrypted-media" allowfullscreen=""></iframe>
<p>First, we briefly introduce the Lightning network (LN) and the channel balance probing attack. Then, we propose an enhanced probing technique that allows an attacker to extract more private information faster. We run simulations based on real-world data and conclude that the proposed probing method is indeed better than prior art. Finally, we discuss potential countermeasures and their trade-offs.</p>

<ol>
  <li><a href="#lightning-network-101">Lightning Network 101</a></li>
  <li><a href="#channel-balance-probing">Channel balance probing</a></li>
  <li><a href="#probing-model">Probing model</a></li>
  <li><a href="#probing-multi-channel-hops">Probing multi-channel hops</a></li>
  <li><a href="#jamming-enhanced-probing">Jamming-enhanced probing</a></li>
  <li><a href="#evaluation">Evaluation</a></li>
  <li><a href="#conclusion">Conclusion</a></li>
</ol>

<h1 id="lightning-network-101">Lightning Network 101</h1>

<p>The Lightning Network (LN) is a layer-two protocol for fast bitcoin payments. It is considered the major scaling solution for Bitcoin. As of January 2022, its publicly announced part consists of 17 thousand nodes and 79 thousand payment channels.</p>

<p>A payment channel is a cryptographic protocol for off-chain bitcoin payments between two parties. A useful mental model to visualize a channel is “beads on a string”. The beads cannot leave the string, they can only move back and forth.</p>

<p><img width="600" src="../images/lightning-probing-2/channel-simple.png" alt="A payment channel between Alice and Bob with capacity of five coins." /></p>

<p>The total number of coins in a channel is called its <em>capacity</em>, and the number of coins currently owned by Alice and Bob are their respective <em>balances</em>. The two balances sum up to the capacity, so we can infer one balance from the other. We define the <em>channel</em> balance to be the balance of the node with alphabetically smaller name (in this example, that would be “Alice”). We refer to a pair of adjacent nodes together with all channels that they share as a <em>hop</em>.</p>

<p>Alice doesn’t have to establish a direct channel to Charlie to send him payments. Instead, she can use a multi-hop path (in this example, via Bob). Multi-hop payments work as follows. Alice offers Bob one coin under the condition that he forwards one coin to Charlie. Bob forwards one coin to Charlie, who uses the payment secret known only to him to redeem the coin. Bob can then use the same secret to redeem the coin from Alice. Hence, one coin has effectively moved from Alice to Charlie.</p>

<p><img width="800" src="../images/lightning-probing-2/multi-hop.png" alt="A payment path allowing Alice to send one coin to Charlie via Bob." /></p>

<p>The key issue with this process is that Alice doesn’t know in advance whether Bob has sufficient balance in the channel towards Charlie. If Alice tries to send another coin along the same path, the payment would fail. Therefore, Lightning follows the trial-and-error approach. The sender may have to make several payment attempts until one of them succeeds.</p>

<p>As we will see, one can exploit the error reporting mechanism of the Lightning network in an attack called balance probing.</p>

<h1 id="channel-balance-probing">Channel balance probing</h1>

<p>The attacker (also referred to as the Prober) wants to learn remote channels balances (which is private information). To achieve this goal, it sends fake payments, or <em>probes</em>, and observes where they fail. If a probe reaches the final destination, all channels along the path have sufficient balances. Otherwise, if the probe fails somewhere along the path, the Prober learns that the erring node lacks balance.</p>

<p>The attacker’s knowledge may be visualized as follows. The outer interval denotes the target channel. The star shows the true balance. The colored area is the set of all points where, according to the attacker’s current knowledge, the balance may be. b^l and b^u are the current balance bounds.</p>

<p><img width="400" src="../images/lightning-probing-2/1dim.png" alt="The attacker's knowledge when probing one channel." /></p>

<p>Initially, the colored area covers the whole interval. By making a series of probes, the attacker updates the balance estimates and shrinks the colored interval. Assuming that balances are equally likely to take any value between zero and the channel capacity, the optimal strategy is to divide the colored interval in half with every probe.</p>

<p>The probing algorithm doesn’t always work perfectly. Consider a hop with two channels. Such channels are called <em>parallel</em>.</p>

<p><img width="600" src="../images/lightning-probing-2/probing-parallel.png" alt="Probing a two-channel hop." /></p>

<p>A routing node (Alice) is free to choose any of the parallel channels to forward the probe. After receiving the error message, the attacker doesn’t know which channel it applies to. As a result, the classic probing algorithm becomes inapplicable.</p>

<p>Note that while the prober cannot update <em>individual</em> balance bounds, it does get some information about the hop as a whole. We need a new probing model to describe what exactly the attacker learns in this case.</p>

<h1 id="probing-model">Probing model</h1>

<p>We propose a new geometrical model that describes probing in the general case, for any number of parallel channels. To introduce our model, let’s use a two-dimensional example. Consider a two-channel hop with the capacities of both channels equal to C. It can be represented as a square with corners (0,0), (0,C), (C,C), (C,0).</p>

<p><img width="300" src="../images/lightning-probing-2/2dim-balance.png" alt="A geometrical representation of a two-channel hop." /></p>

<p>Each point within the square corresponds to a possible vector of channel balances. The star denotes the true balance point: the first channel has balance b_1, and the second channel has balance b_2.</p>

<p>The attacker sends the first probe of amount a_1.</p>

<p><img width="600" src="../images/lightning-probing-2/2dim.png" alt="Probing a two-channel hop." /></p>

<p>The probe doesn’t reach the destination, so the prober concludes that <em>all</em> channel balances are less than the probe amount: b_1 &lt; a_1 and b_2 &lt; a_1. Geometrically, this means that the true balance is <em>inside</em> the a_1-sided square that the probe “cuts” from the lower-left corner of the larger square. Now, the attacker sends another probe with amount a_2 that does reach the destination. This means that <em>at least one</em> of the channels has sufficient balance: either b_1 &gt; a_2 or b_2 &gt; a_2. Geometrically, it means that the true balance is <em>outside</em> of the a_2-sided square. As a result of there two probes, the attacker has obtained the upper and lower bounds that correspond geometrically to the colored L-shaped figure (the difference of two squares).</p>

<p>What do these bounds bound, by the way? As mentioned before, the prober doesn’t necessarily learn anything about individual balances. Instead, it learns how much a hop can forward in the probe direction – simply speaking, the maximum of the balances. We refer to this value as h = max(b_i). The analogous value in the opposite direction is denoted as g.</p>

<p>Probes in the opposite direction have a similar representation in the geometrical model: instead of “cutting” squares from the lower-left corner from the larger square, they cut squares from the upper-right corner. Consider the state of probing after four probes have been done:</p>

<p><img width="600" src="../images/lightning-probing-2/2dim-full.png" alt="Attacker's knowledge after four probes for a two-channel hop." /></p>

<p>The attacker’s knowledge is comprised of four values – the lower and upper bounds on h and g. The bounds on h – h^l and h^u – define an L-shape “looking north-east”, whereas g^l and g^u define an analogous L-shape “looking south-west”. The intersection of these shapes defines the attacker’s knowledge: the smaller the area of the resulting figure, the more precisely the prober knows there the true balances are.</p>

<p>Here is where our first contribution comes in. We suggest choosing each next probe amount such as the probe cuts the colored figure in half by area. Prior algorithms chose the probe amount as the mid-point between the current balance estimates, which may be sub-optimal in the multi-channel case. Instead, our generalized approach is optimized for hops with any number of channels.</p>

<p>The following figures illustrate the process of probing a 2-channel hop step by step. (We only go through the first four steps explicitly.)</p>

<p><img width="800" src="../images/lightning-probing-2/2dim-steps.png" alt="Probing a two-channel hop step by step." /></p>

<p>Note that at some point the colored area splits into two disjoint diagonally symmetric rectangles. This is a representation of the fact that balances can only be probed up to permutation, because the model assigns channels to axes randomly.</p>

<p>The same model naturally applies to any number of channels (and therefore, dimensions).</p>

<p>Now consider a question. Given enough probes, can the attacker probe any hop, with any number of channels?</p>

<h1 id="probing-multi-channel-hops">Probing multi-channel hops</h1>

<p>Consider a 3-channel hop with equal-capacity channels.</p>

<p><img width="400" src="../images/lightning-probing-2/3dim.png" alt="Probing a three-channel hop with equal capacities." /></p>

<p>Analogously to the two-dimensional case, each probe now cuts a <em>cube</em> (instead of a square) from the lower-left vertex of the larger <em>cube</em> that represents the hop. The two bounds on h correspond to two surfaces, each composed of three perpendicular faces of the respective cube. The true balance must be above the smaller (purple) surface representing the lower bound, and below the larger (orange) surface representing the upper bound.</p>

<p>What happens when the attacker learn h precisely? The two surfaces collapse into one:</p>

<p><img width="400" src="../images/lightning-probing-2/3dim-final.png" alt="A three-channel hop fully probed from one direction." /></p>

<p>The balance must be somewhere <em>on</em> the colored surface.</p>

<p>Probes from the opposite direction produce a symmetrical surface, also composed of three perpendicular squares. The true balance must be somewhere on the intersection of these two surfaces. However, in the general case, two such <em>surfaces</em> intersect along a <em>line</em> composed of six intervals. The attacker cannot learn exactly where on this line the balance is! Compare it to the two-dimensional case, where instead of surfaces we had linear L-shapes, which neatly intersected at exactly two points, reflecting the true balance vector (modulo permutation).</p>

<p><img width="400" src="../images/lightning-probing-2/3dim-intersect.png" alt="Fully probing a three-channel hop doesn't reveal exact balances." /></p>

<p><img width="400" src="../images/lightning-probing-2/2dim-intersect.png" alt="Fully probing a two-channel hop reveals exact balances." /></p>

<p>An intuitive interpretation of this difference could be as follows. There are only two directions that a channel can be probed in. Probing in each direction decreases the dimensionality by one. That is, if the hop in question has only one or two channels, the final result would only contain one or two points. In the 3-dimensional case, the best the attacker can achieve is a <em>line</em>, that is, a one-dimensional figure. In the 4-dimensional case, the end result would be some <em>surface</em>, in the 5-dimensional case – some 3-dimensional <em>volume</em>, and so on.</p>

<p>(There is another scenario when a multi-channel hop may not always be probed fully. Can you guess what the reason is? If you want the answer, see Appendix B in the paper.)</p>

<p>The key issue that prevents full information extraction is that the prober cannot influence which of the parallel channels the probes go through. Instead, probes only reveal information about the <em>aggregate</em> of the balances. If only there were a way to force probes go through a specific channel…</p>

<h1 id="jamming-enhanced-probing">Jamming-enhanced probing</h1>

<p>We suggest combining jamming with probing to extract more information from multi-channel hops.</p>

<p>Jamming is a type of denial-of-service attacks on Lightning channels. The attacker sends a payment to itself (either via a circular route or simply to another node that it also controls) and deliberately delays finalizing the payment. As a result, the funds along the route are left “in-flight” and are unavailable for other payments.</p>

<p>There are two types of jamming (by capacity and by slots), discussing them is outside the scope of this post (please refer to the Background section of the paper and references therein). For our purposes, it’s sufficient to understand that an attacker can temporarily disable a victim channel.</p>

<p><img width="600" src="../images/lightning-probing-2/jamming.png" alt="Jamming the Alice-Bob hop via a circular route." /></p>

<p>We suggest combining jamming and probing to overcome the dimensionality issue described above. In particular, the attacker can jam all channels in a multi-channel hop except one, and then probe the remaining channel. In other words, while the attacker cannot influence how a routing node chooses a channel to forward a probe, it is possible to decrease the set of <em>suitable</em> channels the node picks from.</p>

<p>Geometrically, jamming-enhanced probing boils down to revealing each channel individually. In the 3-dimensional case, the prober first reveals b_1, then b_2, and then b_3. Each balance is represented by a plane parallel to the corresponding axis. The intersection of three perpendicular planes is a single point representing the true balance vector.</p>

<p><img width="400" src="../images/lightning-probing-2/3dim-enhanced.png" alt="The end result of jamming-enhanced probing a three-channel hop." /></p>

<p>To recap: our contributions are as follows. We introduce a new probing model that accurately describes the attacker’s knowledge when probing multi-channel hops. We propose jamming-enhanced probing to overcome the limitation on information extraction in multi-channel hops. Finally, we suggest using an optimized algorithm (generalized binary search) to select probe amounts for multi-channel hops.</p>

<p>The question now is: how do we measure the benefits that our proposed improvements provide?</p>

<h1 id="evaluation">Evaluation</h1>

<p>We evaluate our approach using <a href="https://github.com/s-tikhomirov/ln-probing-simulator">our own simulator</a> written in Python. We capture a snapshot of the network using our own Lightning node and assign balances to channels uniformly at random. We then pick 20 target hops with a given number of channels (from 1 to 5) and probe them in the simulator.</p>

<p>We use two metrics to access the success of the attack: information gain and probing speed. Information gain reflects the share (from 0 to 1) of uncertainty about channel balances in target hops that the attacker was able to resolve. (By uncertainty we mean the binary logarithm of the number of points contained in the final figure describing the attackers knowledge.) Probing speed shows how much information the prober gets per message sent (a message is either a probe of a jam).</p>

<p>We alter the probing algorithm in three ways:</p>

<ul>
  <li>Jamming-enhanced probing vs non-enhanced probing</li>
  <li>Optimized amount selection vs simple binary search</li>
  <li>Direct vs remote probing</li>
</ul>

<p>In direct probing, the attacker established a channel to the target hop directly. In the real network, this requires on-chain fee but, on the other hand, all probes reach the target hop. In remote probing, the attacker sends probes along multi-channel paths. This allows for amortizing the cost of channel openings across many target hops but some probes are wasted due to insufficient balances in intermediary hops.</p>

<p>For each alteration of the probing algorithm, we run the simulation 100 times and average the results.</p>

<p><img width="800" src="../images/lightning-probing-2/gains_snapshot.png" alt="Information gain graphs." /></p>

<p>For information gain, we observe that:</p>
<ol>
  <li>for non-enhanced probing (the left graph), the information gain decreases as the number of channels increases (due to the dimensionality issue);</li>
  <li>jamming-enhanced probing (the right graph) overcomes this limitation, achieving nearly full information extraction for multi-channel hops;</li>
  <li>all else equal, remote probing performs slightly worse than direct probing.</li>
</ol>

<p><img width="800" src="../images/lightning-probing-2/speed_snapshot.png" alt="Probing speed graphs." /></p>

<p>For probing speed, we observe that:</p>
<ol>
  <li>Direct probing with optimized amount selection (the left graph, blue line) achieves nearly perfect probing speed of 1 bit / message;</li>
  <li>Remote probing is always slightly slower than direct probing;</li>
  <li>Optimized amount selection is always faster than non-optimized amount selection.</li>
</ol>

<p>In summary: we confirm that jamming-enhanced probing yields more balance information, and that optimized amount selection allows for faster probing.</p>

<p>Potential countermeasures may be divided into node-level policies (something a single node can apply) or network-level protocol changes. On the node level, popular routing nodes may batch payments (so that payments in the opposite directions cancel each other out), split payments among their parallel channels, establish unannounced channels in parallel to public ones, or drop or forge error messages (which would, however, decrease reliability). Measures on the network level largely intersect with <a href="https://blog.bitmex.com/preventing-channel-jamming/">potential anti-jamming proposals</a>, for instance, upfront fees for both successful and failed payment attempts.</p>

<h1 id="conclusion">Conclusion</h1>

<p>In summary, we have introduced an enhanced probing technique for Lightning channels and confirmed using simulations that it reveals channel balances better and faster.</p>

<p>More generally, the issue we’ve been discussing illustrates the dilemma for Lightning. As long as Lightning is permissionless and privacy-focused (in particular, it uses onion routing), bad actors would be able to abuse it by mounting attacks on reliability (such as jamming) or privacy (such as probing). The key challenge for LN development is to limit the negative effects of unwanted network activity while preserving the permissionless nature of the network. We hope this work helps advance the understanding of the relevant trade-offs and be a basis of future protocol improvements.</p>

<p>For more details, see the <a href="https://eprint.iacr.org/2021/384">full paper</a> (to be presented at <a href="https://fc22.ifca.ai/">Financial Cryptography 2022</a>). <a href="https://docs.google.com/presentation/d/1IPZdpSVX2B636G6m4o66jQCk8RAO5HUy_HD_ITgR_-M/edit?usp=sharing">Slides</a>, a <a href="https://youtu.be/ZiD7NqQ1YZc">video presentation</a> (roughly based on this post), and the <a href="https://github.com/s-tikhomirov/ln-probing-simulator">source code of the simulator</a> are also available.</p>]]></content><author><name></name></author><summary type="html"><![CDATA[In this post, we summarize our paper on channel balance probing in the Lightning network. It supersedes our earlier work on this topic. A video presentation based on this post (a longer version is also available): First, we briefly introduce the Lightning network (LN) and the channel balance probing attack. Then, we propose an enhanced probing technique that allows an attacker to extract more private information faster. We run simulations based on real-world data and conclude that the proposed probing method is indeed better than prior art. Finally, we discuss potential countermeasures and their trade-offs.]]></summary></entry><entry><title type="html">Clustering transactions in Bitcoin and other cryptocurrencies</title><link href="https://s-tikhomirov.github.io//transaction-clustering-bitcoin/" rel="alternate" type="text/html" title="Clustering transactions in Bitcoin and other cryptocurrencies" /><published>2019-11-25T00:00:00+00:00</published><updated>2019-11-25T00:00:00+00:00</updated><id>https://s-tikhomirov.github.io//transaction-clustering-bitcoin</id><content type="html" xml:base="https://s-tikhomirov.github.io//transaction-clustering-bitcoin/"><![CDATA[<p>For the significant portion of 2018, as part of my PhD studies in <a href="https://www.cryptolux.org/">CryptoLUX</a> group at the <a href="https://wwwen.uni.lu/">University of Luxembourg</a>, I’ve been working on network-level privacy attack on Bitcoin and other cryptocurrencies with professor Alex Biryukov.
This blog post summarizes our findings, which have been published in 2019 (“<a href="http://hdl.handle.net/10993/39724">Deanonymization and linkability of cryptocurrency transactions based on network analysis</a>”).
You can watch my presentation at EuroS&amp;P 2019 in Stockholm (press CC for subtitles; <a href="https://orbilu.uni.lu/bitstream/10993/39724/2/biryukov-tikhomirov-deanonymization-and-linkability-slides.pdf">slides</a> are also available):</p>
<iframe width="560" height="315" src="https://www.youtube-nocookie.com/embed/XXO3FBqwwO8" frameborder="0" allow="encrypted-media" allowfullscreen=""></iframe>
<p>You though it was enough to use mixers or privacy-focused altcoins to preserve the privacy of your cryptocurrency transactions?
Think again…</p>

<h1 id="privacy-in-cryptocurrencies">Privacy in cryptocurrencies</h1>

<p>Bitcoin is the first successful implementation of decentralized digital money.
Its key innovation is using proof-of-work to make modifying the ledger difficult.
But in order to be really decentralized, Bitcoin and other cryptocurrencies must provide at least some level of privacy.
Otherwise, those in power would still have some influence over cryptocurrency users.</p>

<p>Unlike a bank account, generating a Bitcoin address doesn’t require having passport.
You can, and are in fact advised to, generate a new address for every transaction.
This doesn’t guarantee full privacy though.</p>

<p>The fundamental trade-off here is between privacy and verifiability.
Bitcoin tries to be decentralized: users must be able to independently validate incoming transactions.
But to be able to <em>verify</em> incoming bitcoins, I must know <em>where they are coming from</em>.</p>

<p>Bitcoin transactions, once included in a block, are stored on thousands of nodes worldwide.
Anyone can download the whole Bitcoin transaction history and analyze it.
Turns out, there are techniques to extract quite a bit of information from the blockchain.</p>

<p>A Bitcoin transaction <em>consumes</em> (spends) a number of unspent transaction outputs (UTXOs) and creates new UTXOs.
One heuristic is that all UTXOs in a transaction belong to the same entity.
This is not generally true due to multisigs etc, but good enough for a heuristic.
Another heuristic is that one of the outputs is the <em>change</em> address.
Just as with cash, you usually don’t pay the exact amount, but pay a bit extra and receive change back.
Multiple papers have been published along those ideas, and there are commercial companies which offer services of deanonymizing cryptocurrency users.</p>

<p>Alternative cryptocurrencies aim at stronger privacy.
More established ones include Monero, Zcash, and (to a lesser degree) Dash.
Newer ones, Beam and Grin, are based on the MimbleWimle protocol.
They do indeed prevent or at least hinder blockchain analysis.
While flaws have been described in all of them, one cannot simply open a blockchain explorer and look where certain coins come from.</p>

<p>The information stored on the blockchain is stored there forever.
But what about the ephemeral information in the peer-to-peer network?
Can an attacker extract any useful deanonymizing data from observing the network traffic?
What makes the idea even more interesting is that privacy focused cryptocurrencies use P2P networks similar to Bitcoin…</p>

<h1 id="randomization-of-transaction-propagation">Randomization of transaction propagation</h1>

<p>Cryptocurrencies rely on peer-to-peer networks to propagate data.
In particular, a Bitcoin node connects to 8 random nodes (and may accept incoming connections) and relays transactions to them.
They, in turn, relay it to their neighbors, and so on.
After a few seconds, nearly all nodes become aware of the new transaction, and miners can include it in a block.</p>

<p>If Alice knows a new transaction, she first announces its hash in an inventory (INV) message to Bob.
Then only if Bob is interested in this transaction he replies with a GETDATA message, and Alice replies with a TX message for a transaction.
If this gossiping mechanism was implemented just naively, it would be dangerous for privacy: an adversary could listen to the network and try to estimate the “rumor source”.
That’s why there are certain <em>broadcast randomization</em> techniques used in Bitcoin and other cryptocurrencies: trickling and diffusion.</p>

<p>Trickling was used in Bitcoin before 2015 and is still used in Zcash.
In trickling, a node chooses a random neighbor subset and announces the transaction only to them.
Then, after a certain delay, another random subset is chosen, and the transaction is announced to its members.
In diffusion, for each neighbor, a node announces the transaction to each neighbor after a random delay.
As we will show, these methods hinder but do not completely eliminate network analysis.</p>

<h1 id="transaction-clustering-based-on-network-level-information">Transaction clustering based on network-level information</h1>

<p>The general idea behind our attack is to connect to many nodes and log the timestamps of transaction announcements.
The intuition is that transactions which originate from the same node propagate in a <em>similar</em> fashion.
(We will define the notion of similarity later.)
First, we have to overcome some technical difficulties to collect the data.</p>

<h2 id="parallel-connections">Parallel connections</h2>

<p>As described earlier, Bitcoin uses a mechanism known as diffusion to  network-level attacks on privacy difficult.
If we collect data by only connecting to each node once, as the reference software allows us to do, we won’t gain much.
Ideally, we want to be the first to receive a new transaction announcement from the peer which generated it.</p>

<p>A typical full node maintains 8 outgoing and allows up to 117 incoming connections.
Say, it has 12 incoming connections, for a total of 20 connections.
This means that if we connect to it, we have only a 1 in 20 chance of being the first to hear about a new transaction.</p>

<p>To overcome broadcast randomization, we use an alternative implementation of the Bitcoin networking stack called <a href="https://github.com/ivanpustogarov/bcclient">bcclient</a>.
It was developed here at <a href="https://www.cryptolux.org/index.php/Home">CryptoLUX</a> as part of the research resulting in “<a href="https://arxiv.org/abs/1410.6079">Bitcoin over Tor isn’t a good Idea</a>” and “<a href="https://arxiv.org/abs/1405.7418">Deanonymisation of Clients in Bitcoin P2P Network</a>”.
Bcclient connects to Bitcoin nodes with multiple <em>parallel</em> connections.
As Zcash inherited most of networking properties of Bitcoin, it was relatively straightforward to adapt bcclient for Zcash.</p>

<h2 id="weighting-ip-addresses">Weighting IP addresses</h2>

<p>Next, we want to find a metric to compare transactions.
Each transaction can be characterized by a vector of IP addresses which announced it to us.
For each transaction, we consider a <em>vector</em> of IP addresses which were the the first to announce it to us, assign <em>weights</em> to them.</p>

<p>Intuitively, the first IP to announce the transaction to us is the most likely to be the sender.
For each subsequent IP, the probability of it being “close” to the sender decreases.
In real network some transactions get to us very quickly, some of them get to us a little bit slower.
For every transaction, we want the weight of IP addresses to drop neither too quickly nor too slowly.
Here is an example of weights assigned to three (made up) timestamp vectors by our parameterized weight function (see the paper for details):</p>

<p><img src="../images/posts/2019-09-30-transaction-clustering-weight.png" width="800" alt="Example weight vectors" /></p>

<h2 id="calculating-correlations">Calculating correlations</h2>

<p>Next, we calculate the correlation coefficient for each pair of weight vectors and depict them as a matrix.
We expect it to exhibit a special structure: with the right permutation of rows and columns, clusters would be visible along the main diagonal.
That would mean that some subsets of transactions are closer related to each other than to other transactions.</p>

<p>To measure the effect of our attack, we use the <a href="https://en.wikipedia.org/wiki/Degree_of_anonymity">anonymity degree</a> proposed by Diaz et al in 2002.
This metric, originally introduced for messaging systems, reflects the amount of information that the attacker can obtain regarding who was the author of each message.
The anonymity degree varies from 0 to 1, where one means perfect anonymity, and zero means no anonymity at all.</p>

<h2 id="recap-what-we-do-step-by-step">Recap: what we do step by step</h2>

<p>We connect to many nodes from servers on three continents (Europe, Asia, and North America) to get a better view of the network.
We log transaction announcements and assign weights to vectors of timestamps for each IP address.
Then we calculate the pairwise correlations between these weight vectors and apply the spectral co-clustering algorithm, which tries to find the permutation of rows and columns in the matrix such that the internal clustering structure would be visible.
It is <a href="https://scikit-learn.org/stable/modules/biclustering.html#biclustering">implemented</a> in a popular Python sklearn library.
Then we calculate the anonymity degree using our own transactions as ground truth.</p>

<h1 id="results">Results</h1>

<p>On the Bitcoin testnet, we did the full-scale experiment connecting to all nodes with as many connections as possible.
We obtained the following picture (see the paper for full results):</p>

<p><img src="../images/posts/2019-09-30-transaction-clustering-bitcoin-testnet.png" width="800" alt="Result for Bitcoin testnet" /></p>

<p>We clearly see that our own transactions (marked with black lines) form a cluster, and we see some other clusters forming along the main diagonal.
We also did the experiment on the Bitcoin mainnet, but in order not to disrupt the real network, we limited ourselves to only connecting to 1,000 nodes, which is about 1/10 of the nodes available, and didn’t try to occupy all connection slots.
The results on mainnet are significantly worse: our transactions are scattered around the clusters.</p>

<p>We also did an experiment on Zcash, and there the picture is not that clear, but still our transactions form a cluster.
It’s also important to note that in Zcash transactions can be shielded or transparent.
Transparent transactions have the same structure as in Bitcoin and give no privacy enhancements.
Shielded transactions take advantage of sophisticated cryptography (zk-SNARKs).
In the following picture, the longer black lines indicate the shielded transactions, and shorter ones indicate transparent transactions.
This shows that our method doesn’t care whether you use zk-SNARKs or not because we only take transaction hashes into account:</p>

<p><img src="../images/posts/2019-09-30-transaction-clustering-zcash-mainnet.png" width="800" alt="Result for Zcash mainnet" /></p>

<p>The traffic for Dash and Monero also exhibits some cluster properties (see the paper for the full results).</p>

<p>We also estimated the original sender IP address for a given cluster.
It is only possible in certain circumstances.
When a node connects to a network, it advertises its IP address in an ADDR message.
If our listener nodes are online at that moment, they can compare the IP address announcement in the address message with the IP addresses which are highly ranked in certain clusters.
We show that, at least for the Bitcoin testnet, an adversary can narrow down the search of the source IP address to about five IP addresses.</p>

<p>What about countermeasures?
From a cryptocurrency user’s point of view, you shouldn’t issue many transactions during the same session.
If you do, you may want to run your nodes with an increased number of connections and also periodically drop and re-establish connections.</p>

<p>Two recent proposals in Bitcoin, <a href="https://arxiv.org/abs/1805.11060">Dandelion++</a> and <a href="https://arxiv.org/abs/1905.10518">Erlay</a>, would defeat our attack if implemented.
In both cases, the key change compared to the existing P2P protocol is that the first stage of propagation is performed with outbound connections only.
This ensures that an adversary has a zero probability of receiving a new transaction announcement early, if the victim did not choose to connect to the adversarial IP.
This probability can not be increased by establishing <em>incoming</em> connections, as we have done in this work.
Note that the Erlay paper explicitly states: “The decision to relay through outbound connections, but not the inbound ones, was made to defend against timing attacks”.</p>

<h1 id="conclusion">Conclusion</h1>

<p>Many blockchain developers think of the network layer as a black box: it broadcasts transactions, what else do we need?
As we have demonstrated, timing of transaction announcements reveals information on related transactions.
This data is invisible on the application level.
Randomization techniques, as they exist today, are not 100% efficient.
The issue is especially crucial for privacy focused cryptocurrencies.
Novel P2P protocols may help alleviate the attack by preferring outbound connections for the initial announcement phase.</p>

<p>See the full papers for more details:</p>

<ul>
  <li><a href="http://hdl.handle.net/10993/39724">Deanonymization and linkability of cryptocurrency transactions based on network analysis</a> (IEEE EuroS&amp;P 2019)</li>
  <li><a href="http://hdl.handle.net/10993/39728">Transaction Clustering Using Network Traffic Analysis for Bitcoin and Derived Blockchains</a> (CryBlock workshop at IEEE INFOCOM 2019) – a shorter version of “Deanonymization and linkability”</li>
  <li><a href="http://hdl.handle.net/10993/39729">Security and Privacy of Mobile Wallet Users in Bitcoin, Dash, Monero, and Zcash</a> (in a special issue of “Pervasive and Mobile Computing” on blockchain technologies) – studying the networking aspects of mobile wallets and applying the clustering technique to transactions issued from smartphones.</li>
</ul>]]></content><author><name></name></author><summary type="html"><![CDATA[For the significant portion of 2018, as part of my PhD studies in CryptoLUX group at the University of Luxembourg, I’ve been working on network-level privacy attack on Bitcoin and other cryptocurrencies with professor Alex Biryukov. This blog post summarizes our findings, which have been published in 2019 (“Deanonymization and linkability of cryptocurrency transactions based on network analysis”). You can watch my presentation at EuroS&amp;P 2019 in Stockholm (press CC for subtitles; slides are also available): You though it was enough to use mixers or privacy-focused altcoins to preserve the privacy of your cryptocurrency transactions? Think again…]]></summary></entry><entry><title type="html">Thoughts on Web3. Part 1</title><link href="https://s-tikhomirov.github.io//thoughts-on-web3-part-1/" rel="alternate" type="text/html" title="Thoughts on Web3. Part 1" /><published>2019-09-04T00:00:00+00:00</published><updated>2019-09-04T00:00:00+00:00</updated><id>https://s-tikhomirov.github.io//thoughts-on-web3-part-1</id><content type="html" xml:base="https://s-tikhomirov.github.io//thoughts-on-web3-part-1/"><![CDATA[<p>This August, Berlin was the global center of all things decentralized.
Thousands of blockchain enthusiasts gathered for the <a href="https://www.blockchainweek.berlin">Berlin Blockchain Week</a> – a series of conferences, meetups, and a <a href="https://ethberlinzwei.com">hackathon</a>.
In this post, I’ll share my thoughts on Web3, which was the primary topic of the the first major event of the week – Web3 Summit. [<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>]</p>

<p><img src="../images/2019-09-04-thoughts-on-web3-part-1-funkhaus.jpg" alt="Web3 Summit 2019. Funkhaus, Berlin. 2019-08-19" /></p>

<p style="text-align:right">"The next Mark Zuckerberg won't start a social network company." – Peter Thiel</p>

<h1 id="what-is-web3">What is Web3?</h1>

<p>The “3” in Web3 here refers to the “versions” of the web.
Web 1.0 consisted of static, read-only websites.
Web 2.0 brought the dynamic, Javascript-powered interfaces and, more importantly, user generated content.
Facebook is the primary example of a Web2 service.[<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>]
Billions of users use it for free to stay in touch, share photos, and much more.
More and more people are feeling upset though.
The major reason for criticism is Facebook’s business model.</p>

<h1 id="what-is-wrong-with-web-20">What is wrong with Web 2.0?</h1>

<p>Contrary to radio and television, the Internet is a dual channel.
Users consume data but also report what they’ve consumed back to the server.
The cost of storing and processing data has been declining rapidly (Moore’s law plus economies of scale) – at least compared to the value of the data if processed properly.
This led to the current Facebook’s business model: it collects comprehensive user profiles and provides precise targeting to advertisers.</p>

<p>Is it that bad if someone extracts enough value from my clicks to let me connect to friends for free?
Advertising <a href="https://commons.wikimedia.org/wiki/File:Woman%27s_Home_Companion_1919_-_Campbell_Vegetable_Soup.png">has been around for centuries</a>, why are people angry at Facebook?</p>

<p>Say, every morning you drop by the same cafe to grab some coffee.
The barista greets you with a smile: “A double espresso as usual, Mister Smith? We have delicious chocolate cupcakes today, would you like one?”
You smile back and feel glad that you live in such a lovely neighborhood.</p>

<p>Now imagine you bought a coffee machine on Amazon.
A minute later, you see an ad: “Bought a coffee machine? Order our extra-special coffee beans! Best for your favorite morning double espresso!”
Feels a bit creepy, right?
What if the ad greets you by name?</p>

<p>People fear the unknown.
A monster jumping from behind the corner in a horror movie makes you scared for a second, but the character walking around the haunted house anticipating a monster jumping from behind <em>any</em> corner generates lasting suspense.
Nuclear power, which is statistically very safe, is perceived as dangerous because but involves deadly yet invisible radiation.
People also don’t like giving up control.
Many people “feel” that <a href="https://en.wikipedia.org/wiki/List_of_countries_by_traffic-related_death_rate">driving</a> is safer than <a href="https://en.wikipedia.org/wiki/Aviation_safety#Statistics">flying</a>, because they feel in control behind the wheel.</p>

<p>Uncertainty and lack of control affect personal data as well.
Sure, Facebook collects data on me, but <em>what exactly</em>? Does it come only from my Facebook usage or from other websites too?
Who has access to the data? How long is it stored and where? What can they understand about me if they analyze it? What if they analyze it in ten years with a hundred times more powerful computer?
Who do they share it with, and what to <em>those</em> parties do?
Can I control what data they collect and what they use it for?</p>

<p>Facebook isn’t keen to answer these questions.
The rare PR-department-style announcements start with the obligatory “We value your privacy” only to continue with a huge “<strong>but</strong>”, carefully wrapped in layers of unconvincing legalspeak.
Facebook is one of the least trusted brands in the US, at the bottom of the list alongside big banks and The Worst Brand Ever in The Land Of The Free: the Government.</p>

<p>Speaking of banks and the government…</p>

<p>Money is another area of life with similar dynamics.
It’s very scary to have your card blocked, especially if you have no cash.
The rationale behind such decisions (is your transaction pattern strange so we block you just in case?) is hidden.
The financial system is complex, controlled by someone else, without an opt-out option….</p>

<p>…until 2009.</p>

<p>Bitcoin showed that it is possible to disrupt the seemingly all-mighty fiat money system.
Understandably, people angry at Facebook started wondering: if Bitcoin found a way to give people a more free and private money, can we use similar technologies to achieve the same goals for data?</p>

<p>Without going into too much technical details, let’s unpack what makes Bitcoin work.</p>

<h1 id="why-bitcoin-works">Why Bitcoin works</h1>

<p>Bitcoin, famously, is a <a href="https://bitcoinrabbithole.org/">rabbit hole</a>.
As you start thinking about what makes Bitcoin work, every insight opens up new questions.
After some time, you stare into the abyss, asking yourself: what is value? what is energy? what is time?
In this chapter, I’ll just focus on three points which I find important for the web3 discussion: what is digital scarcity, value as a special content type, and how this enables a closed reward system in Bitcoin.</p>

<h2 id="digital-scarcity">Digital scarcity</h2>

<p>Bitcoin is the first implementation of <em>digital scarcity</em> without a trusted party.</p>

<p>Digital information works very well for many purposes, but one characteristic delayed the (proper) digitalization of money for nearly three decades (from David Chaum’s <a href="https://en.wikipedia.org/wiki/David_Chaum#Digital_cash">digital cash</a> in 1982 to Bitcoin in 2008): <strong>bits are not scarce</strong>.
Whatever sequence of bits you give me, I can copy it, diluting the value you aimed at transferring.
A simple but dirty way of solving this problem was to appoint a central party, which we all <em>trust</em> to not let people copy value-representing bits.
(This is called fraud or counterfeiting and is punishable off-protocol, that is, by armed people putting you in a cage.)</p>

<p>Bitcoin is digital scarcity without people with guns.
Where does its value come from?</p>

<p>Well, where does value of anything come from?
It comes from people.</p>

<p>Imagine a universe without humans.
How much is a star worth? How much is an atom of hydrogen worth?
These questions are absurd because value is subjective.
“Something is valuable” literally means “some <em>people</em> find it useful”.</p>

<p>People need money – a system of value transfer though time (store of value) and space (medium of exchange).
(Unit of account is the third function of money, but it arguably emerges if the other two work well.)
Bitcoin satisfies the demand for a money system for enough people to be worth around $10k apiece at the time of this writing.
Turns out, to achieve this, it has to be very “inefficient” and make the harder choice at every turn.
But as the type of information Bitcoin handles is so valuable (it is pure <em>value</em> itself), the expenses are worth it.</p>

<h2 id="is-money-just-another-content-type">Is money just another content type?</h2>

<p>I’m a long time fan of Andreas Antonopoulos.
His passion and dedication helped me comprehend the incredible beauty of Bitcoin design.
One of my favorite metaphors by Andreas is “<a href="https://youtu.be/3NqDNetEqmI">money as a content type</a>”.
However, while catchy and indeed applicable in programming contexts, it is not entirely accurate.</p>

<p>Content types like MP3 or JPG are just sets of rules that let a computer <em>interpret</em> a sequence of bits.
You may <em>try</em> to interpret a JPG file as text – this will most likely result in pages of unreadable symbols.
But fundamentally JPG bits are the same as TXT bits and even the same as EXE bits.</p>

<p>But you can’t <em>interpret bitcoin</em> as text.</p>

<p>The <a href="https://bitcoin.org/bitcoin.pdf">Bitcoin whitepaper</a> defines a coin as a chain of digital signatures.
Signatures, of course, can be represented as text and printed in hexadecimal.
But not <em>every</em> coin is a bitcoin.
To be <em>valid</em>, a coin must stem from the coinbase of some valid block in the <em>heaviest chain</em>, which started from a <em>particular</em> genesis block.</p>

<p>A digital representation of value does not work in vacuum.
It must <em>relate</em> to some agreed upon money system.
In the same way it makes no sense to say “pay me 100” without specifying the currency, it makes no sense to consider a chain of signatures without linking it to <em>the</em> Bitcoin system.
In fiat, we establish this relation (PKI, banking licenses, etc) and <em>then</em> trust that the system won’t fail, while it technically can.
I trust that the numbers in my bank account there are, all fiat sins notwithstanding, not part of an outright scam plot.
In Bitcoin, we establish this relation (full nodes, SPV nodes, trusted servers – whatever fits our security model) but no additional trust is required.
The validity of blocks can be verified independently, and the probabilistic uniqueness of history is established with proof-of-work.</p>

<h2 id="a-soviet-porcelain-factory">A Soviet porcelain factory</h2>

<p>When the Soviet Union <a href="https://en.wikipedia.org/wiki/Dissolution_of_the_Soviet_Union">collapsed</a>, its economic system was completely dysfunctional. [<sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>]
It was not just <em>inefficient</em> as you might expect from a centrally planned economy – it was impossible to perform basic financial tasks.
Instead of money, a porcelain factory would pay its employees in dishes and cups, which they sold for next to nothing on the streets, just to bump into empty shelves in a grocery store. [<sup id="fnref:4" role="doc-noteref"><a href="#fn:4" class="footnote" rel="footnote">4</a></sup>]</p>

<p>Bitcoin is a porcelain factory.
It pays salaries to its “employees” with what it produces. [<sup id="fnref:5" role="doc-noteref"><a href="#fn:5" class="footnote" rel="footnote">5</a></sup>]
However, this is that single case where this makes sense – the factory literally <em>produces money</em>.
The money-like objects Bitcoin spits out every ten minutes on average <em>turn out</em> to be exactly what people generally expect to be paid in.
Therefore it is possible to embed the reward for maintaining the system into the system itself.</p>

<p>Regular employees trust their employer to pay them and rely on an <em>external</em> legal system to enforce their job contract if they don’t.
Bitcoin miners, on the other hand, are not <em>expecting</em> a salary from anyone.
Their pay is indivisible from their performance.
There is no “boss” who could fail to pay, no company to go bankrupt.
This closed cryptoeconomic loop makes the whole thing working and independent of any external party.</p>

<p>To summarize: value is a special type of information.
Money is a system for transferring value.
Before Bitcoin, digital money required trust.
Satoshi leveraged a very special property of <em>value</em> to embed a reward mechanism into Bitcoin, creating a <em>closed</em> (hence independent) system.
I’d argue that this is the absolutely critical piece of the Bitcoin puzzle.
Let’s call these self-sustaining systems with built-in economic motivations – <em>cryptoeconomic</em> systems.</p>

<h1 id="why-is-my-facebook-data-valuable">Why is my Facebook data valuable?</h1>

<p>Let’s get back to the Web2 / Web3 discussion.</p>

<p>Facebook is one of the largest companies in the world, yet I don’t pay anything to use it.
How come?
This is quite a cliché already, but if you’re not paying for the product, you are the product.
More precisely, <a href="https://rakhim.org/2018/09/you-are-a-data-point/">you are a data point</a> which lets Facebook develop better products for its true clients – advertisers.
To attract data point (like you), Facebook maintains all those data centers and hires best developers to lure people into scrolling their feeds for hours on end.</p>

<p>Users’ data has no <em>immediate</em> value however.
It is more of a debt instrument: it promises returns, if you derive insights from it and sell them to someone.
The value of Facebook’s ad targeting system stems from a number of ingredients:</p>
<ul>
  <li>the huge amount of data (<a href="https://www.businessinsider.com/facebook-grew-monthly-average-users-in-q1-2019-4?r=US&amp;IR=T">1.5 billion people use Facebook daily</a>, tracked every split second);</li>
  <li>the monopoly on data (a Facebook’s competitor can hire smart engineers, but can’t train the algorithms on Facebook’s datasets);</li>
  <li>the proprietary nature of the algorithms.</li>
</ul>

<p>The loop is self-enforcing.
Facebook’s algorithms are developed by the best-in-generation engineers and trained on the world’s largest dataset.
Facebook can afford to hire the best engineers and run powerful servers because of high revenue, enabled by the best data and algorithms, and so on.</p>

<p>The Web3 question then is: can we break that circle?
Can we build a self-sustained, decentralized <em>cryptoeconomic</em> system on top of <em>data</em> to atone for the <a href="https://podcastnotes.org/2019/08/31/andreessen-crypto/">Internet’s original sin</a>?</p>

<p>I’ll try to answer this question in the next post.</p>

<div class="footnotes" role="doc-endnotes">
  <ol>
    <li id="fn:1" role="doc-endnote">
      <p>For Russian speakers: you can <a href="https://www.youtube.com/playlist?list=PLU_YQeud2Sg6gcV6HNg47vUU4dnCD5W0V">watch</a> and <a href="https://basicblockradio.com/e086/">listen to</a> our coverage of Berlin Blockchain Week in Basic Block podcast. I also participated in the ETHBerlin Zwei hackathon, helping with <a href="https://github.com/smartkek/dai-insured">a project around Maker DAO</a>. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:2" role="doc-endnote">
      <p>I will Facebook as metaphor for a Web2 service throughout this post, but the same applies to Google and others. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:3" role="doc-endnote">
      <p>Let me suggest <a href="https://www.goodreads.com/book/show/21351284-collapse-of-an-empire">Collapse of an Empire</a> by Yegor Gaidar if you’re interested in why that happened. <a href="#fnref:3" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:4" role="doc-endnote">
      <p>People also used all kinds of money substitutes. I remember mid-1990s Russian TV ads with prices in “<a href="https://ru.wikipedia.org/wiki/Условная_единица">у.е.</a>”, an euphemism for “USD converted to rubles at the date of purchase” invented due to a legal ban on price listings in <del>stablecoins</del> foreign currencies. <a href="#fnref:4" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:5" role="doc-endnote">
      <p>An interesting question to ponder is who are the employees of Bitcoin? The obvious question is miners, but it is only miners? What about early adopters who invested at $10 per BTC and have the resources to go Bitcoin full time or develop another blockchain? What about Coindesk journalists? Or countless <a href="https://github.com/s-tikhomirov/blockchain-podcasts">blockchain podcasters</a>, myself included? In some indirect way, all people working in the space have been paid, essentially, by a DAO named Bitcoin. Mind = blown. A nice topic for another post. <a href="#fnref:5" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
  </ol>
</div>]]></content><author><name></name></author><summary type="html"><![CDATA[This August, Berlin was the global center of all things decentralized. Thousands of blockchain enthusiasts gathered for the Berlin Blockchain Week – a series of conferences, meetups, and a hackathon. In this post, I’ll share my thoughts on Web3, which was the primary topic of the the first major event of the week – Web3 Summit. [1] For Russian speakers: you can watch and listen to our coverage of Berlin Blockchain Week in Basic Block podcast. I also participated in the ETHBerlin Zwei hackathon, helping with a project around Maker DAO. &#8617;]]></summary></entry><entry><title type="html">Eltoo</title><link href="https://s-tikhomirov.github.io//eltoo/" rel="alternate" type="text/html" title="Eltoo" /><published>2019-04-25T00:00:00+00:00</published><updated>2019-04-25T00:00:00+00:00</updated><id>https://s-tikhomirov.github.io//eltoo</id><content type="html" xml:base="https://s-tikhomirov.github.io//eltoo/"><![CDATA[<p>Continuing the journey through layer-two technologies, here is a summary of the paper “<a href="https://blockstream.com/eltoo.pdf">eltoo: A Simple Layer2 Protocol for Bitcoin</a>” by Christian Decker et al (see also: <a href="https://blockstream.com/2018/04/30/en-eltoo-next-lightning/">a summary in the Blockstream blog</a>).
Eltoo proposes a new construction for payment channels.[<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>]
It is not a fully-fledged protocol, rather, it only describes one crucial building block – state revocation mechanism.
As you might remember from <a href="/sok-off-the-chain-transactions">my summary</a> of “<a href="https://eprint.iacr.org/2019/360">SoK: Off the chain transactions</a>”, the crucial challenge in L2 protocol design is old state invalidation.
Lightning uses replace by revocation (in the SoK paper terms) which works in practice but has its drawbacks.
The construction is rather complex, and the intermediate states held by the two parties are different.
This inherent asymmetry prevents easily extending the protocol to support multi-party channels.
Eltoo suggest another, symmetric state revocation mechanism, which is arguably better modulo one crucial limitation: it depends on a non-existent SIGHASH_NOINPUT signature flag.
The good news is, this change can be implemented relatively easily via a soft fork and doesn’t seem to be very contentions.
If that happens, it would be possible to replace state revocation mechanism in the live Lightning network to Eltoo while preserving all other aspects (channel synchronization vis HTLCs, routing algorithms, etc).</p>

<h1 id="history-nsequense">History: nSequense</h1>

<p>It’s worth noting that Satoshi himself tinkered the problem of re-negotiating a transaction without broadcasting it.
Bitcoin transactions have an <em>nSequence</em> field, which was initially meant to act as a counter.
Miners were assumed to give priority to transactions with higher sequence numbers.
Honestly, it’s hard for me to grasp how Satoshi, who invented the brilliant economic game that has been securing Bitcoin for over a decade now, seriously thought sequence numbers could work.
Miners as rational agents simply choose the transaction with the highest fee in case of a double-spend attempt. [<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>]</p>

<p>In any case, miners even have some degree of plausible deniability: they may claim that they simply haven’t heard of the transaction with a higher sequence number.[<sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>]</p>

<h1 id="bitcoin-scripts">Bitcoin scripts</h1>

<p>Bitcoin is essentially a replicated state machine.
The <em>state</em> here is the set of unspent transaction outputs (<em>UTXO set</em>).
Transactions modify this set by removing some outputs from the set (“consuming”) and creating new ones.
Validity rules ensure that all transaction adhere to the same rules:</p>
<ul>
  <li>the sum of output values is not greater than the sum of input values;</li>
  <li>the script executes to <em>true</em>.</li>
</ul>

<p>On the protocol level, Bitcoin has no accounts and no balances, only scripts and transactions.
Bitcoin’s scripting system is not particularly intuitive.[<sup id="fnref:4" role="doc-noteref"><a href="#fn:4" class="footnote" rel="footnote">4</a></sup>]
A transaction pends outputs by providing <em>input scripts</em> which when concatenated with some unspent outputs evaluate to true.
Conditions under which bitcoins in the newly created outputs can be spent are described with <em>output scripts</em>.
A popular script template is “pay to public key hash” (P2PKH), which means “a valid signature corresponding to this public key”.[<sup id="fnref:5" role="doc-noteref"><a href="#fn:5" class="footnote" rel="footnote">5</a></sup>]</p>

<p>For example, for P2PKH the output script is “OP_DUP OP_HASH160 &lt;Bob’s Public Key Hash&gt; OP_EQUALVERIFY OP_CHECKSIG”.
The input script is “Bob’s Signature&gt; &lt;Bob’s Public Key&gt;”.[<sup id="fnref:6" role="doc-noteref"><a href="#fn:6" class="footnote" rel="footnote">6</a></sup>]
If you concatenate these scripts, imagine you’re a stack machine, and execute the commands, you’ll arrive at <em>true</em>, which means this input-output pair is legit (<a href="https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch06.asciidoc">Chapter 6 of Mastering Bitcoin</a> explains the process in detail).</p>

<p>More complex conditions can be encoded with pay-to-script-hash (P2SH) outputs.
Here, the output only specifies the <em>hash</em> of the script necessary to unlock the coins.
The spender provides the script, and the interpreter <em>first</em> checks the hash, and <em>then</em> executes the script.[<sup id="fnref:7" role="doc-noteref"><a href="#fn:7" class="footnote" rel="footnote">7</a></sup>]</p>

<h1 id="eltoo-on-chain-version">Eltoo: on-chain version</h1>

<p>The previous section was there just to give the necessary background.
Let’s return at the task at hand: a secure L2 state replacement mechanism.
The authors first introduce an <em>on-chain</em> version of Eltoo, where all intermediary states must be confirmed on-chain.
Then the authors “lift the protocol off the blockchain” and show how just with one little tweak to Bitcoin’s transaction engine enables fully-fledged Eltoo.</p>

<p>Similar to Lightning network, Eltoo is a three-stage protocol consisting of a setup phase, a negotiating phase, and a settlement phase.
The setup phase consists of transferring some coins to a 2-of-2 multisig address. [<sup id="fnref:8" role="doc-noteref"><a href="#fn:8" class="footnote" rel="footnote">8</a></sup>]</p>

<p>Both Alice and Bob generate and exchange <em>two</em> public-private key pairs: a settlement keypair and am update keypair.
Assume Alice funds the channel.
She creates a transaction spending her coins to a 2-of-2 multisig with either both update keys, or both settlement keys.
Before sending it to Bob, she requires him to sign a initial settlement transaction which spends the coins from the multisig back to Alice.[<sup id="fnref:9" role="doc-noteref"><a href="#fn:9" class="footnote" rel="footnote">9</a></sup>]</p>

<p>The funding output, as well as all successive update transaction outputs, can be spent in two ways (expressed as OP_IF branches in the script).
The first (true) branch is the settlement branch: it requires two signatures with <em>settlement</em> keys <strong>and</strong> imposes a <em>relative</em> timelock (i.e., is valid only after a certain number of blocks are created after the transaction which created the output was confirmed).
The second (false) branch is the update branch: it spends the output of the previous update transaction without any timelock.</p>

<p>Note the differences with Lightning:</p>
<ul>
  <li>information held by parties is <em>symmetric</em>;</li>
  <li>parties use <em>different</em> sets of keys for update and for settlement;</li>
  <li>a new update transaction spends the output of the <em>previous</em> update transaction (whereas in Lightning all update transaction re-spend the same 2-of-2 output of the funding transaction).</li>
</ul>

<p>The parties have the time until the timelock in the settlement branch expires to spend this output with a new update transaction (signed with their <em>update</em> keys).</p>

<p>Consider Figure 1:</p>

<p><img src="../images/posts/2019-04-25-eltoo-figure-1.png" alt="Christian Decker, Rusty Russell, Olaoluwa Osuntokun" /></p>

<p>I should confess that I often have trouble wrapping my head around such diagrams, as the meaning of an edge if often defined implicitly.
Here a rectangle is a transaction, a circle is a script, a (transaction – script) edge means “contains”, and a (script – transaction) edge means “spends”.
Note the key insight: <strong>many transactions can spend the same output</strong>.
Of course, at most one of them is ever confirmed on-chain, but multiple valid transactions can spend the same (P2SH / P2WPKH) output (probably executing different branches of the script).
So the parties can co-sign <em>one</em> transaction which spends a given output, exchange it without broadcasting it, and then co-sign <em>another</em> transaction spending <em>the same</em> output.
In fact this is exactly what they do: until the timeout in the settlement branch expires, they spend the latest output with a new update transaction.
If they want to close the channel, they either simply wait till the timeout expires and broadcast the latest settlement transaction, or co-sign a new update transaction and broadcast it immediately (cooperative close).</p>

<p>Here is how the update phase happens:</p>

<blockquote>
  <p>The update transaction effectively doublespends the settlement transaction before it becomes valid. As with the funding transaction, before signing and broadcasting the new update transaction, the two endpoints negotiate a new settlement transaction that spends the newly created contract output.</p>
</blockquote>

<p>Note that we are still dependent on on-chain confirmations of all intermediary update transactions!
So this is not an L2 protocol yet.
If only we had a way to preserve the security guarantees without broadcasting all intermediary update transactions…</p>

<h1 id="3-2-1-lift-off">3, 2, 1, lift-off!</h1>

<p>That is exactly what the authors do in what they call “lifting the protocol off the chain”.
Citing the <a href="https://blockstream.com/2018/04/30/en-eltoo-next-lightning/">blog post</a>,</p>

<blockquote>
  <p>The key insight in eltoo is that we can skip intermediate updates, basically connecting the final update transaction to the contract creation.</p>
</blockquote>

<p>But how is that even possible?
How can a transaction be valid if it doesn’t specify exactly which outputs it spends?</p>

<h2 id="sighash_noinput">SIGHASH_NOINPUT</h2>

<p>Bitcoin transaction includes the transaction hash and the output index of the outputs it spends.
Translated into English, a typical transaction looks like this:</p>
<ul>
  <li>I’m spending output number 0 from transaction 0xdead, here is the input script;</li>
  <li>I’m spending output number 1 from transaction 0xbeef, here is the input script;</li>
  <li>I’m creating an output, here is the output script.</li>
</ul>

<p>But what if a transaction didn’t have to commit to the exact outputs it’s spending?
In fact, it is possible:</p>

<blockquote>
  <p>signatures in Bitcoin transactions can be parameterized with the sighash-flag that specifies which parts of the transaction are committed to in the signature. By introducing a new sighash flag, SIGHASH_NOINPUT, it is possible to selectively mark a transaction as a floating transaction.</p>
</blockquote>

<p>(A transaction is called <em>floating</em> if it “can be bound to any previous transaction with matching scripts”.)</p>

<p>Let’s visualize an output as a secure vault.
A transaction opens a vault and re-distributes the money to other vaults.
A vault says: “whoever has the key can open me and do whatever they want with the content”.
A usual transaction says: “I have the key from <strong>this</strong> vault, so I’m opening it (and distributing the content into other vaults)”.
A SIGHASH_NOINPUT transaction says: “I have a key, but I don’t yet know which vault I will open with it – might be <strong>any</strong> vault where the key works”.
Hence, the <em>same</em> update transaction, once signed, can be <em>modified</em> to <em>attach</em> it to the required settlement transaction.</p>

<p>Is that all?
Not quite…</p>

<h2 id="ordering">Ordering</h2>

<p>As the authors point out,</p>

<blockquote>
  <p>Using the SIGHASH_NOINPUT flag for update transaction adds a lot of flexibility, however they are now too flexible.</p>
</blockquote>

<p>In particular, the current scheme allows replacing a new update transaction with an older one.
This unfortunate circumstance is explained by the fact that we completely discarded any notion of order.
Without committing to outputs, any update transaction can be attached to any other update transaction.
But this is not what we wanted:</p>

<blockquote>
  <p>by using SIGHASH_NOINPUT we have removed any commitment to the state we are replacing. We therefore have to selectively re-introduce some of the previous transaction’s details into the validation.</p>
</blockquote>

<p>Timelocks come to the rescue (again).</p>

<p>To check for timelocks, Bitcoin has two opcodes with totally intuitive names: OP_CHECKSEQUENCEVERIFY (aka OP_CSV) and OP_CHECKTIMELOCKVERIFY (aka OP_CTLV).
The former one is <em>relative</em>, and the latter one is <em>absolute</em>.
Eltoo uses absolute timelocks to order update transactions.
Currently, OP_CSV ensures that the current time is later than the one encoded in the output.
There are two ways to define absolute timelocks: as blockchain height, or as a UNIX timestamp.
In yet another example of technical elegance, Bitcoin uses the same field for these two cases.
The semantics is determined by the value of the field.
The current timestamp, at the time of writing, is around 1.5 billion (1 billion seconds is approximately 32 years).
The current block height at the time of this writing is a bit over 573 thousand.
So if the value in the nLocktime field is above 0.5 billion it is <em>interpreted</em> as a timestamp, otherwise as a block height (note: billion with a b, that is, Bitcoin can exist for another 10 thousand years until timelocks are broken).</p>

<p>Just if that wasn’t enough of a dirty hack, the authors suggest the following.
There are around 1 billion (and counting!) timestamps between 0.5 billion and <a href="https://www.unixtimestamp.com/">the current moment</a>.
All these timestamps are in the past, so absolute timelocks are irrelevant for them.
“I define this transaction to only be valid after 15:43 UTC on 8 October 1997” – well, it <em>is</em> already valid.
Therefore, we can repurpose the poor nLocktime field once more and assign the values in the 0.5 billion – 1.5 billion range the semantics of Eltoo state counters. [<sup id="fnref:10" role="doc-noteref"><a href="#fn:10" class="footnote" rel="footnote">10</a></sup>]</p>

<p>Anyway, just assigning some semantics to some field doesn’t make it enforceable (just as was the case with nSequence).
The authors suggest a workaround:</p>

<blockquote>
  <p>In order to achieve the limited binding for settlement transaction a new set of public keys A s,i and B s,i is derived that is specific to each state number. The key-pair derivation can easily be done with hierarchical deterministic key derivation as used by many existing Bitcoin wallets. This ensures that a settlement transaction can only be bound to the matching update transaction.</p>
</blockquote>

<p>The paper more fine-grained details on the compatibility with P2SH and P2WSH transactions and on fees, but the general construction can be summarized as follows: parties update the state using floating update transactions, which double-spend the output of the previous update transaction before the corresponding settlement transaction is valid.</p>

<h1 id="eltoo-vs-lightning">Eltoo vs Lightning</h1>

<h2 id="structure-of-intermediary-transactions">Structure of intermediary transactions</h2>

<p>The Eltoo authors put an emphasis on the symmetric nature of their protocol.
Thinking about it a bit, I came to realize the fundamental distinction between Eltoo and Lightning revocation protocols.</p>

<p>Lightning transactions re-spend the same output (of the funding transaction) again and again.
To prevent an old state from being confirmed, the <em>toxic</em> information is used: if Alice broadcasts an old state, Bob has the revocation key allowing him to take all the money from the channel.
In Eltoo, on the contrary, intermediary transactions are organized <em>linearly</em>.
The first update transaction spends the output from the funding transaction, and the settlement transaction spends the output of the latest update transaction.
We don’t need toxic information anymore, because transaction are (temporarily) linked linearly.
Only when the channel closes, the latest update transaction is bound to the initial funding output.
This allows for the symmetry of the information held by the parties.</p>

<p>Note that in Eltoo, as well as in Lightning, the victim must be online to present a “double-spending” update transaction if an old settlement transaction is broadcast.
And, similarly to Lightning, only payments larger than the L1 fee are economically secure.</p>

<h2 id="the-cost-of-timelocks">The cost of timelocks</h2>

<p>In Eltoo, the settlement part of every output script is time-locked.
The authors acknowledge the related trade-off:</p>

<blockquote>
  <p>Notice that choosing the correct timeout for the settlement branch is a trade-off. It must be chosen high enough to guarantee that any subsequent update is confirmed before the settlement transaction becomes valid. On the other hand this timeout is also the time participants have to wait before funds are returned to their sole control should the other participant stop cooperating.</p>
</blockquote>

<p>In my opinion, this is the biggest drawback of Eltoo: a channel expires after some time if both parties do nothing.
Of course, if both parties are cooperative and want to maintain the channel, they just won’t broadcast the now-valid settlement transaction, but this can not be guaranteed.
Lightning, on the other hand, maintains the state of a channel indefinitely: both parties can go offline, return in a year, and continue updating the channel.
Does this lead to inapplicability of Eltoo in any of the Lightning use cases, or vice versa?
If state revocation technique is just a replaceable module in a larger L2 protocol, can we envision a future where we have multiple implementations of all L2 building blocks (networking, routing, state invalidation…) and can combine them to better suit our particular set of constraints?..</p>

<h1 id="conclusion">Conclusion</h1>

<p>Eltoo is a cleverly designed protocol.
I’m looking forward to the <a href="https://github.com/bitcoin/bips/blob/master/bip-0118.mediawiki">SIGHASH_NOINPUT BIP</a> being discussed and hopefully implemented, enabling Eltoo implementation.</p>

<p>Eltoo and SIGHASH_NOINUT is an interesting example of blockchain protocol design evolution.
As far as I can tell, the initial code published by Satoshi was not of enterprise quality.
It took years of engineering effort to separate it logically (wallet stuff from networking stuff from consensus stuff) and fix bugs.
But also on a higher level, Bitcoin initially was probably too “strict”.
For example, transactions must commit to outputs they are spending.
How could it be otherwise?
Seems logical, until someone explores the protocol deeply enough to realize that we can remove this restriction and enable new functionality without sacrificing exiting security guarantees.
Segwit is another improvement of this kind.
What else can we strip from the protocol to enable new use cases without weakening the security model?</p>

<p>Another lesson from reading about Bitcoin scripts is that Bitcoin is full of dirty hacks.
I understand the desire of some talented teams in the space to take the brilliant idea by Nakamoto, derive a new system based on it, and implement it cleanly from scratch (e.g., Cardano with its heavy emphasis on formal verification).
For what it’s worth, I have a feeling that Bitcoin’s network effects are already so strong, that it would be really hard to compete with it in its niche, despite all its inefficiencies.
The question is, what are other niches for blockchains, apart from digital money?</p>
<div class="footnotes" role="doc-endnotes">
  <ol>
    <li id="fn:1" role="doc-endnote">
      <p>The authors seem to prefer stylized inscription “eltoo”, in all small letters, but I prefer capitalizing proper nouns for easier reading, sorry. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:2" role="doc-endnote">
      <p>At least in the basic model without non-monetary incentives like a wish to harm Bitcoin for political reasons, or fees outside the protocol (also known as bribes). <a href="#fnref:2" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:3" role="doc-endnote">
      <p>Another aspect of the original Bitcoin design which makes me cringe is “send to IP” – see <a href="https://letstalkbitcoin.com/blog/post/lets-talk-bitcoin-393-open-neutral-decentralized-borderless">this LTB episode</a> where Andreas Antonopoulos explains why this is a very, very bad idea. <a href="#fnref:3" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:4" role="doc-endnote">
      <p>Which may explain why Ethereum gained traction so quickly: it appealed to a large community of web developers who were not willing to dive into peculiarities of stack-based languages inspired by Forth. <a href="#fnref:4" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:5" role="doc-endnote">
      <p>More precisely, to the public key hash – the key is hashed to prevent certain privacy attacks and to add a layer of protection against the quantum threat: quantum computers may be able to derive a private key from the public key, but can’t reverse hashes. <a href="#fnref:5" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:6" role="doc-endnote">
      <p>As the authors node, “due to this case being so common, the spending condition is commonly referred to as <em>scriptPubKey</em>, and the input script is referred to as <em>scriptSig</em>”. Oh, I love how intuitive Bitcoin terminology is! Was it supposed to act as a filter against people without at least a Master’s degree? <a href="#fnref:6" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:7" role="doc-endnote">
      <p>By the way, P2SH is recognized by the script format only. That is, if the interpreter sees a script in the form “check that the hash of the argument is equal to this”, it <em>also</em> interprets the argument itself as a script! Oh, how much engineering elegance! Wait till we get to timelocks, this will get <em>curiouser and curiouser</em>. <a href="#fnref:7" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:8" role="doc-endnote">
      <p>While the final protocol is extensible to more parties, the authors describe the 2-party version first. <a href="#fnref:8" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:9" role="doc-endnote">
      <p>Segwit enabled the ability to build protocols including spending not-yet-confirmed outputs, as their identifiers are not malleable anymore. <a href="#fnref:9" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
    <li id="fn:10" role="doc-endnote">
      <p>Reminds me of <a href="http://hackingdistributed.com/2016/04/05/how-software-gets-bloated/">this post</a> by Emin Gün Sirer on “clever” hacks in multi-layered protocols and the following software bloat… <a href="#fnref:10" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
  </ol>
</div>]]></content><author><name></name></author><summary type="html"><![CDATA[Continuing the journey through layer-two technologies, here is a summary of the paper “eltoo: A Simple Layer2 Protocol for Bitcoin” by Christian Decker et al (see also: a summary in the Blockstream blog). Eltoo proposes a new construction for payment channels.[1] It is not a fully-fledged protocol, rather, it only describes one crucial building block – state revocation mechanism. As you might remember from my summary of “SoK: Off the chain transactions”, the crucial challenge in L2 protocol design is old state invalidation. Lightning uses replace by revocation (in the SoK paper terms) which works in practice but has its drawbacks. The construction is rather complex, and the intermediate states held by the two parties are different. This inherent asymmetry prevents easily extending the protocol to support multi-party channels. Eltoo suggest another, symmetric state revocation mechanism, which is arguably better modulo one crucial limitation: it depends on a non-existent SIGHASH_NOINPUT signature flag. The good news is, this change can be implemented relatively easily via a soft fork and doesn’t seem to be very contentions. If that happens, it would be possible to replace state revocation mechanism in the live Lightning network to Eltoo while preserving all other aspects (channel synchronization vis HTLCs, routing algorithms, etc). The authors seem to prefer stylized inscription “eltoo”, in all small letters, but I prefer capitalizing proper nouns for easier reading, sorry. &#8617;]]></summary></entry><entry><title type="html">SoK: Off the chain transactions</title><link href="https://s-tikhomirov.github.io//sok-off-the-chain-transactions/" rel="alternate" type="text/html" title="SoK: Off the chain transactions" /><published>2019-04-17T00:00:00+00:00</published><updated>2019-04-17T00:00:00+00:00</updated><id>https://s-tikhomirov.github.io//sok-off-the-chain-transactions</id><content type="html" xml:base="https://s-tikhomirov.github.io//sok-off-the-chain-transactions/"><![CDATA[<p>Here is papers I’ve been waiting for for quite a while.
Thinking about it, I’d be happy to have (co-)written it, had I directed my research into this topic a bit earlier.
Today’s summary is based on a systematization of knowledge (SoK) paper by Lewis Gudgeon et al entitled “<a href="https://eprint.iacr.org/2019/360">SoK: Off the chain transactions</a>”.
The readers of this blog must be familiar with some of the challenges in layer-two protocols which I outlined in previous paper summaries.
But as in any rapidly developing field, information is dispersed across various media and is getting outdated quickly.
The authors summarize and systematize the challenges faced by the developers of layer-two protocols and compare the existing solutions.
I definitely gained at least some of the mental clarity essential for digging deeper and contributing to the field.
Read the whole thing to get general view of what problems are out there and how various proposals are tackling them.
And the 136 references will fill your “read later” folder with papers for months to come!</p>

<h1 id="layers-and-myths">Layers and myths</h1>

<p>Blockchains scale poorly.
There are multiple approaches to improve the efficiency of the base layer (consensus algorithm).
But updating an existing blockchain is hard (if it’s decentralized).
Layer-two protocols are a class of scaling solutions orthogonal to consensus improvements.
Their main feature is that they don’t require any modifications on layer-one, taking it’s security guarantees as assumptions.</p>

<p>The authors outline the four “myths” regarding layer two:</p>
<ol>
  <li>blockchains can’t scale without advances in layer-one;</li>
  <li>layer-two solutions are secure only if fully collateralized;</li>
  <li>off-chain transactions are private by default;</li>
  <li>blockchain transaction fees depend on transaction size or computational complexity, not its monetary value.</li>
</ol>

<p>The very word “myth” calls for the verb “debunk”, but this is not what will happen: rather, the authors provide insights regarding the “myths” throughout the paper.</p>

<p>What are these “layers” anyway?
The authors propose the following classification:</p>
<ul>
  <li>hardware layer (layer “minus-one”) account for the fact that hardware may provide “trusted execution environment” like Intel SGX (which shifts the security assumptions towards the hardware manufacturer and alleviates many problems of protocols designed to be run on general purpose hardware);</li>
  <li>layer-zero is the peer-to-peer network (though the paper focuses on permissionless networks, the authors point out that the insights regarding layer-two are also applicable to the permissioned ones);</li>
  <li>layer-one is the consensus algorithm;</li>
  <li>layer-two are protocols built on top of layer-one, of which the authors define three types.</li>
</ul>

<p>What do we mean by “built on top”?</p>

<blockquote>
  <p>Layer-two protocols typically assume two properties from the blockchain layer: integrity (i.e. only valid transactions are added to the ledger) and eventual synchronicity with an upper time bound (i.e. a valid transaction is eventually added to the ledger, before a critical timeout).</p>
</blockquote>

<p>The authors separate layer-two protocols into two groups: channels (payment and state) and commit-chains.</p>

<h1 id="channels">Channels</h1>

<p>A channel is a type of a second-layer protocol where parties</p>

<blockquote>
  <p>consent to state updates unanimously by exchanging authenticated state transitions off-chain.</p>
</blockquote>

<p>There are two kinds of channels (though the division seems a bit artificial from the purely theoretical standpoint, though it does reflect the current reality): payment channels and state channels.</p>

<p>The workflow of a payment channel consists of three phases:</p>

<ol>
  <li>channel establishment: the parties lock up collateral on-chain;</li>
  <li>channel transitions: the parties co-sign state updates and exchange them off-chain;</li>
  <li>channel closure: the parties finalize the latest state on-chain (collaboratively or via a dispute mechanism).</li>
</ol>

<p>The key design challenge for both payment and state channels is to prevent parties from submitting an outdated state to the blockchain.
There are four state replacement techniques:</p>
<ul>
  <li>replace-by-incentive (RbI). In a one-way payment channel where Alice pays Bob, Bob will only submit the latest state to the blockchain because it will yield him the highest value. Clearly, it doesn’t work with a bidirectional channel: if Bob pays back to Alice, he still possesses an outdated but valid state and can submit it to the blockchain effectively stealing from Alice.</li>
  <li>replace-by-timelock (RbT). In this scheme, each new update is timelocked (is valid only after a certain timestamp). Every new update must have the timelock closer to the present by a safe margin. This guarantees that the interested party will be able to submit the latest state to the blockchain before any outdated states are even valid. The drawback is that the lifetime of RbT is limited by the first timelock.</li>
  <li>replace-by-revocation (RbR). This is what Lightning is built upon. When agreeing on a new balance distribution in a channel, Alice and Bob effectively invalidate the previous state. If one of them tries to submit an outdated state to the blockchain, another party will be able to take the whole collateral. The drawback here is that parties are supposed to be constantly monitoring the blockchain.</li>
  <li>replace-by-version (RbV). This approach works best if the layer-one is stateful (like Ethereum). Each new state has an incrementing counter; the version with the highest counter value is valid (disputes can be settled by a smart contract).</li>
</ul>

<p><img src="../images/2019-04-17-sok-off-the-chain-transactions-table-1.png" alt="Lewis Gudgeon, Pedro Moreno-Sanchez, Stefanie Roos, Patrick McCorry, Arthur Gervais" style="width:600px;" /></p>

<p>The first three techniques are used in payment channels; the fourth one is the basic idea behind state channels.</p>

<h2 id="payment-vs-state-channels">Payment vs state channels</h2>

<p>The authors outline multiple proposed payment channel constructions, most of them are, in my opinion, only of a historical value.
The very first ideas for unidirectional channels date back to Satoshi himself, then there was a construction known as Spilman channels, then Decker and Wattenhofer proposed <a href="https://www.tik.ee.ethz.ch/file/716b955c130e6c703fac336ea17b1670/duplex-micropayment-channels.pdf">Duplex micropayment channels</a>…
Eventually, the only implemented payment channel scheme is Lightning (<a href="/how-lightning-works">see part 1 of my series on it</a>).
The authors point out the key drawbacks in the RbR-based Lightning construction:</p>

<blockquote>
  <p>RbR is the first channel design to require both parties to remain online and fully synchronized with the blockchain to observe  malicious closure attempts. &lt;…&gt; RbR introduces unfavorable implications for third-party watching services &lt;…&gt; entails O(N) storage.</p>
</blockquote>

<p>Can we improve this by using RbV in payment channels somehow? Decker et al propose</p>

<blockquote>
  <p>Eltoo to support RbV in UTXO-based blockchains through the use of floating transactions</p>
</blockquote>

<p>State channels generalized ideas pioneered by payment channels and apply similar construction to arbitrary computations.
The authors distinguish state channel constructions with <em>closure disputes</em> and <em>command disputes</em>.
To be honest, I didn’t understand this paragraph, as the notion of “installing / uninstalling” an on-chain application needs clarification.
The two most prominent constructions (at least those with formal security proofs) are <a href="https://eprint.iacr.org/2017/635">Perun</a> (closure disputes) and <a href="https://arxiv.org/abs/1702.05812">Sprites</a> (command disputes).</p>

<h1 id="channel-synchronization">Channel synchronization</h1>

<p>Up to this point, we were only talking about single channels, but what about channel networks?
The authors suggest the term <em>channel synchronization</em> to denote techniques to logically connect updates to multiple channels.
A well-known example are hash-time-locked contracts (HTLCs) used in Lightning: channels along the path get atomically rebalanced if the receiver reveals a hash preimage or a timeout expires.
Another approach, suited for stateful blockchains, is a global <em>preimage manager</em> – a smart contract which keeps track of the revealed preimages (see the Sprites paper).
The key advantage of the preimage manager is that it lowers the requirements for capital lockup from quadratic to linear in the number of hops (in the worst case).
In Lightning, the timelock at each next hop must be different from the previous one by a security margin to allow an intermediary node to confirm the correct balance on-chain in case of a dispute.
In a stateful blockchain with a preimage manager, the deadline can be the same for all channels in the path.
Other approaches to channel synchronization include <em>scriptless multi-hop locks</em> (see “General state channel networks”) and <em>virtual channels</em>.</p>

<h1 id="routing">Routing</h1>

<p>An important question in channel networks is how to find a path to the receiver capable of delivering the payment.
This tasks becomes tricky if we consider the resource limitations of end-user devices (smartphones) and privacy requirements (it would be bad to reveal all intermediate states in every channel, hence failures due to insufficient capacity are inevitable).</p>

<p>Shall we use onion routing for better privacy, one might suggest?
Here is a surprising insight:</p>

<blockquote>
  <p>Some algorithms involve onion routing, which requires the random selection of nodes in a path to achieve its anonymity guarantees. As routing algorithms do not select the nodes randomly, it remains unclear if onion routing provides privacy in the context of payment channels.</p>
</blockquote>

<p>Turns out, onion routing in Lightning may end up being just security theater?..
Mind = blown.
Looks like yet another issue stemming from the fundamental “information vs value” distinction.
Worth further investigation!</p>

<p>The authors define two approaches to routing: with global and local view.
Lightning and Raiden use source routing with global view: the sender is expected to have a full (potentially slightly outdated) snapshot of the network.
Local view routing algorithm fall into four categories: distributed hash tables (<a href="https://www.celer.network/">Celer</a>’s cRoute), flow algorithms, landmark routing (<a href="/silentwhispers">SilentWhispers</a>, Flare), and network embeddings (<a href="/speedymurmurs">SpeedyMurmurs</a>).
Another dimension to compare routing algorithms is along the <em>circuit switching</em> (atomic payments) vs <em>packet switching</em> (splitting a payment into smaller chunks and transferring them through different paths).
Routing for channel networks seems to be a very promising area of research, as</p>

<blockquote>
  <p>no routing algorithm fulfills all desired criteria.</p>
</blockquote>

<p><img src="../images/2019-04-17-sok-off-the-chain-transactions-table-2.png" alt="Lewis Gudgeon, Pedro Moreno-Sanchez, Stefanie Roos, Patrick McCorry, Arthur Gervais" /></p>

<h1 id="commit-chains">Commit-chains</h1>

<p>Commit chains are the second category of layer-two protocols, alongside channels.
In contrast to channels,</p>

<blockquote>
  <p>commit-chains are maintained by one single party that acts as an intermediary for transactions.</p>
</blockquote>

<p>The two proposals the authors review are <a href="https://eprint.iacr.org/2018/642">NOCUST</a> (an account-based commit-chain) and Plasma (a UTXO-based commit-chain).
More specifically, they focus on Plasma Cash as the most mature Plasma flavor (I love <a href="https://youtu.be/suMDB34GXZ4">this video</a> of Karl Floersch explaining it!).</p>

<p>The general  workflow for commit-chains is as follows.
An <em>operator</em> collects commit-chain transactions and periodically <em>commits</em> to the latest state.
There is no three-step lifecycle (open – transact – close) as in channels: the application is always on once launched.
Users can anytime withdraw their funds to the layer-one chain.</p>

<p>There are two ways for users to verify that their transactions are reflected correctly in the latest state commitment: Merkle proofs and zero-knowledge proofs.
The distinction is that Merkle root commitments “do not self-enforce”, whereas</p>

<blockquote>
  <p>ZKPs enforce consistent state transitions on-chain.</p>
</blockquote>

<p>If I understood this correctly, we can theoretically encode the rules into the layer-one contract saying that the state transition is not valid unless accompanied by a zero-knowledge proof of its correctness.
On the other hand, I don’t quite see how this can prevent censorship (an operator refusing to process chosen transactions as if they were not requested).</p>

<h1 id="security-and-privacy">Security and privacy</h1>

<p>Layer-two protocols introduce new security / privacy challenges.
Despite a common belief, L2 transactions are not absolutely private by default (hardly anything is).
Sure, they are not permanently recorded in a globally replicated database for all chainalyses of the future to analyze.
But the privacy problem is far from solved.
Some of the relevant proposals mentioned in the paper are <a href="https://eprint.iacr.org/2016/575">TumbleBit</a>, <a href="https://eprint.iacr.org/2016/701">Bolt</a> (the anonymous channels from the Zcash team, not the Lightning specification), and Rayo / Fulgor (this blog has a <a href="/concurrency-privacy-payment-channel-networks/">summary of those</a>).</p>

<p>The listed security threats include:</p>
<ul>
  <li>the requirement to keep keys in a hot wallet;</li>
  <li>the requirement to constantly be online;</li>
  <li>the problem of mass exits (bank runs for the new age – everybody is trying to exit the malicious layer-two system, but the layer-one can’t handle the load, timeouts expire, preventing honest dispute resolution);</li>
  <li>high cost of on-chain proof verification (650k gas on Ethereum for a single ZKP!);</li>
  <li>wormhole attack (two malicious nodes along the same path short-cut the preimage off the protocol, taking the fees from nodes between them);</li>
  <li>capital lock-up attack (start a payment but never reveal the preimage; all intermediary nodes pay the opportunity cost of locked channel capacities).</li>
</ul>

<p>An interesting observation is the finality vs collateral trade-off between channels and commit-chains:</p>

<blockquote>
  <p>Unlike previously discussed layer-two protocols, the intermediary commit-chain operator does not require on-chain collateral to securely route a payment &lt;…&gt; commit-chain transactions do not offer instant transaction finality (as in channels) but eventual finality after commit-chain transactions are recorded securely in an on-chain checkpoint.</p>
</blockquote>

<p>The way I think about this is that in channels intermediary nodes provide <em>capacity</em>, therefore, they must lock it up.
The meaning of the phrase “Alice sent 1 coin to Bob via Charlie” is actually: Alice send 1 coin to Bob, <em>and</em> Bob sent 1 coin to Charlie, with atomicity enabled by the protocol.
But Charlie must <em>have</em> at least 1 coin to transfer to Bob while he waits to be able to pull 1 coin from Alice!
On the contrary, in commit-chains, user have to <em>trust</em> the operator for all actions after the latest commitment.
Because of these additional trust assumptions, the system can operate <em>without collateral</em>: a Plasma operator implementing a payment service doesn’t need to have any coins to be able to update its internal database and commit to the next state.
On the other hand, operators may optionally put up collateral which they would lose in case of fraud to boost users’ confidence.
Moreover, by introducing some trust, we mitigate the mass exit problem:</p>

<blockquote>
  <p>commit-chains do not require a deadline for users to withdraw their coins, mitigating the transaction fee bidding war.</p>
</blockquote>

<p>This is possible because the operator doesn’t hold any collateral which is locked in case of a dispute and must be unlocked eventually (hence timelocks)!</p>

<p>Another very interesting observation is that layer-two makes fees <del>great</del> dependent on the transaction <em>value</em> again.
In traditional finance, it’s common to charge fees as a percentage of the transaction value.
Bitcoin has a completely different model: transactions compete for space in blocks which is limited in terms of <em>bytes</em>, not coins.
Therefore, a common unit of account for Bitcoin transaction is satoshis per byte.
It costs about the same to transfer one dollar or a million dollars, if the transaction script has the same structure.
On the contrary, layer-two re-introduces the <em>value semantics</em> into protocols: relaying a payment requires liquidity, and intermediary nodes pay more in opportunity costs of locked capital when transferring a million dollars as opposed to one dollar.
This suggests that the layer-two fee economics will look very different from layer-one (and we haven’t fully explored the latter yet…).</p>

<h1 id="conclusion">Conclusion</h1>

<p>I highly recommend reading the whole paper (maybe multiple times, as it’s quite dense).
There is just so much work to do in layer-two!
We have barely scratched the surface.
I hope to take part in this exploration soon.</p>]]></content><author><name></name></author><summary type="html"><![CDATA[Here is papers I’ve been waiting for for quite a while. Thinking about it, I’d be happy to have (co-)written it, had I directed my research into this topic a bit earlier. Today’s summary is based on a systematization of knowledge (SoK) paper by Lewis Gudgeon et al entitled “SoK: Off the chain transactions”. The readers of this blog must be familiar with some of the challenges in layer-two protocols which I outlined in previous paper summaries. But as in any rapidly developing field, information is dispersed across various media and is getting outdated quickly. The authors summarize and systematize the challenges faced by the developers of layer-two protocols and compare the existing solutions. I definitely gained at least some of the mental clarity essential for digging deeper and contributing to the field. Read the whole thing to get general view of what problems are out there and how various proposals are tackling them. And the 136 references will fill your “read later” folder with papers for months to come!]]></summary></entry><entry><title type="html">Routing cryptocurrency with the Spider network</title><link href="https://s-tikhomirov.github.io//spider-network/" rel="alternate" type="text/html" title="Routing cryptocurrency with the Spider network" /><published>2019-04-13T00:00:00+00:00</published><updated>2019-04-13T00:00:00+00:00</updated><id>https://s-tikhomirov.github.io//spider-network</id><content type="html" xml:base="https://s-tikhomirov.github.io//spider-network/"><![CDATA[<p>Let’s continue our journey though recent paper which suggest ways to optimize routing in payment channel networks.
In previous posts, we looked an <a href="/silentwhispers">SilentWhispers</a> and <a href="/speedymurmurs">SpeedyMurmurs</a>.
Both approaches emphasized privacy as an important goal, but employed different constructions: landmark-based routing (SW) and embedding-based routing (SM).
Today let’s look into a paper entitled “<a href="https://arxiv.org/abs/1809.05088">Routing cryptocurrency with the Spider network</a>” (2018) by Sivaraman et al.</p>

<h1 id="introduction">Introduction</h1>

<p>I won’t spend much time summarizing the introduction sections, as the points the authors underline are more or less the same in every paper in this area: blockchains are cool but not scalable, second-layer solutions are coming to the rescue, but how do we find paths?</p>

<p>The authors outline the existing approaches to routing in PCNs, listing all the usual suspects: the generic max-flow algorithm, Flare, SilientWhispers, and SpeedyMurmurs.
In all previous approaches, however, payment atomicity was considered “a red line”: a payment must either go through or fail, tertium non datur.
The key idea of the authors is to weaken this requirement, and optimize for two separate variables: the success ratio (the share of transactions successfully processed) and the success volume (the total monetary value of the transactions successfully processed).
More concretely, the Spider routing algorithm</p>

<blockquote>
  <p>actively accounts for the cost of channel imbalance by preferring routes that rebalance channels.</p>
</blockquote>

<p>The authors identify the key challenge to continuous operation of PCNs: if a channel is consistently utilized in one direction more then in the other, it eventually gets depleted, and requires an on-chain transaction (with an on-chain fee) to continue operating.
To address the challenge, they introduce another approach to modeling the whole problem, which I find clever and insightful.</p>

<p>The authors also emphasize one crucial security assumption in PCNs, which is <a href="https://medium.com/@peter_r/visualizing-htlcs-and-the-lightning-networks-dirty-little-secret-cb9b5773a0">often overlooked</a>:</p>

<blockquote>
  <p>the underlying cryptography backing payment channels assumes that transactions on the payment channels are larger than the blockchain transaction fee to ensure that broadcasting the true balance is profitable.</p>
</blockquote>

<h1 id="model">Model</h1>

<p>Previous approaches, such as SilentWhispers (SW) and SpeedyMurmurs (SM), operate under the paradigm of <em>periodic rebalancing</em>.
Nodes establish channels and start using them.
The network as a whole only knows the initial total channel balances.
As channels are being used, their capacity distributions deviate more and more from the initial state.
As a consequence, the share of payments failed due to insufficient capacity also rises.
To make the network useful again, rebalancing is required.
Rebalancing either happens once every epoch (SW), or “on-demand” (SM).
In both cases, this is a distinct process, separated from usual transaction routing.</p>

<p>The authors of Spider have a more ambitions vision: a payment network which doesn’t need rebalancing at all!
Rebalancing in Spider, instead of being a separate process, happens naturally as payments are routed.</p>

<h2 id="an-example">An example</h2>

<p>The usual way to model a PCN is a graph where nodes represent peers and edges represent channels (usually directed and weighted according to capacity).
Let’s call it the network graph, as it represents the existing topology of the network.
The authors separate the two questions:</p>

<ol>
  <li>What is the current structure of the network? The answer is the <em>network</em> graph.</li>
  <li>How do the participants <em>want</em> to use the network? The answer is the <em>payment</em> graph.</li>
</ol>

<p>The payment graph shows the intentions of the peers: how much value they collectively want to transfer in which directions.
Money flows are modeled not as individual atomic transactions, but as constant flows (which can be approximated with a series of <em>unit payments</em>).
The key question is, given a <em>topology</em> graph <strong>and</strong> a <em>payment</em> graph, how many payments can we satisfy?
Alternatively, given a payment graph, what is the optimal topology graph?
Or, given a topology graph, can we influence the payment graph using fees to make the network balanced?</p>

<p>Consider a fully connected network of 3 nodes: Alice, Bob, and Charlie.
Say, the initial state is three channels with capacities of 1 bitcoin on one side: Alice to Bob, Bob to Charlie, and Charlie to Alice.
Alice wants to transfer 1 bitcoin per day to Bob, Bob – 1 bitcoin per day to Charlie, Charlie – 1 bitcoin per day to Alice.
If every node chooses the shortest path, all 3 channels will be depleted after 1 day.
But if, instead, one of the “payment streams” is routed sub-optimally (Charlie – Bob – Alice instead of Charlie – Alice), the flow of funds in one direction would offset that in the other, and the network would be able to run indefinitely (assuming zero fees).</p>

<p>The authors give a somewhat more elaborate “motivating example” in the beginning of Section 5 to illustrate this point:</p>

<p><img src="../images/2019-04-13-spider-network-payment-network-graphs.png" alt="Vibhaalakshmi Sivaraman, Shaileshh Bojja Venkatakrishnan, Mohammad Alizadeh, Giulia Fanti, Pramod Viswanath" /></p>

<p>The goal is to maintain the network in a “balanced” condition, that is, such that every node has equal incoming and outgoing value flows.
Let’s define “balanced” transaction rate as the sum of flows through all nodes which can be maintained indefinitely.
Turns out, if every source node chooses the shortest path as a destination, the overall throughput wouldn’t be optimal!
In a sample network of 5 nodes, they show how the “balanced” transaction rate is smaller than the maximum possible if all nodes choose optimal routes for their transactions (similar to my previous example with 3 nodes).</p>

<p>The authors than show that any payment graph can be decomposed into two components: the <em>circulation</em> and the <em>DAG</em>.
These two graphs have the same nodes as the payment graph, but the weight of each edge is split between the two components (it may be zero in one of them).</p>

<p><img src="../images/2019-04-13-spider-networ-circulation-and-DAG.png" alt="Vibhaalakshmi Sivaraman, Shaileshh Bojja Venkatakrishnan, Mohammad Alizadeh, Giulia Fanti, Pramod Viswanath" /></p>

<p>All flows in circulation are balanced at every node (incoming and outgoing flows are equal).
All what’s left goes into the DAG.
The authors show that for every payment graph there is a <em>maximum circulation graph</em> (with the highest <em>transaction rate</em>), and for this circulation there is a network graph which achieves the maximum transaction rate.
The total throughput would depend only on the locked-up capacity compared to the “settlement delay” – the time it takes for the receiver’s secret to propagate to the routing node. Until that moment, the funds are “in flight” and can’t be use for other transfers.
Consequently, no network graph achieves transaction rate higher than that of the circulation.</p>

<h2 id="adding-on-chain-rebalancing">Adding on-chain rebalancing</h2>

<p>What happens if a payment channel does get depleted?</p>

<ol>
  <li>If I lack capacity on the local side (I want to send more), I can top up my side of the channel with an on-chain transactions (not sure this is currently implemented in Lightning, but this assumption is useful for modeling).</li>
  <li>If I lack capacity on the remote side (I want to receive more), I can either spend some coins, or ask my counterparty to top up their side of the channel. Assuming there is no fiat world, and my counterparty is not particularly generous, I’m expected to reimburse them with, again, an on-chain transaction (see 1).</li>
</ol>

<p>To reflect these options, the authors add the cost of on-chain rebalancing to the picture.
Putting all parts together (and accounting for the off-chain as well as on-chain fees), they come up with a system of equations which form a linear programming (LP) problem: our goal is to maximize the throughput given a set of constraints.
Solving the equations with the primal-dual algorithm, the authors derive the formula for the optimal fee structure that achieves the maximal transaction rate.
They also note that the used algorithm may converge too slowly if the network conditions are changing rapidly, and suggest a less precise but faster-converging alternative (“waterfilling”).</p>

<h1 id="evaluation">Evaluation</h1>

<p>The authors compare their approach (with both the LP and waterfilling algorithms) with alternatives: SW, SM, and max-flow.
The set of transaction from Ripple acts as a real-world dataset (as in the previous papers on the subject).</p>

<p>I’ll come back to it later.</p>

<p>The first experiment compared the performance of the six algorithms (Spider LP, Spider Waterfilling, Max-flow, Shortest path, SW, and SM) for atomic payments only with two network topologies: <a href="http://www.topology-zoo.org/">ISP-like</a> (not clear what it means exactly though) and a subset of Ripple.
See the results in the figure below; I’m surprised that SM performed so poorly and can’t come up with a reason why (comments welcome!).</p>

<p><img src="../images/2019-04-13-spider-network-evaluation.png" alt="Vibhaalakshmi Sivaraman, Shaileshh Bojja Venkatakrishnan, Mohammad Alizadeh, Giulia Fanti, Pramod Viswanath" /></p>

<p>Note also that Spider only operates in benign conditions (i.e., in a permissioned network):</p>

<blockquote>
  <p>[O]ur design does not address incentives &lt;…&gt;. It also doesn’t account for adversarial routers relaying wrong values to the sender.</p>
</blockquote>

<p>Isn’t it the most important point?
How does Lightning handle it by the way?
Wouldn’t be surprised if it also doesn’t (looks exiting: so much space for research!).</p>

<h1 id="conclusion-and-the-road-traffic-analogy">Conclusion and the road traffic analogy</h1>

<p>The Spider network incorporates one of the key ideas from information networks (packet switching), which allowed the Internet to scale massively, to value networks.
This definitely looks promising.
But though the authors compare their approach to SW and SM, it still seems to me that they are answering different questions.</p>

<p>Citing the paper (page 12, Section 5.3, emphasis mine):</p>

<blockquote>
  <p>To select paths, end-hosts monitor the total price <strong>on different paths</strong> and choose the cheapest option.</p>
</blockquote>

<p>And in Section 6.1, describing the experimental setup:</p>

<blockquote>
  <p>We restrict both algorithms to use <strong>4 disjoint shortest paths</strong> for every source-destination pair.</p>
</blockquote>

<p>Well, but how do I get the list of these “different paths” / “disjoint shortest paths”?
Aren’t we back at square one, in search for an efficient routing algorithm?</p>

<p>I think the following analogy might be useful.
Imagine a city suffering from road congestion.
People are commuting by car from home to work and back every day, and there is not enough road capacity.</p>

<p>Imagine an ideal situation:</p>
<ul>
  <li>the city consists of two areas A and B;</li>
  <li>half of the population lives in A and works in B, and the other half lives in B and works in A</li>
  <li>the working hour are distributed equally throughout the day.</li>
</ul>

<p>Under these rather unrealistic conditions, roads are equally occupied throughout the day.
But in reality, people mostly work in the city center and live outside it, work mostly during the day and sleep at night, so the flow is not balanced.</p>

<p>There are two related, but distinct questions:</p>

<ol>
  <li>For an individual: given all preconditions (the home address, the work address, the traffic conditions), how do I choose the best route?</li>
  <li>For the city administration (or whoever owns the resources): how much do we charge for roads to balance supply and demand?</li>
</ol>

<p>Max-flow, SW, and SM answer the first question.
What makes the task difficult is that the map of the city is too big for any individual to fully possess.
SilentWhispers find the shortest way to a highway (landmark), and from the highway (navigation on the highway is considered trivial).
SpeedyMurmurs, on the other hand, at every crossing turn to the direction which brings you closer to your destination.</p>

<p>Spider network seems to answer the second question: given a payment graph (where people want to go) and the network graph (road capacities), how much should entering each portion of a road cost so that equal amounts of traffic would go in the opposite directions?
If the most direct highway to my destination costs a lot, I’d be happy to take a detour via less congested roads.
If just the right share of commuters does so, the traffic will be balanced.
But is this even the right goal to strive for?
Shouldn’t we aim for getting more people to their destinations using as little fuel as possible?
(Public transport, anyone?)</p>

<p>Maybe we’re hitting the limits of usefulness of this analogy, but maybe we aren’t.</p>]]></content><author><name></name></author><summary type="html"><![CDATA[Let’s continue our journey though recent paper which suggest ways to optimize routing in payment channel networks. In previous posts, we looked an SilentWhispers and SpeedyMurmurs. Both approaches emphasized privacy as an important goal, but employed different constructions: landmark-based routing (SW) and embedding-based routing (SM). Today let’s look into a paper entitled “Routing cryptocurrency with the Spider network” (2018) by Sivaraman et al.]]></summary></entry><entry><title type="html">SpeedyMurmurs: applying friend-to-friend routing to payments</title><link href="https://s-tikhomirov.github.io//speedymurmurs/" rel="alternate" type="text/html" title="SpeedyMurmurs: applying friend-to-friend routing to payments" /><published>2019-03-27T00:00:00+00:00</published><updated>2019-03-27T00:00:00+00:00</updated><id>https://s-tikhomirov.github.io//speedymurmurs</id><content type="html" xml:base="https://s-tikhomirov.github.io//speedymurmurs/"><![CDATA[<p>In a <a href="/silentwhispers">previous post</a>, I discussed SilentWhispers – a routing algorithm for credit networks.
Today I’ll dive into a follow-up paper by a partially intersecting group of authors, entitled “<a href="https://arxiv.org/abs/1709.05748">Settling payments fast and private: decentralized routing for path-based transactions</a>”, which presents a routing algorithm called SpeedyMurmurs.</p>

<h1 id="routing-revisited">Routing, revisited</h1>

<p>The <a href="https://eprint.iacr.org/2016/1054">SilentWhispers paper</a> (2016) was mostly dealing with credit networks (and was itself a continuation of the work of some of the co-authors on this subject).
The introduction of the SpeedyMurmurs paper (2017) sets up the stage in the blockchain world.
The authors introduce the notion of path-based transaction (PBT) network, which unifies credit networks such as Ripple and Stellar with L2 solutions on top of open blockchains (Lightning, Raiden).
They then define the three mechanisms that comprise a PBT network:</p>

<ul>
  <li>routing;</li>
  <li>payment;</li>
  <li>accountability.</li>
</ul>

<p>In all PBT networks, payments are executed in an atomic series of elementary operations.
Credit networks use a (somewhat) centralized entity for ensuring atomicity (even “real-world” courts maybe).
Lightning and Raiden rely on the base layer.
The problem of routing, however, is orthogonal to the implementation of enforcement, so it makes sense to unify the credit-like networks under an umbrella term PBT.</p>

<p>Speaking of the routing mechanism specifically, the authors list four dimensions to measure how well it performs:</p>

<ul>
  <li>effectiveness (share of successfully completed transactions);</li>
  <li>efficiency (delays and overhead);</li>
  <li>scalability (how the system responds to the growth of the number of nodes, links, and transactions).</li>
</ul>

<p>On top of that, the authors argue that routing must not compromise users’ privacy.</p>

<p>Ripple and Stellar base routing decisions based on the information stored in a public blockchain.
A number of obviously centralized approaches are mentioned as well, but these are not interesting (with a trusted third party we can do anything).
A more noteworthy proposal, which I haven’t heard of before, is <a href="https://bitfury.com/content/downloads/whitepaper_flare_an_approach_to_routing_in_lightning_network_7_7_2016.pdf">Flare</a> – a 2016 BitFury-designed routing proposal for Lightning which <a href="https://bitcoin.stackexchange.com/q/85356/31712">didn’t eventually go into production</a> (though it does go into my reading list).</p>

<p>SilentWhispers, as the authors modestly note, is</p>

<blockquote>
  <p>the most promising approach in regard to privacy</p>
</blockquote>

<p>Let me just briefly remind you the key ideas:</p>

<ul>
  <li>a number of well-known, highly-connected nodes called landmarks periodically run a breadth-first search and create a spanning tree over all nodes;</li>
  <li>payments are routed via a landmark;</li>
  <li>landmarks calculate path capacities using a multi-party computation, which conceals individual nodes and their link capacities;</li>
  <li>if the path capacity via the first landmark is low, the sender tries another one.</li>
</ul>

<p>However, it suffers from a number of drawbacks:</p>

<ol>
  <li>the spanning tree has to  be re-computed once every epoch, including the parts which have not changed;</li>
  <li>all paths go through a landmark, even if both the sender and the receiver are in the same sub-tree;</li>
  <li>as all nodes along the path must send shares of their local link capacity to all landmarks, the number of messages grows quadratically;</li>
  <li>the protocol doesn’t handle concurrency.</li>
</ol>

<p>SpeedyMurmurs aim to solve these problems by abandoning the landmark routing and using another approach called <em>embedding-based routing</em>.</p>

<h1 id="embedding-based-routing">Embedding-based routing</h1>

<p>What are embeddings, exactly?
I was looking for a definition (a sentence starting with “An embedding is…”) but didn’t find one.
Instead, the notion is introduced as follows:</p>

<blockquote>
  <p>Embeddings rely on assigning coordinates to nodes &lt;…&gt; and having nodes forward packets based on the distances between coordinates known to that node and a destination coordinate.</p>
</blockquote>

<p>For the proposed routing algorithm, <em>greedy</em> embeddings are used.
Greedy embeddings assign coordinates based on the position of the node in the spanning tree.
Then a distance function is defined on pairs of coordinates with the following essential quality: for every (sender, destination) pair, the sender has a neighbor which is closer to the destination than the sender itself.
This means that a greedy algorithm – forwarding a message to the neighbor which is the closest to the destination – will always find a path (and not get stuck in a local minimum).</p>

<p>Consider prefix embeddings – greedy embeddings where a coordinate of a node contains the coordinate of its parent as a prefix.
Imagine a binary tree with three levels.
The root gets an empty string as its coordinate.
The two nodes on the first level get “0” and “1”.
Their children get, rather unsurprisingly, “00”, “01”, “10”, and “11”.
If we want to send a message from “00” to “11”, the shortest path follows the tree up to the root and back to the leaf: 00 – 0 – “” – 1 – 11.
If the sender and the receiver are in the same sub-tree, we don’t have to go all the way up to the root: 00 – 0 – 01.
There might also be “shortcuts” – links between nodes which do not belong to the tree.
If a suitable shortcut exists, a routing algorithm should choose it.</p>

<p>On each step, the next node to forward the message to is chosen greedily among all neighbors (includes shortcuts).
The distance function is defined as d(u,v) = |u| + |v| - 2CPL(u,v), where |u| is the length of the path from u to the root (equivalently, the length of its coordinate vector), and CPL is the common prefix length of the two coordinates.
The formula basically says:</p>

<ol>
  <li>go from u to the root;</li>
  <li>go from the root to v;</li>
  <li>oh, you didn’t actually have to traverse the common part of these paths (twice).</li>
</ol>

<p>It’s not the only way to construct embedding-based routing, but follows the general recipe:</p>

<ol>
  <li>construct a spanning tree;</li>
  <li>assign a pre-defined coordinate to the root;</li>
  <li>let each node derive its coordinate from its parent’s coordinate;</li>
  <li>define a suitable distance function between coordinates.</li>
</ol>

<p>This is all well and good, but what about privacy?
In a routing scheme described above, every intermediary node knows where the message is going.
A privacy-preserving protocol called VOUTE comes to the rescue.</p>

<p>VOUTE uses <em>anonymous return addresses</em>, which allow intermediary nodes to choose the neighbor closest to the destination without revealing its coordinate.
Sounds somewhat like homomorphic encryption which supports distance calculation (or comparison, at least) on encrypted coordinates.
See the <a href="https://arxiv.org/abs/1601.06119">VOUTE paper</a> (and <a href="https://www.freehaven.net/anonbib/cache/roos16anonymous.pdf">a shorter version</a>) for more detail.</p>

<p>By the way, did you notice how we now talk about messages and not transactions?</p>

<h1 id="friend-to-friend-networks">Friend-to-friend networks</h1>

<p>Embedding-based routing was initially conceived for anonymous messaging in <em>friend-to-friend</em> (F2F) networks.
In F2F, links are established and maintained consciously, relying on off-protocol trust relations.
This differs from P2P networks like BitTorrent, where a user connects to whichever peers provide the highest bandwidth (I don’t care where I get my file from as long as the checksum is correct).
In anonymous messaging, maximizing bandwidth is not as important as not letting your data fall into untrusted hands.</p>

<p>F2F lies in between data-based (BitTorrent) and value-based (Lightning) P2P networks.
F2F links are more semantically charged compared to those in filesharing networks, but, contrary to PBT networks, are <strong>undirected</strong> and <strong>unweighted</strong>.
Undirected means that messages can be directly transmitted from Alice to Bob if and only if they can be directly transmitted from Bob to Alice.
Unweighted means that</p>

<blockquote>
  <p>transmitting a message does not affect the ability of the link to transmit future messages</p>
</blockquote>

<p>(I think this is a rather deep observation highlighting the crucial difference between digital representations of <em>information</em> and <em>value</em>.)</p>

<p>In order to adapt routing algorithms from F2F to PBT, we have to account for two crucial features of credit links:</p>

<ul>
  <li>asymmetry: links from Alice to Bob and from Bob to Alice generally differ in capacity;</li>
  <li>weights: all links are weighted, and weight changes must be accounted for during graph re-balancing.</li>
</ul>

<h1 id="speedymurmurs">SpeedyMurmurs</h1>

<p>SpeedyMurmurs adapt VOUTE – a greedy embedding-based routing with anonymous return addresses for F2F messaging – to the PBT model.
The protocol operates in weighted graph model and distinguishes between <strong>unidirectional</strong> and <strong>bidirectional</strong> links.
Alice and Bob are said to share a bidirectional link, if they share two links with positive weights in opposite directions.
I’m a bit skeptical on whether this model accurately reflects the reality of PCNs like Lightning, where the following four types of relationship between Alice and Bob have distinct qualities [<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>]:</p>

<ol>
  <li>sharing <em>no</em> channel;</li>
  <li>sharing an open channel with all capacity <em>on one side</em>;</li>
  <li>sharing a <em>pair</em> of channels with their full capacities on opposite sides;</li>
  <li>sharing a channel with non-zero capacity <em>on both sides</em>.</li>
</ol>

<p>(Formalizing the properties of these states may also be interesting!)</p>

<p>We assume, as in SilentWhispers, that there is a set of well-connected and well-known nodes called <strong>landmarks</strong>.
Each landmark defines its own spanning tree.
For each tree, each nodes is assigned a coordinate based on its parent’s coordinate.
A payment is split into random chunks, and each chunk is sent along the path within a different tree.</p>

<p>The protocol consists of three algorithms:</p>

<ul>
  <li><em>setRoutes</em> creates spanning trees and assigns coordinates to nodes;</li>
  <li><em>setCred</em> reacts to a change in a link capacity;</li>
  <li><em>routePay</em> discovers a suitable path for the requested transaction.</li>
</ul>

<h2 id="setting-the-routes">Setting the routes</h2>

<p>The authors modify the VOUTE’s tree creation algorithm by splitting it into two phases.
During the first phase, the original algorithm runs, considering only bidirectional links.
Then, if any nodes are left outside the tree, they “attach” to it with their unidirectional links.
Note that the algorithm described in the paper (Algorithm 1, page 7) assumes a central coordinator which maintains a queue of nodes not yet in the tree.
In a distributed scenario, the authors acknowledge,</p>

<blockquote>
  <p>starting the second phase is tricky</p>
</blockquote>

<p>The nodes that are not yet in the tree are not sure whether they should wait for an invitation from a node with a bidirectional link, or just be satisfied with a unidirectional one.
But the problem can be circumvented by choosing a proper timeout, after which a node assumes the second phase has started.</p>

<h2 id="setting-the-credit">Setting the credit</h2>

<p>The key problem is how to make the network react to changes in link capacities.
The authors suggest the following algorithm.
The network reacts to one of the following events:</p>

<ul>
  <li>a new unidirectional link: as one of the node is not yet part of the tree, it chooses the other as the parent;</li>
  <li>a now non-zero bidirectional link: if one of the nodes has only a unidirectional link to its current parent, it should choose the newly connected node with a bidirectional link as a new parent instead; this leads to the tree replacing unidirectional links with bidirectional ones whenever possible, leading to higher potential throughput;</li>
  <li>removed link: as one of the two nodes in question is a child of another, the child selects a new parent.</li>
</ul>

<p>Every time a node changes its parent, all its neighbors are notified, they then also choose a new parent and a corresponding coordinate.</p>

<p>Note that setCred doesn’t react to changes in capacities of existing links!</p>

<h2 id="routing-the-payment">Routing the payment</h2>

<p>RoutePay discovers the path between a sender and a receiver capable of transferring the required amount.
To improve anonymity, the sender splits the payment in random chunks and sends them along paths in different trees.
This allows to avoid a costly multi-party computation of SilentWhispers and also gives a passive attacker less information on the lower band of the payment.</p>

<p>The routing accounts for weighted links but doesn’t actually do much to optimize for this new model.
Routing fails if there is no neighbor with a coordinate closer to the receiver <em>and</em> sufficient available credit.
There may be a path with sufficient credit which temporarily goes “the wrong way”, but <em>greedy</em> routing wouldn’t consider it.
Can we make routing a bit less greedy to account for some combination of the qualities “being close to the destination” and “having sufficient capacity”?
Another open question worth investigating!</p>

<h2 id="privacy-analysis">Privacy analysis</h2>

<p>The authors dedicate a separate sub-section to privacy guarantees of SpeedyMurmurs.
Most of the privacy properties follow from proofs in the VOUTE paper, but one assumption regarding value privacy looks suspicious:</p>

<blockquote>
  <p>we say that a PBT network achieves value privacy, if the adversary cannot determine the value c &lt;…&gt;, if the adversary is not sitting in <strong>any</strong> of the involved routing paths.</p>
</blockquote>

<p>A rather weak assumption, isn’t it?
In credit networks with an external identity systems, where establishing Sybil nodes requires social engineering attacks at scale, this might be reasonable.
But for a PCN over an open blockchain, where a resourceful attacker can easily launch a well-connected, well funded node and route everyone’s payments while spying on them…
This just doesn’t seem right.
Note that the reason why the sender splits the value into random chunks supposed to be a countermeasure against this attack.
The authors note:</p>

<blockquote>
  <p>when the adversary corrupts some of the paths &lt;…&gt; we cannot prevent the adversary from estimating c.</p>
</blockquote>

<p>Indeed, the adversary at least learns the lower bound for the total value, and as the value is shared uniformly, and the number of landmarks is known, the total value may be estimated as L * c_i, where L is the number of landmarks.</p>

<h1 id="evaluation">Evaluation</h1>

<p>The authors identified multiple “axes” along which routing algorithms for P2P networks may differ:</p>

<ul>
  <li>routing: landmark-based, greedy embedding-based, or “tree-only”;</li>
  <li>stabilization method: periodic or on-demand;</li>
  <li>assignment of credit on paths: multi-party computation or random;</li>
  <li>landmark selection: highest degree or random;</li>
</ul>

<p>and five metrics: fraction of successful transactions, delay, messages sent per transaction path length, path length, and messages related to stabilization per epoch.</p>

<p>SilentWhispers are landmark-based, with periodic stabilization, and multi-party computation.
SpeedyMurmurs are greedy embedding-based, with on-demand stabilization, and random credit assignment.
Both use highest-degree nodes as landmarks.</p>

<p>Using the GTNA graph analysis framework and the dataset of Ripple transactions, the authors compared multiple combinations of the parameters listed above.
Unsurprisingly, Ford-Fulkenson (a generic max-flow algorithm) “exhibited prohibitive delays”.
More interestingly, SpeedyMurmurs performed better than SilentWhispers across all metrics in the “static scenario” (without stabilization).
In a dynamic scenario, during “normal operation”, SpeedyMurmurs were also superior, but during “intervals of frequent change” the opposite was the case.
All algorithms except for Ford-Fulkenson showed success ratio substantially lower than 100%, none of them higher than 91%.</p>

<p>I fully agree with the authors in that</p>

<blockquote>
  <p>users might not be willing to accept a failure rate of 10%</p>
</blockquote>

<p>Lots of work lies ahead before we make payment networks appealing to the general public (or, more realistically, at least to developers who would create payment-network-based applications targeting the general public).</p>

<h1 id="summary-and-questions">Summary and questions</h1>

<p>SpeedyMurmurs are an interesting proposal, but I can’t get over the feeling that we can’t just apply relatively minor tweaks to algorithms from <em>data transfer</em> networks and apply them to <em>value transfer</em>.
We definitely can and should borrow ideas from existing research, but routing algorithms for P2P messaging may need more radical modifications to be useful in a PBT setting, especially in PCNs like Lightning and Raiden.</p>

<p>Another faucet of the same issue: most algorithms in the paper are introduced in a “centralized” manner (assuming there is a central coordinator who runs the protocol).
Then a paragraph follows, starting with something along the lines of: “the distributed version of the algorithm is basically the same, but nodes additionally do this and that”.
I’m not convinced that patching centralized algorithms in such a manner does not introduce vulnerabilities or substantial inefficiencies.
I’d prefer algorithms to be introduced in a decentralized setting directly.</p>

<p>The same applies to the details of the graph model.
How well does modeling a bidirectional link as a pair of independent unidirectional ones reflect the reality (of networks being implemented in practice, as they are the most useful things to model)?
Do differences between stateless (Lightning, Bitcoin-style) vs stateful (Raiden, Ethereum-style) models play a role here?</p>

<p>Moreover, as I’m undoubtedly spoiled by the importance of the Bitcoin’s incentive layer, every time I see a protocol description containing a phrase like “every node does X and notifies all its neighbors”, certain questions immediately start popping up in my head:</p>

<ul>
  <li>what if it doesn’t?</li>
  <li>why would it want to?</li>
  <li>what if it does it but incorrectly?</li>
  <li>what if it sends equivocating messages?</li>
  <li>what if an attacker launches 100x more nodes than the network currently contains?</li>
</ul>

<p><a href="https://youtu.be/hiq1lturAao?t=91">Und so weiter, und so weiter</a>…</p>

<p>Yet, in the end, isn’t it always the same question – and always the same answer?</p>
<div class="footnotes" role="doc-endnotes">
  <ol>
    <li id="fn:1" role="doc-endnote">
      <p>Cases 3 and 4, though they both enable payments in both directions, differ, at least, in that they require, respectively, two or one on-chain transactions to redeem the balances on layer-1. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p>
    </li>
  </ol>
</div>]]></content><author><name></name></author><summary type="html"><![CDATA[In a previous post, I discussed SilentWhispers – a routing algorithm for credit networks. Today I’ll dive into a follow-up paper by a partially intersecting group of authors, entitled “Settling payments fast and private: decentralized routing for path-based transactions”, which presents a routing algorithm called SpeedyMurmurs.]]></summary></entry></feed>