This is what I mean when I say you have to be bug-for-bug compatible with Bitcoin Core. Compatibility with any subset of Bitcoin Core, including its blessed consensus checks, is insufficient to demonstrate compatibility with Bitcoin Core.
(Consider the browser analogy -- rendering compatibility between IE and Chrome is moot if there exist some HTTP headers which cause IE to choke and Chrome not to. Bitcoin's security guarantee is, essentially, "every webpage in history, in the future, and which can be created by our adversaries renders in exactly the single way it would in our reference client, across all clients.")
This dynamic plus the fact that chains with less hashpower behind them tend to be attached to the economic losers in an attack virtually guarantees that the Bitcoin network/protocol is in practice a client monoculture. (That client monoculture can still fork against itself, though. Fun stuff when that happens.)
This is why the Bitcoin Core developers have been trying to factor out all of the code that affects consensus into a separate library. I think it's likely that this library, libbitcoinconsensus, will be at the core of most clients in future, but its existence and maturation will also allow other clients to emerge and replace Bitcoin Core for some applications.
In your analogy, libbitcoinconsensus is like a minimal rendering engine that implements the standards and can be used by anybody to build a full web browser.
I'm not sure. His statement that all clients must be bug-for-bug compatible with Bitcoin Core is simply false. There is a big chunk of behavior that can differ from client to client without the network fragmenting and without bad consequence to the clients whose behavior diverges from the majority's.
For example, clients have complete choice about which nodes to talk to and which received transactions to propagate. In fact, clients have complete choice about all functionality that depends on data that is not (or not yet) incorporated into the blockchain.
That's why we have the notion of consensus-relevance. There is a lot of stuff in Bitcoin Core that simply is not consensus-relevant, and Bitcoin Core's developers intend to continue to segregate the consensus-relevant stuff from the stuff that any client is free to make its own choice about. Yes, in the future it might be true that there's a big kernel of code that every client needs to use, but that kernel will not be all of Bitcoin Core as we know it today.
Because of bugs like this, the bitcoin developers strongly discourage using reimplementations of consensus code. Consensus code is complex, and a reimplementation cannot reject any blocks that the majority implementation (the implementation used by the most hashing power) would accept.
To be safe for mining, the behavior must match exactly. This includes bugs in the code, errors introduced by compiler optimization, bugs in any libraries, and quirks of the programming language.
This is the reason they created libbitcoinconsensus. The library allows you to innovate around non consensus-critical features of bitcoin without introducing forking risks.
"The difficulty of writing consensus critical code: the SIGHASH_SINGLE bug"
Five pages of detail about a single bug^H^H^H feature, and just the other day I found yet another edge case in the implementation that I hadn't noticed before.
I don't believe that you are really Peter Todd, can you please sign the challenge "1" with one of your known bitcoin addresses?
Joking aside, I used some of your scripts to seed the fuzzer:
https://github.com/jonasnick/bitcoinconsensus_testcases/tree....
If you come up with more scripts I'd love to feed them to the fuzzer. I'm still surprised that a javascript implementation (bitcore) is withstanding so far.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
https://news.ycombinator.com/item?id=9526884
> I don't believe that you are really Peter Todd, can you please sign the
> challenge "1" with one of your known bitcoin addresses?
Pff, PGP all the way.
> Joking aside, I used some of your scripts to seed the fuzzer:
> https://github.com/jonasnick/bitcoinconsensus_testcases/tree.... If you come
> up with more scripts I'd love to feed them to the fuzzer. I'm still surprised
> that a javascript implementation (bitcore) is withstanding so far.
Oh, by scripts you mean the test cases I've added in Bitcoin Core? Thanks!
-----BEGIN PGP SIGNATURE-----
iQGrBAEBCACVBQJVUP6SXhSAAAAAABUAQGJsb2NraGFzaEBiaXRjb2luLm9yZzAw
MDAwMDAwMDAwMDAwMDAwMjE5MjMyYzdkYTRjMTk2NjllMjY1NTc3ODkzMDVlODE0
MzQ2OWQyMGZiZGRjODYvFIAAAAAAFQARcGthLWFkZHJlc3NAZ251cGcub3JncGV0
ZUBwZXRlcnRvZC5vcmcACgkQJIFAPaXwkfv+tAf/WDq99I+Sn54//hEzmX21WV6d
A1OvMhfQ2xalf6O8l5gwzQkcvADcjlN00emc09GI2s4iQE2U85mkFhH/s+LWB8q6
K0qLTZjUvZMlSEpf6kGSGiClJpWt/K6vHewwWRZr6gJ/iqYper87SUpDqaV71QjU
cXdFFNx2tCoqSHuLRddLaYhP3p30bu/LWdcawG16hKCWiKp/it5dqPsMT1bm6QAQ
aRoGPAgx8xB/g9Jw1dXZOmtEjLCrBimq7TdDY58RgRlqQk8oyox5T8Ypsro+uPg/
7v5U9lA5SS2ggXURpQpBwRAtDNEFMMHr2r0pu913Cea/wAG1iOPlUwvVpgnHRQ==
=ETM3
-----END PGP SIGNATURE-----
Interesting results, but the post seemed a bit liberal in its application of mathematical notation. A fork can be explained entirely in English, with no use of numbers, let alone subscripting or negation...
> I was curious to find interesting scripts, i.e. scripts that trigger unusual edge cases.
In my limited experience with afl-fuzz, it seems to record test inputs which cause crashes (at least by default). Did you change the criteria for it to consider a test failed/crashed or otherwise worth recording?
It also records every input that triggers a new edge of the execution graph in the 'queue' folder. After I'm done with fuzzing I minimize it with afl-cmin.
(Consider the browser analogy -- rendering compatibility between IE and Chrome is moot if there exist some HTTP headers which cause IE to choke and Chrome not to. Bitcoin's security guarantee is, essentially, "every webpage in history, in the future, and which can be created by our adversaries renders in exactly the single way it would in our reference client, across all clients.")
This dynamic plus the fact that chains with less hashpower behind them tend to be attached to the economic losers in an attack virtually guarantees that the Bitcoin network/protocol is in practice a client monoculture. (That client monoculture can still fork against itself, though. Fun stuff when that happens.)