I’m not asking anyone to implement it. I just want to talk about it to know if it’s really useful or not or if it has problems that I have failed to anticipate.

If you are a slow miner and the correct nonce is not one of your allowed ones, then too bad for you but you will maybe be more lucky for the next block. It is not less fair than the current PoW system where nonces are chosen randomly. And besides, I keep saying ‘the correct nonce’ but in fact there is an infinity of them. Each miner has at least one correct nonce in his allowed nonces.

The only way to improve the blockchain is not to improve security. The 2 main issues I see right now are the increasing size of the blockchain (200gb+ as of 2019) and the electric consumption of miners, that I try to tackle here.

@josselinchevalay
I don’t understand PoS as much as I understand PoW, but from what I can tell, to validate a new block in a PoS context there is only one validator (the counterpart of miner in PoW). My idea is just an upgrade to PoW so that there can never be 2 miners doing the same cryptographic calculation to mine a new block and thus wasting electricity uselessly. So the first difference is that in PoS only one person ‘mines’ a new block whereas in my idea every miner works ‘hand in hand’ to mine the new block. Secondly there is no stake in my idea, if the miners approve a fraudulent transaction, the network just rejects it, whereas in PoS the validator loses money if it happens. Thirdly, the cryptographic complexity of the calculation that the validator has to do in PoS is far lesser than in my idea (and in PoW in general) because if it was the case, a single validator would have to do the work that thousands and thousands of miners do in PoW in 10 minutes in order to validate a new block.

@JacopoStanchi The main issue with your idea is that every node out there is trying to solve a different equation. Simplified, you are trying to calculate a result below a specific value for the following

Hash(transaction blocks + my wallet id + nonce)

As each node has a unique wallet id, the required values of the nonce will be different for each node so making sure that they don’t calculate the same nonces could actually result in blocks not being found.

This algorithm could only be used in mining pools (where it probably already is) because the way they work is that you use the wallet id of the pool, and then trust that to pay you a “share” of what has been mined over a period of time

I think there might be confusion about some facts:

As @MattewSteeples pointed out, you have to mine on the same wallet ID to “share the load”

Exploring a given walletID “nonce space” is not better (or worse) than exploring several individual “nonce space”. More on that below.

The difficulty (aka the number of leading zero of the output hash you should have to get a reward) is dynamically adjusted so that blocks are mined every 10 min (on average)

(1) tells us that to cooperate, you have to trust the wallet iD of the group. Whoever have control over it have power. If you get rewarded only when you find, you have no interest in cooperating (see below) but have the risk of being robbed by the pool leader (mathematically it’s a pure loss). If you share the reward, you have the same amount on average, but the income is much more regular. That’s why mining pools exist in the first place, at the cost of having to trust the pool leader.

(2) For a given block, any pair (walletID, nonce) have the same chance of giving a winning hash and give you a reward. This is counter-intuitive but important to understand: Having explored 10^20 nonces for your wallet ID without having found a winning nonce DOES NOT increase your chances of finding a good one in the next N tries. Exactly as loosing 10 times at the roulette wheel in a casino DOES NOT increase (or decrease) your chances of winning the next round.

(3) but all that is not really important: the network will adjust its difficulty so that a winning (walletID, nonce) is found every 10 min. Wether its because of dumb uncooperative nodes doing the same computation again and again, or a fully cooperative network all computing on the same wallet ID and never trying the same nonce twice (will never happen, but you get the point).

No matter what, the energy spent for a block (network-wide) will be 10min * electrical power of ALL the mining nodes… Whatever their strategy. Having a good strategy only increase your cake share at the expense of the others. No effect on electrical consumption.

So. If you want to optimize electrical consumption, you only have two options: reduce the number of nodes (nothing you can do about that) or reduce the energy required for computing one hash (via new hardware, etc).

I agree that the miners try to solve different equations and the nonce is different for each miner, but I don’t understand why you say that it can result in blocks not being found. With this mechanism, more nonces are being tried each minute than with current PoW. Besides, if there was only one correct nonce, then maybe the set of allowed nonces of all the miners of the network may not contain this nonce. But that’s not the case, there is an infinity of correct nonces and each miner has an infinity of them in their allowed nonces. So in my opinion blocks should always be found eventually.

Your formula is not quite right. It should be more like hash({previous block hash, block data, getOffset(wallet id) + n*2^37})
Come to think of it but instead of using a getOffset function we can just do wallet id + n*total number of possible wallet ids.

But in my idea the miners who didn’t find the nonce don’t get a share of the money, whereas in a pool they do.

@Akita
I’m also adressing the addendums you made at the end of your message.

Miners in my idea don’t share the load. If they don’t cooperate (i.e. try unallowed nonces) their reward is denied by the network.

I’m not arguing that, but the use of partioning into wallet ids nonce spaces is to not have overlap between the sets of nonces tried by 2 miners. I’m not saying that we have a better chance of finding the correct nonce at each test, but that the rate of different nonces tried in one minute by all the miners is higher.

Why the limit of one block every 10 minutes? It is an arbitrary figure, each cryptocurrency has a different limit. The reason is to not create new blocks too frequently and thus saturate the blockchain. So we have to change the complexity of mining accordingly to match the 10 minutes of mining for Bitcoin, so that enough transactions can be added to the pending list during mining.
But with my idea, for the same complexity, you find a new block in less time. What it means is, for a given complexity, instead of finding a new block in 10 minutes, you find one in 5 minutes. What you can do is either increase the complexity so that it becomes 10 minutes again OR miners stop to consume electricity for 5 minutes before the new block is opened. Either you have more security OR less power wasted.

Reduction of calculations in order to cut power consumption is certainly an honorable means to an end but not the only means. If environmentally detrimental power consumption is the problem to be solved, then it should also be solved by environmentally benign power generation. Power your mining operation with alternative sources to fossil fuels.

You are right, I was referring to the Bitcoin Blockchain as an example. Sorry for not having made it clear. I thought you wanted to find a new way to mine existing blockchain, not designing a new one.

Miners won’t stop for five minutes. They will start processing the new block, try to find the next one (trying nonces in their own nonce-space, of course) and submit it as soon as they can (with transactions). You have actually 2 pathes: more security (at the expense of throughput), more throughtput (at the expense of security). If you have a dynamic difficulty, then it will go back to that value, no matter what. If you don’t, blocks gets mined faster and faster with technical evolution, and you have a problem of centralisation/security very soon.

NB: You could argue that network would deny publishing too soon this block N (before 10 min), but your would be shifting the competing from the better hash rate to a better network access. Miners will precompute block N+1 and publish it at the beginning of the cycle. The one with better access (more open connections with other nodes, more bandwidth) will have it sent and accepted by a lot of nodes and have better chance of it being mined upon for round N+2 and be selected permanently with the rule of the longer chain. Network access is even more unequal than CPU/GPU/ASIC, so you’re actually decreasing security by discouraging nodes with high latency because them finding early is not an advantage anymore.

NB:

That’s the contrary. We change the difficulty so that not too many transactions per day are validated when processing power rise, so that the blockchain (the list of transactions, not the network) doesn’t grow too fast. Why do we want that? For decentralisation of validation: every crapy mobile app is able to catch up the validation of every transaction onchain, even if unable to mine, but that’s less of a problem: you can trust that the records are healthy without mining.

There aren’t an infinite number of nonces, correct or otherwise. In Bitcoin (for example) it’s a 32-bit field.

The problem is that there is no “overlap of nonces” as far as the calculation is concerned. The nonce is part of the calculation, as is the nodes own wallet id. For example, you could calculate using a nonce of 10 and not get an acceptable value. I could calculate with a nonce of 10 and, because my wallet id is different, my value would be accepted.

Yes I have heard about miners going to Iceland to exploit geothermic energy. But given that most miners are based in China and that China has a power shortage right now, I can’t bring myself to think that most of the energy used on mining today is clean. Besides, even the clean energy could be used for a better purpose. That’s a really interesting subject but I admit that I don’t have enough knowledge on this.

No, the current block would be published as soon as it has been found and the miner would get their reward, however the next block would be opened at 10 minutes. Miners wouldn’t be able to precompute it because they would need to wait until 9 minutes and 59 seconds to be sure that every transaction in the block has been broadcasted to the network. Nonetheless I admit that this idea is very experimental, and I suspect that it may facilitate a 51% attack. But if it is really a bad idea, we can still use the other option which is increase the complexity of mining, and thus increase the security of the blockchain (failing a reduction of mining’s electrical consumption).

Yes sorry that was a mistake of me. I know there is a finite amount of nonces, what I meant is that all miners will “statistically” always have a correct nonce in their “allowed nonce space”. So the blocks will always be found eventually.
But 32 bits seems strange. It only amounts to 4 billion possible values. Even my PC with a 3GHz processor can compute this much values in a relatively short time.

What I mean by overlap is all the nonces tried by 2 different miners, and I’m pretty sure there are overlaps with mining currently. But as the nonces are chosen randomly to minimize such overlaps I cannot assess how big they are. If the overlap is negligible, my idea is completely useless.
The correct nonce doesn’t depend on the wallet id. If you get an acceptable value with a nonce of 10, then I also get an acceptable value with a nonce of 10. The difference is who is able to claim the reward. If miner A finds an acceptable value with a nonce that ‘belongs’ to miner B, then he gets nothing and B gets the reward even though he did nothing.

They don’t need every transaction, they need enough to fill the block or at least enough to make it worth it to begin 5 min earlier. On a very saturated blockchain like bitcoin where even big juicy transactions are in the queue for a long time, you always have some to process. And even if the queue is empty (which basically never happens) or you don’t want to wait for it to fill up, nothing stops you from mining an empty block (!) You won’t have transaction fees, but you get the reward, so it’s still worth not waiting to start finding a good nonce. You would still have to wait to publish it, but you can pretend you just started to compute, and it was the very first nonce you tried (pretty lucky, right? )

In that case, you are back to square one about electrical consumption, network speed, etc, etc. the only difference is that the miners didn’t tried the same nonces, but it won’t make much of a difference, would it? (Unless the crypto puzzle is useful by itself. Some crypto-currencies tried that, and you would compute to simulate protein folding, simulating physics, etc. There, the blockchaincouldn’t care less what you compute, but humans do and society benefits from miners never testing real-world hypotesis twice. These coins never reached the critical size, unfortunately… )

An additional note: for miner to explore the same nonce space (but different part of it), you will have to remove the wallet ID from the equation. You can change the consensus rule from “hash(previous block hash, wallet ID, nonce) must have N leading zero” to “walletID.sign(hash(previous block hash, nonce) with N leading zeros)”, so that miners explore different parts of the same space, but you can still verify who is the winner thanks to their signature.

But the protocol of this hypothetical cryptocurrency would reject this block. The nodes would only accept a block with all the transactions available at 10 minutes.

Miners would try different nonces but they would also try MORE nonces. How many more? That I would like to know. If there is a lot, then we could at least raise the complexity of mining and thus increase the security, but if there isn’t, then my idea is useless.

A Proof of Work problem must be NP-difficult. What it means is that an answer can be found in an exponential time but the verification of this answer takes a polynomial time. In other words, slowly resolved but quickly verified.

In a blockchain, the data of the block and the hash of the previous block must be part of this problem, because otherwise an attacker could not only tamper with the last block of the blockchain but also all the other blocks. The fact that the blockchain is a simply linked list is what gives us its security.

And another thing, to follow the blockchain philosophy, verifying a block shouldn’t rely on a centralised third-party. For example, if instead of trying useless nonces, miners lent their computing power to the NASA for 9 minutes and then solved a cryptographic puzzle that takes on average 1 minute, then we would rely on the NASA API to be sure that “this miner has indeed done the job and can now go to the next step”.

The problem with a useful mining is that it must meet all these requirements, but I’m pretty sure someone will come up with something interesting in the future (or maybe it is already the case).

But I want each miner to have their own nonce space. My way to authenticate the winner is to check who is the owner of the correct nonce, and this way we get their walled id.

What do you mean by “all the transactions available”? Once a transaction is published, it takes a long for every node to be aware of it. That is why Nakamoto chose a long time (10min) between blocks(well, they wanted blocks to have enough time to propagate, but the same logic applies, only thousands of time per round). With a never-ending stream of new transactions, 2 distant nodes will never see the same set of to-be-processed transactions. And how would you know that you are aware of all the transactions and you can start mine the block?
Another problem is that if you store all the transactions and don’t limit them, you will have a 20-GB block every 10 min and your blockchain will grow super fast, excluding small nodes and leading to centralization. A good strategy for big miners would actually be to spam the network with transactions, since they are among the few who can keep up.

There is something why would exploring more nonces would lead to more security?
If a valid block hash needs N leading-zeros, you will need 2^N different tries on average to get one. No matter what you throw in it, be it previous hash + your wallet ID + nonce or previous hash + nonce, or random string, or hash of a random wikipedia page.

Yes, I know. There were such problems about protein folding, where we knew that a particular protein folds, and its potential chemical energy falls from X to X’, following an ever decreasing path which we don’t know. Miners have to iterate over possible rotations of each part of the molecule. If someone finds a solution it’s trivial to check that the protein indeed reached its final form and that its potential energy is always decreasing along the path. Hard to find, easy to check. No need of NASA API for other miners to check that the solution is correct.

I know that too. The miner just submits (previous hash, its solution, its walletID).
This gives a new hash to include and gives the next molecule to work on.
(In this particular example, project devs published a pre-shared list of millions of molecules, and the hash of the previous block would point to the next molecule to study. The weak point is that the public researchers having made this beauty could have precomputed all or some of the solutions in the list. Consequently, it’s not a zero-trust system, but it illustrates that useful PoW is possible.)

@Akita
Wow indeed the idea of protein folding is brilliant. But one of the problems is that the mining complexity is constant whereas in Bitcoin it is proportional to the amount of computing power of the miners. What it means is that the time to create a new block will decrease more and more as new miners enter the network.

I was referring to the NASA API because I heard a suggestion to make an useful PoW: each time a new block is created, the miners lend their computing power to Facebook, NASA or Google for 9 minutes and then do ‘useless’ PoW for 1 minute. I think it is a bad idea because it wouldn’t be decentralized (it would rely on trusted third-parties like APIs).

Currently, PoW operates like that:
When a miner finds a new block, he broadcasts it to the network and directly after he opens the next block. In this block, the previous hash is the one he just found, and in the data he puts all the pending transactions (the ones which have been added to the queue during the 10 minutes) plus his reward. Then he broadcasts this block to the network in order to start the mining, so he is the source of authority for “which transactions must be included in the next block to mine”.

My suggestion was: The winning miner should first broadcast the block he has just found to the nodes of the network, then wait the remaining time before 10 minutes to open the next block. During this time, new transactions are added to the queue. If he doesn’t wait and sends directly the next block, all the other nodes of the network would reject this block.

What I meant by “all the transactions available” is all the transactions in the queue at 10 minutes when the new block is opened.

To know that you have all the transactions needed to start mining, you just have to wait until you receive the next block sent by the winning miner, as he is the source of authority for this block.

But I understand this idea of waiting until 10 minutes has lots of problems so I will just abandon it.

But what prevents this currently in Bitcoin, Ethereum etc?

Let’s say we find hashes with 30 leading zeros in 10 minutes with current PoW, but with my idea in 5 minutes (I’m (maybe) exaggerating), because if you explore more nonces in as much time you will incidentally find the correct hash in less time. To go back to 10 minutes, we increase the complexity of mining, i.e. the number of leading zeros. For example, if we use my idea, we would have 31 leading zeros instead of 30. But now, 51% attacks would be way harder, because an attacker would have to create new blocks with 31 leading zeros faster than the rest of the miners, which is harder than with 30 zeros using the current PoW. Hence the system is safer.

This can be solved by “Now a miner must submit the solution for N molecules at once to win”.

Yes, I don’t like it either. But Ethereum uses this idea with a trick: the winner gets to execute the snippet submitted by users (for example the smart contracts). Everyone can submit them… The “transaction queue” is replaced by a “run that” queue. “transaction fee” is replaced by gas, which is proportionnal to the amount of computation needed to complete the task. The more computation you need, the more “fee” you have to pay the miner. this is to prevent “crack 2048 bit encryption for a few cents, plz”.

No. They put 10Mb-worth of transaction (for example), which is the block size of the blockchain. They get to chose which transactions in the much bigger “transaction queue” they fit in the block. It’s in their best interest to fill the block as much as possible (not more or their block will be rejected by the network) and to select the ones with the highest fees (or rather, the best ratio fee/byte). That is why, as a Bitcoin user, you have to put a higher reward if you want your transaction to be processed fast so that it is select by the miner. If there are too many transactions, you have to pay a lot to get your transaction through, but the number of transaction is limited by the block size, which is hardcoded in most PoW blockchains. That’s why back at the time we said “Bitcoin can’t scale!” and people tried to come with solutions (hardfork to increase the block size, compress signatures, Lightning Network to register most transactions off-chain, etc.).
This size limitation severely limit the performance and the throughput, but also limit the blockchain (the registry, not the network) and enable small nodes to be able to check ALL transactions (mining is hard, but check 10Mb every 10 min is easy). Nakamoto chose decentralization and security over throughput and low cost of transacting.

For Ethereum, I’m less sure, but I think there is no block size limit. The “gas” has a price, so users only submit if it’s worth it. The gas price is dynamic, but I can’t remember how. Mining is of course way harder than validating, but the blockchain is still growing too fast for validators to keep up with common hardware. This is a tremendous problem for decentralization and security. It means that you, as a user, can no longer check the integrity of the chain since it grows faster than you validate it (!) So you have to trust someone that it is indeed valid. They try to hide it with confusing different roles for nodes (that they all refer to as “nodes”), being less and less transparent about the health of the network, and they buy timewith sharding. I was a big fan of Ethereum, and I considered Bitcoin as a “crutial but now outdated experiment”, but it’s no longer the case. (Bitcoin-like PoW is still an energetic catastrophe that must end soon, though!)

You won’t decrease or increase the average time to find a good hash.
Traditional PoW is: Miners all plays with random number in the casino’s roulette. Whoever hits first a N-win streak wins the game.
Your Pow is: each miner plays on a specific number, different for each player. Whoever hits first a N-win streak wins the game.

In both case, an attacker with more than 50% of the total hashrate wins and takes over. No matter what hashes pattern the network asks them to try.

It will be harder for the attacker, and harder for the defenders. If you don’t adjust the difficulty, you’ve just improved the throuput at the expense of the decentralization. If you do, you’re back to square one : the same hash power computing non-stop for ten minutes, and whoever controls 51% takes over.

I may have missed something myself, and I would love to be proved wrong with a simulation with low difficulty, iterated 1000 times, in the case of 1000 nodes computing random nonces on their own walletID, and 1000 nodes computing only their allowed nonces following your method. Don’t mesure the time needed by your program to switch from one walletID to the other or from one modulo to the next of course.

Yes, you missed what I call the overlap, the amount of nonces that are tested several times during mining. I’m sorry, I can’t do a benchmark right now because all I have is a smartphone, but I will try to explain my reasoning. Let’s simplify the problem:

We have only 2 miners (in the case where we use my idea, let’s say that the first one mines all the even numbers and the other one all the odd numbers)

The nonces are 3 bits long (only 8 possible values, we have to simplify)

The complexity of mining is 2 leading zeros in the hash

=> The number of correct nonces is 2^3/2^2=2 (on average)
=> The probability to get a correct hash in 1 test is 2/8=1/4 (on average)

Method 1: Random tests

Probability to get a correct hash in the 1st try: 1/4
Probability to get a correct hash in the 2nd try: 1/4
Probability to get a correct hash in the 3rd try: 1/4
…

Method 2: Optimized tests (my method)

Probability to get a correct hash in the 1st try: 1/4
Probability to get a correct hash in the 2nd try: 1/3 (we are sure that we won’t try the same nonce as in the first try, so we reduce the list of nonces that may be correct)
Probability to get a correct hash in the 3rd try: 1/2
Probability to get a correct hash in the 4th try: 1

So if we do the math, the probability to get a correct hash in 2 tries:

with method 1: (1/4)+(3/4)(1/4)=7/16=0.4375

with method 2: (1/4)+(3/4)(1/3)=1/2=0.5

So we have actually more chances to get a correct hash in 2 tries using my method than using the random method. And imagine how much more overlap there must be with as many miners as in Bitcoin.

If all the miners of the network try nonces randomly without making sure that they haven’t tried them before (i.e. the stupid way) then we can actually calculate the amount of overlap because we can consider them as one unique miner who tries nonces the stupid way. For example if there are only 64 possible values of nonces (6 bits nonce) then the probability of having overlap on the second try is 1/64, the probability to have overlap on the third try is (1/64)(1/64)+(63/64)(2/64) etc. To do the calculations, we have to know the 3 parameters: the number of possible nonces (apparently it’s 2^32 for Bitcoin), the hashrate of the miners and the complexity of mining (number of leading zeros in the hash).

So yes, we can still break my system with 51% of the mining power, but in fact you can have less than 51% of the mining power to break the current PoW system! (assuming the attacker does an efficient exploration of nonces and all the other miners do a random exploration)

Besides, I assume miners currently use LFSR (linear-feedback shift registers) to generate pseudo-randomly the nonce they will try, and if they do I’m practically sure that using my method is quicker to generate a nonce. So even if my method wasn’t more efficient (which I’m almost sure it is), at least it would be a little bit faster because we would generate nonces slightly faster.

But in the end I agree with you, the Bitcoin PoW system has to disappear. So even if my idea was useful to increase the security of the blockchain, it is just a patch to an obsolete system.

I don’t have time to answer at length now, but I challenge the assumption that probability of finding a good nonce will be 1/4, then 1/3, then 1/2 and finally 1 with your method. You seem to consider that 2 nonces cannot produce the same hash, but it’s not true. Your 4th nonce can output the same hash as the first one for example. It has 1/8 chance to do so. You don’t necessarily exhaust all possible outputs by testing the first 8 hashes.