provider.getTransaction discussion (v5 vs v6 changes)

[NaliFinance]

Ethers Version

6.7.1

Search Terms

keccak v6

Describe the Problem

The keccak256 function gives a different result upgrading to v6

V6 code throw the exception.. what is changed from V5 to V6?

Code Snippet

//V5 code
if (ethers.utils.keccak256(raw) !== tx.hash)
    throw new Error("Invalid tx signature");

//V6 code
if (ethers.keccak256(raw) !== tx.hash)
    throw new Error("Invalid tx signature");

Contract ABI

No response

Errors

No response

Environment

Ethereum (mainnet/ropsten/rinkeby/goerli), node.js (v12 or newer), Geth

Environment (Other)

No response

[ricmoo]

Can you please include the raw input and result of tx.hash? Just use console.log to output them. :)

[NaliFinance]

Yes! of course! what a fast answer! anyway, i posted directly because this exact code is working with V5 but not on V6 and i would like to update my app

raw:
0x02f902ba01068402faf0808508d8f9fc008303c047943fc91a3afd70395cd496c647d5a6cc9d4b2b7fad8806f05b59d3b20000b902843593564c000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000065a6ec3300000000000000000000000000000000000000000000000000000000000000020b080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000040000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000006f05b59d3b200000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000006f05b59d3b20000000000000000000000000000000000000000000000002b48cb92ceacaf0ed61600000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000c02aaa39b223fe8d0a0e5c4f27ead9083c756cc20000000000000000000000008edc62648236c20e528adb675ad3f3047697186fc0808080

txHash: '0x65723d375b20024371f80b97d9e89e3260356546b5715263839d66d2f04fbdba',
keccakV6: '0x7ecbfd837a76a70d446beed59b3d8cbd75da7037611a5e0d8c0a382c43f7680e'

i also tried to don't serialize the transaction but result is the same.. transactions raw come from:
let txData = stringifyBN(tx, true);

switch (tx.type) {
case 0: {
transaction = Transaction.fromTxData(txData, {common});
break;
}
case 1: {
transaction = AccessListEIP2930Transaction.fromTxData(txData, {common});
break;
}
case 2: {
transaction = FeeMarketEIP1559Transaction.fromTxData(txData, {common});
break;
}
default: {
throw new Error("Invalid tx type");
}
}

if (sign) transaction = transaction.sign(privateKey);

const raw = '0x' + transaction.serialize().toString('hex')

[ricmoo]

I get "0x7ecbfd837a76a70d446beed59b3d8cbd75da7037611a5e0d8c0a382c43f7680e" in both v5 and v6 for that hexstring.

I also get the correct hash in both v5 and v6:

// v5
parseTransaction(data).hash
// "0x7ecbfd837a76a70d446beed59b3d8cbd75da7037611a5e0d8c0a382c43f7680e"

// v6
Transaction.from(data).hash
// "0x7ecbfd837a76a70d446beed59b3d8cbd75da7037611a5e0d8c0a382c43f7680e"

I'm not quite sure what the above code is doing; are you using the EthereumJS library for that? I'm not super familiar with their API, but it looks like it.

If you want to produce a signed transaction, you should be able to do something like:

// Create a Transaction object; starts off unsigned
const txParams = { type, to, value, data }; // and such
const tx = Transaction.from(txParams);

// Or if you have the serialized transaction already
// const tx = Transaction.from(txData)

// Add a signature by signing the unsignedHash
tx.signature = signingKey.sign(tx.unsignedHash);

// The signed transaction hex:
tx.serialized

If you provide concrete examples of the inputs and outputs, I can provide examples with the values interleaved. :)

[ricmoo]

Random note; in v6 the Transaction object is a live object; you can update explicit values and the derived values are getters that will automatically be updated:

> t = new Transaction()
> t.unsignedSerialized
'0x02c98080808080808080c0'
> t.value = 123
> t.unsignedSerialized
'0x02c98080808080807b80c0'
> t.to = "0x1234567890123456789012345678901234567890"
> t.unsignedSerialized
'0x02dd80808080809412345678901234567890123456789012345678907b80c0'
> t.type = 1
> t.unsignedSerialized
'0x01dc808080809412345678901234567890123456789012345678907b80c0'

[NaliFinance]

i imported both ethers v5 and v6 in the project.. and this is what i discovered

hashes from mempool are different because txs are different, but the focus is on that V5 gives me always the same calculated keccak of tx hash from mempool and a tx read from V6 doesn't

tx read from V5 provider (provider = new ethers.providers.WebSocketProvider(WSS);)
hashFromMempool: '0x5d6cb064b5a40c03d7358023b0a9f8c74074848621bb2b6a0f9b9dab7141c1de',
ethers5keccak: '0x5d6cb064b5a40c03d7358023b0a9f8c74074848621bb2b6a0f9b9dab7141c1de',
ethers6keccak: '0x5d6cb064b5a40c03d7358023b0a9f8c74074848621bb2b6a0f9b9dab7141c1de'

tx read from V6 provider (provider = new ethers.WebSocketProvider(WSS);)
hashFromMempool: '0x86bfee38299d4aee5096ec9eb21b2fbae42a543106ccfbedb07fcde6aa2215d6',
ethers5keccak: '0xf9fddee0a2d495c6995fe6f19c1cfb29235697f6cfa738a2e7c49859b4e82ca3',
ethers6keccak: '0xf9fddee0a2d495c6995fe6f19c1cfb29235697f6cfa738a2e7c49859b4e82ca3'

when i get a raw transaction from provider created with V5 the keccak of:
const raw = '0x' + transaction.serialize().toString('hex')

is the same as the hash i get from mempool, when i use the provider created with V6 is different. Keccak calc is the same for V5 and V6, verified. Something else is happening, the codebase is the same 😱

i'm trying to understand what is causing this difference digging more, but it's clear that there's a difference, i just still don't know what it is

[ricmoo]

Oh! Can you also make sure you update to the latest version, 6.10.0? There is an issue where nodes return different values for things in the mempool.

One thing we noticed is that yParity for pending transactions is ”0x0” and "0x1" (a hexstring) but for mined transactions it is 0 and 1 (a number).

[NaliFinance]

Yes, i tried also with 6.10.0, i found the problem... i think that there's absolutely no bugs, it's just the type changed from V5 provider to V6, so passing the tx to ethereumjs is not valid anymore.. i didn't find this mentioned in the migration guide
https://docs.ethers.org/v6/migrating/

This is a V5 provider transaction from getTransaction method:
tx {
hash: '0x5d6cb064b5a40c03d7358023b0a9f8c74074848621bb2b6a0f9b9dab7141c1de',
type: 2,
accessList: [],
blockHash: '0xb2f9f97d507644def76fe78d0504e6dbe2a8d2a91df1ff5b5c431474f7275cd3',
blockNumber: 19022505,
transactionIndex: 1,
confirmations: 92,
from: '0xA1Fb6F42CAdBCC8B4c1276DF222e4EfD320Ea03E',
gasPrice: BigNumber { _hex: '0x085a4071e6', _isBigNumber: true },
maxPriorityFeePerGas: BigNumber { _hex: '0x05f5e100', _isBigNumber: true },
maxFeePerGas: BigNumber { _hex: '0x0a87a20149', _isBigNumber: true },
gasLimit: BigNumber { _hex: '0x031ce9', _isBigNumber: true },
to: '0x3fC91A3afd70395Cd496C647d5a6CC9D4B2b7FAD',
value: BigNumber { _hex: '0x058d15e176280000', _isBigNumber: true },
nonce: 671,
data: '0x3593564c000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000065a7088f00000000000000000000000000000000000000000000000000000000000000020b080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000058d15e17628000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000058d15e17628000000000000000000000000000000000000000000000000f02206ff59ec574b421200000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000c02aaa39b223fe8d0a0e5c4f27ead9083c756cc20000000000000000000000000c5cb676e38d6973837b9496f6524835208145a2',
r: '0x26554f7972686716409a8c7e1672fe17017625959f94dbf61e8cbb5ca4ec0c70',
s: '0x3cbe7f0cbecf2a51d5f5d37f24893d0c60416c7c0fa153fc56e4cd96124844d3',
v: 0,
creates: null,
chainId: 1,
wait: [Function (anonymous)]
}

this is V6:
tx TransactionResponse {
provider: WebSocketProvider {},
blockNumber: 19022505,
blockHash: '0xb2f9f97d507644def76fe78d0504e6dbe2a8d2a91df1ff5b5c431474f7275cd3',
index: undefined,
hash: '0x5d6cb064b5a40c03d7358023b0a9f8c74074848621bb2b6a0f9b9dab7141c1de',
type: 2,
to: '0x3fC91A3afd70395Cd496C647d5a6CC9D4B2b7FAD',
from: '0xA1Fb6F42CAdBCC8B4c1276DF222e4EfD320Ea03E',
nonce: 671,
gasLimit: 204009n,
gasPrice: 35873911270n,
maxPriorityFeePerGas: 100000000n,
maxFeePerGas: 45225214281n,
data: '0x3593564c000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000065a7088f00000000000000000000000000000000000000000000000000000000000000020b080000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000400000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000058d15e17628000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000058d15e17628000000000000000000000000000000000000000000000000f02206ff59ec574b421200000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000c02aaa39b223fe8d0a0e5c4f27ead9083c756cc20000000000000000000000000c5cb676e38d6973837b9496f6524835208145a2',
value: 400000000000000000n,
chainId: 1n,
signature: Signature { r: "0x26554f7972686716409a8c7e1672fe17017625959f94dbf61e8cbb5ca4ec0c70", s: "0x3cbe7f0cbecf2a51d5f5d37f24893d0c60416c7c0fa153fc56e4cd96124844d3", yParity: 0, networkV: null },
accessList: []
}

Same transaction of course, values are the same but structure is different.. i wonder if there is some method to get the tx in the old format, or i need to update my method to check the hash, maybe updating the ethereumjs lib

[ricmoo]

Oh, I see.

My guess is because the various Signature properties (r, s, v, yParity, etc) are wrapped up in a separate signature property now. Can you try this out to see if it fills in everything you need for EthereumJS transactions:

const flatTx = Object.assign({ }, tx.toJSON(), tx.signature?.toJSON());

The other option, I wonder, is there a way to pass the tx.serialized into a method of EthereumJS that reconstructs the object? The serialized form contains everything. :)

[NaliFinance]

I tried both, the final solution is this (this is quick&dirty, just to test it)
transaction = Object.assign({ }, transaction.toJSON(), transaction.signature?.toJSON());
transaction.chainId = new BigNumber(transaction.chainId);
transaction.gasLimit = new BigNumber(transaction.gasLimit); // all bignumbers because etherumjs wants hex strings or bignumber for these properties
transaction.gasPrice = new BigNumber(transaction.gasPrice);
transaction.maxFeePerGas = new BigNumber(transaction.maxFeePerGas);
transaction.maxPriorityFeePerGas = new BigNumber(transaction.maxPriorityFeePerGas);
transaction.value = new BigNumber(transaction.value);
transaction.v = parseInt(transaction.signature?.yParity);

transaction is the TransactionResponse from pending event, it works like the old v5 response

but, with the serialized i get undefined on transaction.serialized (using v6.10.0)

anyway, i got one working solution, but i don't know if it's really optimized to make those operations or switch back to ethers-v5 and use the transaction as is.. (i need to scan 500-1000 txs every block)

maybe there is a better way to accomplish the result

[ricmoo]

Oh!

I think you don't have a Transaction object, but a TransactionResponse? Are you getting this from provider.getTransaction(hash)? You need to convert it to a Transaction object first to use the live features.

For example:

const provider = new ethers.InfuraProvider();

const txHash = "0xa8e82eed99f8266999decb5ee4ac0fed95ceb1519581d4d3c079c4029c775acc";

// This is just a response from the server; it has only had minor validation done and is a regular object
const txResp = await provider.getTransaction(txHash);

// This object is the fully live instance
const tx = Transaction.from(txResp);

console.log(tx.serialized);
// "0x02f8760183057889843b9aca00853c89352800830186a094b745a4f78f9c468cb9c8e4b74d039dc0ca2ecacb87e880f8a4ae401380c080a0fc739c87ecb5302390c056f7372f7c2b6aff0df3a3e987b154b07797b3ff49d0a07d1db03dede03327046d51d315b4b2a72b3f5ab988973fdf2eabdfa06e6d7c1f"

console.log(tx.serialized);
// "0xa8e82eed99f8266999decb5ee4ac0fed95ceb1519581d4d3c079c4029c775acc"

Does that make sense?

[NaliFinance]

yes, i wrote that is a TransactionResponse, anyway will try that also, but what's the performance gain using ethers-v6 in the project? is the Transaction.from() and expensive operation if i have to scan the mempool and analyze every tx?
Because now i have to choose if continue with ethers-v5 or the v6..

[ricmoo]

Transaction.from is basically just doing what you were doing, copying over the values. But it computes derived values lazily, so it doesn’t compute anything you don’t need.

There hasn’t been a v5 release in over a year. I am planning one to fix a few small things over the next week, so there will be a minor version bump of v5 soon, but using v6 will ensure you get the latest bug fixed and blockchain features.

I haven’t done any benchmarks but v6 uses Nobel/BigInt for all internal operations while v5 uses elliptic/BN.js, so I would expect a much faster from address calculation. Would love feedback on that though. The hashing library used by v6 should be faster too.

That said, v5 remains stable, which is why it hasn’t required any updates in a year. :)

[NaliFinance]

I don't know why but passing the TransactionResponse object to Transaction.from i get an empty object, anyway it seems that the from() takes a few milliseconds to run, maybe it's better for me to keep the V5 until i sort out all the things, i'm in the middle of a refactoring, that's why i wanted to try out the V6 (i'm always happy to use the latest available versions)

time: 3.8791000000055647 tx Transaction {}
time: 4.014899999994668 tx Transaction {}
time: 3.8870000000024447 tx Transaction {}
time: 5.064299999998184 tx Transaction {}
time: 4.975999999995111 tx Transaction {}

while converting with the object.assign seems a lot faster (and working!)
time: 0.04690000000005057
time: 0.010999999999967258
time: 0.024100000000089494
time: 0.020400000000108776
time: 0.01999999999998181

As soon as i finish the refactoring i will test the whole code with v5 and v6 because it's important for me to have the fastest code possible

[NaliFinance]

i changed the title so people don't get confused reading it

[ricmoo]

I've moved it to Q&A discussions as well. ;)

[ricmoo]

@NaliFinance The object isn't empty. The console.log in Node.js cannot safely access getters. But it is not empty; if you try tx.to you will see it's present. If you use a Browser, it will show it as a getter.

If you spot anything in the Transactin object you believe could be causing it to be slower too, please let me know. :)