By Mikhail Vladimirov
Ethereum is a programmable blockchain, whose functionality could be extended by publishing pieces of executable code, known as smart contracts, into the blockchain itself. This distinguishes Ethereum from the first generation of blockchains, where new functionality requires client software to be modified, nodes to be upgraded, and the whole blockchain to be forked.
A smart contract is a piece of executable code published on-chain, that has a unique blockchain address assigned to it. Smart contract controls all the assets belonging to its address and may act on behalf of this address when interacting with other smart contracts. Each smart contract has persistent storage that is used to preserve the smart contract state between invocations.
Solidity is the primary programming language for smart contract development on Ethereum, as well as on several other blockchain platforms that use Ethereum Virtual Machine (EVM).
Programming was always about math, blockchain was always about finance, and finance was about math since ancient times (or maybe math was about finance). Being the primary programming language for Ethereum blockchain, Solidity has to do math well.
In this series, we discuss various aspects of how Solidity does the math, and how developers do math in Solidity. The first topic to discuss is numbers.
Numeric Types in Solidity
In comparison to mainstream programming languages, Solidity has quite many numeric types: namely 5,248. Yes, according to the documentation, there are 32 signed integer, 32 unsigned integer, 2592 signed fixed-point, and 2592 unsigned fixed-point types. JavaScript has only two numeric types. Python 2 used to have four, but in Python 3 type “long” was dropped, so now there are only three. Java has seven and C++ has something about fourteen.
With so many numeric types, Solidity should have proper type for everybody, right? Not so fast. Lets look at these numeric types a bit closer.
We will start with the following question:
Why Do We Need Multiple Numeric Types?
Spoiler: we don’t.
There are no numeric types in pure math. A number may be integer or non-integer, rational or irrational, positive or negative, real or imaginary etc, but these are just properties, the number may or may not have, and single number may have several such properties at once.
Many high-level programming languages have single numeric type. JavaScript had only “number” until “BigInt” was introduced in 2019.
Unless doing hardcore low-level stuff, developers don’t really need multiple numeric types, they just need pure numbers with arbitrary range and precision. However, such numbers are not natively supported by hardware, and are somewhat expensive to emulate in software.
That’s why low-level programming languages and languages aimed at high performance usually have multiple numeric types, such as signed/unsigned, 8/16/32/64/128 bits wide, integer/floating-point etc. These types are natively supported by hardware and are widely used in file formats, network protocols etc, thus low-level code benefits from them.
However, for performance reasons, these types usually inherit all the weird semantics of underlying CPU instructions, such as silent over- and underflow, asymmetric range, binary fractions, byte ordering issues etc. This makes them painful in high-level business logic code. Straightforward usage often appears insecure, and secure usage often becomes cumbersome and unreadable.
So, the next question is:
Why Does Solidity Has So Many Numeric Types?
Spoiler: it doesn’t.
EVM natively supports two data types: 256-bit word and 8-bit byte. Stack elements, storage keys and values, instruction and memory pointers, timestamps, balances, transaction and block hashes, addresses etc are 256-bit words. Memory, byte code, call data, and return data consist of bytes. Most of the the EVM opcodes deal with words, including all math operations. Some of the math operations treat words as signed integers, some as unsigned integers, while other operations just work the same way regardless of whether arguments are signed on unsigned.
So EVM natively supports two numeric types: signed 256-bit integer and unsigned 256-bit integer. These types are known in Solidity as int and uint respectively.
Apart from these two types (and their aliases int256 and uint256) Solidity has 62 integer types int<N>, and uint<N>, where <N> could be any multiple of 8 from 8 to 248, i.e. 8, 16, …, 248. On EVM levels, all these types are backed by the same 256-bit words, but result of every operation is truncated to N bits. They could be useful for specific cases, when particular bit width is needed, but for general calculations these types are just less powerful and less efficient (truncating after every operation is not free) versions of int and uint.
Finally, Solidity has 5184 fixed-point types fixedNxM and ufixedNxM where N is multiple of 8 from 8 to 256 and N is an integer number from 0 to 80 inclusive. These types are supposed to implement decimal fixed-point arithmetic of various range and precision, but as of now (Solidity 0.6.2) the documentation says that:
Fixed point numbers are not fully supported by Solidity yet. They can be declared, but cannot be assigned to or from.
So fixed-point numbers as well as fractions numbers in general are not currently supported.
Then, the next question is:
What If We Need Fraction Numbers or Integers Bigger Than 256 Bit?
Spoiler: you have to emulate them.
One would say, that 256 bit ought to be enough for anybody. However, once most of the numbers in Ethereum are 256 bit wide, even simple sum of two numbers may be as wide as 257 bit, and product of two numbers may be up to 512 bit wide.
Common way to emulate fixed or variable width integers numbers, that are wider, than types natively supported by programming language, is to represent them as fixed or variable length sequences of shorter, natively supported integer numbers. So bit image of wide integer is the concatenation of bit images of shorter integers.
In Solidity, wide integers may be represented as fixed or dynamic arrays whose elements are either bytes or uint vales.
For fractions situation is a bit more complicated, as as there are different flavors of them, each having its own advantages and drawback.
The most basic are simple fractions: just one integer, called “numerator”, divided by another integer, called “denominator”. In Solidity simple fraction could be represented as a pair of two integers, or as a single integer, whose bit image is the concatenation of bit images of numerator and denominator. In the latter case, numerator and denominator has to be of the same width.
Another popular format for fractions is fixed-point numbers. Fixed-point number is basically a simple fraction whose denominator is a predefined constant, usually power of 2 or 10. The former case is known as “binary” fixed-point, while the latter is known as “decimal” fixed-point. As long as denominator is predefined, there is no need to specify is explicitly, so only the numerator need to be specified. In Solidity fixed-point numbers are usually represented as a single integer numerator, while commonly used denominators are 10¹⁸, 10²⁷, 2⁶⁴, and 2¹²⁸.
Yet another well-known format for fraction numbers is floating-point. Basically, floating point number could be described as following: m×B^e, where m (mantissa) and e (exponent) are integers, while B (base) is a predefined integer constant, usually 2 or 10. The former case is known as “binary” floating-point, and the latter case is known as “decimal” floating-point.
IEEE-754 standardizes several common floating-point formats, including five binary formats known as “half”, “single”, “double”, “quadruple”, and “octuple” precision. Each of these formats packs both, mantissa and exponent, into single sequence of 16, 32, 64, 128, or 256 bits respectively. In Solidity, these standard formats could be represented by binary types bytes2, bytes4, bytes8, bytes16, and bytes32. Alternatively, mantissa and exponent could be represented separately as a pair of integers.
And the file question for this section:
Do We Have to Implement All This by Ourselves?
Spoiler: not necessary.
The good news is that there are Solidity libraries for various number formats, such as: fixidity (decimal fixed-point with arbitrary number of decimals), DSMath (decimal fixed-point with 18 or 27 decimals), BANKEX Library (IEEE-754 octuple precision floating-point), ABDK Libraries (binary fixed-point and quadruple precision floating-point) etc.
The bad news is that different libraries use different formats, so it is really hard to combine them. The roots of this problem will be discussed in the next section.
Numeric Literals in Solidity
In the previous section we discussed how numbers are represented at run time. Here we will look at how they are represented at the development time, i.e. in the code itself.
Compared to mainstream languages, Solidity has quite a rich syntax for numeric literals. First of all, good old decimal integers are supported, such as 42. As in other C-like languages, there are hexadecimal integer literals, like 0xDeedBeef. So far so good.
In Solidity, literals may have unit suffix, such as 6 ether, or 3 days. A unit, is basically a factor, the literal is multiplied by. Here ether is 10¹⁸ and days is 86,400 (24 hours × 60 minutes × 60 seconds).
Apart from this, Solidity supports scientific notation for integer literals, such as 2.99792458e8. This is quite unusual, as mainstream languages support scientific notation for fractional literals only.
But probably the most unique feature of the whole Solidity language, is its support for rational literal expressions. Virtually every mature compiler is able to evaluate constant expressions at compile time, so x = 2 + 2 does not generate add opcode, but is rather equivalent to x = 4. Solidity is able to do this as well, but actually, it goes far beyond that.
In mainstream languages, compile-time evaluation of constant expression is just an optimization, so constant expression is evaluated at compile time exactly the same way as it would be evaluated at run time. This makes it possible to replace any part of such expression with named constant or variable holding the same value, and get exactly the same result. However, for Solidity this is not the case.
At run time, division in Solidity rounds result towards zero, and other arithmetic operations wraps on overflow, while at compile time, expressions are evaluated using simple fractions with arbitrary large numerator and denominator. So, at run time, expression ((7 / 11 + 3 / 13) * 22 + 1) * 39 would be evaluated to 39, while at compile time the very same expression is evaluated to 705. The difference is because at run time, 7 / 11 and 3 / 13 are rounded to zero, but at compile time, the whole expression is evaluated in simple fractions without any rounding at all.
Even more interesting, the following expression is is valid in Solidity: 7523 /48124631 * 6397, while this is not valid: 7523 / 48125631 * 6397. The difference is that the former evaluates to integer number, while the latter evaluates to non-integer. Remember, that Solidity do not support fractions at run time, so all literals have to be integer.
While fractional numbers and big integers may be represented in Solidity at run time, as described in the previous sections, there is no convenient way to represent them in the code. This makes any code, that performs operations with such numbers, rather cryptic.
As long as Solidity does not have a standard fixed-point nor floating-point format, every library uses its own, which makes libraries incompatible with each other.
Overflow
Every time I see +, *, or ** doing audit of another Solidity smart contract, I start writing the following comment: “overflow is possible here”. I need a few seconds to write these four words, and during these seconds I observe nearby lines trying to find a reason, why overflow is not possible, or why overflow should be allowed in this particular case. If the reason is found, I delete the comment, but most often the comment remains in the final audit report.
Things aren’t meant to be this way. Arithmetic operators supposed to allow writing compact and easy to read formulas such as a**2 + 2*a*b + b**2. However, this expression would almost definitely raise a bunch of security concerns, and the real code is more likely to look like this:
add (add (pow (a, 2), mul (mul (2, a), b)), pow (b, 2))
Here add, mul, and pow are functions implementing “safe” versions of +, *, and ** respectively.
Concise and convenient syntax is discouraged, plain arithmetic operators are marginally used (and not more than one at a time), cumbersome and unreadable functional syntax is everywhere. In this article we analyse the problem, that made things so weird, whose infamous name is: overflow.
We Took a Wrong Turn Somewhere
One would say, that overflow was always there, and all programming languages suffer from it. But is this really true? Did you ever see something like SafeMath library implemented for C++, Python, or JavaScript? Do you really think, that every + or * is a security breach, until the opposite is proven? Most probably, your answer for both questions is “no”. So,
Why Overflow in Solidity Is So Much Painful?
Spoiler: nowhere to run, nowhere to hide.
Numbers do not overflow in pure math. One may add two arbitrary large numbers and get precise result. Numbers do not overflow in high-level programming languages such as JavaScript and Python. In some cases the result could fall into infinity, but at least adding two positive numbers may never produce negative result. In C++ and Java integer numbers do overflow, but floating-point numbers don’t.
In those languages, where integer types do overflow, plain integers are used primarily for indexes, counters, and buffer sizes, i.e. for values limited by the size of data being processed. For values, that potentially may exceed range of plain integers, there are floating-point, big integer, and big decimal data types, either built-in or implemented via libraries.
Basically, when the result of an arithmetic operation does not fit into the type of the arguments, there are a few options what compiler may do: i) use wider result type; ii) return truncated result and use side channel to notify the program about overflow; iii) throw an exception; and iv) just silently return truncated result.
The first option is implemented in Python 2 when handling int type overflows. The second option is what carry/overflow flags in CPUs are for. The third option is implemented for Solidity by SafeMath library. The fourth option is what Solidity implements by itself.
The fourth option is probably the worst one, as it makes arithmetic operations error-prone, and at the same time makes overflow detection quite expensive, especially for multiplication case. One needs to perform additional division after every multiplication to be on the safe side.
So, Solidity neither has safe types, one could run to, nor it has safe operations, one could hide behind. Having nowhere to run and nowhere to hide, developers have to meet overflows face to face and fight them all throughout the code.
Then, the next question is:
Why Doesn’t Solidity Have Safe Types Nor Operations?
Spoiler: because EVM don’t have them.
Smart contracts have to be secure. Bugs and vulnerabilities in them cost millions of dollars, as we’ve already learned the hard way. Being the primary language for smart contracts development, Solidity takes security very seriously. If has many features supposed to prevent developers from shooting themselves in the feet. We mean features like payable keyword, type cast limitations etc. Such features are added with every major release, often breaking backward compatibility, but the community tolerates this for the sake of better security.
However, basic arithmetic operations are so unsafe that almost nobody use them directly nowadays, and the situation doesn’t improve. The only operation that became a bit safer is division: division by zero used to return zero, but now it throws an exception, but even division didn’t become fully safe, as it still may overflow. Yes, in Solidity int type division overflows when -2¹²⁷ is being divided by -1, as correct answer (2¹²⁷) does not fit into int. All other operations, namely +, -, *, and ** are still prone to over- or underflow and thus are intrinsically unsafe.
Arithmetic operations in Solidity replicate the behavior of corresponding EVM opcodes, and making these operations safe at compiler level would increase gas consumption by several times. Plain ADD opcode costs 3 gas. The cheapest opcode sequence for implementing safe add the author of the article managed to find is:
DUP2(3) DUP2(3) NOT(3) LT(3) <overflow>(3) JUMPI(10) ADD(3)
Here <overflow> is the address to jump on overflow. Numbers in brackets are gas costs of the operations, and these numbers give us 28 gas in total. Almost 10 times more, than plain ADD. Too much, right? It depends on what you compare with. Say, calling add function from SafeMath library would cost about 88 gas.
So, safe arithmetic at library or compiler level costs much, but
Why Doesn’t EVM Have Safe Arithmetic Opcodes?
Spoiler: for no good reason.
One would say that arithmetic semantic in EVM replicates that of CPU for performance reasons. Yes, some modern CPUs have opcodes for 256-bit arithmetic, however mainstream EVM implementations don’t seem to use these opcodes. Geth uses big.Int type from the standard library of Go programming language. This type implements arbitrary wide big integers backed by arrays of native words. Parity uses its own library implementing fixed-width big integers on top of native 64-bit words.
For both implementations, additional cost of arithmetic overflow detection would virtually be zero. Thus, once EVM would have versions of arithmetic opcodes, that revert on overflow, their gas cost could be made the same as for existing unsafe versions, or just marginally higher.
Even more useful would be opcodes that do not overflow at all, but return the whole result instead. Such opcodes would permit efficient implementation of arbitrary wide big integers at compiler or library level.
We don’t know why EVM doesn’t have the opcodes described above. Maybe just because other mainstream virtual machines don’t have them?
So far we were telling about real overflow: a situation when calculation result is too big to fit into the result data type. Now it is time to discover the other side of the problem:
Phantom Overflows
How one would calculate 3% of x in Solidity? In mainstream languages one just writes 0.03*x, but Solidity doesn’t support fractions. What about x*3/100? Well, this will work in most cases, but what if x is so large, that x*3 will overflow? From the previous section we know what to do, right? Just use mul from SafeMath and be in the safe side: mul (x, 3) / 100… Not so fast.
The latter version is somewhat more secure, as it reverts where the former version returns incorrect result. This is good, but… Why on earth calculating 3% of something may ever overflow? 3% of something is guaranteed to be less that original value: in both, nominal and absolute terms. So, as long as x fits into 256-bit word, then 3% of x should also fit, shouldn’t it?
Well, I call this “phantom overflow”: a situation when final calculation result would fit into the result data type, but some intermediate operation overflows.
Phantom overflows are much harder to detect and address than real overflows. One solution is to use wider integer type or even floating-point type for intermediate values. Another is to refactor the expression in order to make phantom overflow impossible. Let’s try to do the latter with our expression.
Arithmetic laws tell us that the following formulas should produce the same result:
(x * 3) / 100
(3 * x) / 100
(x / 100) * 3
(3 / 100) * x
However, integer division in Solidity is not the same as division in pure math, as in Solidity it rounds the result toward zero. The first two variants are basically equivalent, and both suffer from phantom overflow. The third variant does not have phantom overflow problem, but is somewhat less precise, especially for small x. The fourth variant is more interesting, as it surprisingly leads to a compilation error:
browser/Junk.sol:5:18: TypeError: Operator * not compatible with types rational_const 3 / 10 and uint256browser/Junk.sol:5:18: TypeError: Operator * not compatible with types rational_const 3 / 10 and uint256
We already described this behavior in our previous article. To make the fourth expression compile we need to change it like this:
(uint (3) / 100) * x
However this does not help much, as the result of corrected expression is always zero, because 3 / 100 rounded towards zero is zero.
Via the third variant we managed to to solve phantom overflow problem at the cost of precision. Actually, precision loss is significant only for small x, while for large x it is negligible. Remember, that for the original expression, phantom overflow problem arises for large x only, so it seems that we may combine both variants like this:
x > SOME_LARGE_NUMBER ? x / 100 * 3 : x * 3 / 100
Here SOME_LARGE_NUMBER could be calculated as (2²⁵⁶-1)/3 and rounding this value down. Now for small x we use original formula, while for large x we use modified formula that do no permit phantom overflow. Looks like we solved phantom overflow problem without significant loss of precision now. Great work, right?
In this particular case, probably yes. But what if we need to calculate not 3%, but rather 3.1415926535%? The formula would be:
x > SOME_LARGE_NUMBER ?
x / 1000000000000 * 31415926535 :
x * 31415926535 / 1000000000000
Our SOME_LARGE_NUMBER will become (2²⁵⁶-1)/31415926535. Not so large then. And what about 3.141592653589793238462643383279%? Being good for simple cases, this approach does not seem to scale well.
Via Coinmonks
