The idea is to have a Decimal data type, for every use where decimals are needed but binary floating point is too inexact.
The Decimal data type will support the Python standard functions and operations, and must comply with the decimal arithmetic ANSI standard X3.274-1996 [1].
Decimal will be floating point (as opposed to fixed point) and will have bounded precision (the precision is the upper limit on the number of significant digits in a result). However, precision is user-settable, and a notion of significant trailing zeroes is supported so that fixed-point usage is also possible.
This work is based on code and test functions written by Eric Price, Aahz and Tim Peters. Just before Python 2.4a1, the decimal.py reference implementation was moved into the standard library; along with the documentation and the test suite, this was the work of Raymond Hettinger. Much of the explanation in this PEP is taken from Cowlishaw’s work [2], comp.lang.python and python-dev.
Here I’ll expose the reasons of why I think a Decimal data type is needed and why other numeric data types are not enough.
I wanted a Money data type, and after proposing a pre-PEP in comp.lang.python, the community agreed to have a numeric data type with the needed arithmetic behaviour, and then build Money over it: all the considerations about quantity of digits after the decimal point, rounding, etc., will be handled through Money. It is not the purpose of this PEP to have a data type that can be used as Money without further effort.
One of the biggest advantages of implementing a standard is that someone already thought out all the creepy cases for you. And to a standard GvR redirected me: Mike Cowlishaw’s General Decimal Arithmetic specification [2]. This document defines a general purpose decimal arithmetic. A correct implementation of this specification will conform to the decimal arithmetic defined in ANSI/IEEE standard 854-1987, except for some minor restrictions, and will also provide unrounded decimal arithmetic and integer arithmetic as proper subsets.
In decimal math, there are many numbers that can’t be represented with a fixed number of decimal digits, e.g. 1/3 = 0.3333333333…….
In base 2 (the way that standard floating point is calculated), 1/2 = 0.1, 1/4 = 0.01, 1/8 = 0.001, etc. Decimal 0.2 equals 2/10 equals 1/5, resulting in the binary fractional number 0.001100110011001… As you can see, the problem is that some decimal numbers can’t be represented exactly in binary, resulting in small roundoff errors.
So we need a decimal data type that represents exactly decimal numbers. Instead of a binary data type, we need a decimal one.
So we go to decimal, but why floating point?
Floating point numbers use a fixed quantity of digits (precision) to represent a number, working with an exponent when the number gets too big or too small. For example, with a precision of 5:
(note that in the last line the number got rounded to fit in five digits).
In contrast, we have the example of a long integer with infinite precision, meaning that you can have the number as big as you want, and you’ll never lose any information.
In a fixed point number, the position of the decimal point is fixed. For a fixed point data type, check Tim Peter’s FixedPoint at SourceForge [4]. I’ll go for floating point because it’s easier to implement the arithmetic behaviour of the standard, and then you can implement a fixed point data type over Decimal.
But why can’t we have a floating point number with infinite precision? It’s not so easy, because of inexact divisions. E.g.: 1/3 = 0.3333333333333… ad infinitum. In this case you should store an infinite amount of 3s, which takes too much memory, ;).
John Roth proposed to eliminate the division operator and force the user to use an explicit method, just to avoid this kind of trouble. This generated adverse reactions in comp.lang.python, as everybody wants to have support for the / operator in a numeric data type.
With this exposed maybe you’re thinking “Hey! Can we just store the 1 and the 3 as numerator and denominator?”, which takes us to the next point.
Rational numbers are stored using two integer numbers, the numerator and the denominator. This implies that the arithmetic operations can’t be executed directly (e.g. to add two rational numbers you first need to calculate the common denominator).
Quoting Alex Martelli:
Anyway, if you’re interested in this data type, you maybe will want to take a look at PEP 239: Adding a Rational Type to Python.
The result is a Decimal data type, with bounded precision and floating point.
Will it be useful? I can’t say it better than Alex Martelli:
There are several uses for such a data type. As I said before, I will use it as base for Money. In this case the bounded precision is not an issue; quoting Tim Peters:
Here I’ll include information and descriptions that are part of the specification [2] (the structure of the number, the context, etc.). All the requirements included in this section are not for discussion (barring typos or other mistakes), as they are in the standard, and the PEP is just for implementing the standard.
Because of copyright restrictions, I can not copy here explanations taken from the specification, so I’ll try to explain it in my own words. I firmly encourage you to read the original specification document [2] for details or if you have any doubt.
The specification is based on a decimal arithmetic model, as defined by the relevant standards: IEEE 854 [3], ANSI X3-274 [1], and the proposed revision [5] of IEEE 754 [6].
The model has three components:
Numbers may be finite or special values. The former can be represented exactly. The latter are infinites and undefined (such as 0/0).
Finite numbers are defined by three parameters:
The numerical value of a finite number is given by:
Special values are named as following:
The context is a set of parameters and rules that the user can select and which govern the results of operations (for example, the precision to be used).
The context gets that name because it surrounds the Decimal numbers, with parts of context acting as input to, and output of, operations. It’s up to the application to work with one or several contexts, but definitely the idea is not to get a context per Decimal number. For example, a typical use would be to set the context’s precision to 20 digits at the start of a program, and never explicitly use context again.
These definitions don’t affect the internal storage of the Decimal numbers, just the way that the arithmetic operations are performed.
The context is mainly defined by the following parameters (see Context Attributes for all context attributes):
The specification defines two default contexts, which should be easily selectable by the user.
Basic Default Context:
Extended Default Context:
The table below lists the exceptional conditions that may arise during the arithmetic operations, the corresponding signal, and the defined result. For details, see the specification [2].
| Clamped | clamped | see spec [2] |
| Division by zero | division-by-zero | [sign,inf] |
| Inexact | inexact | unchanged |
| Invalid operation | invalid-operation | [0,qNaN] (or [s,qNaN] or [s,qNaN,d] when the cause is a signaling NaN) |
| Overflow | overflow | depends on the rounding mode |
| Rounded | rounded | unchanged |
| Subnormal | subnormal | unchanged |
| Underflow | underflow | see spec [2] |
Note: when the standard talks about “Insufficient storage”, as long as this is implementation-specific behaviour about not having enough storage to keep the internals of the number, this implementation will raise MemoryError.
Regarding Overflow and Underflow, there’s been a long discussion in python-dev about artificial limits. The general consensus is to keep the artificial limits only if there are important reasons to do that. Tim Peters gives us three:
Virtually all implementations of 854 use (and as IBM’s standard even suggests) “forbidden” exponent values to encode non-finite numbers (infinities and NaNs). A bounded exponent can do this at virtually no extra storage cost. If the exponent is unbounded, then additional bits have to be used instead. This cost remains hidden until more time- and space- efficient implementations are attempted.
Big as it is, the IBM standard is a tiny start at supplying a complete numeric facility. Having no bound on exponent size will enormously complicate the implementations of, e.g., decimal sin() and cos() (there’s then no a priori limit on how many digits of pi effectively need to be known in order to perform argument reduction).
Edward Loper give us an example of when the limits are to be crossed: probabilities.
That said, Robert Brewer and Andrew Lentvorski want the limits to be easily modifiable by the users. Actually, this is quite possible:
round-down: The discarded digits are ignored; the result is unchanged (round toward 0, truncate):
round-half-up: If the discarded digits represent greater than or equal to half (0.5) then the result should be incremented by 1; otherwise the discarded digits are ignored:
round-half-even: If the discarded digits represent greater than half (0.5) then the result coefficient is incremented by 1; if they represent less than half, then the result is not adjusted; otherwise the result is unaltered if its rightmost digit is even, or incremented by 1 if its rightmost digit is odd (to make an even digit):
round-ceiling: If all of the discarded digits are zero or if the sign is negative the result is unchanged; otherwise, the result is incremented by 1 (round toward positive infinity):
round-floor: If all of the discarded digits are zero or if the sign is positive the result is unchanged; otherwise, the absolute value of the result is incremented by 1 (round toward negative infinity):
round-half-down: If the discarded digits represent greater than half (0.5) then the result is incremented by 1; otherwise the discarded digits are ignored:
round-up: If all of the discarded digits are zero the result is unchanged, otherwise the result is incremented by 1 (round away from 0):
I must separate the requirements in two sections. The first is to comply with the ANSI standard. All the requirements for this are specified in the Mike Cowlishaw’s work [2]. He also provided a very large suite of test cases.
The second section of requirements (standard Python functions support, usability, etc.) is detailed from here, where I’ll include all the decisions made and why, and all the subjects still being discussed.
The explicit construction does not get affected by the context (there is no rounding, no limits by the precision, etc.), because the context affects just operations’ results. The only exception to this is when you’re Creating from Context.
There’s no loss and no need to specify any other information:
Strings containing Python decimal integer literals and Python float literals will be supported. In this transformation there is no loss of information, as the string is directly converted to Decimal (there is not an intermediate conversion through float):
Also, you can construct in this way all special values (Infinity and Not a Number):
The initial discussion on this item was what should happen when passing floating point to the constructor:
Several people alleged that (1) is the better option here, because it’s what you expect when writing Decimal(1.1). And quoting John Roth, it’s easy to implement:
But If I really want my number to be Decimal('110000000000000008881784197001252...e-51'), why can’t I write Decimal(1.1)? Why should I expect Decimal to be “rounding” it? Remember that 1.1 is binary floating point, so I can predict the result. It’s not intuitive to a beginner, but that’s the way it is.
Anyway, Paul Moore showed that (1) can’t work, because:
which is wrong, because if I write Decimal('1.1') it is exact, not D(1.1000000000000001). He also proposed to have an explicit conversion to float. bokr says you need to put the precision in the constructor and mwilson agreed:
But Alex Martelli says that:
So, the accepted solution through c.l.p is that you can not call Decimal with a float. Instead you must use a method: Decimal.from_float(). The syntax:
where floatNumber is the float number origin of the construction and decimal_places are the number of digits after the decimal point where you apply a round-half-up rounding, if any. In this way you can do, for example:
Based on later discussions, it was decided to omit from_float() from the API for Py2.4. Several ideas contributed to the thought process:
Aahz suggested to construct from tuples: it’s easier to implement eval()’s round trip and “someone who has numeric values representing a Decimal does not need to convert them to a string.”
The structure will be a tuple of three elements: sign, number and exponent. The sign is 1 or 0, the number is a tuple of decimal digits and the exponent is a signed int or long:
Of course, you can construct in this way all special values:
No mystery here, just a copy.
where value1 can be int, long, string, 3-tuple or Decimal, value2 can only be float, and decimal_places is an optional non negative int.
This item arose in python-dev from two sources in parallel. Ka-Ping Yee proposes to pass the context as an argument at instance creation (he wants the context he passes to be used only in creation time: “It would not be persistent”). Tony Meyer asks from_string to honor the context if it receives a parameter “honour_context” with a True value. (I don’t like it, because the doc specifies that the context be honored and I don’t want the method to comply with the specification regarding the value of an argument.)
Tim Peters gives us a reason to have a creation that uses context:
Casey Duncan wants to use another method, not a bool arg:
In the process of deciding the syntax of that, Tim came up with a better idea: he proposes not to have a method in Decimal to create with a different context, but having instead a method in Context to create a Decimal instance. Basically, instead of:
it will be:
From Tim:
So, we decided to use a context method to create a Decimal that will use (only to be created) that context in particular (for further operations it will use the context of the thread). But, a method with what name?
Tim Peters proposes three methods to create from diverse sources (from_string, from_int, from_float). I proposed to use one method, create_decimal(), without caring about the data type. Michael Chermside: “The name just fits my brain. The fact that it uses the context is obvious from the fact that it’s Context method”.
The community agreed with that. I think that it’s OK because a newbie will not be using the creation method from Context (the separate method in Decimal to construct from float is just to prevent newbies from encountering binary floating point issues).
So, in short, if you want to create a Decimal instance using a particular context (that will be used just at creation time and not any further), you’ll have to use a method of that context:
Example:
As the implicit construction is the consequence of an operation, it will be affected by the context as is detailed in each point.
John Roth suggested that “The other type should be handled in the same way the decimal() constructor would handle it”. But Alex Martelli thinks that
So, here I define the behaviour again for each data type.
An int or long is a treated like a Decimal explicitly constructed from Decimal(str(x)) in the current context (meaning that the to-string rules for rounding are applied and the appropriate flags are set). This guarantees that expressions like Decimal('1234567') + 13579 match the mental model of Decimal('1234567') + Decimal('13579'). That model works because all integers are representable as strings without representation error.
Everybody agrees to raise an exception here.
Aahz is strongly opposed to interact with float, suggesting an explicit conversion:
The example of the valid python expression, 35 + 1.1, seems to suggest that Decimal(35) + 1.1 should also be valid. However, a closer look shows that it only demonstrates the feasibility of integer to floating point conversions. Hence, the correct analog for decimal floating point is 35 + Decimal(1.1). Both coercions, int-to-float and int-to-Decimal, can be done without incurring representation error.
The question of how to coerce between binary and decimal floating point is more complex. I proposed allowing the interaction with float, making an exact conversion and raising ValueError if exceeds the precision in the current context (this is maybe too tricky, because for example with a precision of 9, Decimal(35) + 1.2 is OK but Decimal(35) + 1.1 raises an error).
This resulted to be too tricky. So tricky, that c.l.p agreed to raise TypeError in this case: you could not mix Decimal and float.
There isn’t any issue here.
In the last pre-PEP I said that “The Context must be omnipresent, meaning that changes to it affects all the current and future Decimal instances”. I was wrong. In response, John Roth said:
In comp.lang.python, Aahz explained that the idea is to have a “context per thread”. So, all the instances of a thread belongs to a context, and you can change a context in thread A (and the behaviour of the instances of that thread) without changing nothing in thread B.
Also, and again correcting me, he said:
Arguing about special cases when there’s need to perform operations with other rules that those of the current context, Tim Peters said that the context will have the operations as methods. This way, the user “can create whatever private context object(s) it needs, and spell arithmetic as explicit method calls on its private context object(s), so that the default thread context object is neither consulted nor modified”.
There’s been some discussion in python-dev about the behaviour of hash(). The community agrees that if the values are the same, the hashes of those values should also be the same. So, while Decimal(25) == 25 is True, hash(Decimal(25)) should be equal to hash(25).
The detail is that you can NOT compare Decimal to floats or strings, so we should not worry about them giving the same hashes. In short:
Regarding str() and repr() behaviour, Ka-Ping Yee proposes that repr() have the same behaviour as str() and Tim Peters proposes that str() behave like the to-scientific-string operation from the Spec.
This is possible, because (from Aahz): “The string form already contains all the necessary information to reconstruct a Decimal object”.
And it also complies with the Spec; Tim Peters:
This section explains all the public methods and attributes of Decimal and Context.
Decimal has no public attributes. The internal information is stored in slots and should not be accessed by end users.
Following are the conversion and arithmetic operations defined in the Spec, and how that functionality can be achieved with the actual implementation.
Following are other methods and why they exist:
These are the attributes that can be changed to modify the context.
The following methods comply with Decimal functionality from the Spec. Be aware that the operations that are called through a specific context use that context and not the thread context.
To use these methods, take note that the syntax changes when the operator is binary or unary, for example:
So, the following are the Spec operations and conversions and how to achieve them through a context (where d is a Decimal instance and n a number that can be used in an Implicit construction):
The divmod(d, n) method supports decimal functionality through Context.
These are methods that return useful information from the Context:
As of Python 2.4-alpha, the code has been checked into the standard library. The latest version is available from:
http://svn.python.org/view/python/trunk/Lib/decimal.py
The test cases are here:
http://svn.python.org/view/python/trunk/Lib/test/test_decimal.py
This document has been placed in the public domain.
Source: https://github.com/python/peps/blob/main/peps/pep-0327.rst
Last modified: 2025-02-01 08:59:27 UTC