Skip to content

Instantly share code, notes, and snippets.

@xwu
Last active August 3, 2017 02:07
Show Gist options
  • Select an option

  • Save xwu/d68baefaae9e9291d2e65bd12ad51be2 to your computer and use it in GitHub Desktop.

Select an option

Save xwu/d68baefaae9e9291d2e65bd12ad51be2 to your computer and use it in GitHub Desktop.
Notes on the user experience of new integer protocols

Notes on the user experience of new integer protocols

Xiaodi Wu
June 17, 2017

Introduction

The design, re-design, and implementation of SE-0104, a proposal to revise integer protocols in Swift, is now largely complete in shipping previews of Swift 4.0. As an exercise, I have used the new APIs to develop a set of additional numeric facilities. Here are some insights gained from that experience and suggestions for improvement based on that experience.

Topics

  1. Performing heterogeneous comparison with integer literals
  2. Conforming to _ExpressibleByBuiltinIntegerLiteral
  3. Overriding heterogeneous comparison and bit shift operators
  4. Masking shifts and arbitrary-width integers
  5. Using ArithmeticOverflow
  6. Initializing from a floating-point source

Heterogeneous comparison with integer literals

SE-0104 added heterogeneous comparison and bit shift operators to the language to improve the user experience (for example, you can now check if an Int value is equal to a UInt value).1

These operators behave as intended with concrete types, but comparisons in generic algorithms behave differently. This was encountered during review of the standard library's implementation of DoubleWidth, which in fact had a bug as a consequence of the following behavior:

func f() -> Bool {
  return UInt.max == ~0
}

func g() -> Bool {
  return UInt.max == .max
}

func h<T : FixedWidthInteger>(_: T.Type) -> Bool {
  return T.max == ~0
}

func i<T : FixedWidthInteger>(_: T.Type) -> Bool {
  return T.max == .max
}

f()          // Returns `true`.
g()          // Returns `true`.
h(UInt.self) // Returns `false`.
i(UInt.self) // Returns `true`.

The reason that h(_:) gives a surprising result is explained as follows:

  • Each concrete integer type implements its own overload of the homogeneous comparison operator, whereas protocol extension methods implement heterogeneous comparison operators. When an integer literal is used on the right-hand side of the comparison, the compiler looks first to the concrete type for an suitable implementation of the operator and finds the homogeneous overload. Therefore, it does not traverse the protocol hierarchy and instead infers the literal to be of the same type as the left-hand side.

  • In generic code, even if the most refined protocol implements its own overload of the homogeneous comparison operator, the compiler will look for all overloads of the operator by traversing the entire protocol hierarchy. Since heterogeneous comparison operators are defined somewhere along the hierarchy, the will always find an overload that accepts the "preferred" integer literal type (Swift.IntegerLiteralType, aka Int) and infers the literal to be of type Int.

Therefore, in the invocation h(UInt.self), we are actually comparing UInt.max to ~(0 as Int). This is a surprising result, as evidenced by the fact that a bug nearly slipped into the standard library itself.


> 1 Similar enhancements for operations such as addition are not yet possible because a design is lacking for how to express the idea of _promotion_. If and when Swift allows integer constants as generic parameters (e.g. `typealias Int64 = _Int<64>`), this may become a more realistic prospect as bit widths could then be expressed in generic constraints (e.g. `func + (lhs: T, rhs: U) -> U where T : FixedWidthInteger, U : FixedWidthInteger, T.BitWidth < U.BitWidth`). This is meant to be an aside and is certainly outside the scope of correctly or incrementally improving upon SE-0104. [↩](#a1)
@dabrahams
Copy link

@xwu what are the “various reasons” you think arbitrary precision integers are best represented as sign-magnitude? As far as I can tell, two's complement has only advantages and no drawbacks in this application.

@dabrahams
Copy link

@xwu regarding making heterogeneous comparison a protocol requirement, I'm a little uncertain. It seems to me that any really good implementation of BigInt will have an inline representation for small integers that doesn't require a dynamic allocation, and so presumably the optimizer should be able to do a pretty good job with x < 42. Have you got some measurements that demonstrate a need for this? That would help justify a change.

@xwu
Copy link
Author

xwu commented Jul 19, 2017

@dabrahams You're right that I don't get notifications on gists. It's been a while since I've thought about these issues, but let's see if I can still answer intelligently:

On the "various reasons" for sign-and-magnitude representation

  • Efficient algorithms for multiplication of arbitrary-precision integers (Karatsuba, Toom-Cook, etc.) have been developed for unsigned values only; unless I'm mistaken, they cannot be applied to two's-complement representations of negative values. Therefore, any multiplication or division with large two's-complement BigInt operands may require complementing one or both operands and/or the result on-the-fly. (By contrast, bitwise operators that treat a BigInt as if it's in two's-complement notation are easy to implement and not computationally inefficient regardless of actual underlying representation.)

  • The sign bit in sign-and-magnitude representation requires one byte (or less, if Swift can be smart about packing the struct); in two's-complement representation, if the bits required to represent the magnitude take up an exact multiple of the "digit" bit width--typically, a "digit" would be a machine word--then the sign bit necessitates an entire extra word.

  • Negating a large BigInt is cheap in sign-and-magnitude representation when the underlying "digits" are copy-on-write; negating a large BigInt is not so much (although this is not, in and of itself, a hugely persuasive argument).

On protocol requirements for heterogeneous comparison

The gold standard, as far as I'm aware, for arbitrary-precision integer libraries is GMP. Julia, Python, R, Ruby all provide arbitrary-precision types by a wrapper to GMP. Although some languages (Ruby, for one, I believe) themselves provide an inline representation for small integers to avoid calling GMP, I'm not aware that GMP itself yet uses an inline representation for small integers other than zero. GMP does provide functions, though, for binary operations with one mpn_t (i.e. big integer) operand and one word-sized operand.

It should be possible to provide a high-quality Swift BigInt type that simply wraps GMP. One aim of these protocols should be to make it easier and not harder for end users to write their own efficient libraries. True, a sufficiently sophisticated wrapper could take the Ruby approach. However, given that the standard library itself has not yet fully solved the issue of efficient conversion in generic code between integer types, I would suggest that this is a heavy task to fall on a third-party author.

@stephentyrone
Copy link

Hi Xiaodi! A couple quick notes:

Asymptotically, conversion between sign-magnitude and twos-complement is O(n), so it disappears into the O(n^(3/2)) or O(n lg* n) of even fast multiplication and division for sufficiently large integers. Also, you can definitely do all of these algorithms directly on twos-complement representations--I'll be happy to help you fill in the details. There is a range between ~128 bits and ~2K bits where conversion is a non-trivial factor, but as a tradeoff you get a simpler add and subtract in that range, as well as the possibility to make fixed-width operations (weakly) constant time without extra inefficiency, which isn't easy with sign-magnitude (this has less value than it sounds like at first, since perf-sensitive bignum computation tends to be done on fixed-width unsigned integer types, where this is a total non-issue).

I think(?) that it should also be possible to use sign-magnitude "under the covers", though this makes naive use of word(at: ) inefficient (I know this last point is something that Dave and I have discussed a little bit in the past).

GMP is superb asymptotically, but pretty bad for smallish fixed-size numbers, due to (a) function call overhead (b) the optimizer can't see through the function calls and (c) there are no fast-paths for small values, even if the optimizer could see through (these are precisely the issues that Dave is hinting at). This is roughly analogous to the distinction between BLAS and Eigen; short of teaching the compiler about BLAS calls as first-class objects for the optimizer, it'll never be competitive with templated optimizations for 3x3 matrices where the compiler can see exactly what it's doing. Not everything is present in Swift to make the later approach viable for bignums (yet?), but the protocols should definitely have that possibility in mind.

@xwu
Copy link
Author

xwu commented Jul 20, 2017

@stephentyrone I stand corrected about direct use of fast multiplication algorithms with two's-complement representations. That is very interesting--one day, I will take you up on the offer of filling in the details (other real-life things are taking up my attention at the moment). Agree that Swift protocols should make it possible to implement bignums that have special representations for small numbers; however, at the moment, I still think it's important that it also doesn't stand in the way of wrapping GMP facilities such as heterogeneous comparison, which in that specific use case mitigates some of the problems you've outlined.

@lorentey
Copy link

(Popping in as an author of a big integer package): it seems to me there is demand for both signed and unsigned big integers, so I want to/need to provide both variants. The sign-magnitude representation allows me to formulate the signed BigInt type in terms of BigUInt, instead of having to maintain two separate, slightly different implementations of all integer operations. It isn't clear to me yet if a two's complement representation for signed integers would allow for code reuse like that -- perhaps with gyb it would.

@stephentyrone: The revised words API allows reasonably efficient on-the fly conversion from sign-magnitude to two's-complement:

https://github.com/lorentey/BigInt/blob/swift4/sources/BigInt.swift#L141-L194

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment