Skip to content

Instantly share code, notes, and snippets.

@yoooov

yoooov/async_swift_proposal.md

Forked from lattner/async_swift_proposal.md
Created May 21, 2020 07:10
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save yoooov/67a0e916ddcfe854ed9b3e55209e5e5b to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save yoooov/67a0e916ddcfe854ed9b3e55209e5e5b to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. @lattner lattner revised this gist Sep 3, 2017. 1 changed file with 42 additions and 7 deletions.
    49 changes: 42 additions & 7 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -349,6 +349,10 @@ class Future<T> {
    let result = .value(value)
    self.result = result
    for awaiter in awaiters {
    // A robust future implementation should probably resume awaiters
    // concurrently into user-controllable contexts. For simplicity this
    // proof-of-concept simply resumes them all serially in the current
    // context.
    awaiter(result)
    }
    awaiters = []
    @@ -420,12 +424,7 @@ func processImageData1a() async -> Image {
    }
    ```

    In the above example, the first two operations will start one after another, and the
    unevaluated computations are wrapped into a `Future` value. This allows all of them to
    happen concurrently (in a way that need not be defined by the language or
    by the `Future` implementation), and the function will wait for completion of them before
    decoding the image. Note that `await` does not block flow of execution: if the value is not
    yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.
    In the above example, the first two operations will start one after another, and the unevaluated computations are wrapped into a `Future` value. This allows all of them to happen concurrently (in a way that need not be defined by the language or by the `Future` implementation), and the function will wait for completion of them before decoding the image. Note that `await` does not block flow of execution: if the value is not yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.

    Other coordination abstractions such as [Communicating Sequential Process channels](https://en.wikipedia.org/wiki/Communicating_sequential_processes) or [Concurrent ML events](https://wingolog.org/archives/2017/06/29/a-new-concurrent-ml) can also be developed as libraries for coordinating coroutines; their implementation is left as an exercise for the reader.

    @@ -459,6 +458,7 @@ There are many details that should be defined as part of this importing process
    - Are multiple result functions common enough to handle automatically?
    - Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration?
    - What should happen with the non-Void-returning completion handler functions (e.g. in `URLSession`)?
    - Should `Void`-returning methods that are commonly used to trigger asynchronous operations in response to events, such as `IBAction` methods, be imported as `async -> Void`?

    Without substantial ObjC importer work, making a clean break and forcing migration in Swift 5 mode would be the most practical way to preserve overridability, but would create a lot of churn in 4-to-5 migration. Alternatively, it may be acceptable to present the `async` versions as `final` wrappers over the underlying callback-based interfaces; this would subclassers to work with the callback-based interface, but there are generally fewer subclassers than callers.

    @@ -666,14 +666,27 @@ The other way to factor the complexity is to make it so that `async` functions d

    This model provides a ton of advantages: it is arguably the right defaults for the vast majority of clients (reducing boilerplate and syntactic noise), provides the ability for the importer and experts to get what they want. The only downside of is that it is a less obvious design than presenting two orthogonal axes, but in the opinion of the proposal authors, this is probably the right set of tradeoffs.

    #### Behavior of `beginAsync` and `suspendAsync` operations

    For async code to be able to interact with synchronous code, we need at least two primitive operations: one to enter a suspendable context, and another to suspend the current context and yield control back to the outer context. Aside from the obvious naming bikeshed, there are some other design details to consider. As proposed, `beginAsync` and continuation closures return `Void` to the calling context, but it may be desirable instead to have them return a value indicating whether the return was because of suspension or completion of the async task, e.g.:

    ```swift
    /// Begin execution of `body`. Return `true` if it completes, or `false` if it
    /// suspends.
    func beginAsync(_ body: () async -> ()) -> Bool
    /// Suspend execution of the current coroutine, passing the current continuation/// into `body` and then returning `false` to the controlling context
    func suspendAsync<T>(_ body: (_ resume: (T) -> Bool) -> Void) async -> T
    ```

    Instead of representing the continuation as a plain function value passed into the `suspendAsync` primitive, a specialized `Continuation<T>` type could be devised. Continuations are one-shot, and a nominal continuation type could statically enforce this by being a move-only type consumed by the resume operation. The continuation could also be returned by `beginAsync` or resuming a continuation instead of being passed into `suspendAsync`, which would put the responsibility for scheduling the continuation into the code that starts the coroutine instead of in the code that causes the suspension. There are tradeoffs to either approach.

    ## Alternatives Considered

    #### Include `Future` or other coordination abstractions in this proposal

    This proposal does not formally propose a `Future` type, or any other coordination abstractions. There are many rational designs for futures, and a lot of experience working with them. On the other hand, there are also completely different coordination primitives that can be used with this coroutine design, and incorporating them into this proposal only makes it larger.

    Furthermore, the shape and functionality of a future may also be affected by Swift's planned evolution; whereas a Future type designed for Swift today would need to be a `class`, and therefore need to guard against potentially multithreaded access, races to fulfill or attempts to fulfill multiple times, and potentially unbounded queueing of awaiting coroutines on the shared future. The introduction of ownership and move-only types would allow us to express futures as a more efficient move-only type requiring exclusive ownership to be forwarded from the fulfilling task to the receiving task, avoiding the threading and queueing problems of a class-based approach, as seen in Rust's [tokio.rs](https://tokio.rs) framework. Whether that is a better design or not needs further discussion.
    Furthermore, the shape and functionality of a future may also be affected by Swift's planned evolution. A `Future` type designed for Swift today would need to be a `class`, and therefore need to guard against potentially multithreaded access, races to fulfill or attempts to fulfill multiple times, and potentially unbounded queueing of awaiting coroutines on the shared future; however, the introduction of ownership and move-only types would allow us to express futures as a more efficient move-only type requiring exclusive ownership to be forwarded from the fulfilling task to the receiving task, avoiding the threading and queueing problems of a class-based approach, as seen in Rust's [tokio.rs](https://tokio.rs) framework. tokio.rs and the C++ coroutine TR also both take the approach of making futures/continuations into templated/generic traits instead of a single concrete implementation, so that the compiler can deeply specialize and optimize state machines for composed async operations. tokio.rs and the C++ coroutine TR also both take the approach of making futures/continuations into templated/generic traits instead of a single concrete implementation, so that the compiler can deeply specialize and optimize state machines for composed async operations. Whether that is a good design for Swift as well needs further exploration.

    #### Have async calls always return a `Future`

    @@ -693,6 +706,28 @@ Despite this model being widely know, we believe that the proposed design is

    The primary argument for adding async/await (and then generators) to the language as first-class language features is that they are the vastly most common use-case of coroutines. In the author's opinion, the design as proposed gives something that works better than the C# model in practice, while also providing a more useful/general language model.

    #### Have a generalized "do notation" for monadic types

    Another approach to avoiding the one-true-future-type problem of C# could be to have a general language feature for chaining continuations through a monadic interface. Although this provides a more general language feature, it still has many of the shortcomings discussed above; it would still perform only a shallow transform of the current function body and introduce a temporary value at every point the coroutine is "awaited". Monads also compose poorly with each other, and require additional lifting and transformation logic to plumb through higher-order operations, which were some of the reasons we also chose not to base Swift's error handling model on sugar over `Result` types. Note that the delimited continuation primitives offered in this proposal are general purpose and can in fact be used to represent monadic unwrapping operations for types like `Optional` or `Result`:

    ```swift
    func doOptional<T>(_ body: (_ unwrap: (T?) async -> T) async -> T?) -> T? {
    var result: T?

    func unwrap(_ value: T?) async -> T {
    if let value = value {
    return value
    }
    suspendAsync { _ in result = nil }
    }

    beginAsync {
    body(unwrap)
    }
    }
    ```

    Monads that represent repeated or nondeterministic operations would not be representable this way due to the one-shot constraint on continuations, but representing such computations as straight-line code in an imperative language with shared mutable state seems like a recipe for disaster to us.

    ## Potential Future Directions

  2. @lattner lattner revised this gist Sep 3, 2017. 1 changed file with 26 additions and 49 deletions.
    75 changes: 26 additions & 49 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -230,8 +230,11 @@ In the common case, async code ought to be invoking other async code that has be
    enable entering and suspending an async context:

    ```swift
    // NB: Names and exact forms subject to bikeshedding, see section under
    // "Alternate syntax options"
    // NB: Names subject to bikeshedding. These are low-level primitives that most
    // users should not need to interact with directly, so namespacing them
    // and/or giving them verbose names unlikely to collide or pollute code
    // completion (and possibly not even exposing them outside the stdlib to begin
    // with) would be a good idea.

    /// Begins an asynchronous coroutine, transferring control to `body` until it
    /// either suspends itself for the first time with `suspendAsync` or completes,
    @@ -302,11 +305,18 @@ func getStuff() async -> Stuff {
    }
    ```

    Functionality of concurrency libraries such as libdispatch can
    Functionality of concurrency libraries such as libdispatch and pthreads can
    also be presented in coroutine-friendly ways:

    ```swift
    extension DispatchQueue {
    /// Move execution of the current coroutine synchronously onto this queue.
    func syncCoroutine() async -> Void {
    await suspendAsync { continuation in
    sync { continuation }
    }
    }

    /// Enqueue execution of the remainder of the current coroutine
    /// asynchronously onto this queue.
    func asyncCoroutine() async -> Void {
    @@ -318,14 +328,14 @@ extension DispatchQueue {

    func queueHopping() async -> Void {
    doSomeStuff()
    await DispatchQueue.main.asyncCoroutine()
    await DispatchQueue.main.syncCoroutine()
    doSomeStuffOnMainThread()
    await backgroundQueue.asyncCoroutine()
    doSomeStuffInBackground()
    }
    ```

    Generalized abstractions for coordinating coroutines can also be built. The simplest of these is a [future](https://en.wikipedia.org/wiki/Futures_and_promises), a value that represents the eventual result of an ongoing computation that may not have completed yet. The exact design for a future type deserves its own proposal, but a proof of concept could look like this:
    Generalized abstractions for coordinating coroutines can also be built. The simplest of these is a [future](https://en.wikipedia.org/wiki/Futures_and_promises), a value that represents a future value which may not be resolved yet. The exact design for a Future type is out of scope for this proposal (it should be its own follow-on proposal), but an example proof of concept could look like this:

    ```swift
    class Future<T> {
    @@ -339,10 +349,6 @@ class Future<T> {
    let result = .value(value)
    self.result = result
    for awaiter in awaiters {
    // A robust future implementation should probably resume awaiters
    // concurrently into user-controllable contexts. For simplicity this
    // proof-of-concept simply resumes them all serially in the current
    // context.
    awaiter(result)
    }
    awaiters = []
    @@ -397,7 +403,9 @@ class Future<T> {
    }
    ```

    Futures can allow parallel execution, by moving `await` from the call to the result when it is needed, and wrapping the parallel calls in individual `Future` objects:
    To reiterate, it is well known that this specific implementation has performance and API weaknesses, the point is merely to sketch how an abstraction like this could be built on top of `async`/`await`.

    Futures allow parallel execution, by moving `await` from the call to the result when it is needed, and wrapping the parallel calls in individual `Future` objects:

    ```swift
    func processImageData1a() async -> Image {
    @@ -412,7 +420,12 @@ func processImageData1a() async -> Image {
    }
    ```

    In the above example, the first two operations will start one after another, and the unevaluated computations are wrapped into a `Future` value. This allows all of them to happen concurrently (in a way that need not be defined by the language or by the `Future` implementation), and the function will wait for completion of them before decoding the image. Note that `await` does not block flow of execution: if the value is not yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.
    In the above example, the first two operations will start one after another, and the
    unevaluated computations are wrapped into a `Future` value. This allows all of them to
    happen concurrently (in a way that need not be defined by the language or
    by the `Future` implementation), and the function will wait for completion of them before
    decoding the image. Note that `await` does not block flow of execution: if the value is not
    yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.

    Other coordination abstractions such as [Communicating Sequential Process channels](https://en.wikipedia.org/wiki/Communicating_sequential_processes) or [Concurrent ML events](https://wingolog.org/archives/2017/06/29/a-new-concurrent-ml) can also be developed as libraries for coordinating coroutines; their implementation is left as an exercise for the reader.

    @@ -446,7 +459,6 @@ There are many details that should be defined as part of this importing process
    - Are multiple result functions common enough to handle automatically?
    - Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration?
    - What should happen with the non-Void-returning completion handler functions (e.g. in `URLSession`)?
    - Should `Void`-returning methods that are commonly used to trigger asynchronous operations in response to events, such as `IBAction` methods, be imported as `async -> Void`?

    Without substantial ObjC importer work, making a clean break and forcing migration in Swift 5 mode would be the most practical way to preserve overridability, but would create a lot of churn in 4-to-5 migration. Alternatively, it may be acceptable to present the `async` versions as `final` wrappers over the underlying callback-based interfaces; this would subclassers to work with the callback-based interface, but there are generally fewer subclassers than callers.

    @@ -654,27 +666,14 @@ The other way to factor the complexity is to make it so that `async` functions d

    This model provides a ton of advantages: it is arguably the right defaults for the vast majority of clients (reducing boilerplate and syntactic noise), provides the ability for the importer and experts to get what they want. The only downside of is that it is a less obvious design than presenting two orthogonal axes, but in the opinion of the proposal authors, this is probably the right set of tradeoffs.

    #### Behavior of `beginAsync` and `suspendAsync` operations

    For async code to be able to interact with synchronous code, we need at least two primitive operations: one to enter a suspendable context, and another to suspend the current context and yield control back to the outer context. Aside from the obvious naming bikeshed, there are some other design details to consider. As proposed, `beginAsync` and continuation closures return `Void` to the calling context, but it may be desirable instead to have them return a value indicating whether the return was because of suspension or completion of the async task, e.g.:

    ```swift
    /// Begin execution of `body`. Return `true` if it completes, or `false` if it
    /// suspends.
    func beginAsync(_ body: () async -> ()) -> Bool
    /// Suspend execution of the current coroutine, passing the current continuation/// into `body` and then returning `false` to the controlling context
    func suspendAsync<T>(_ body: (_ resume: (T) -> Bool) -> Void) async -> T
    ```

    Instead of representing the continuation as a plain function value passed into the `suspendAsync` primitive, a specialized `Continuation<T>` type could be devised. Continuations are one-shot, and a nominal continuation type could statically enforce this by being a move-only type consumed by the resume operation. The continuation could also be returned by `beginAsync` or resuming a continuation instead of being passed into `suspendAsync`, which would put the responsibility for scheduling the continuation into the code that starts the coroutine instead of in the code that causes the suspension. There are tradeoffs to either approach.

    ## Alternatives Considered

    #### Include `Future` or other coordination abstractions in this proposal

    This proposal does not formally propose a `Future` type, or any other coordination abstractions. There are many rational designs for futures, and a lot of experience working with them. On the other hand, there are also completely different coordination primitives that can be used with this coroutine design, and incorporating them into this proposal only makes it larger.

    Furthermore, the shape and functionality of a future may also be affected by Swift's planned evolution. A `Future` type designed for Swift today would need to be a `class`, and therefore need to guard against potentially multithreaded access, races to fulfill or attempts to fulfill multiple times, and potentially unbounded queueing of awaiting coroutines on the shared future; however, the introduction of ownership and move-only types would allow us to express futures as a more efficient move-only type requiring exclusive ownership to be forwarded from the fulfilling task to the receiving task, avoiding the threading and queueing problems of a class-based approach, as seen in Rust's [tokio.rs](https://tokio.rs) framework. tokio.rs and the C++ coroutine TR also both take the approach of making futures/continuations into templated/generic traits instead of a single concrete implementation, so that the compiler can deeply specialize and optimize state machines for composed async operations. Whether that is a good design for Swift as well needs further exploration.
    Furthermore, the shape and functionality of a future may also be affected by Swift's planned evolution; whereas a Future type designed for Swift today would need to be a `class`, and therefore need to guard against potentially multithreaded access, races to fulfill or attempts to fulfill multiple times, and potentially unbounded queueing of awaiting coroutines on the shared future. The introduction of ownership and move-only types would allow us to express futures as a more efficient move-only type requiring exclusive ownership to be forwarded from the fulfilling task to the receiving task, avoiding the threading and queueing problems of a class-based approach, as seen in Rust's [tokio.rs](https://tokio.rs) framework. Whether that is a better design or not needs further discussion.

    #### Have async calls always return a `Future`

    @@ -694,28 +693,6 @@ Despite this model being widely know, we believe that the proposed design is

    The primary argument for adding async/await (and then generators) to the language as first-class language features is that they are the vastly most common use-case of coroutines. In the author's opinion, the design as proposed gives something that works better than the C# model in practice, while also providing a more useful/general language model.

    #### Have a generalized "do notation" for monadic types

    Another approach to avoiding the one-true-future-type problem of C# could be to have a general language feature for chaining continuations through a monadic interface. Although this provides a more general language feature, it still has many of the shortcomings discussed above; it would still perform only a shallow transform of the current function body and introduce a temporary value at every point the coroutine is "awaited". Monads also compose poorly with each other, and require additional lifting and transformation logic to plumb through higher-order operations, which were some of the reasons we also chose not to base Swift's error handling model on sugar over `Result` types. Note that the delimited continuation primitives offered in this proposal are general purpose and can in fact be used to represent monadic unwrapping operations for types like `Optional` or `Result`:

    ```swift
    func doOptional<T>(_ body: (_ unwrap: (T?) async -> T) async -> T?) -> T? {
    var result: T?

    func unwrap(_ value: T?) async -> T {
    if let value = value {
    return value
    }
    suspendAsync { _ in result = nil }
    }

    beginAsync {
    body(unwrap)
    }
    }
    ```

    Monads that represent repeated or nondeterministic operations would not be representable this way due to the one-shot constraint on continuations, but representing such computations as straight-line code in an imperative language with shared mutable state seems like a recipe for disaster to us.

    ## Potential Future Directions

    @@ -784,4 +761,4 @@ completion handler on the original queue.

    ## Thanks

    Thanks to @oleganza for the original draft which influenced this!
    Thanks to @oleganza for the original draft which influenced this!
  3. @lattner lattner revised this gist Aug 22, 2017. 1 changed file with 49 additions and 24 deletions.
    73 changes: 49 additions & 24 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -230,11 +230,8 @@ In the common case, async code ought to be invoking other async code that has be
    enable entering and suspending an async context:

    ```swift
    // NB: Names subject to bikeshedding. These are low-level primitives that most
    // users should not need to interact with directly, so namespacing them
    // and/or giving them verbose names unlikely to collide or pollute code
    // completion (and possibly not even exposing them outside the stdlib to begin
    // with) would be a good idea.
    // NB: Names and exact forms subject to bikeshedding, see section under
    // "Alternate syntax options"

    /// Begins an asynchronous coroutine, transferring control to `body` until it
    /// either suspends itself for the first time with `suspendAsync` or completes,
    @@ -305,18 +302,11 @@ func getStuff() async -> Stuff {
    }
    ```

    Functionality of concurrency libraries such as libdispatch and pthreads can
    Functionality of concurrency libraries such as libdispatch can
    also be presented in coroutine-friendly ways:

    ```swift
    extension DispatchQueue {
    /// Move execution of the current coroutine synchronously onto this queue.
    func syncCoroutine() async -> Void {
    await suspendAsync { continuation in
    sync { continuation }
    }
    }

    /// Enqueue execution of the remainder of the current coroutine
    /// asynchronously onto this queue.
    func asyncCoroutine() async -> Void {
    @@ -328,14 +318,14 @@ extension DispatchQueue {

    func queueHopping() async -> Void {
    doSomeStuff()
    await DispatchQueue.main.syncCoroutine()
    await DispatchQueue.main.asyncCoroutine()
    doSomeStuffOnMainThread()
    await backgroundQueue.asyncCoroutine()
    doSomeStuffInBackground()
    }
    ```

    Generalized abstractions for coordinating coroutines can also be built. The simplest of these is a [future](https://en.wikipedia.org/wiki/Futures_and_promises), a value that represents a future value which may not be resolved yet. The exact design for a future type deserves its own proposal, but a proof of concept could look like this:
    Generalized abstractions for coordinating coroutines can also be built. The simplest of these is a [future](https://en.wikipedia.org/wiki/Futures_and_promises), a value that represents the eventual result of an ongoing computation that may not have completed yet. The exact design for a future type deserves its own proposal, but a proof of concept could look like this:

    ```swift
    class Future<T> {
    @@ -349,6 +339,10 @@ class Future<T> {
    let result = .value(value)
    self.result = result
    for awaiter in awaiters {
    // A robust future implementation should probably resume awaiters
    // concurrently into user-controllable contexts. For simplicity this
    // proof-of-concept simply resumes them all serially in the current
    // context.
    awaiter(result)
    }
    awaiters = []
    @@ -403,7 +397,7 @@ class Future<T> {
    }
    ```

    Futures allow parallel execution, by moving `await` from the call to the result when it is needed, and wrapping the parallel calls in individual `Future` objects:
    Futures can allow parallel execution, by moving `await` from the call to the result when it is needed, and wrapping the parallel calls in individual `Future` objects:

    ```swift
    func processImageData1a() async -> Image {
    @@ -418,12 +412,7 @@ func processImageData1a() async -> Image {
    }
    ```

    In the above example, the first two operations will start one after another, and the
    unevaluated computations are wrapped into a `Future` value. This allows all of them to
    happen concurrently (in a way that need not be defined by the language or
    by the `Future` implementation), and the function will wait for completion of them before
    decoding the image. Note that `await` does not block flow of execution: if the value is not
    yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.
    In the above example, the first two operations will start one after another, and the unevaluated computations are wrapped into a `Future` value. This allows all of them to happen concurrently (in a way that need not be defined by the language or by the `Future` implementation), and the function will wait for completion of them before decoding the image. Note that `await` does not block flow of execution: if the value is not yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.

    Other coordination abstractions such as [Communicating Sequential Process channels](https://en.wikipedia.org/wiki/Communicating_sequential_processes) or [Concurrent ML events](https://wingolog.org/archives/2017/06/29/a-new-concurrent-ml) can also be developed as libraries for coordinating coroutines; their implementation is left as an exercise for the reader.

    @@ -457,6 +446,7 @@ There are many details that should be defined as part of this importing process
    - Are multiple result functions common enough to handle automatically?
    - Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration?
    - What should happen with the non-Void-returning completion handler functions (e.g. in `URLSession`)?
    - Should `Void`-returning methods that are commonly used to trigger asynchronous operations in response to events, such as `IBAction` methods, be imported as `async -> Void`?

    Without substantial ObjC importer work, making a clean break and forcing migration in Swift 5 mode would be the most practical way to preserve overridability, but would create a lot of churn in 4-to-5 migration. Alternatively, it may be acceptable to present the `async` versions as `final` wrappers over the underlying callback-based interfaces; this would subclassers to work with the callback-based interface, but there are generally fewer subclassers than callers.

    @@ -664,14 +654,27 @@ The other way to factor the complexity is to make it so that `async` functions d

    This model provides a ton of advantages: it is arguably the right defaults for the vast majority of clients (reducing boilerplate and syntactic noise), provides the ability for the importer and experts to get what they want. The only downside of is that it is a less obvious design than presenting two orthogonal axes, but in the opinion of the proposal authors, this is probably the right set of tradeoffs.

    #### Behavior of `beginAsync` and `suspendAsync` operations

    For async code to be able to interact with synchronous code, we need at least two primitive operations: one to enter a suspendable context, and another to suspend the current context and yield control back to the outer context. Aside from the obvious naming bikeshed, there are some other design details to consider. As proposed, `beginAsync` and continuation closures return `Void` to the calling context, but it may be desirable instead to have them return a value indicating whether the return was because of suspension or completion of the async task, e.g.:

    ```swift
    /// Begin execution of `body`. Return `true` if it completes, or `false` if it
    /// suspends.
    func beginAsync(_ body: () async -> ()) -> Bool
    /// Suspend execution of the current coroutine, passing the current continuation/// into `body` and then returning `false` to the controlling context
    func suspendAsync<T>(_ body: (_ resume: (T) -> Bool) -> Void) async -> T
    ```

    Instead of representing the continuation as a plain function value passed into the `suspendAsync` primitive, a specialized `Continuation<T>` type could be devised. Continuations are one-shot, and a nominal continuation type could statically enforce this by being a move-only type consumed by the resume operation. The continuation could also be returned by `beginAsync` or resuming a continuation instead of being passed into `suspendAsync`, which would put the responsibility for scheduling the continuation into the code that starts the coroutine instead of in the code that causes the suspension. There are tradeoffs to either approach.

    ## Alternatives Considered

    #### Include `Future` or other coordination abstractions in this proposal

    This proposal does not formally propose a `Future` type, or any other coordination abstractions. There are many rational designs for futures, and a lot of experience working with them. On the other hand, there are also completely different coordination primitives that can be used with this coroutine design, and incorporating them into this proposal only makes it larger.

    Furthermore, the shape and functionality of a future may also be affected by Swift's planned evolution; whereas a Future type designed for Swift today would need to be a `class`, and therefore need to guard against potentially multithreaded access, races to fulfill or attempts to fulfill multiple times, and potentially unbounded queueing of awaiting coroutines on the shared future. The introduction of ownership and move-only types would allow us to express futures as a more efficient move-only type requiring exclusive ownership to be forwarded from the fulfilling task to the receiving task, avoiding the threading and queueing problems of a class-based approach, as seen in Rust's [tokio.rs](https://tokio.rs) framework. Whether that is a better design or not needs further discussion.
    Furthermore, the shape and functionality of a future may also be affected by Swift's planned evolution. A `Future` type designed for Swift today would need to be a `class`, and therefore need to guard against potentially multithreaded access, races to fulfill or attempts to fulfill multiple times, and potentially unbounded queueing of awaiting coroutines on the shared future; however, the introduction of ownership and move-only types would allow us to express futures as a more efficient move-only type requiring exclusive ownership to be forwarded from the fulfilling task to the receiving task, avoiding the threading and queueing problems of a class-based approach, as seen in Rust's [tokio.rs](https://tokio.rs) framework. tokio.rs and the C++ coroutine TR also both take the approach of making futures/continuations into templated/generic traits instead of a single concrete implementation, so that the compiler can deeply specialize and optimize state machines for composed async operations. Whether that is a good design for Swift as well needs further exploration.

    #### Have async calls always return a `Future`

    @@ -691,6 +694,28 @@ Despite this model being widely know, we believe that the proposed design is

    The primary argument for adding async/await (and then generators) to the language as first-class language features is that they are the vastly most common use-case of coroutines. In the author's opinion, the design as proposed gives something that works better than the C# model in practice, while also providing a more useful/general language model.

    #### Have a generalized "do notation" for monadic types

    Another approach to avoiding the one-true-future-type problem of C# could be to have a general language feature for chaining continuations through a monadic interface. Although this provides a more general language feature, it still has many of the shortcomings discussed above; it would still perform only a shallow transform of the current function body and introduce a temporary value at every point the coroutine is "awaited". Monads also compose poorly with each other, and require additional lifting and transformation logic to plumb through higher-order operations, which were some of the reasons we also chose not to base Swift's error handling model on sugar over `Result` types. Note that the delimited continuation primitives offered in this proposal are general purpose and can in fact be used to represent monadic unwrapping operations for types like `Optional` or `Result`:

    ```swift
    func doOptional<T>(_ body: (_ unwrap: (T?) async -> T) async -> T?) -> T? {
    var result: T?

    func unwrap(_ value: T?) async -> T {
    if let value = value {
    return value
    }
    suspendAsync { _ in result = nil }
    }

    beginAsync {
    body(unwrap)
    }
    }
    ```

    Monads that represent repeated or nondeterministic operations would not be representable this way due to the one-shot constraint on continuations, but representing such computations as straight-line code in an imperative language with shared mutable state seems like a recipe for disaster to us.

    ## Potential Future Directions

    @@ -759,4 +784,4 @@ completion handler on the original queue.

    ## Thanks

    Thanks to @oleganza for the original draft which influenced this!
    Thanks to @oleganza for the original draft which influenced this!
  4. @lattner lattner revised this gist Aug 18, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -68,7 +68,7 @@ func processImageData2(completionBlock: (result: Image?, error: Error?) -> Void)
    completionBlock(nil, error)
    return
    }
    return imageResult
    completionBlock(imageResult)
    }
    }
    }
  5. @lattner lattner revised this gist Aug 18, 2017. 1 changed file with 7 additions and 1 deletion.
    8 changes: 7 additions & 1 deletion async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -452,7 +452,13 @@ func processImageData() async throws -> (half1: Image, half2: Image)
    func processImageData() async throws
    ```

    There are many details that should be defined as part of this importing process - for example, what are the exact rules for the transformation? Are multiple result functions common enough to handle automatically? Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration? Without substantial ObjC importer work, making a clean break and forcing migration in Swift 5 mode would be the most practical way to preserve overridability, but would create a lot of churn in 4-to-5 migration. Alternatively, it may be acceptable to present the `async` versions as `final` wrappers over the underlying callback-based interfaces; this would subclassers to work with the callback-based interface, but there are generally fewer subclassers than callers.
    There are many details that should be defined as part of this importing process - for example:
    - What are the exact rules for the transformation?
    - Are multiple result functions common enough to handle automatically?
    - Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration?
    - What should happen with the non-Void-returning completion handler functions (e.g. in `URLSession`)?

    Without substantial ObjC importer work, making a clean break and forcing migration in Swift 5 mode would be the most practical way to preserve overridability, but would create a lot of churn in 4-to-5 migration. Alternatively, it may be acceptable to present the `async` versions as `final` wrappers over the underlying callback-based interfaces; this would subclassers to work with the callback-based interface, but there are generally fewer subclassers than callers.

    ## Interaction with existing features

  6. @lattner lattner revised this gist Aug 17, 2017. 1 changed file with 592 additions and 1182 deletions.
    1,774 changes: 592 additions & 1,182 deletions async_swift_proposal.md
    View file
    592 additions, 1,182 deletions not shown because the diff is too large. Please use a local Git client to view these changes.
  7. @lattner lattner revised this gist Aug 17, 2017. 1 changed file with 1182 additions and 592 deletions.
    1,774 changes: 1,182 additions & 592 deletions async_swift_proposal.md
    View file
    1,182 additions, 592 deletions not shown because the diff is too large. Please use a local Git client to view these changes.
  8. @lattner lattner revised this gist Aug 16, 2017. 1 changed file with 48 additions and 13 deletions.
    61 changes: 48 additions & 13 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -139,7 +139,7 @@ func processImageData5(recipient:Person, completionBlock: (result: Image?, error

    #### Problem 5: Because completion handlers are awkward, too many APIs are defined synchronously

    This is hard to quantify, but the author believes that the awkwardness of defining and using asynchronous APIs (using completion handlers) has led to many APIs being defined with apparently synchronous behavior, even when they can block. This can lead to problematic performance and responsiveness problems in UI applications - e.g. spinning cursor. It can also lead to the definition of APIs that cannot be used when asynchrony is critical to achieve scale, e.g. on the server.
    This is hard to quantify, but the authors believe that the awkwardness of defining and using asynchronous APIs (using completion handlers) has led to many APIs being defined with apparently synchronous behavior, even when they can block. This can lead to problematic performance and responsiveness problems in UI applications - e.g. spinning cursor. It can also lead to the definition of APIs that cannot be used when asynchrony is critical to achieve scale, e.g. on the server.

    #### Problem 6: Other "resumable" computations are awkward to define

    @@ -177,8 +177,8 @@ These problem have been faced in many systems and many languages, and the abstra
    This proposal adds general coroutine support to Swift, biasing the nomenclature and terminology towards the most common use-case: defining and using asynchronous APIs, eliminating many of the problems working with completion handlers. The choice of terminology (`async` vs `yields`) is a bikeshed topic which needs to be addressed, but isn't pertinent to the core semantics of the model. See [Alternate Syntax Options](#alternate-syntax-options) at the end for an exploration of syntactic options in this space.

    It is important to understand up-front, that the proposed coroutine model does not interface
    with any particularly concurrency primitives on the system: you can think of it as syntactic
    sugar for completion handlers. This means that, the introduction of coroutines would not
    with any particular concurrency primitives on the system: you can think of it as syntactic
    sugar for completion handlers. This means that the introduction of coroutines would not
    change the queues that completion handlers are called on, as happens in some other systems.

    ### Async semantics
    @@ -263,7 +263,35 @@ func suspendAsync<T>(
    ) async throws -> T
    ```

    These are similar to the "shift" and "reset" primitives of [delimited continuations](https://en.wikipedia.org/wiki/Delimited_continuation). These primitives enable callback-based APIs to be wrapped up as async coroutine APIs:
    These are similar to the "shift" and "reset" primitives of [delimited continuations](https://en.wikipedia.org/wiki/Delimited_continuation). These enable a non-async function to call an `async` function. For example, consider this
    `@IBAction` written with completion handlers:

    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    processImage(completionHandler: {(image) in
    // 2
    imageView.image = image
    })
    // 3
    }
    ```

    This is an essential pattern, but is itself sort of odd: an `async` operation is being fired off immediately (#1), then runs the subsequent code (#3), and the completion handler (#2) runs at some time later -- on some queue (often the main one). This pattern frequently leads to mutation of global state (as in this example) or to making assumptions about which queue the completion handler is run on. Despite these problems, it is essential that the model encompasses this pattern, because it is a practical necessity in Cocoa development. With this proposal, it would look like this:

    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    beginAsync {
    // 2
    let image = await processImage()
    imageView.image = image
    }
    // 3
    }
    ```

    These primitives enable callback-based APIs to be wrapped up as async coroutine APIs:

    ```swift
    // Legacy callback-based API
    @@ -491,13 +519,16 @@ of cleaning up an abandoned coroutine:

    ```swift
    func processImageData() async throws -> Image {
    startSpinner()
    defer { stopSpinner() } // will be called when error is thrown, when all
    // operations complete and a result is returned,
    // or when the coroutine is abandoned. We don't
    // want to leave the cursor spinning if work has
    // stopped.
    let dataResource = try await loadWebResource("dataprofile.txt")
    startProgressBar()
    defer {
    // This will be called when error is thrown, when all operations
    // complete and a result is returned, or when the coroutine is
    // abandoned. We don't want to leave the progress bar animating if
    // work has stopped.
    stopProgressBar()
    }

    let dataResource = try await loadWebResource("dataprofile.txt")
    let imageResource = try await loadWebResource("imagedata.dat")
    do {
    let imageTmp = try await decodeImage(dataResource, imageResource)
    @@ -652,7 +683,7 @@ Despite this model being widely know, we believe that the proposed design is
    - Requiring a future object to be instantiated at every `await` point adds overhead. Since a major use case for this feature is to adapt existing Cocoa APIs, which already use callbacks, queues, target-action, or other mechanisms to coordinate the scheduling of the continuation of an async task, introducing a future into the mix would be an additional unnecessary middleman incurring overhead when wrapping these APIs, when in most cases there is already a direct consumer for the continuation point.
    - A design that directly surfaces a monadic type like `Future` as the result of an async computation heavily implies a compiler-driven coroutine transform, whereas this design is more implementation-agnostic. Compiler-transformed coroutines are a great compromise for integrating lightweight tasks into an existing runtime model that's already heavily callstack-dependent, or one aims to maintain efficient interop with C or other languages that heavily constrain the implementation model, and Swift definitely has both. It is conceivable that, in the eventual future, a platform such as Swift-on-the-server could provide a pure- or predominantly-Swift ABI where enough code is pure Swift to make cheap relocatable stacks the norm and overhead on C interop acceptable, as has happened with the Go runtime. This could make `async` a no-op at runtime, and perhaps allow us to consider eliminating the annotation altogether. The semantic presence of a future object between every layer of an async process would be an obstacle to the long-term efficiency of such a platform.

    The primary argument for adding async/await (and then generators) to the language as first-class language features is that they are the vastly most common use-case of coroutines. In the authors opinion, the design as proposed gives something that works better than the C# model in practice, while also providing a more useful/general language model.
    The primary argument for adding async/await (and then generators) to the language as first-class language features is that they are the vastly most common use-case of coroutines. In the author's opinion, the design as proposed gives something that works better than the C# model in practice, while also providing a more useful/general language model.


    ## Potential Future Directions
    @@ -663,6 +694,10 @@ This proposal has been kept intentionally minimal, but there are many possible w

    Given the availability of convenient asynchrony in Swift, it would make sense to introduce new APIs to take advantage of it. Filesystem APIs are one example that would be great to see. The Swift on Server working group would also widely adopt these features. GCD could also provide new helpers for allowing `() async -> Void` coroutines to be enqueued, or for allowing a running coroutine to move its execution onto a different queue.

    #### Documentation

    As part of this introduction it makes sense to extend the Swift API design guidelines and other documentation to describe and encourage best practices in asynchronous API design.

    #### `rethrows` could be generalized to support potentially `async` operations

    The `rethrows` modifier exists in Swift to allow limited abstraction over function types by higher order functions. It would be possible to define a similar mechanism to allow abstraction over `async` operations as well. More generally, by modeling both `throws` and `async` as effects on function types, we can eventually provide common abstraction tools to abstract over both effects in protocols and generic code, simultaneously addressing the "can't have a Sequence that `throws`" and "can't have a Sequence that's `async`" kinds of limitations in the language today.
    @@ -700,7 +735,7 @@ condition):

    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    detachedFuture {
    beginAsync {
    let image = await processImageData()
    // Do the update on the main thread/queue since it owns imageView.
    mainQ.async {
  9. @lattner lattner revised this gist Aug 16, 2017. 1 changed file with 134 additions and 125 deletions.
    259 changes: 134 additions & 125 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -11,31 +11,32 @@ This paper introduces a first class [Coroutine model](https://en.wikipedia.org/w

    It is important to understand that this is proposing compiler support that is completely concurrency runtime-agnostic. This proposal does not include a new runtime model (like "actors") - it works just as well with GCD as with pthreads or another API. Furthermore, unlike designs in other languages, it is independent of specific coordination mechanisms, such as futures or channels, allowing these to be built as library feature. The only runtime support required is compiler support logic for transforming and manipulating the implicitly generated closures.

    This is derived from an earlier proposal written by [Oleg Andreev](https://github.com/oleganza), available [here](https://gist.github.com/oleganza/7342ed829bddd86f740a). It has been significantly rewritten by [Chris Lattner](https://github.com/lattner) and [Joe Groff](https://github.com/jckarter).
    This draws some inspiration from an earlier proposal written by [Oleg Andreev](https://github.com/oleganza), available [here](https://gist.github.com/oleganza/7342ed829bddd86f740a). It has been significantly rewritten by [Chris Lattner](https://github.com/lattner) and [Joe Groff](https://github.com/jckarter).

    ## Motivation: Completion handlers are suboptimal

    To provide motivation for why it is important to do something here, lets look at some of the problems that Cocoa (and server/cloud) programmers frequently face.

    #### Problem 1: Pyramid of doom

    Sequence of simple operations is unnaturally composed in the nested blocks.
    Sequence of simple operations is unnaturally composed in the nested blocks. Here is a
    made up example showing this:

    ```swift
    func makeSandwich1(completionBlock: (result: Sandwich) -> Void) {
    cutBread { bread in
    cutCheese { cheese in
    cutHam { ham in
    cutTomato { tomato in
    completionBlock(Sandwich([bread, cheese, ham, tomato]))
    func processImageData1(completionBlock: (result: Image) -> Void) {
    loadWebResource("dataprofile.txt") { dataResource in
    loadWebResource("imagedata.dat") { imageResource in
    decodeImage(dataResource, imageResource) { imageTmp in
    dewarpAndCleanupImage(imageTmp) { imageResult in
    completionBlock(imageResult)
    }
    }
    }
    }
    }

    makeSandwich1 { sandwich in
    eat(sandwich)
    processImageData1 { image in
    display(image)
    }
    ```

    @@ -46,40 +47,40 @@ This "pyramid of doom" makes it difficult to keep track of code that is running,
    Handling errors becomes difficult and very verbose. Swift 2 introduced an error handling model for synchronous code, but callback-based interfaces do not derive any benefit from it:

    ```swift
    func makeSandwich2(completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    cutBread { bread, error in
    guard let bread = bread else {
    func processImageData2(completionBlock: (result: Image?, error: Error?) -> Void) {
    loadWebResource("dataprofile.txt") { dataResource, error in
    guard let dataResource = dataResource else {
    completionBlock(nil, error)
    return
    }
    cutCheese { cheeseSlice, error in
    guard let cheeseSlice = cheeseSlice else {
    loadWebResource("imagedata.dat") { imageResource, error in
    guard let imageResource = imageResource else {
    completionBlock(nil, error)
    return
    }
    cutHam { hamSlice, error in
    guard let hamSlice = hamSlice else {
    decodeImage(dataResource, imageResource) { imageTmp, error in
    guard let imageTmp = imageTmp else {
    completionBlock(nil, error)
    return
    }
    cutTomato { tomatoSlice in
    guard let tomatoSlice = tomatoSlice else {
    dewarpAndCleanupImage(imageTmp) { imageResult in
    guard let imageResult = imageResult else {
    completionBlock(nil, error)
    return
    }
    completionBlock(Sandwich([bread, cheeseSlice, hamSlice, tomatoSlice]), nil)
    return imageResult
    }
    }
    }
    }
    }

    makeSandwich2 { sandwich, error in
    guard let sandwich = sandwich else {
    error("No sandwich today")
    processImageData2 { image, error in
    guard let image = image else {
    error("No image today")
    return
    }
    eat(sandwich)
    display(image)
    }
    ```

    @@ -88,15 +89,15 @@ makeSandwich2 { sandwich, error in
    Conditionally executing an asynchronous function is a huge pain. Perhaps the best approach is to write half of the code in a helper "continuation" closure that is conditionally executed, like this:

    ```swift
    func makeSandwich3(recipient: Person, completionBlock: (result: Sandwich) -> Void) {
    let continuation: (contents: SandwichContents) -> Void = {
    func processImageData3(recipient: Person, completionBlock: (result: Image) -> Void) {
    let continuation: (contents: image) -> Void = {
    // ... continue and call completionBlock eventually
    }
    if !recipient.isVegetarian {
    continuation(mushroom)
    if recipient.hasProfilePicture {
    continuation(recipient.profilePicture)
    } else {
    cutHam { hamSlice in
    continuation(hamSlice)
    decodeImage { image in
    continuation(image)
    }
    }
    }
    @@ -107,13 +108,13 @@ func makeSandwich3(recipient: Person, completionBlock: (result: Sandwich) -> Voi
    It's easy to bail out by simply returning without calling the appropriate block. When forgotten, the issue is very hard to debug:

    ```swift
    func makeSandwich4(completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    cutBread { bread, error in
    guard let bread = bread else {
    func processImageData4(completionBlock: (result: Image?, error: Error?) -> Void) {
    loadWebResource("dataprofile.txt") { dataResource, error in
    guard let dataResource = dataResource else {
    return // <- forgot to call the block
    }
    cutCheese { cheese, error in
    guard let cheese = cheese else {
    loadWebResource("imagedata.dat") { imageResource, error in
    guard let imageResource = imageResource else {
    return // <- forgot to call the block
    }
    ...
    @@ -126,10 +127,10 @@ When you do not forget to call the block, you can still forget to return after t
    Thankfully `guard` syntax protects against that to some degree, but it's not always relevant.

    ```swift
    func makeSandwich5(recipient:Person, completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    if recipient.isVegetarian {
    if let sandwich = cachedVegetarianSandwich {
    completionBlock(cachedVegetarianSandwich) // <- forgot to return after calling the block
    func processImageData5(recipient:Person, completionBlock: (result: Image?, error: Error?) -> Void) {
    if recipient.hasProfilePicture {
    if let image = recipient.profilePicture {
    completionBlock(image) // <- forgot to return after calling the block
    }
    }
    ...
    @@ -173,8 +174,12 @@ It is the responsibility of the compiler to transform the function into a form t

    These problem have been faced in many systems and many languages, and the abstraction of [coroutines](https://en.wikipedia.org/wiki/Coroutine) is a standard way to address them. Without delving too much into theory, coroutines are an extension of basic functions that allow a function to return a value *or be suspended*. They can be used to implement generators, asynchronous models, and other capabilities - there is a large body of work on the theory, implementation, and optimization of them.

    This proposal adds general coroutine support to Swift, biasing the nomenclature and terminology towards the most common use-case: defining and using asynchronous APIs, eliminating many of the problems working with completion handlers. The choice of terminology (`async` vs `yields`) is a bikeshed topic which needs to be addressed, but isn't pertinent to the core semantics of the model. See [Alternate Syntax Options][#alternate-syntax-options] at the end for an exploration of syntactic options in this space.
    This proposal adds general coroutine support to Swift, biasing the nomenclature and terminology towards the most common use-case: defining and using asynchronous APIs, eliminating many of the problems working with completion handlers. The choice of terminology (`async` vs `yields`) is a bikeshed topic which needs to be addressed, but isn't pertinent to the core semantics of the model. See [Alternate Syntax Options](#alternate-syntax-options) at the end for an exploration of syntactic options in this space.

    It is important to understand up-front, that the proposed coroutine model does not interface
    with any particularly concurrency primitives on the system: you can think of it as syntactic
    sugar for completion handlers. This means that, the introduction of coroutines would not
    change the queues that completion handlers are called on, as happens in some other systems.

    ### Async semantics

    @@ -192,30 +197,29 @@ Just as a normal function (#1) will implicitly convert to a throwing function (#
    On the function declaration side of the things, you can declare a function as being asynchronous just as you declare it to be throwing, but use the `async` keyword:

    ```swift
    func makeSandwich() async -> Sandwich { ... }
    func processImageData() async -> Image { ... }

    // Semantically similar to this:
    func makeSandwich(completionHandler: (result: Sandwich) -> Void) { ... }
    func processImageData(completionHandler: (result: Image) -> Void) { ... }
    ```

    Calls to `async` functions can implicitly suspend the current coroutine. To make this apparent to maintainers of code, you are required to "mark" expressions that call `async` functions with the new `await` keyword (exactly analogously to how `try` is used to mark subexpressions that contain throwing calls). Putting these pieces together, the first example (from the pyramid of doom explanation, above) can be rewritten in a more natural way:

    ```swift
    func cutBread() async -> Bread
    func cutCheese() async -> Cheese
    func cutHam() async -> Ham
    func cutTomato() async -> Vegetable

    func makeSandwich1() async -> Sandwich {
    let bread = await cutBread()
    let cheese = await cutCheese()
    let ham = await cutHam()
    let tomato = await cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    func loadWebResource(_ path: String) async -> Resource
    func decodeImage(_ r1: Resource, _ r2: Resource) async -> Image
    func dewarpAndCleanupImage(_ i : Image) async -> Image

    func processImageData1() async -> Image {
    let dataResource = await loadWebResource("dataprofile.txt")
    let imageResource = await loadWebResource("imagedata.dat")
    let imageTmp = await decodeImage(dataResource, imageResource)
    let imageResult = await dewarpAndCleanupImage(imageTmp)
    return imageResult
    }
    ```

    Under the hood, the compiler rewrites this code using nested closures like in example `makeSandwich1` above. Note that every operation starts only after the previous one has completed, but each call site to an `async` function could suspend execution of the current function.
    Under the hood, the compiler rewrites this code using nested closures like in example `processImageData1` above. Note that every operation starts only after the previous one has completed, but each call site to an `async` function could suspend execution of the current function.

    Finally, you are only allowed to invoke an `async` function from within another `async` function or closure. This follows the model of Swift 2 error handling, where you cannot call a throwing function unless you're in a throwing function or inside of a `do/catch` block.

    @@ -303,7 +307,7 @@ func queueHopping() async -> Void {
    }
    ```

    Generalized abstractions for coordinating coroutines can also be built. The simplest of these is a [future](https://en.wikipedia.org/wiki/Futures_and_promises), a value that represents a future value which may not be resolved yet. The exact design for a future type deserves its own proposal (and would be influenced by the availability of planned Swift features like ownership and move-only types), but a proof of concept could look like this:
    Generalized abstractions for coordinating coroutines can also be built. The simplest of these is a [future](https://en.wikipedia.org/wiki/Futures_and_promises), a value that represents a future value which may not be resolved yet. The exact design for a future type deserves its own proposal, but a proof of concept could look like this:

    ```swift
    class Future<T> {
    @@ -374,18 +378,24 @@ class Future<T> {
    Futures allow parallel execution, by moving `await` from the call to the result when it is needed, and wrapping the parallel calls in individual `Future` objects:

    ```swift
    func makeSandwich1a() async -> Sandwich {
    let bread = Future { await cutBread() }
    let cheese = Future { await cutCheese() }
    let ham = Future { await cutHam() }
    let tomato = Future { await cutTomato() }
    ... stuff ...
    return Sandwich([await bread.get(), await cheese.get(), await ham.get(), await tomato.get())
    func processImageData1a() async -> Image {
    let dataResource = Future { await loadWebResource("dataprofile.txt") }
    let imageResource = Future { await loadWebResource("imagedata.dat") }

    // ... other stuff can go here to cover load latency...

    let imageTmp = await decodeImage(dataResource.get(), imageResource.get())
    let imageResult = await dewarpAndCleanupImage(imageTmp)
    return imageResult
    }
    ```

    In the above example all four operations will start one after another, and the unevaluated computation is wrapped into a `Future` value. This allows all of them to happen concurrently (in a way that need not be defined by the language or
    by the `Future` implementation), and the function will wait for completion of every one of them before returning a `Sandwich`. Note that `await` does not block flow of execution: if the value is not yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.
    In the above example, the first two operations will start one after another, and the
    unevaluated computations are wrapped into a `Future` value. This allows all of them to
    happen concurrently (in a way that need not be defined by the language or
    by the `Future` implementation), and the function will wait for completion of them before
    decoding the image. Note that `await` does not block flow of execution: if the value is not
    yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.

    Other coordination abstractions such as [Communicating Sequential Process channels](https://en.wikipedia.org/wiki/Communicating_sequential_processes) or [Concurrent ML events](https://wingolog.org/archives/2017/06/29/a-new-concurrent-ml) can also be developed as libraries for coordinating coroutines; their implementation is left as an exercise for the reader.

    @@ -397,21 +407,21 @@ There are multiple possible designs for this with different tradeoffs. The maxi

    ```objc
    // Before
    - (void) makeSandwich:(void(^)())completionHandler;
    - (void) makeSandwich:(void(^)(Sandwich* __nonnull sandwich))completionHandler;
    - (void) makeSandwich:(void(^)(Sandwich* __nullable sandwich, NSError* __nullable error))completionHandler;
    - (void) makeSandwich:(void(^)(Sandwich* __nullable half1, Sandwich* __nullable half2, NSError* __nullable error))completionHandler;
    - (void) makeSandwich:(void(^)(NSError* __nullable error))completionHandler;
    - (void) processImageData:(void(^)())completionHandler;
    - (void) processImageData:(void(^)(Image* __nonnull image))completionHandler;
    - (void) processImageData:(void(^)(Image* __nullable image1, NSError* __nullable error))completionHandler;
    - (void) processImageData:(void(^)(Image* __nullable half1, Image* __nullable half2, NSError* __nullable error))completionHandler;
    - (void) processImageData:(void(^)(NSError* __nullable error))completionHandler;
    ```

    The declarations above are imported both in their normal completion handler form, but also in their nicer `async` forms:

    ```swift
    func makeSandwich() async
    func makeSandwich() async -> Sandwich
    func makeSandwich() async throws -> Sandwich
    func makeSandwich() async throws -> (half1: Sandwich, half2: Sandwich)
    func makeSandwich() async throws
    func processImageData() async
    func processImageData() async -> Image
    func processImageData() async throws -> Image
    func processImageData() async throws -> (half1: Image, half2: Image)
    func processImageData() async throws
    ```

    There are many details that should be defined as part of this importing process - for example, what are the exact rules for the transformation? Are multiple result functions common enough to handle automatically? Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration? Without substantial ObjC importer work, making a clean break and forcing migration in Swift 5 mode would be the most practical way to preserve overridability, but would create a lot of churn in 4-to-5 migration. Alternatively, it may be acceptable to present the `async` versions as `final` wrappers over the underlying callback-based interfaces; this would subclassers to work with the callback-based interface, but there are generally fewer subclassers than callers.
    @@ -426,26 +436,25 @@ Error handling syntax introduced in Swift 2 composes naturally with this asynchr

    ```swift
    // Could throw or be interrupted:
    func makeSandwich() async throws -> Sandwich
    func processImageData() async throws -> Image

    // Semantically similar to:
    func makeSandwich(completionHandler: (result: Sandwich?, error: Error?)->Void)
    func processImageData(completionHandler: (result: Image?, error: Error?) -> Void)
    ```

    Our example thus becomes (compare with the example `makeSandwich2`):
    Our example thus becomes (compare with the example `processImageData2`):

    ```swift
    func cutBread() async throws -> Bread
    func cutCheese() async throws -> Cheese
    func cutHam() async throws -> Ham
    func cutTomato() async throws -> Vegetable

    func makeSandwich2() async throws -> Sandwich {
    let bread = try await cutBread()
    let cheese = try await cutCheese()
    let ham = try await cutHam()
    let tomato = try await cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    func loadWebResource(_ path: String) async throws -> Resource
    func decodeImage(_ r1: Resource, _ r2: Resource) async throws -> Image
    func dewarpAndCleanupImage(i: Image) async throws -> Vegetable

    func processImageData2() async throws -> Image {
    let dataResource = try await loadWebResource("dataprofile.txt")
    let imageResource = try await loadWebResource("imagedata.dat")
    let imageTmp = try await decodeImage(dataResource, imageResource)
    let imageResult = try await dewarpAndCleanupImage(imageTmp)
    return imageResult
    }
    ```

    @@ -481,37 +490,36 @@ extension to add the guarantee that `defer`-ed statements also execute as part
    of cleaning up an abandoned coroutine:

    ```swift
    func makeSandwich() async throws -> Sandwich {
    func processImageData() async throws -> Image {
    startSpinner()
    defer { stopSpinner() } // will be called when error is thrown, when all
    // operations complete and a result is returned,
    // or when the coroutine is abandoned. We don't
    // want to leave the cursor spinning if work has
    // stopped.
    let bread = try await cutBread()
    let cheese = try await cutCheese()
    let dataResource = try await loadWebResource("dataprofile.txt")
    let imageResource = try await loadWebResource("imagedata.dat")
    do {
    let ham = try await cutHam()
    } catch _ as Trichinosis {
    // Better shut the shop down until the health inspector arrives
    let imageTmp = try await decodeImage(dataResource, imageResource)
    } catch _ as CorruptedImage {
    // Give up hope now.
    await abandon()
    }
    let tomato = try await cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    return try await dewarpAndCleanupImage(imageTmp)
    }
    ```

    This fills in another gap in the expressivity of callback-based APIs, where it is difficult to express cleanup code that must execute at some point regardless of whether the callback closure is really called. However, abandonment should not be taken as a fully-baked "cancellation" feature; if cancellation is important, it should continue to be implemented by the programmer where needed, and there are many standard patterns that can be applied. Particularly when coupled with error handling, common cancellation patterns become very elegant:

    ```
    @IBAction func makeSandwich(sender: AnyObject) {
    @IBAction func processImageData(sender: AnyObject) {
    beginAsync {
    do {
    let bread = try await kitchen.cutBread()
    let cheese = try await kitchen.cutCheese()
    let ham = try await kitchen.cutHam()
    let tomato = try await kitchen.cutTomato()
    eat(Sandwich(bread, cheese, ham, tomato]))
    let dataResource = try await imageProcessor.loadWebResource("dataprofile.txt")
    let imageResource = try await imageProcessor.loadWebResource("imagedata.dat")
    let imageTmp = try await imageProcessor.decodeImage(dataResource, imageResource)
    let imageResult = try await imageProcessor.dewarpAndCleanupImage(imageTmp)
    display(imageResult)
    } catch CocoaError.userCancelled {
    // Ignore, user quit the kitchen.
    } catch {
    @@ -521,28 +529,28 @@ This fills in another gap in the expressivity of callback-based APIs, where it i
    }
    }
    @IBAction func exitKitchen(sender: AnyObject) {
    kitchen.cancel()
    @IBAction func stopImageProcessing(sender: AnyObject) {
    imageProcessor.cancel()
    }
    ```

    Internally, `kitchen` may use `NSOperation` or a custom `cancelled` flag. The intent of this section is to give a single example of how to approach this, not to define a normative or all-encompassing approach that should be used in all cases.
    Internally, `imageProcessor` may use `NSOperation` or a custom `cancelled` flag. The intent of this section is to give a single example of how to approach this, not to define a normative or all-encompassing approach that should be used in all cases.

    #### Completion handlers with multiple return values

    Completion handler APIs may have multiple result arguments (not counting an error argument). These are naturally represented by tuple results in `async` functions:

    ```swift
    // Before
    func makeClubSandwich(completionHandler: (part1: Sandwich?, part2: Sandwich?, error: Error?) -> Void)
    func processImageHalves(completionHandler: (part1: Image?, part2: Image?, error: Error?) -> Void)

    // After
    func makeSandwich() async throws -> (Sandwich, Sandwich)
    func processImageHalves() async throws -> (Image, Image)
    ```

    ## Source Compatibility

    This is a generally additive feature, but it does turn take `async` and `await` as keywords, so it will break code that uses them as identifiers. This is expected to have very minor impact: the most pervasive use of `async` as an identifier occurs in code that works with dispatch queues, but fortunately keywords are allowed as qualified member names, so code like this doesn't need any change:
    This is a generally additive feature, but it does take `async` and `await` as keywords, so it will break code that uses them as identifiers. This is expected to have very minor impact: the most pervasive use of `async` as an identifier occurs in code that works with dispatch queues, but fortunately keywords are allowed as qualified member names, so code like this doesn't need any change:

    ```swift
    myQueue.async { ... }
    @@ -572,22 +580,21 @@ Here are a couple of syntax level changes to the proposal that are worth discuss

    #### Spelling of `async` keyword

    Instead of spelling the function type modifier as `async`, it could be spelled as `yields`, since the functionality really is about coroutines, not about asynchrony by itself. The recommendation to use `async/await` biases towards making sure that the most common use case (asynchrony) uses industry standard terms. The other coroutine use cases would be much less common, at least according to the unscientific opinion of the proposal author.
    Instead of spelling the function type modifier as `async`, it could be spelled as `yields`, since the functionality really is about coroutines, not about asynchrony by itself. The recommendation to use `async/await` biases towards making sure that the most common use case (asynchrony) uses industry standard terms. The other coroutine use cases would be much less common, at least according to the unscientific opinion of the proposal authors.

    To give an idea of what this could look like, here's the example from above resyntaxed:

    ```swift
    func cutBread() yields -> Bread
    func cutCheese() yields -> Cheese
    func cutHam() yields -> Ham
    func cutTomato() yields -> Vegetable

    func makeSandwich1() yields -> Sandwich {
    let bread = yield cutBread()
    let cheese = yield cutCheese()
    let ham = yield cutHam()
    let tomato = yield cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    func loadWebResource(_ path: String) yields -> Resource
    func decodeImage(_ r1: Resource, _ r2: Resource) yields -> Image
    func dewarpAndCleanupImage(_ i : Image) yields -> Image

    func processImageData1() yields -> Image {
    let dataResource = yield loadWebResource("dataprofile.txt")
    let imageResource = yield loadWebResource("imagedata.dat")
    let imageTmp = yield decodeImage(dataResource, imageResource)
    let imageResult = yield dewarpAndCleanupImage(imageTmp)
    return imageResult
    }
    ```

    @@ -618,14 +625,16 @@ The other way to factor the complexity is to make it so that `async` functions d
    (Int) async(nonthrowing) -> Int // Asynchronous function, doesn't throw.
    ```

    This model provides a ton of advantages: it is arguably the right defaults for the vast majority of clients (reducing boilerplate and syntactic noise), provides the ability for the importer and experts to get what they want. The only downside of is that it is a less obvious design than have two orthogonal axes, but in the opinion of the proposal author, this is probably the right set of tradeoffs.
    This model provides a ton of advantages: it is arguably the right defaults for the vast majority of clients (reducing boilerplate and syntactic noise), provides the ability for the importer and experts to get what they want. The only downside of is that it is a less obvious design than presenting two orthogonal axes, but in the opinion of the proposal authors, this is probably the right set of tradeoffs.


    ## Alternatives Considered

    #### Include `Future` or other coordination abstractions in this proposal

    This proposal does not formally propose a `Future` type, or any other coordination abstractions. There are many rational designs for futures, and a lot of experience working with them. On the other hand, there are also completely different coordination primitives that can be used with this coroutine design, and blessing `Future` out of the gate with standard library support may suppress valuable experimentation in this space. Furthermore, the shape and functionality of a future may also be affected by Swift's planned evolution; whereas a Future type designed for Swift today would most likely need to be a `class`, and therefore need to guard against potentially multithreaded access, races to fulfill or attempts to fulfill multiple times, and potentially unbounded queueing of awaiting coroutines on the shared future. Ownership and move-only types would allow us to express futures as a more efficient move-only type requiring exclusive ownership to be forwarded from the fulfilling task to the receiving task, avoiding the threading and queueing problems of a class-based approach, as seen in Rust's [tokio.rs](https://tokio.rs) framework.
    This proposal does not formally propose a `Future` type, or any other coordination abstractions. There are many rational designs for futures, and a lot of experience working with them. On the other hand, there are also completely different coordination primitives that can be used with this coroutine design, and incorporating them into this proposal only makes it larger.

    Furthermore, the shape and functionality of a future may also be affected by Swift's planned evolution; whereas a Future type designed for Swift today would need to be a `class`, and therefore need to guard against potentially multithreaded access, races to fulfill or attempts to fulfill multiple times, and potentially unbounded queueing of awaiting coroutines on the shared future. The introduction of ownership and move-only types would allow us to express futures as a more efficient move-only type requiring exclusive ownership to be forwarded from the fulfilling task to the receiving task, avoiding the threading and queueing problems of a class-based approach, as seen in Rust's [tokio.rs](https://tokio.rs) framework. Whether that is a better design or not needs further discussion.

    #### Have async calls always return a `Future`

    @@ -692,10 +701,10 @@ condition):
    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    detachedFuture {
    let sandwich = await makeSandwich()
    let image = await processImageData()
    // Do the update on the main thread/queue since it owns imageView.
    mainQ.async {
    imageView.image = sandwich.preview
    imageView.image = image
    }
    }
    }
    @@ -709,4 +718,4 @@ completion handler on the original queue.

    ## Thanks

    @oleganza thanked @groue and @pierlo for their feedback on his original proposal. Thanks to @oleganza for the original draft which influenced this!
    Thanks to @oleganza for the original draft which influenced this!
  10. @lattner lattner revised this gist Aug 16, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,7 +1,7 @@
    # Async/Await for Swift

    * Proposal: SE-XXXX
    * Author: [Chris Lattner](https://github.com/lattner), [Joe Groff](https://github.com/jckarter)
    * Authors: [Chris Lattner](https://github.com/lattner), [Joe Groff](https://github.com/jckarter)

    ## Introduction

  11. @lattner lattner revised this gist Aug 16, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -5,7 +5,7 @@

    ## Introduction

    Modern Cocoa development involves a lot of asynchronous programming using closures and completion handlers, but these APIs are hard to use. This gets particularly problematic when many asynchronous operations are used, error handling is required, or control flow between asyncronous calls gets complicated. This proposal describes a language extension to make this a lot more natural and less error prone.
    Modern Cocoa development involves a lot of asynchronous programming using closures and completion handlers, but these APIs are hard to use. This gets particularly problematic when many asynchronous operations are used, error handling is required, or control flow between asynchronous calls gets complicated. This proposal describes a language extension to make this a lot more natural and less error prone.

    This paper introduces a first class [Coroutine model](https://en.wikipedia.org/wiki/Coroutine) to Swift. Functions can opt into to being *async*, allowing the programmer to compose complex logic involving asynchronous operations, leaving the compiler in charge of producing the necessary closures and state machines to implement that logic.

  12. @lattner lattner revised this gist Aug 16, 2017. 1 changed file with 233 additions and 202 deletions.
    435 changes: 233 additions & 202 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,17 +1,17 @@
    # Async/Await for Swift

    * Proposal: SE-XXXX
    * Author: [Chris Lattner](https://github.com/lattner)
    * Author: [Chris Lattner](https://github.com/lattner), [Joe Groff](https://github.com/jckarter)

    ## Introduction

    Modern Cocoa development involves a lot of asynchronous programming using closures and completion handlers, but these APIs are hard to use. This gets particularly problematic when many asynchronous operations are used, error handling is required, or control flow between asyncronous calls gets complicated. This proposal describes a language extension to make this a lot more natural and less error prone.

    This paper introduces a first class [Coroutine model](https://en.wikipedia.org/wiki/Coroutine) to Swift. Further, it provides standard library affordances to expose these as *asynchronous semantics* and *Futures* to Swift, which are concepts [available in many other languages now](https://en.wikipedia.org/wiki/Await) (including C#, Javascript, Python, and others). Functions can opted into to being *async*, allowing the programmer can compose complex logic involving asynchronous operations - while the compiler produces necessary closures and state machines to implement that logic.
    This paper introduces a first class [Coroutine model](https://en.wikipedia.org/wiki/Coroutine) to Swift. Functions can opt into to being *async*, allowing the programmer to compose complex logic involving asynchronous operations, leaving the compiler in charge of producing the necessary closures and state machines to implement that logic.

    It is important to understand that this is proposing compiler support that is completely concurrency runtime-agnostic. This proposal does not include a new runtime model (like "actors") - it works just as well with GCD as with pthreads or another API. The only runtime support required is the definition of a couple of types and compiler support logic for manipulating the implicitly generated closures.
    It is important to understand that this is proposing compiler support that is completely concurrency runtime-agnostic. This proposal does not include a new runtime model (like "actors") - it works just as well with GCD as with pthreads or another API. Furthermore, unlike designs in other languages, it is independent of specific coordination mechanisms, such as futures or channels, allowing these to be built as library feature. The only runtime support required is compiler support logic for transforming and manipulating the implicitly generated closures.

    This is derived from an earlier proposal written by [Oleg Andreev](https://github.com/oleganza), available [here](https://gist.github.com/oleganza/7342ed829bddd86f740a). It has been significantly rewritten by [Chris Lattner](https://github.com/lattner), with input from [Joe Groff](https://github.com/jckarter).
    This is derived from an earlier proposal written by [Oleg Andreev](https://github.com/oleganza), available [here](https://gist.github.com/oleganza/7342ed829bddd86f740a). It has been significantly rewritten by [Chris Lattner](https://github.com/lattner) and [Joe Groff](https://github.com/jckarter).

    ## Motivation: Completion handlers are suboptimal

    @@ -41,9 +41,9 @@ makeSandwich1 { sandwich in

    This "pyramid of doom" makes it difficult to keep track of code that is running, and the stack of closures leads to many second order effects.

    #### Problem 2: Verbosity and old-style error handling
    #### Problem 2: Error handling

    Handling errors becomes difficult and very verbose.
    Handling errors becomes difficult and very verbose. Swift 2 introduced an error handling model for synchronous code, but callback-based interfaces do not derive any benefit from it:

    ```swift
    func makeSandwich2(completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    @@ -92,7 +92,7 @@ func makeSandwich3(recipient: Person, completionBlock: (result: Sandwich) -> Voi
    let continuation: (contents: SandwichContents) -> Void = {
    // ... continue and call completionBlock eventually
    }
    if !recipient.isVegeterian {
    if !recipient.isVegetarian {
    continuation(mushroom)
    } else {
    cutHam { hamSlice in
    @@ -127,16 +127,16 @@ Thankfully `guard` syntax protects against that to some degree, but it's not alw

    ```swift
    func makeSandwich5(recipient:Person, completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    if recipient.isVegeterian {
    if let sandwich = cachedVegeterianSandwich {
    completionBlock(cachedVegeterianSandwich) // <- forgot to return after calling the block
    if recipient.isVegetarian {
    if let sandwich = cachedVegetarianSandwich {
    completionBlock(cachedVegetarianSandwich) // <- forgot to return after calling the block
    }
    }
    ...
    }
    ```

    #### Problem 5: Because Completion handlers are awkward, too many APIs are defined synchronously
    #### Problem 5: Because completion handlers are awkward, too many APIs are defined synchronously

    This is hard to quantify, but the author believes that the awkwardness of defining and using asynchronous APIs (using completion handlers) has led to many APIs being defined with apparently synchronous behavior, even when they can block. This can lead to problematic performance and responsiveness problems in UI applications - e.g. spinning cursor. It can also lead to the definition of APIs that cannot be used when asynchrony is critical to achieve scale, e.g. on the server.

    @@ -169,9 +169,9 @@ It is the responsibility of the compiler to transform the function into a form t



    ## Proposed Solution: The Coroutine transformation
    ## Proposed Solution: Coroutines

    These problem have been faced in many systems and many languages, and the abstraction of [Coroutines](https://en.wikipedia.org/wiki/Coroutine) is a standard way to address them. Without delving too much into theory, Coroutines are an extension of basic functions that allow a function to return a value *or be suspended*. They can be used to implement generators, asynchronous models, and other capabilities - there is a large body of work on the theory, implementation, and optimization of them.
    These problem have been faced in many systems and many languages, and the abstraction of [coroutines](https://en.wikipedia.org/wiki/Coroutine) is a standard way to address them. Without delving too much into theory, coroutines are an extension of basic functions that allow a function to return a value *or be suspended*. They can be used to implement generators, asynchronous models, and other capabilities - there is a large body of work on the theory, implementation, and optimization of them.

    This proposal adds general coroutine support to Swift, biasing the nomenclature and terminology towards the most common use-case: defining and using asynchronous APIs, eliminating many of the problems working with completion handlers. The choice of terminology (`async` vs `yields`) is a bikeshed topic which needs to be addressed, but isn't pertinent to the core semantics of the model. See [Alternate Syntax Options][#alternate-syntax-options] at the end for an exploration of syntactic options in this space.

    @@ -220,51 +220,178 @@ Under the hood, the compiler rewrites this code using nested closures like in ex
    Finally, you are only allowed to invoke an `async` function from within another `async` function or closure. This follows the model of Swift 2 error handling, where you cannot call a throwing function unless you're in a throwing function or inside of a `do/catch` block.


    ### Futures
    #### Entering and leaving async code

    Because the coroutine transformation described in this design lives at the level of function types, it can be applied to many different applications that benefit from the inversion of control that the coroutine transformation provides (e.g. generators). That said, the most important first application is to provide `async`/`await` as a better way to work with asynchronous functions. Similar APIs could be used for other applications, but those are beyond the scope of this proposal.
    In the common case, async code ought to be invoking other async code that has been dispatched by the framework the app is built on top of, but at some point, an async process needs to spawn from a controlling synchronous context, and the async process needs to be able to suspend itself and allow its **continuation** to be scheduled by the controlling context. We need a couple of primitives to
    enable entering and suspending an async context:

    Asynchronous semantics are implemented with [futures](https://en.wikipedia.org/wiki/Futures_and_promises) — types that represent a future value which may not be resolved yet. Extending the standard library with the following declarations allow programmers to reason about this and work with them:
    ```swift
    // NB: Names subject to bikeshedding. These are low-level primitives that most
    // users should not need to interact with directly, so namespacing them
    // and/or giving them verbose names unlikely to collide or pollute code
    // completion (and possibly not even exposing them outside the stdlib to begin
    // with) would be a good idea.

    /// Begins an asynchronous coroutine, transferring control to `body` until it
    /// either suspends itself for the first time with `suspendAsync` or completes,
    /// at which point `beginAsync` returns. If the async process completes by
    /// throwing an error before suspending itself, `beginAsync` rethrows the error.
    func beginAsync(_ body: () async throws -> Void) rethrows -> Void

    /// Suspends the current asynchronous task and invokes `body` with the task's
    /// continuation closure. Invoking `continuation` will resume the coroutine
    /// by having `suspendAsync` return the value passed into the continuation.
    /// It is a fatal error for `continuation` to be invoked more than once.
    func suspendAsync<T>(
    _ body: (_ continuation: @escaping (T) -> ()) -> ()
    ) async -> T

    /// Suspends the current asynchronous task and invokes `body` with the task's
    /// continuation and failure closures. Invoking `continuation` will resume the
    /// coroutine by having `suspendAsync` return the value passed into the
    /// continuation. Invoking `error` will resume the coroutine by having
    /// `suspendAsync` throw the error passed into it. Only one of
    /// `continuation` and `error` may be called; it is a fatal error if both are
    /// called, or if either is called more than once.
    func suspendAsync<T>(
    _ body: (_ continuation: @escaping (T) -> (),
    _ error: @escaping (Error) -> ()) -> ()
    ) async throws -> T
    ```

    These are similar to the "shift" and "reset" primitives of [delimited continuations](https://en.wikipedia.org/wiki/Delimited_continuation). These primitives enable callback-based APIs to be wrapped up as async coroutine APIs:

    ```swift
    class ThrowingFuture<T> {
    func get() async throws -> T {...}
    // Legacy callback-based API
    func getStuff(completion: (Stuff) -> Void) { ... }

    // Swift wrapper
    func getStuff() async -> Stuff {
    return await suspendAsync { continuation in
    getStuff(completion: continuation)
    }
    }
    class Future<T> : ThrowingFuture<T> {
    override func get() async -> T {...}
    ```

    Functionality of concurrency libraries such as libdispatch and pthreads can
    also be presented in coroutine-friendly ways:

    ```swift
    extension DispatchQueue {
    /// Move execution of the current coroutine synchronously onto this queue.
    func syncCoroutine() async -> Void {
    await suspendAsync { continuation in
    sync { continuation }
    }
    }

    /// Enqueue execution of the remainder of the current coroutine
    /// asynchronously onto this queue.
    func asyncCoroutine() async -> Void {
    await suspendAsync { continuation in
    async { continuation }
    }
    }
    }

    func future<T>(fn : () async -> T) -> Future<T> { ... }
    func future<T>(fn : () async throws -> T) -> ThrowingFuture<T> { ... }
    func queueHopping() async -> Void {
    doSomeStuff()
    await DispatchQueue.main.syncCoroutine()
    doSomeStuffOnMainThread()
    await backgroundQueue.asyncCoroutine()
    doSomeStuffInBackground()
    }
    ```

    There are two types to represent a value that will be computed in the future: `Future<T>` for non-throwing asynchronous functions and `ThrowingFuture<T>` for throwing ones. `Future` derives from `ThrowingFuture` to reflect the subtype relationship between throwing functions and non-throwing ones.
    Generalized abstractions for coordinating coroutines can also be built. The simplest of these is a [future](https://en.wikipedia.org/wiki/Futures_and_promises), a value that represents a future value which may not be resolved yet. The exact design for a future type deserves its own proposal (and would be influenced by the availability of planned Swift features like ownership and move-only types), but a proof of concept could look like this:

    ```swift
    class Future<T> {
    private enum Result { case error(Error), value(T) }
    private var result: Result? = nil
    private var awaiters: [(Result) -> Void] = []

    // Fulfill the future, and resume any coroutines waiting for the value.
    func fulfill(_ value: T) {
    precondition(self.result == nil, "can only be fulfilled once")
    let result = .value(value)
    self.result = result
    for awaiter in awaiters {
    awaiter(result)
    }
    awaiters = []
    }

    `Future` and `ThrowingFuture` are classes because they are non-copyable. This requires that they have reference (or move-only) semantics. In many trivial cases, instances of these classes could be completely optimized away, assuming the definition of these types are `fragile` (in the resilience model sense).
    // Mark the future as having failed to produce a result.
    func fail(_ error: Error) {
    precondition(self.result == nil, "can only be fulfilled once")
    let result = .error(error)
    self.result = result
    for awaiter in awaiters {
    awaiter(result)
    }
    awaiters = []
    }

    The definition of these types is intentionally kept as narrow as possible (while still being functional) in this proposal. We expect subsequent proposals to expand the API surface area.
    func get() async throws -> T {
    switch result {
    // Throw/return the result immediately if available.
    case .error(let e)?:
    throw e
    case .value(let v)?:
    return v
    // Wait for the future if no result has been fulfilled.
    case nil:
    return await suspendAsync { continuation, error in
    awaiters.append({
    switch $0 {
    case .error(let e): error(e)
    case .value(let v): continuation(v)
    }
    })
    }
    }
    }

    #### Example Using Futures
    // Create an unfulfilled future.
    init() {}

    // Begin a coroutine by invoking `body`, and create a future representing
    // the eventual result of `body`'s completion.
    convenience init(_ body: () async -> T) {
    self.init()
    beginAsync {
    do {
    self.fulfill(await body())
    } catch {
    self.fail(error)
    }
    }
    }
    }
    ```

    To allow parallel execution, move `await` from the call to the result when it is needed, and wrap the asynchronous calls in a "future", which can itself be `await`d on later.
    Futures allow parallel execution, by moving `await` from the call to the result when it is needed, and wrapping the parallel calls in individual `Future` objects:

    ```swift
    func makeSandwich1a() async -> Sandwich {
    let bread = future { await cutBread() }
    let cheese = future { await cutCheese() }
    let ham = future { await cutHam() }
    let tomato = future { await cutTomato() }
    let bread = Future { await cutBread() }
    let cheese = Future { await cutCheese() }
    let ham = Future { await cutHam() }
    let tomato = Future { await cutTomato() }
    ... stuff ...
    return Sandwich([await bread.get(), await cheese.get(), await ham.get(), await tomato.get())
    }
    ```

    In the above example all four operations will start one after another, and the unevaluated computation is wrapped into a `Future` value. This allows all of them to happen concurrently (in a way that need not be defined by the language), and the function will wait for completion of every one of them before returning a `Sandwich`. Note that `await` does not block flow of execution: if the value is not yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.
    In the above example all four operations will start one after another, and the unevaluated computation is wrapped into a `Future` value. This allows all of them to happen concurrently (in a way that need not be defined by the language or
    by the `Future` implementation), and the function will wait for completion of every one of them before returning a `Sandwich`. Note that `await` does not block flow of execution: if the value is not yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.

    Other coordination abstractions such as [Communicating Sequential Process channels](https://en.wikipedia.org/wiki/Communicating_sequential_processes) or [Concurrent ML events](https://wingolog.org/archives/2017/06/29/a-new-concurrent-ml) can also be developed as libraries for coordinating coroutines; their implementation is left as an exercise for the reader.

    ## Conversion of Objective-C APIs
    ## Conversion of imported Objective-C APIs

    Full details are beyond the scope of this proposal, but it is important to enhance the importer to project Objective-C completion-handler based APIs into `async` forms. This is a transformation comparable to how `NSError**` functions are imported as `throws` functions. Having the importer do this means that many Cocoa APIs will be modernized en-mass.
    Full details are beyond the scope of this proposal, but it is important to enhance the importer to project Objective-C completion-handler based APIs into `async` forms. This is a transformation comparable to how `NSError**` functions are imported as `throws` functions. Having the importer do this means that many Cocoa APIs will be modernized en masse.

    There are multiple possible designs for this with different tradeoffs. The maximally source compatible way to do this is to import completion handler-based APIs in two forms: both the completion handler and the `async` form. For example, given:

    @@ -287,7 +414,7 @@ func makeSandwich() async throws -> (half1: Sandwich, half2: Sandwich)
    func makeSandwich() async throws
    ```

    There are many details that should be defined as part of this importing process - for example, what are the exact rules for the transformation? Are multiple result functions common enough to handle automatically? How do overrides work when there are two definitions for the same method? Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration?
    There are many details that should be defined as part of this importing process - for example, what are the exact rules for the transformation? Are multiple result functions common enough to handle automatically? Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration? Without substantial ObjC importer work, making a clean break and forcing migration in Swift 5 mode would be the most practical way to preserve overridability, but would create a lot of churn in 4-to-5 migration. Alternatively, it may be acceptable to present the `async` versions as `final` wrappers over the underlying callback-based interfaces; this would subclassers to work with the callback-based interface, but there are generally fewer subclassers than callers.

    ## Interaction with existing features

    @@ -322,27 +449,10 @@ func makeSandwich2() async throws -> Sandwich {
    }
    ```

    Internally, compiler rewrites the code using closures and exits early if any intermediate call throws an error. Every `try await` will serialize execution, so `cutCheese` will only begin after `cutBread` completes.
    Coroutines address one of the major shortcomings of the Swift 2 error model,
    that it did not interoperate well with callback-oriented asynchronous APIs and
    required clumsy boilerplate to propagate errors across callback boundaries.

    To enable `throw`ing `async` calls execute in parallel, `try await` must be used after all necessary operations has begun.

    ```swift
    func makeSandwich2a() async throws -> Sandwich {
    let breadFuture = future { try await cutBread() }
    let cheeseFuture = future { try await cutCheese() }
    let hamFuture = future { try await cutHam() }
    let tomatoFuture = future { try await cutTomato() }

    let bread = try await breadFuture.get()
    let cheese = try await cheeseFuture.get()
    let ham = try await hamFuture.get()
    let tomato = try await tomatoFuture.get()

    return Sandwich([bread, cheese, ham, tomato])
    }
    ```

    In this example we begin all operations at once and then first wait for completion of `cutBread` and exit before checking the results of other operations. Note that this example throws at the point of the first `try await` whose asynchronous operation threw an error, but does not fail with the earliest operation.

    #### Closure type inference

    @@ -352,123 +462,50 @@ Because the `await` keyword is used at all points where execution may be suspend
    let myClosure = { () async -> () in ... }
    ```

    #### `defer`
    #### `defer` and abandonment

    There is no change to the semantics of `defer`, it continues to be bound by the programmer visible syntactic scope. The deferred statement is executed when the the current scope is exited. For example, when asynchronous operations complete and a return is hit, when an error is thrown, etc.
    Coroutines can be suspended, and while suspended, there is the potential for a coroutine's execution to be **abandoned** if all references to its continuation
    closure(s) are released without being executed:

    ```swift
    func makeSandwich() async throws -> Sandwich {
    startSpinner()
    defer stopSpinner() // will be called when error is thrown or when all operations complete and a result is returned.
    let bread = try await cutBread()
    let cheese = try await cutCheese()
    let ham = try await cutHam()
    let tomato = try await cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    }
    ```

    #### Completion handlers with multiple return values

    Completion handler APIs may have multiple result arguments (not counting an error argument). These are naturally represented by tuple results in `async` functions:

    ```swift
    // Before
    func makeClubSandwich(completionHandler: (part1: Sandwich?, part2: Sandwich?, error: Error?) -> Void)

    // After
    func makeSandwich() async throws -> (Sandwich, Sandwich)
    ```

    ### Sync and async functions interoperability

    We have already seen how asynchronous functions call synchronous and asynchronous ones, but we haven't explored how synchronous ones (e.g. event handlers) call `async` ones and how `async` ones can wrap old-fashioned functions still written with completion handlers.

    #### Calling an async function from a non-async function

    Because `async/await` fundamentally transforms the control flow within a function, you're only generally allowed to call an `async` function from within another `async` function. This works well until you get all the way to the top of the stack, e.g. an event handler (like an `IBAction`). Those are `Void`-returning functions which are not themselves `async`, and it is important to be able to perform asynchronous work from within them. For example, if you were using a completion handler, your code might look like this:

    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    makeSandwich(completionHandler: {(sandwich) in
    // 2
    imageView.image = sandwich.preview
    })
    // 3
    }
    ```

    This is an essential pattern, but is itself sort of odd: an `async` operation is being fired off immediately (#1), then runs the subsequent code (#3), and the completion handler (#2) runs at some time later -- on some queue (often the main one). This pattern frequently leads to mutation of global state (as in this example) or to making assumptions about which queue the completion handler is run on. Despite these problems, it is essential that the model encompasses this pattern, because it is a practical necessity in Cocoa development.

    To address these use-cases, there could be a function that takes an async closure, starts it, but ignores the result. Technically speaking, the design outlined above is sufficient, just use the existing `Future` type for this:

    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    _ = future {
    // 2
    let sandwich = await makeSandwich()
    imageView.image = sandwich.preview
    }
    // 3
    /// Shut down the current coroutine and give its memory back to the
    /// shareholders.
    func abandon() async -> Never {
    await suspendAsync { _ = $0 }
    }
    ```

    However, it may make sense to add something more explicit, such as a new standard library function:
    It is to be expected that, upon abandonment, any references captured in wait by the continuation should be released, as with any closure. However, there may be other cleanup that must be guaranteed to occur. `defer` serves the general
    role of "guaranteed cleanup" in synchronous code, and it would be a natural
    extension to add the guarantee that `defer`-ed statements also execute as part
    of cleaning up an abandoned coroutine:

    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    detachedFuture {
    // 2
    let sandwich = await makeSandwich()
    imageView.image = sandwich.preview
    func makeSandwich() async throws -> Sandwich {
    startSpinner()
    defer { stopSpinner() } // will be called when error is thrown, when all
    // operations complete and a result is returned,
    // or when the coroutine is abandoned. We don't
    // want to leave the cursor spinning if work has
    // stopped.
    let bread = try await cutBread()
    let cheese = try await cutCheese()
    do {
    let ham = try await cutHam()
    } catch _ as Trichinosis {
    // Better shut the shop down until the health inspector arrives
    await abandon()
    }
    // 3
    let tomato = try await cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    }
    ```

    The exact naming of this operation is sure to require extensive bikeshedding. For the purpose of this proposal, we simply observe that this is easy to build the standard library if desirable.


    #### Wrapping async function in a block-based function

    With something like `detachedFuture`, it is easy to wrap a result of asynchronous call in a classic completion handler API.

    ```swift
    func makeSandwichOldWay(completionHandler: (Sandwich) -> Void) {
    detachedFuture {
    let sandwich = await makeSandwich()
    completionHandler(sandwich)
    }
    }
    This fills in another gap in the expressivity of callback-based APIs, where it is difficult to express cleanup code that must execute at some point regardless of whether the callback closure is really called. However, abandonment should not be taken as a fully-baked "cancellation" feature; if cancellation is important, it should continue to be implemented by the programmer where needed, and there are many standard patterns that can be applied. Particularly when coupled with error handling, common cancellation patterns become very elegant:

    func makeSandwichWithErrorOldWay(completionHandler: (Sandwich?, Error?) -> Void) {
    detachedFuture {
    do {
    let sandwich = try await makeSandwichWithError()
    completionHandler(sandwich, nil)
    } catch {
    completionHandler(nil, error)
    }
    }
    }
    ```

    #### Wrapping completion handler functions in `async` function

    Most Objective-C APIs should be automatically imported as `async` by the compiler (described below), however some APIs may not be automatically importable. If so, it would make sense to add an API (possibly in the GCD layer) to enable this. This is a framework level change though, so it is beyond the scope of this proposal.

    ### Cancellation Example

    As in Swift 4, this proposal includes no built-in language support for scheduling tasks or cancellation: once an asyncronous operation is started, it cannot be implicitly stopped, even if (e.g.) the last reference to a future containing the computation is dropped.

    If cancellation is important, it should continue to be implemented by the programmer where needed, and there are many standard patterns that can be applied. However, particularly coupled with improved error handling, common cancellation patterns become very elegant:

    ```swift
    @IBAction func makeSandwich(sender: AnyObject) {
    detachedFuture {
    beginAsync {
    do {
    let bread = try await kitchen.cutBread()
    let cheese = try await kitchen.cutCheese()
    @@ -489,8 +526,19 @@ If cancellation is important, it should continue to be implemented by the progra
    }
    ```

    Internally, `kitchen` may use `NSOperations` or custom `cancelled` flag. The intent of this section is to give a single example of how to approach this, not to define a normative or all-encompassing approach that should be used in all cases.
    Internally, `kitchen` may use `NSOperation` or a custom `cancelled` flag. The intent of this section is to give a single example of how to approach this, not to define a normative or all-encompassing approach that should be used in all cases.

    #### Completion handlers with multiple return values

    Completion handler APIs may have multiple result arguments (not counting an error argument). These are naturally represented by tuple results in `async` functions:

    ```swift
    // Before
    func makeClubSandwich(completionHandler: (part1: Sandwich?, part2: Sandwich?, error: Error?) -> Void)

    // After
    func makeSandwich() async throws -> (Sandwich, Sandwich)
    ```

    ## Source Compatibility

    @@ -545,18 +593,19 @@ func makeSandwich1() yields -> Sandwich {

    #### Make `async` be a subtype of `throws` instead of orthogonal to it

    It would be a great simplification of the language model to make the `async` modifier on a function imply that the function is `throw`ing, instead of making them orthogonal modifiers. From an intuitive perspective, this makes sense because many of the sorts of operations that are asynchronous (e.g. loading a resource, talking to the network, etc) can also fail. There is also precedent from many other systems that use `async`/`await` for this. Instead of four kinds of function type, we'd only have three:
    It would be a great simplification of the language model to make the `async` modifier on a function imply that the function is `throw`ing, instead of making them orthogonal modifiers. From an intuitive perspective, this makes sense because many of the sorts of operations that are asynchronous (e.g. loading a resource, talking to the network, etc) can also fail. There is also precedent from many other systems that use `async`/`await` for this; for example, .NET `Task`s and Javascript promises both combine error handling with async sequencing. One could argue that that's because .NET and Javascript's established runtimes both feature pervasive implicit exceptions; however, popular async frameworks for the Rust programming language, such as [tokio.rs](https://tokio.rs), have also chosen to incorporate error handling directly into their `Future` constructs, because doing so was found to be more practical and ergonomic than trying to compose theoretically-orthogonal `Future<T>` and `Result<T>` constructs.

    If we made `async` a subtype of `throws`, then instead of four kinds of function type, we'd only have three:

    ```swift
    (Int) -> Int // Normal function
    (Int) throws -> Int // Throwing function
    (Int) async -> Int // Asynchronous function, can also throw
    ```

    The `try` marker could also be dropped from `try await`, because all `await`s would be known to throw. For user code, you would never need the ugly `async throws` modifier stack.

    Going this direction would simplify a lot of things: `ThrowingFuture` would go away, as would the overload of `future`. Arguably the `try` marker could be dropped from `try await`, because all `await`s would be known to throw. For user code, it would directly reduce the number of times you'd see the ugly `async throws` modifier stack.

    There are two downsides to doing this: the first of which is that Cocoa has a number of completion handler APIs that cannot throw, and not having the ability to express that would make the importer story more complex. The second is that uses of these APIs would now cause pointless `do`/`catch` blocks which would be dead.
    A downside to doing this is that Cocoa in practice does have a number of completion handler APIs that do not take error arguments, and not having the ability to express that would make the importer potentially lose type information. Many of these APIs express failure in more limited ways, such as passing `nil` into the completion closure, passing in a `BOOL` to indicate success, or communicating status via side properties of the coordinating object; auditing for and recognizing all of these idioms would complicate the importer and slow the SDK modernization process. Even then, Swift subclassers overriding the `async` forms of these APIs would be allowed by the language to throw errors even though the error cannot really be communicated across the underlying Objective-C interface.

    #### Make `async` default to `throws`

    @@ -574,24 +623,25 @@ This model provides a ton of advantages: it is arguably the right defaults for t

    ## Alternatives Considered

    #### Remove `future` from this proposal, and add it as a follow-on
    #### Include `Future` or other coordination abstractions in this proposal

    This proposal includes a intentionally minimal `future` design to show an important use-case. However, there are many rational designs for futures, and a lot of experience working with them. If the discussion and design of futures ends up slowing down the proposal, it would make sense to split it out to a completely separate proposal: after all, it is a simple additive feature on top of the basic language support being added.
    This proposal does not formally propose a `Future` type, or any other coordination abstractions. There are many rational designs for futures, and a lot of experience working with them. On the other hand, there are also completely different coordination primitives that can be used with this coroutine design, and blessing `Future` out of the gate with standard library support may suppress valuable experimentation in this space. Furthermore, the shape and functionality of a future may also be affected by Swift's planned evolution; whereas a Future type designed for Swift today would most likely need to be a `class`, and therefore need to guard against potentially multithreaded access, races to fulfill or attempts to fulfill multiple times, and potentially unbounded queueing of awaiting coroutines on the shared future. Ownership and move-only types would allow us to express futures as a more efficient move-only type requiring exclusive ownership to be forwarded from the fulfilling task to the receiving task, avoiding the threading and queueing problems of a class-based approach, as seen in Rust's [tokio.rs](https://tokio.rs) framework.

    #### Change async calls to always return a `Future`
    #### Have async calls always return a `Future`

    The most commonly cited alternative design is to follow the model of (e.g.) C#, where calls to
    async functions return a future (aka `Task` in C#), instead of futures being a library feature separable from the
    core language. Going this direction adds async/await to the language instead of adding a
    more general coroutine feature.

    Despite this model being widely know, we believe that the proposed design
    is superior for a number of reasons:
    The most commonly cited alternative design is to follow the model of (e.g.) C#, where calls to async functions return a future (aka `Task` in C#), instead of futures being a library feature separable from the core language. Going this direction adds async/await to the language instead of adding a more general coroutine feature.

    Despite this model being widely know, we believe that the proposed design is
    superior for a number of reasons:

    - Coroutines (and the coroutine transformation that inverts control flow) are generally useful language features beyond the domain of async/await. For example, building async/await into the compiler would require building generators in as well.
    - Coroutines are generally useful language features beyond the domain of async/await. For example, building async/await into the compiler would require building generators in as well.
    - The proposed design eliminates the problem of calling an API (without knowing it is async) and getting a `Future<T>` back instead of the expected `T` result type. C# addresses this by suggesting that all `async` methods have their name be suffixed with `Async`, which is suboptimal.
    - By encoding `async` as a first-class part of function types, closure literals can also be transparently `async` by contextual type inference. In the future, mechanisms like `rethrows` can be extended to allow polymorphism over asynchrony for higher-level operations like `map` to work as expected without creating intermediate collections of `Future<T>`, although this proposal does not propose any such abstraction mechanisms in the short term.
    - The C# model for await is a unary prefix keyword, which does not compose well in the face of chaining. Wherein C# you may have to write something like `x = await (await foo()).bar()`, with the proposed design you can simply write `x = await foo().bar()` for the same reasons that you don't have to write `try` on every single call in a chain that can throw.
    - There are multiple different interesting abstractions (like future) to consider. By putting the details of them in the standard library, other people can define and use their own abstractions where it makes sense.
    - It is useful to be able to design and discuss futures as an independent standard library feature without tying the entire success or failure of coroutines as a language proposal to `Future`'s existence.
    - There are multiple different interesting abstractions besides futures to consider. By putting the details of them in the standard library, other people can define and use their own abstractions where it makes sense.
    - Requiring a future object to be instantiated at every `await` point adds overhead. Since a major use case for this feature is to adapt existing Cocoa APIs, which already use callbacks, queues, target-action, or other mechanisms to coordinate the scheduling of the continuation of an async task, introducing a future into the mix would be an additional unnecessary middleman incurring overhead when wrapping these APIs, when in most cases there is already a direct consumer for the continuation point.
    - A design that directly surfaces a monadic type like `Future` as the result of an async computation heavily implies a compiler-driven coroutine transform, whereas this design is more implementation-agnostic. Compiler-transformed coroutines are a great compromise for integrating lightweight tasks into an existing runtime model that's already heavily callstack-dependent, or one aims to maintain efficient interop with C or other languages that heavily constrain the implementation model, and Swift definitely has both. It is conceivable that, in the eventual future, a platform such as Swift-on-the-server could provide a pure- or predominantly-Swift ABI where enough code is pure Swift to make cheap relocatable stacks the norm and overhead on C interop acceptable, as has happened with the Go runtime. This could make `async` a no-op at runtime, and perhaps allow us to consider eliminating the annotation altogether. The semantic presence of a future object between every layer of an async process would be an obstacle to the long-term efficiency of such a platform.

    The primary argument for adding async/await (and then generators) to the language as first-class language features is that they are the vastly most common use-case of coroutines. In the authors opinion, the design as proposed gives something that works better than the C# model in practice, while also providing a more useful/general language model.

    @@ -602,7 +652,15 @@ This proposal has been kept intentionally minimal, but there are many possible w

    #### New Foundation, GCD, and Server APIs

    Given the availability of convenient asynchrony in Swift, it would make sense to introduce new APIs to take advantage of it. Filesystem APIs are one example that would be great to see. The Swift on Server working group would also widely adopt these features.
    Given the availability of convenient asynchrony in Swift, it would make sense to introduce new APIs to take advantage of it. Filesystem APIs are one example that would be great to see. The Swift on Server working group would also widely adopt these features. GCD could also provide new helpers for allowing `() async -> Void` coroutines to be enqueued, or for allowing a running coroutine to move its execution onto a different queue.

    #### `rethrows` could be generalized to support potentially `async` operations

    The `rethrows` modifier exists in Swift to allow limited abstraction over function types by higher order functions. It would be possible to define a similar mechanism to allow abstraction over `async` operations as well. More generally, by modeling both `throws` and `async` as effects on function types, we can eventually provide common abstraction tools to abstract over both effects in protocols and generic code, simultaneously addressing the "can't have a Sequence that `throws`" and "can't have a Sequence that's `async`" kinds of limitations in the language today.

    #### Blocking calls

    Affordances could be added to better call blocking APIs from `async` functions and to hard wait for an `async` function to complete. There are significant tradeoffs and wide open design space to explore here, and none of it is necessary for the base proposal.

    #### Fix queue-hopping Objective-C completion handlers

    @@ -649,33 +707,6 @@ is being run on a different queue than the function was invoked on, and if so, e
    completion handler on the original queue.


    #### `rethrows` could be generalized to support potentially `async` operations

    The `rethrows` modifier exists in Swift to allow limited abstraction over function types by higher order functions. It would be possible to define a similar mechanism to allow abstraction over `async` operations as well.

    #### Array form of `await` and other operations

    There are many interesting APIs that could be added to support different sort of patterns,
    e.g. "wait on a list of pending operations, continuing when the first is available".

    Standard `map`, `flatMap` and `reduce` methods could be provided on `Future` and `ThrowingFuture`. The runtime could implement parallel execution of all mapping operations on futures, allowing you to write something like this:

    ```swift
    var urls : [URL]
    // ...
    urls.map(fetch).map(parse).map(analyze).reduce([]){ $0.append($1) }
    ```

    In the example above, all invocations of `fetch` will happen at once, then `parse`, `analyze` and `append` will happen in order of completion of every operation. Therefore, the data will be processed as soon as it is available, but the order of the resulting array is not guaranteed.

    These have not been included in the basic proposal because they are not necessary, require runtime support, and should arguably be part of the GCD API.

    #### Blocking calls

    Affordances could be added to better call blocking APIs from `async` functions and to hard wait for an `async` function to complete. There are significant tradeoffs and wide open design space to explore here, and none of it is necessary for the base proposal.

    ## Thanks

    @oleganza thanked @groue and @pierlo for their feedback on his original proposal. Thanks to @oleganza for the original draft which influenced this!

    Thanks to [Joe Groff](https://github.com/jckarter) for his extensive feedback which helped reshape this proposal into something that is a lot more general than it started as.
  13. @lattner lattner revised this gist Aug 10, 2017. 1 changed file with 137 additions and 94 deletions.
    231 changes: 137 additions & 94 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -207,11 +207,11 @@ func cutHam() async -> Ham
    func cutTomato() async -> Vegetable

    func makeSandwich1() async -> Sandwich {
    let bread = await cutBread()
    let cheese = await cutCheese()
    let ham = await cutHam()
    let tomato = await cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    let bread = await cutBread()
    let cheese = await cutCheese()
    let ham = await cutHam()
    let tomato = await cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    }
    ```

    @@ -250,12 +250,12 @@ To allow parallel execution, move `await` from the call to the result when it is

    ```swift
    func makeSandwich1a() async -> Sandwich {
    let bread = future { await cutBread() }
    let cheese = future { await cutCheese() }
    let ham = future { await cutHam() }
    let tomato = future { await cutTomato() }
    ... stuff ...
    return Sandwich([await bread.get(), await cheese.get(), await ham.get(), await tomato.get())
    let bread = future { await cutBread() }
    let cheese = future { await cutCheese() }
    let ham = future { await cutHam() }
    let tomato = future { await cutTomato() }
    ... stuff ...
    return Sandwich([await bread.get(), await cheese.get(), await ham.get(), await tomato.get())
    }
    ```

    @@ -289,8 +289,6 @@ func makeSandwich() async throws

    There are many details that should be defined as part of this importing process - for example, what are the exact rules for the transformation? Are multiple result functions common enough to handle automatically? How do overrides work when there are two definitions for the same method? Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration?



    ## Interaction with existing features

    This proposal dovetails naturally with existing language features in Swift, here are a few examples:
    @@ -316,11 +314,11 @@ func cutHam() async throws -> Ham
    func cutTomato() async throws -> Vegetable

    func makeSandwich2() async throws -> Sandwich {
    let bread = try await cutBread()
    let cheese = try await cutCheese()
    let ham = try await cutHam()
    let tomato = try await cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    let bread = try await cutBread()
    let cheese = try await cutCheese()
    let ham = try await cutHam()
    let tomato = try await cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    }
    ```

    @@ -330,17 +328,17 @@ To enable `throw`ing `async` calls execute in parallel, `try await` must be used

    ```swift
    func makeSandwich2a() async throws -> Sandwich {
    let breadFuture = future { try await cutBread() }
    let cheeseFuture = future { try await cutCheese() }
    let hamFuture = future { try await cutHam() }
    let tomatoFuture = future { try await cutTomato() }
    let bread = try await breadFuture.get()
    let cheese = try await cheeseFuture.get()
    let ham = try await hamFuture.get()
    let tomato = try await tomatoFuture.get()
    return Sandwich([bread, cheese, ham, tomato])
    let breadFuture = future { try await cutBread() }
    let cheeseFuture = future { try await cutCheese() }
    let hamFuture = future { try await cutHam() }
    let tomatoFuture = future { try await cutTomato() }

    let bread = try await breadFuture.get()
    let cheese = try await cheeseFuture.get()
    let ham = try await hamFuture.get()
    let tomato = try await tomatoFuture.get()

    return Sandwich([bread, cheese, ham, tomato])
    }
    ```

    @@ -360,13 +358,13 @@ There is no change to the semantics of `defer`, it continues to be bound by the

    ```swift
    func makeSandwich() async throws -> Sandwich {
    startSpinner()
    defer stopSpinner() // will be called when error is thrown or when all operations complete and a result is returned.
    let bread = try await cutBread()
    let cheese = try await cutCheese()
    let ham = try await cutHam()
    let tomato = try await cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    startSpinner()
    defer stopSpinner() // will be called when error is thrown or when all operations complete and a result is returned.
    let bread = try await cutBread()
    let cheese = try await cutCheese()
    let ham = try await cutHam()
    let tomato = try await cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    }
    ```

    @@ -392,12 +390,12 @@ Because `async/await` fundamentally transforms the control flow within a functio

    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    makeSandwich(completionHandler: {(sandwich) in
    // 2
    imageView.image = sandwich.preview
    })
    // 3
    // 1
    makeSandwich(completionHandler: {(sandwich) in
    // 2
    imageView.image = sandwich.preview
    })
    // 3
    }
    ```

    @@ -407,27 +405,27 @@ To address these use-cases, there could be a function that takes an async closur

    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    _ = future {
    // 2
    let sandwich = await makeSandwich()
    imageView.image = sandwich.preview
    }
    // 3
    // 1
    _ = future {
    // 2
    let sandwich = await makeSandwich()
    imageView.image = sandwich.preview
    }
    // 3
    }
    ```

    However, it may make sense to add something more explicit, such as a new standard library function:

    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    detachedFuture {
    // 2
    let sandwich = await makeSandwich()
    imageView.image = sandwich.preview
    }
    // 3
    // 1
    detachedFuture {
    // 2
    let sandwich = await makeSandwich()
    imageView.image = sandwich.preview
    }
    // 3
    }
    ```

    @@ -440,21 +438,21 @@ With something like `detachedFuture`, it is easy to wrap a result of asynchronou

    ```swift
    func makeSandwichOldWay(completionHandler: (Sandwich) -> Void) {
    detachedFuture {
    let sandwich = await makeSandwich()
    completionHandler(sandwich)
    }
    detachedFuture {
    let sandwich = await makeSandwich()
    completionHandler(sandwich)
    }
    }

    func makeSandwichWithErrorOldWay(completionHandler: (Sandwich?, Error?) -> Void) {
    detachedFuture {
    do {
    let sandwich = try await makeSandwichWithError()
    completionHandler(sandwich, nil)
    } catch {
    completionHandler(nil, error)
    }
    detachedFuture {
    do {
    let sandwich = try await makeSandwichWithError()
    completionHandler(sandwich, nil)
    } catch {
    completionHandler(nil, error)
    }
    }
    }
    ```

    @@ -470,24 +468,24 @@ If cancellation is important, it should continue to be implemented by the progra

    ```swift
    @IBAction func makeSandwich(sender: AnyObject) {
    detachedFuture {
    do {
    let bread = try await kitchen.cutBread()
    let cheese = try await kitchen.cutCheese()
    let ham = try await kitchen.cutHam()
    let tomato = try await kitchen.cutTomato()
    eat(Sandwich(bread, cheese, ham, tomato]))
    } catch CocoaError.userCancelled {
    // Ignore, user quit the kitchen.
    } catch {
    // Some really interesting error happened
    presentError(error)
    }
    detachedFuture {
    do {
    let bread = try await kitchen.cutBread()
    let cheese = try await kitchen.cutCheese()
    let ham = try await kitchen.cutHam()
    let tomato = try await kitchen.cutTomato()
    eat(Sandwich(bread, cheese, ham, tomato]))
    } catch CocoaError.userCancelled {
    // Ignore, user quit the kitchen.
    } catch {
    // Some really interesting error happened
    presentError(error)
    }
    }
    }

    @IBAction func exitKitchen(sender: AnyObject) {
    kitchen.cancel()
    kitchen.cancel()
    }
    ```

    @@ -499,18 +497,18 @@ Internally, `kitchen` may use `NSOperations` or custom `cancelled` flag. The in
    This is a generally additive feature, but it does turn take `async` and `await` as keywords, so it will break code that uses them as identifiers. This is expected to have very minor impact: the most pervasive use of `async` as an identifier occurs in code that works with dispatch queues, but fortunately keywords are allowed as qualified member names, so code like this doesn't need any change:

    ```swift
    myQueue.async { ... }
    myQueue.async { ... }
    ```

    That said, there could be obscure cases that break. One example that occurs in the Swift testsuite is of the form:

    ```swift
    extension DispatchQueue {
    func myThing() {
    async {
    ...
    }
    func myThing() {
    async {
    ...
    }
    }
    }
    ```

    @@ -537,11 +535,11 @@ func cutHam() yields -> Ham
    func cutTomato() yields -> Vegetable

    func makeSandwich1() yields -> Sandwich {
    let bread = yield cutBread()
    let cheese = yield cutCheese()
    let ham = yield cutHam()
    let tomato = yield cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    let bread = yield cutBread()
    let cheese = yield cutCheese()
    let ham = yield cutHam()
    let tomato = yield cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    }
    ```

    @@ -606,6 +604,51 @@ This proposal has been kept intentionally minimal, but there are many possible w

    Given the availability of convenient asynchrony in Swift, it would make sense to introduce new APIs to take advantage of it. Filesystem APIs are one example that would be great to see. The Swift on Server working group would also widely adopt these features.

    #### Fix queue-hopping Objective-C completion handlers

    One unfortunate reality of the existing Cocoa stack is that many asynchronous methods are
    unclear about which queue they run the completion handler on. In fact, one of the top hits
    for implementing completion handlers on Stack Overflow includes this Objective-C code:

    ```objective-c
    - (void)asynchronousTaskWithCompletion:(void (^)(void))completion;
    {
    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{

    // Some long running task you want on another thread

    dispatch_async(dispatch_get_main_queue(), ^{
    if (completion) {
    completion();
    }
    });
    });
    }
    ```

    Note that it runs the completion handler on the main queue, not on the queue which it was
    invoked on. This disparity causes numerous problems for Cocoa programmers, who would
    probably defensively write the `@IBAction` above like this (or else face a possible race
    condition):

    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    detachedFuture {
    let sandwich = await makeSandwich()
    // Do the update on the main thread/queue since it owns imageView.
    mainQ.async {
    imageView.image = sandwich.preview
    }
    }
    }
    ```

    This can be fixed in the Objective-C importer, which is going to be making thunks for the
    completion-handler functions anyway: the thunk could check to see if the completion handler
    is being run on a different queue than the function was invoked on, and if so, enqueue the
    completion handler on the original queue.


    #### `rethrows` could be generalized to support potentially `async` operations

    The `rethrows` modifier exists in Swift to allow limited abstraction over function types by higher order functions. It would be possible to define a similar mechanism to allow abstraction over `async` operations as well.
    @@ -618,9 +661,9 @@ e.g. "wait on a list of pending operations, continuing when the first is availab
    Standard `map`, `flatMap` and `reduce` methods could be provided on `Future` and `ThrowingFuture`. The runtime could implement parallel execution of all mapping operations on futures, allowing you to write something like this:

    ```swift
    var urls : [URL]
    // ...
    urls.map(fetch).map(parse).map(analyze).reduce([]){ $0.append($1) }
    var urls : [URL]
    // ...
    urls.map(fetch).map(parse).map(analyze).reduce([]){ $0.append($1) }
    ```

    In the example above, all invocations of `fetch` will happen at once, then `parse`, `analyze` and `append` will happen in order of completion of every operation. Therefore, the data will be processed as soon as it is available, but the order of the resulting array is not guaranteed.
  14. @lattner lattner revised this gist Aug 10, 2017. 1 changed file with 28 additions and 10 deletions.
    38 changes: 28 additions & 10 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -5,7 +5,7 @@

    ## Introduction

    Modern Cocoa development involves a lot of asynchronous programming using closures and completion handlers, but completion handlers APIs are hard to use when number of operations grows and dependencies between them become more complicated. This proposal describes a language extension to make this a lot more natural and less error prone.
    Modern Cocoa development involves a lot of asynchronous programming using closures and completion handlers, but these APIs are hard to use. This gets particularly problematic when many asynchronous operations are used, error handling is required, or control flow between asyncronous calls gets complicated. This proposal describes a language extension to make this a lot more natural and less error prone.

    This paper introduces a first class [Coroutine model](https://en.wikipedia.org/wiki/Coroutine) to Swift. Further, it provides standard library affordances to expose these as *asynchronous semantics* and *Futures* to Swift, which are concepts [available in many other languages now](https://en.wikipedia.org/wiki/Await) (including C#, Javascript, Python, and others). Functions can opted into to being *async*, allowing the programmer can compose complex logic involving asynchronous operations - while the compiler produces necessary closures and state machines to implement that logic.

    @@ -15,7 +15,7 @@ This is derived from an earlier proposal written by [Oleg Andreev](https://githu

    ## Motivation: Completion handlers are suboptimal

    To provide motivation for why it is important to do something here, lets look at a number of gross problems that Cocoa (and server/cloud) programmers frequently face.
    To provide motivation for why it is important to do something here, lets look at some of the problems that Cocoa (and server/cloud) programmers frequently face.

    #### Problem 1: Pyramid of doom

    @@ -401,17 +401,17 @@ Because `async/await` fundamentally transforms the control flow within a functio
    }
    ```

    This is an essential pattern, but is itself sort of odd: an `async` operation is being fired off immediately (#1), then runs the subsequent code (#3), and the completion handler (#2) runs at some time later -- on some queue (often the main one). This pattern frequently leads to mutation of global state (as in this example) or to making assumptions about which queue the completion handler is run on. While this may not be very safe, it is essential that the model encompasses this pattern, because it is a practical necessity in Cocoa development.
    This is an essential pattern, but is itself sort of odd: an `async` operation is being fired off immediately (#1), then runs the subsequent code (#3), and the completion handler (#2) runs at some time later -- on some queue (often the main one). This pattern frequently leads to mutation of global state (as in this example) or to making assumptions about which queue the completion handler is run on. Despite these problems, it is essential that the model encompasses this pattern, because it is a practical necessity in Cocoa development.

    To address this, there should be a function that takes an async closure, starts it, but ignores the result. The design outlined above is sufficient, just use the existing `Future` type for this:
    To address these use-cases, there could be a function that takes an async closure, starts it, but ignores the result. Technically speaking, the design outlined above is sufficient, just use the existing `Future` type for this:

    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    _ = future {
    // 2
    let sandwich = await makeSandwich()
    imageView.image = sandwich.preview
    // 2
    let sandwich = await makeSandwich()
    imageView.image = sandwich.preview
    }
    // 3
    }
    @@ -423,16 +423,17 @@ However, it may make sense to add something more explicit, such as a new standar
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    detachedFuture {
    // 2
    let sandwich = await makeSandwich()
    imageView.image = sandwich.preview
    // 2
    let sandwich = await makeSandwich()
    imageView.image = sandwich.preview
    }
    // 3
    }
    ```

    The exact naming of this operation is sure to require extensive bikeshedding. For the purpose of this proposal, we simply observe that this is easy to build the standard library if desirable.


    #### Wrapping async function in a block-based function

    With something like `detachedFuture`, it is easy to wrap a result of asynchronous call in a classic completion handler API.
    @@ -527,6 +528,23 @@ Here are a couple of syntax level changes to the proposal that are worth discuss

    Instead of spelling the function type modifier as `async`, it could be spelled as `yields`, since the functionality really is about coroutines, not about asynchrony by itself. The recommendation to use `async/await` biases towards making sure that the most common use case (asynchrony) uses industry standard terms. The other coroutine use cases would be much less common, at least according to the unscientific opinion of the proposal author.

    To give an idea of what this could look like, here's the example from above resyntaxed:

    ```swift
    func cutBread() yields -> Bread
    func cutCheese() yields -> Cheese
    func cutHam() yields -> Ham
    func cutTomato() yields -> Vegetable

    func makeSandwich1() yields -> Sandwich {
    let bread = yield cutBread()
    let cheese = yield cutCheese()
    let ham = yield cutHam()
    let tomato = yield cutTomato()
    return Sandwich([bread, cheese, ham, tomato])
    }
    ```

    #### Make `async` be a subtype of `throws` instead of orthogonal to it

    It would be a great simplification of the language model to make the `async` modifier on a function imply that the function is `throw`ing, instead of making them orthogonal modifiers. From an intuitive perspective, this makes sense because many of the sorts of operations that are asynchronous (e.g. loading a resource, talking to the network, etc) can also fail. There is also precedent from many other systems that use `async`/`await` for this. Instead of four kinds of function type, we'd only have three:
  15. @lattner lattner revised this gist Aug 10, 2017. 1 changed file with 7 additions and 7 deletions.
    14 changes: 7 additions & 7 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -562,19 +562,19 @@ This model provides a ton of advantages: it is arguably the right defaults for t

    This proposal includes a intentionally minimal `future` design to show an important use-case. However, there are many rational designs for futures, and a lot of experience working with them. If the discussion and design of futures ends up slowing down the proposal, it would make sense to split it out to a completely separate proposal: after all, it is a simple additive feature on top of the basic language support being added.

    #### Change async calls to always return a future
    #### Change async calls to always return a `Future`

    Most commonly cited alternative design is to follow the model of (e.g. C#), where calls to
    async functions return a future (a `Task` in C#), instead of futures being a library feature separable from the
    core language. This design effectively adds async/await to the language, it does not add a
    The most commonly cited alternative design is to follow the model of (e.g.) C#, where calls to
    async functions return a future (aka `Task` in C#), instead of futures being a library feature separable from the
    core language. Going this direction adds async/await to the language instead of adding a
    more general coroutine feature.

    Despite this model being widely know, we believe that the proposed design
    is superior for a number of reasons:

    - Coroutines (and the coroutine transformation that inverts control flow) are generally useful language features beyond the domain of async/await. Building async/await into the compiler would require building generators in as well.
    - The proposed design eliminates the problem of calling an API (without knowing it is async) and getting a future back instead of the expected result type. C# addresses this by suggesting that all `async` methods have their name be suffixed with `Async`, which is suboptimal.
    - The C# model for await is a unary prefix keyword, which does not compose well in the face of chaining. Wherein C# you may have to write something like `x = await (await foo()).bar()`, with this design you can simply write `x = await foo().bar()` for the same reasons that you don't have to write `try` on every single call in a chain that can throw.
    - Coroutines (and the coroutine transformation that inverts control flow) are generally useful language features beyond the domain of async/await. For example, building async/await into the compiler would require building generators in as well.
    - The proposed design eliminates the problem of calling an API (without knowing it is async) and getting a `Future<T>` back instead of the expected `T` result type. C# addresses this by suggesting that all `async` methods have their name be suffixed with `Async`, which is suboptimal.
    - The C# model for await is a unary prefix keyword, which does not compose well in the face of chaining. Wherein C# you may have to write something like `x = await (await foo()).bar()`, with the proposed design you can simply write `x = await foo().bar()` for the same reasons that you don't have to write `try` on every single call in a chain that can throw.
    - There are multiple different interesting abstractions (like future) to consider. By putting the details of them in the standard library, other people can define and use their own abstractions where it makes sense.

    The primary argument for adding async/await (and then generators) to the language as first-class language features is that they are the vastly most common use-case of coroutines. In the authors opinion, the design as proposed gives something that works better than the C# model in practice, while also providing a more useful/general language model.
  16. @lattner lattner revised this gist Aug 9, 2017. 1 changed file with 18 additions and 22 deletions.
    40 changes: 18 additions & 22 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -215,32 +215,12 @@ func makeSandwich1() async -> Sandwich {
    }
    ```

    Under the hood, the compiler rewrites this code using nested closures like in example `makeSandwich1` above. Note that every operation starts only after the previous one has completed.

    To allow parallel execution, move `await` from the call to the result when it is needed, and wrap the asynchronous calls in a "future", which can itself be `await`d on later.

    ```swift
    func makeSandwich1a() async -> Sandwich {
    let bread = future { await cutBread() }
    let cheese = future { await cutCheese() }
    let ham = future { await cutHam() }
    let tomato = future { await cutTomato() }
    return Sandwich([await bread.get(), await cheese.get(), await ham.get(), await tomato.get())
    }
    ```

    In the above example all four operations will start one after another, and the unevaluated computation is wrapped into a `Future` value through the following standard library function:

    ```swift
    func future<T>(fn : () async -> T) -> Future<T> { ... }
    ```

    ... the function will wait for completion of every one of them before returning a sandwich. Note that `await` does not block flow of execution: if the value is not yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.
    Under the hood, the compiler rewrites this code using nested closures like in example `makeSandwich1` above. Note that every operation starts only after the previous one has completed, but each call site to an `async` function could suspend execution of the current function.

    Finally, you are only allowed to invoke an `async` function from within another `async` function or closure. This follows the model of Swift 2 error handling, where you cannot call a throwing function unless you're in a throwing function or inside of a `do/catch` block.


    ### Future types
    ### Futures

    Because the coroutine transformation described in this design lives at the level of function types, it can be applied to many different applications that benefit from the inversion of control that the coroutine transformation provides (e.g. generators). That said, the most important first application is to provide `async`/`await` as a better way to work with asynchronous functions. Similar APIs could be used for other applications, but those are beyond the scope of this proposal.

    @@ -264,6 +244,22 @@ There are two types to represent a value that will be computed in the future: `F

    The definition of these types is intentionally kept as narrow as possible (while still being functional) in this proposal. We expect subsequent proposals to expand the API surface area.

    #### Example Using Futures

    To allow parallel execution, move `await` from the call to the result when it is needed, and wrap the asynchronous calls in a "future", which can itself be `await`d on later.

    ```swift
    func makeSandwich1a() async -> Sandwich {
    let bread = future { await cutBread() }
    let cheese = future { await cutCheese() }
    let ham = future { await cutHam() }
    let tomato = future { await cutTomato() }
    ... stuff ...
    return Sandwich([await bread.get(), await cheese.get(), await ham.get(), await tomato.get())
    }
    ```

    In the above example all four operations will start one after another, and the unevaluated computation is wrapped into a `Future` value. This allows all of them to happen concurrently (in a way that need not be defined by the language), and the function will wait for completion of every one of them before returning a `Sandwich`. Note that `await` does not block flow of execution: if the value is not yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.


    ## Conversion of Objective-C APIs
  17. @lattner lattner revised this gist Aug 9, 2017. 1 changed file with 33 additions and 31 deletions.
    64 changes: 33 additions & 31 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -265,9 +265,39 @@ There are two types to represent a value that will be computed in the future: `F
    The definition of these types is intentionally kept as narrow as possible (while still being functional) in this proposal. We expect subsequent proposals to expand the API surface area.



    ## Conversion of Objective-C APIs

    Full details are beyond the scope of this proposal, but it is important to enhance the importer to project Objective-C completion-handler based APIs into `async` forms. This is a transformation comparable to how `NSError**` functions are imported as `throws` functions. Having the importer do this means that many Cocoa APIs will be modernized en-mass.

    There are multiple possible designs for this with different tradeoffs. The maximally source compatible way to do this is to import completion handler-based APIs in two forms: both the completion handler and the `async` form. For example, given:

    ```objc
    // Before
    - (void) makeSandwich:(void(^)())completionHandler;
    - (void) makeSandwich:(void(^)(Sandwich* __nonnull sandwich))completionHandler;
    - (void) makeSandwich:(void(^)(Sandwich* __nullable sandwich, NSError* __nullable error))completionHandler;
    - (void) makeSandwich:(void(^)(Sandwich* __nullable half1, Sandwich* __nullable half2, NSError* __nullable error))completionHandler;
    - (void) makeSandwich:(void(^)(NSError* __nullable error))completionHandler;
    ```

    The declarations above are imported both in their normal completion handler form, but also in their nicer `async` forms:

    ```swift
    func makeSandwich() async
    func makeSandwich() async -> Sandwich
    func makeSandwich() async throws -> Sandwich
    func makeSandwich() async throws -> (half1: Sandwich, half2: Sandwich)
    func makeSandwich() async throws
    ```

    There are many details that should be defined as part of this importing process - for example, what are the exact rules for the transformation? Are multiple result functions common enough to handle automatically? How do overrides work when there are two definitions for the same method? Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration?



    ## Interaction with existing features

    This proposal dovetails naturally with existing language features in Swift, here are a few examples.
    This proposal dovetails naturally with existing language features in Swift, here are a few examples:

    #### Error handling

    @@ -356,35 +386,7 @@ func makeClubSandwich(completionHandler: (part1: Sandwich?, part2: Sandwich?, er
    func makeSandwich() async throws -> (Sandwich, Sandwich)
    ```

    ## Conversion of Objective-C APIs

    Full details are beyond the scope of this proposal, but it is important to enhance the importer to project Objective-C completion-handler based APIs into `async` forms. This is a transformation comparable to how `NSError**` functions are imported as `throws` functions. Having the importer do this means that many Cocoa APIs will be modernized en-mass.

    There are multiple possible designs for this with different tradeoffs. The maximally source compatible way to do this is to import completion handler-based APIs in two forms: both the completion handler and the `async` form. For example, given:

    ```objc
    // Before
    - (void) makeSandwich:(void(^)())completionHandler;
    - (void) makeSandwich:(void(^)(Sandwich* __nonnull sandwich))completionHandler;
    - (void) makeSandwich:(void(^)(Sandwich* __nullable sandwich, NSError* __nullable error))completionHandler;
    - (void) makeSandwich:(void(^)(Sandwich* __nullable half1, Sandwich* __nullable half2, NSError* __nullable error))completionHandler;
    - (void) makeSandwich:(void(^)(NSError* __nullable error))completionHandler;
    ```

    The declarations above are imported both in their normal completion handler form, but also in their nicer `async` forms:

    ```swift
    func makeSandwich() async
    func makeSandwich() async -> Sandwich
    func makeSandwich() async throws -> Sandwich
    func makeSandwich() async throws -> (half1: Sandwich, half2: Sandwich)
    func makeSandwich() async throws
    ```

    There are many details that should be defined as part of this importing process - for example, what are the exact rules for the transformation? Are multiple result functions common enough to handle automatically? How do overrides work when there are two definitions for the same method? Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration?


    ## Sync and async functions interoperability
    ### Sync and async functions interoperability

    We have already seen how asynchronous functions call synchronous and asynchronous ones, but we haven't explored how synchronous ones (e.g. event handlers) call `async` ones and how `async` ones can wrap old-fashioned functions still written with completion handlers.

    @@ -463,7 +465,7 @@ func makeSandwichWithErrorOldWay(completionHandler: (Sandwich?, Error?) -> Void)

    Most Objective-C APIs should be automatically imported as `async` by the compiler (described below), however some APIs may not be automatically importable. If so, it would make sense to add an API (possibly in the GCD layer) to enable this. This is a framework level change though, so it is beyond the scope of this proposal.

    ## Cancellation Example
    ### Cancellation Example

    As in Swift 4, this proposal includes no built-in language support for scheduling tasks or cancellation: once an asyncronous operation is started, it cannot be implicitly stopped, even if (e.g.) the last reference to a future containing the computation is dropped.

  18. @lattner lattner revised this gist Aug 9, 2017. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -176,7 +176,7 @@ These problem have been faced in many systems and many languages, and the abstra
    This proposal adds general coroutine support to Swift, biasing the nomenclature and terminology towards the most common use-case: defining and using asynchronous APIs, eliminating many of the problems working with completion handlers. The choice of terminology (`async` vs `yields`) is a bikeshed topic which needs to be addressed, but isn't pertinent to the core semantics of the model. See [Alternate Syntax Options][#alternate-syntax-options] at the end for an exploration of syntactic options in this space.


    ## Async semantics
    ### Async semantics

    Today, function types can be normal or `throw`ing. This proposal extends them to also be allowed to be `async`. These are all valid function types:

    @@ -240,7 +240,7 @@ func future<T>(fn : () async -> T) -> Future<T> { ... }
    Finally, you are only allowed to invoke an `async` function from within another `async` function or closure. This follows the model of Swift 2 error handling, where you cannot call a throwing function unless you're in a throwing function or inside of a `do/catch` block.


    ## Future types
    ### Future types

    Because the coroutine transformation described in this design lives at the level of function types, it can be applied to many different applications that benefit from the inversion of control that the coroutine transformation provides (e.g. generators). That said, the most important first application is to provide `async`/`await` as a better way to work with asynchronous functions. Similar APIs could be used for other applications, but those are beyond the scope of this proposal.

  19. @lattner lattner revised this gist Aug 9, 2017. 1 changed file with 12 additions and 13 deletions.
    25 changes: 12 additions & 13 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -88,7 +88,7 @@ makeSandwich2 { sandwich, error in
    Conditionally executing an asynchronous function is a huge pain. Perhaps the best approach is to write half of the code in a helper "continuation" closure that is conditionally executed, like this:

    ```swift
    func makeSandwich5(recipient: Person, completionBlock: (result: Sandwich) -> Void) {
    func makeSandwich3(recipient: Person, completionBlock: (result: Sandwich) -> Void) {
    let continuation: (contents: SandwichContents) -> Void = {
    // ... continue and call completionBlock eventually
    }
    @@ -107,7 +107,7 @@ func makeSandwich5(recipient: Person, completionBlock: (result: Sandwich) -> Voi
    It's easy to bail out by simply returning without calling the appropriate block. When forgotten, the issue is very hard to debug:

    ```swift
    func makeSandwich3(completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    func makeSandwich4(completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    cutBread { bread, error in
    guard let bread = bread else {
    return // <- forgot to call the block
    @@ -126,7 +126,7 @@ When you do not forget to call the block, you can still forget to return after t
    Thankfully `guard` syntax protects against that to some degree, but it's not always relevant.

    ```swift
    func makeSandwich4(recipient:Person, completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    func makeSandwich5(recipient:Person, completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    if recipient.isVegeterian {
    if let sandwich = cachedVegeterianSandwich {
    completionBlock(cachedVegeterianSandwich) // <- forgot to return after calling the block
    @@ -220,7 +220,7 @@ Under the hood, the compiler rewrites this code using nested closures like in ex
    To allow parallel execution, move `await` from the call to the result when it is needed, and wrap the asynchronous calls in a "future", which can itself be `await`d on later.

    ```swift
    func makeSandwich2() async -> Sandwich {
    func makeSandwich1a() async -> Sandwich {
    let bread = future { await cutBread() }
    let cheese = future { await cutCheese() }
    let ham = future { await cutHam() }
    @@ -271,26 +271,25 @@ This proposal dovetails naturally with existing language features in Swift, here

    #### Error handling

    Error handling syntax introduced in Swift 2 composes naturally with asynchronous model.
    Error handling syntax introduced in Swift 2 composes naturally with this asynchronous model.

    ```swift
    // Could throw or be interrupted:
    func makeSandwich() async throws -> Sandwich

    // Semantically similar to:
    func makeSandwich(completionHandler: (result: Sandwich?, error: Error?)->Void)

    ```

    Our example thus becomes (compare with the example `makeSandwich1`):
    Our example thus becomes (compare with the example `makeSandwich2`):

    ```swift
    func cutBread() async throws -> Bread
    func cutCheese() async throws -> Cheese
    func cutHam() async throws -> Ham
    func cutTomato() async throws -> Vegetable

    func makeSandwich3() async throws -> Sandwich {
    func makeSandwich2() async throws -> Sandwich {
    let bread = try await cutBread()
    let cheese = try await cutCheese()
    let ham = try await cutHam()
    @@ -304,11 +303,11 @@ Internally, compiler rewrites the code using closures and exits early if any int
    To enable `throw`ing `async` calls execute in parallel, `try await` must be used after all necessary operations has begun.

    ```swift
    func makeSandwich4() async throws -> Sandwich {
    let breadFuture = future { await cutBread() }
    let cheeseFuture = future { await cutCheese() }
    let hamFuture = future { await cutHam() }
    let tomatoFuture = future { await cutTomato() }
    func makeSandwich2a() async throws -> Sandwich {
    let breadFuture = future { try await cutBread() }
    let cheeseFuture = future { try await cutCheese() }
    let hamFuture = future { try await cutHam() }
    let tomatoFuture = future { try await cutTomato() }

    let bread = try await breadFuture.get()
    let cheese = try await cheeseFuture.get()
  20. @lattner lattner revised this gist Aug 9, 2017. 1 changed file with 100 additions and 72 deletions.
    172 changes: 100 additions & 72 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -1,9 +1,10 @@
    Async/Await for Swift
    =====================
    # Async/Await for Swift

    * Proposal: SE-XXXX
    * Author: [Chris Lattner](https://github.com/lattner)

    ## Introduction

    Modern Cocoa development involves a lot of asynchronous programming using closures and completion handlers, but completion handlers APIs are hard to use when number of operations grows and dependencies between them become more complicated. This proposal describes a language extension to make this a lot more natural and less error prone.

    This paper introduces a first class [Coroutine model](https://en.wikipedia.org/wiki/Coroutine) to Swift. Further, it provides standard library affordances to expose these as *asynchronous semantics* and *Futures* to Swift, which are concepts [available in many other languages now](https://en.wikipedia.org/wiki/Await) (including C#, Javascript, Python, and others). Functions can opted into to being *async*, allowing the programmer can compose complex logic involving asynchronous operations - while the compiler produces necessary closures and state machines to implement that logic.
    @@ -12,8 +13,7 @@ It is important to understand that this is proposing compiler support that is co

    This is derived from an earlier proposal written by [Oleg Andreev](https://github.com/oleganza), available [here](https://gist.github.com/oleganza/7342ed829bddd86f740a). It has been significantly rewritten by [Chris Lattner](https://github.com/lattner), with input from [Joe Groff](https://github.com/jckarter).

    Common problems with completion handlers
    ----------------------------------------
    ## Motivation: Completion handlers are suboptimal

    To provide motivation for why it is important to do something here, lets look at a number of gross problems that Cocoa (and server/cloud) programmers frequently face.

    @@ -169,16 +169,14 @@ It is the responsibility of the compiler to transform the function into a form t



    The Coroutine transformation
    ----------------------------
    ## Proposed Solution: The Coroutine transformation

    These problem have been faced in many systems and many languages, and the abstraction of [Coroutines](https://en.wikipedia.org/wiki/Coroutine) is a standard way to address them. Without delving too much into theory, Coroutines are an extension of basic functions that allow a function to return a value *or be suspended*. They can be used to implement generators, asynchronous models, and other capabilities - there is a large body of work on the theory, implementation, and optimization of them.

    This proposal adds general coroutine support to Swift, biasing the nomenclature and terminology towards the most common use-case: defining and using asynchronous APIs, eliminating many of the problems working with completion handlers. The choice of terminology (`async` vs `yields`) is a bikeshed topic which needs to be addressed, but isn't pertinent to the core semantics of the model. See [Alternate Design Options][#alternate-design-options] at the end for an exploration of syntactic options in this space.
    This proposal adds general coroutine support to Swift, biasing the nomenclature and terminology towards the most common use-case: defining and using asynchronous APIs, eliminating many of the problems working with completion handlers. The choice of terminology (`async` vs `yields`) is a bikeshed topic which needs to be addressed, but isn't pertinent to the core semantics of the model. See [Alternate Syntax Options][#alternate-syntax-options] at the end for an exploration of syntactic options in this space.


    Async semantics
    ---------------
    ## Async semantics

    Today, function types can be normal or `throw`ing. This proposal extends them to also be allowed to be `async`. These are all valid function types:

    @@ -242,8 +240,36 @@ func future<T>(fn : () async -> T) -> Future<T> { ... }
    Finally, you are only allowed to invoke an `async` function from within another `async` function or closure. This follows the model of Swift 2 error handling, where you cannot call a throwing function unless you're in a throwing function or inside of a `do/catch` block.


    Error handling
    --------------
    ## Future types

    Because the coroutine transformation described in this design lives at the level of function types, it can be applied to many different applications that benefit from the inversion of control that the coroutine transformation provides (e.g. generators). That said, the most important first application is to provide `async`/`await` as a better way to work with asynchronous functions. Similar APIs could be used for other applications, but those are beyond the scope of this proposal.

    Asynchronous semantics are implemented with [futures](https://en.wikipedia.org/wiki/Futures_and_promises) — types that represent a future value which may not be resolved yet. Extending the standard library with the following declarations allow programmers to reason about this and work with them:

    ```swift
    class ThrowingFuture<T> {
    func get() async throws -> T {...}
    }
    class Future<T> : ThrowingFuture<T> {
    override func get() async -> T {...}
    }

    func future<T>(fn : () async -> T) -> Future<T> { ... }
    func future<T>(fn : () async throws -> T) -> ThrowingFuture<T> { ... }
    ```

    There are two types to represent a value that will be computed in the future: `Future<T>` for non-throwing asynchronous functions and `ThrowingFuture<T>` for throwing ones. `Future` derives from `ThrowingFuture` to reflect the subtype relationship between throwing functions and non-throwing ones.

    `Future` and `ThrowingFuture` are classes because they are non-copyable. This requires that they have reference (or move-only) semantics. In many trivial cases, instances of these classes could be completely optimized away, assuming the definition of these types are `fragile` (in the resilience model sense).

    The definition of these types is intentionally kept as narrow as possible (while still being functional) in this proposal. We expect subsequent proposals to expand the API surface area.


    ## Interaction with existing features

    This proposal dovetails naturally with existing language features in Swift, here are a few examples.

    #### Error handling

    Error handling syntax introduced in Swift 2 composes naturally with asynchronous model.

    @@ -292,16 +318,18 @@ func makeSandwich4() async throws -> Sandwich {
    return Sandwich([bread, cheese, ham, tomato])
    }
    ```
    The above example the unevaluated computation are wrapped into a "future" value through the following standard library function overload:

    In this example we begin all operations at once and then first wait for completion of `cutBread` and exit before checking the results of other operations. Note that this example throws at the point of the first `try await` whose asynchronous operation threw an error, but does not fail with the earliest operation.

    #### Closure type inference

    Because the `await` keyword is used at all points where execution may be suspended, it is simple for the compiler to determine whether a closure is `async` or not: it is if the body includes an `await`. This works exactly the same way that the presence of `try` in a closure causes it to be inferred as a throwing closure. You can also explicitly mark a closure as `async` using the standard form of:

    ```swift
    func future<T>(fn : () async throws -> T) -> ThrowingFuture<T> { ... }
    let myClosure = { () async -> () in ... }
    ```

    In this example we begin all operations at once and then first wait for completion of `cutBread` and exit before checking the results of other operations. Note that this example throws at the point of the first `try await` whose asynchronous operation threw an error, but does not fail with the earliest operation.

    Behavior of `defer`
    -------------------
    #### `defer`

    There is no change to the semantics of `defer`, it continues to be bound by the programmer visible syntactic scope. The deferred statement is executed when the the current scope is exited. For example, when asynchronous operations complete and a return is hit, when an error is thrown, etc.

    @@ -317,8 +345,7 @@ func makeSandwich() async throws -> Sandwich {
    }
    ```

    Returning multiple values
    -------------------------
    #### Completion handlers with multiple return values

    Completion handler APIs may have multiple result arguments (not counting an error argument). These are naturally represented by tuple results in `async` functions:

    @@ -330,31 +357,7 @@ func makeClubSandwich(completionHandler: (part1: Sandwich?, part2: Sandwich?, er
    func makeSandwich() async throws -> (Sandwich, Sandwich)
    ```

    Future types
    ------------

    Because the coroutine transformation described in this design lives at the level of function types, it can be applied to many different applications that benefit from the inversion of control that the coroutine transformation provides (e.g. generators). That said, the most important first application is to provide `async`/`await` as a better way to work with asynchronous functions. Similar APIs could be used for other applications, but those are beyond the scope of this proposal.

    Asynchronous semantics are implemented with [futures](https://en.wikipedia.org/wiki/Futures_and_promises) — types that represent a future value which may not be resolved yet. Extending the standard library with the following declarations allow programmers to reason about this and work with them:

    ```swift
    class ThrowingFuture<T> {
    func get() async throws -> T {...}
    }
    class Future<T> : ThrowingFuture<T> {
    override func get() async -> T {...}
    }

    func future<T>(fn : () async -> T) -> Future<T> { ... }
    func future<T>(fn : () async throws -> T) -> ThrowingFuture<T> { ... }
    ```

    There are two types to represent a value that will be computed in the future: `Future<T>` for non-throwing asynchronous functions and `ThrowingFuture<T>` for throwing ones. `Future` derives from `ThrowingFuture` to reflect the subtype relationship between throwing functions and non-throwing ones.

    `Future` and `ThrowingFuture` are classes because they are non-copyable. This requires that they have reference or move-only semantics. Of course, other designs are possible. In many trivial cases, instances of these classes could be completely optimized away, assuming the definition of these types are `fragile` (in the resilience model sense).

    Conversion of Objective-C APIs
    ------------------------------
    ## Conversion of Objective-C APIs

    Full details are beyond the scope of this proposal, but it is important to enhance the importer to project Objective-C completion-handler based APIs into `async` forms. This is a transformation comparable to how `NSError**` functions are imported as `throws` functions. Having the importer do this means that many Cocoa APIs will be modernized en-mass.

    @@ -382,8 +385,7 @@ func makeSandwich() async throws
    There are many details that should be defined as part of this importing process - for example, what are the exact rules for the transformation? Are multiple result functions common enough to handle automatically? How do overrides work when there are two definitions for the same method? Would it be better to just import completion handler functions only as `async` in Swift 5 mode, forcing migration?


    Sync and async functions interoperability
    -----------------------------------------
    ## Sync and async functions interoperability

    We have already seen how asynchronous functions call synchronous and asynchronous ones, but we haven't explored how synchronous ones (e.g. event handlers) call `async` ones and how `async` ones can wrap old-fashioned functions still written with completion handlers.

    @@ -462,10 +464,11 @@ func makeSandwichWithErrorOldWay(completionHandler: (Sandwich?, Error?) -> Void)

    Most Objective-C APIs should be automatically imported as `async` by the compiler (described below), however some APIs may not be automatically importable. If so, it would make sense to add an API (possibly in the GCD layer) to enable this. This is a framework level change though, so it is beyond the scope of this proposal.

    Cancellation Example
    --------------------
    ## Cancellation Example

    As in Swift 4, this proposal includes no built-in language support for scheduling tasks or cancellation. Cancellation should continued to be implemented by the programmer where needed, and there are many different patterns that can be applied. However, coupled with improved error handling, implementing common cancellation patterns becomes elegant:
    As in Swift 4, this proposal includes no built-in language support for scheduling tasks or cancellation: once an asyncronous operation is started, it cannot be implicitly stopped, even if (e.g.) the last reference to a future containing the computation is dropped.

    If cancellation is important, it should continue to be implemented by the programmer where needed, and there are many standard patterns that can be applied. However, particularly coupled with improved error handling, common cancellation patterns become very elegant:

    ```swift
    @IBAction func makeSandwich(sender: AnyObject) {
    @@ -493,8 +496,35 @@ As in Swift 4, this proposal includes no built-in language support for schedulin
    Internally, `kitchen` may use `NSOperations` or custom `cancelled` flag. The intent of this section is to give a single example of how to approach this, not to define a normative or all-encompassing approach that should be used in all cases.


    Alternate Design Options
    ------------------------
    ## Source Compatibility

    This is a generally additive feature, but it does turn take `async` and `await` as keywords, so it will break code that uses them as identifiers. This is expected to have very minor impact: the most pervasive use of `async` as an identifier occurs in code that works with dispatch queues, but fortunately keywords are allowed as qualified member names, so code like this doesn't need any change:

    ```swift
    myQueue.async { ... }
    ```

    That said, there could be obscure cases that break. One example that occurs in the Swift testsuite is of the form:

    ```swift
    extension DispatchQueue {
    func myThing() {
    async {
    ...
    }
    }
    }
    ```

    This can be addressed by changing the code to use `self.async` or backticks. The compiler should be able to detect a large number of these cases and produce a fixit.

    ## Effect on ABI stability

    This proposal does not change the ABI of any existing language features, but does introduce a new concept that adds to the ABI surface area, including a new mangling and calling convention.

    ## Alternate Syntax Options

    Here are a couple of syntax level changes to the proposal that are worth discussing, these don't fundamentally change the shape of the proposal.

    #### Spelling of `async` keyword

    @@ -529,32 +559,31 @@ The other way to factor the complexity is to make it so that `async` functions d
    This model provides a ton of advantages: it is arguably the right defaults for the vast majority of clients (reducing boilerplate and syntactic noise), provides the ability for the importer and experts to get what they want. The only downside of is that it is a less obvious design than have two orthogonal axes, but in the opinion of the proposal author, this is probably the right set of tradeoffs.


    Source Compatibility
    --------------------
    ## Alternatives Considered

    This is a generally additive feature, but it does turn take `async` and `await` as keywords, so it will break code that uses them as identifiers. This is expected to have very minor impact: the most pervasive use of `async` as an identifier occurs in code that works with dispatch queues, but fortunately keywords are allowed as qualified member names, so code like this doesn't need any change:
    #### Remove `future` from this proposal, and add it as a follow-on

    ```swift
    myQueue.async { ... }
    ```
    This proposal includes a intentionally minimal `future` design to show an important use-case. However, there are many rational designs for futures, and a lot of experience working with them. If the discussion and design of futures ends up slowing down the proposal, it would make sense to split it out to a completely separate proposal: after all, it is a simple additive feature on top of the basic language support being added.

    That said, there could be obscure cases that break. One example that occurs in the Swift testsuite is of the form:
    #### Change async calls to always return a future

    ```swift
    extension DispatchQueue {
    func myThing() {
    async {
    ...
    }
    }
    }
    ```
    Most commonly cited alternative design is to follow the model of (e.g. C#), where calls to
    async functions return a future (a `Task` in C#), instead of futures being a library feature separable from the
    core language. This design effectively adds async/await to the language, it does not add a
    more general coroutine feature.

    Despite this model being widely know, we believe that the proposed design
    is superior for a number of reasons:

    This can be addressed by changing the code to use `self.async` or backticks. The compiler should be able to detect a large number of these cases and produce a fixit.
    - Coroutines (and the coroutine transformation that inverts control flow) are generally useful language features beyond the domain of async/await. Building async/await into the compiler would require building generators in as well.
    - The proposed design eliminates the problem of calling an API (without knowing it is async) and getting a future back instead of the expected result type. C# addresses this by suggesting that all `async` methods have their name be suffixed with `Async`, which is suboptimal.
    - The C# model for await is a unary prefix keyword, which does not compose well in the face of chaining. Wherein C# you may have to write something like `x = await (await foo()).bar()`, with this design you can simply write `x = await foo().bar()` for the same reasons that you don't have to write `try` on every single call in a chain that can throw.
    - There are multiple different interesting abstractions (like future) to consider. By putting the details of them in the standard library, other people can define and use their own abstractions where it makes sense.

    The primary argument for adding async/await (and then generators) to the language as first-class language features is that they are the vastly most common use-case of coroutines. In the authors opinion, the design as proposed gives something that works better than the C# model in practice, while also providing a more useful/general language model.


    Potential Future Directions
    ---------------------------
    ## Potential Future Directions

    This proposal has been kept intentionally minimal, but there are many possible ways to expand this in the future. For example:

    @@ -587,8 +616,7 @@ These have not been included in the basic proposal because they are not necessar

    Affordances could be added to better call blocking APIs from `async` functions and to hard wait for an `async` function to complete. There are significant tradeoffs and wide open design space to explore here, and none of it is necessary for the base proposal.

    Thanks
    ------
    ## Thanks

    @oleganza thanked @groue and @pierlo for their feedback on his original proposal. Thanks to @oleganza for the original draft which influenced this!

  21. @lattner lattner revised this gist Aug 9, 2017. 1 changed file with 4 additions and 4 deletions.
    8 changes: 4 additions & 4 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -338,18 +338,18 @@ Because the coroutine transformation described in this design lives at the level
    Asynchronous semantics are implemented with [futures](https://en.wikipedia.org/wiki/Futures_and_promises) — types that represent a future value which may not be resolved yet. Extending the standard library with the following declarations allow programmers to reason about this and work with them:

    ```swift
    class Future<T> {
    func get() async -> T {...}
    }
    class ThrowingFuture<T> {
    func get() async throws -> T {...}
    }
    class Future<T> : ThrowingFuture<T> {
    override func get() async -> T {...}
    }

    func future<T>(fn : () async -> T) -> Future<T> { ... }
    func future<T>(fn : () async throws -> T) -> ThrowingFuture<T> { ... }
    ```

    There are two types to represent a value that will be computed in the future: `Future<T>` for non-throwing asynchronous functions and `ThrowingFuture<T>` for throwing ones.
    There are two types to represent a value that will be computed in the future: `Future<T>` for non-throwing asynchronous functions and `ThrowingFuture<T>` for throwing ones. `Future` derives from `ThrowingFuture` to reflect the subtype relationship between throwing functions and non-throwing ones.

    `Future` and `ThrowingFuture` are classes because they are non-copyable. This requires that they have reference or move-only semantics. Of course, other designs are possible. In many trivial cases, instances of these classes could be completely optimized away, assuming the definition of these types are `fragile` (in the resilience model sense).

  22. @lattner lattner revised this gist Aug 9, 2017. 1 changed file with 5 additions and 5 deletions.
    10 changes: 5 additions & 5 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -284,10 +284,10 @@ func makeSandwich4() async throws -> Sandwich {
    let hamFuture = future { await cutHam() }
    let tomatoFuture = future { await cutTomato() }

    let bread = try await breadFuture
    let cheese = try await cheeseFuture
    let ham = try await hamFuture
    let tomato = try await tomatoFuture
    let bread = try await breadFuture.get()
    let cheese = try await cheeseFuture.get()
    let ham = try await hamFuture.get()
    let tomato = try await tomatoFuture.get()

    return Sandwich([bread, cheese, ham, tomato])
    }
    @@ -351,7 +351,7 @@ func future<T>(fn : () async throws -> T) -> ThrowingFuture<T> { ... }

    There are two types to represent a value that will be computed in the future: `Future<T>` for non-throwing asynchronous functions and `ThrowingFuture<T>` for throwing ones.

    `Future` and `ThrowingFuture` are classes because they have reference semantics. Of course, other designs are possible. In many trivial cases, instances of these classes could be completely optimized away, assuming the definition of these types are `fragile` (in the resilience model sense).
    `Future` and `ThrowingFuture` are classes because they are non-copyable. This requires that they have reference or move-only semantics. Of course, other designs are possible. In many trivial cases, instances of these classes could be completely optimized away, assuming the definition of these types are `fragile` (in the resilience model sense).

    Conversion of Objective-C APIs
    ------------------------------
  23. @lattner lattner revised this gist Aug 9, 2017. 1 changed file with 30 additions and 30 deletions.
    60 changes: 30 additions & 30 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -21,7 +21,7 @@ To provide motivation for why it is important to do something here, lets look at

    Sequence of simple operations is unnaturally composed in the nested blocks.

    ```
    ```swift
    func makeSandwich1(completionBlock: (result: Sandwich) -> Void) {
    cutBread { bread in
    cutCheese { cheese in
    @@ -45,7 +45,7 @@ This "pyramid of doom" makes it difficult to keep track of code that is running,

    Handling errors becomes difficult and very verbose.

    ```
    ```swift
    func makeSandwich2(completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    cutBread { bread, error in
    guard let bread = bread else {
    @@ -87,7 +87,7 @@ makeSandwich2 { sandwich, error in

    Conditionally executing an asynchronous function is a huge pain. Perhaps the best approach is to write half of the code in a helper "continuation" closure that is conditionally executed, like this:

    ```
    ```swift
    func makeSandwich5(recipient: Person, completionBlock: (result: Sandwich) -> Void) {
    let continuation: (contents: SandwichContents) -> Void = {
    // ... continue and call completionBlock eventually
    @@ -106,7 +106,7 @@ func makeSandwich5(recipient: Person, completionBlock: (result: Sandwich) -> Voi

    It's easy to bail out by simply returning without calling the appropriate block. When forgotten, the issue is very hard to debug:

    ```
    ```swift
    func makeSandwich3(completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    cutBread { bread, error in
    guard let bread = bread else {
    @@ -125,7 +125,7 @@ func makeSandwich3(completionBlock: (result: Sandwich?, error: Error?) -> Void)
    When you do not forget to call the block, you can still forget to return after that.
    Thankfully `guard` syntax protects against that to some degree, but it's not always relevant.

    ```
    ```swift
    func makeSandwich4(recipient:Person, completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    if recipient.isVegeterian {
    if let sandwich = cachedVegeterianSandwich {
    @@ -144,7 +144,7 @@ This is hard to quantify, but the author believes that the awkwardness of defini

    The problems described above are on specific case of a general class of problems involving "resumable" computations. For example, if you want to write code that produces a list of squares of numbers, you might write something like this:

    ```
    ```swift
    for i in 1...10 {
    print(i*i)
    }
    @@ -154,7 +154,7 @@ However, if you want to write this as a Swift sequence, you have to define this

    In contrast, languages that have generators allow you to write something more close to this:

    ```
    ```swift
    func getSequence() -> AnySequence<Int> {
    let seq = sequence {
    for i in 1...10 {
    @@ -182,7 +182,7 @@ Async semantics

    Today, function types can be normal or `throw`ing. This proposal extends them to also be allowed to be `async`. These are all valid function types:

    ```
    ```swift
    (Int) -> Int // #1: Normal function
    (Int) throws -> Int // #2: Throwing function
    (Int) async -> Int // #3: Asynchronous function
    @@ -193,7 +193,7 @@ Just as a normal function (#1) will implicitly convert to a throwing function (#

    On the function declaration side of the things, you can declare a function as being asynchronous just as you declare it to be throwing, but use the `async` keyword:

    ```
    ```swift
    func makeSandwich() async -> Sandwich { ... }

    // Semantically similar to this:
    @@ -202,7 +202,7 @@ func makeSandwich(completionHandler: (result: Sandwich) -> Void) { ... }

    Calls to `async` functions can implicitly suspend the current coroutine. To make this apparent to maintainers of code, you are required to "mark" expressions that call `async` functions with the new `await` keyword (exactly analogously to how `try` is used to mark subexpressions that contain throwing calls). Putting these pieces together, the first example (from the pyramid of doom explanation, above) can be rewritten in a more natural way:

    ```
    ```swift
    func cutBread() async -> Bread
    func cutCheese() async -> Cheese
    func cutHam() async -> Ham
    @@ -221,7 +221,7 @@ Under the hood, the compiler rewrites this code using nested closures like in ex

    To allow parallel execution, move `await` from the call to the result when it is needed, and wrap the asynchronous calls in a "future", which can itself be `await`d on later.

    ```
    ```swift
    func makeSandwich2() async -> Sandwich {
    let bread = future { await cutBread() }
    let cheese = future { await cutCheese() }
    @@ -233,7 +233,7 @@ func makeSandwich2() async -> Sandwich {

    In the above example all four operations will start one after another, and the unevaluated computation is wrapped into a `Future` value through the following standard library function:

    ```
    ```swift
    func future<T>(fn : () async -> T) -> Future<T> { ... }
    ```

    @@ -247,7 +247,7 @@ Error handling

    Error handling syntax introduced in Swift 2 composes naturally with asynchronous model.

    ```
    ```swift
    // Could throw or be interrupted:
    func makeSandwich() async throws -> Sandwich

    @@ -258,7 +258,7 @@ func makeSandwich(completionHandler: (result: Sandwich?, error: Error?)->Void)

    Our example thus becomes (compare with the example `makeSandwich1`):

    ```
    ```swift
    func cutBread() async throws -> Bread
    func cutCheese() async throws -> Cheese
    func cutHam() async throws -> Ham
    @@ -277,7 +277,7 @@ Internally, compiler rewrites the code using closures and exits early if any int

    To enable `throw`ing `async` calls execute in parallel, `try await` must be used after all necessary operations has begun.

    ```
    ```swift
    func makeSandwich4() async throws -> Sandwich {
    let breadFuture = future { await cutBread() }
    let cheeseFuture = future { await cutCheese() }
    @@ -294,7 +294,7 @@ func makeSandwich4() async throws -> Sandwich {
    ```
    The above example the unevaluated computation are wrapped into a "future" value through the following standard library function overload:

    ```
    ```swift
    func future<T>(fn : () async throws -> T) -> ThrowingFuture<T> { ... }
    ```

    @@ -305,7 +305,7 @@ Behavior of `defer`

    There is no change to the semantics of `defer`, it continues to be bound by the programmer visible syntactic scope. The deferred statement is executed when the the current scope is exited. For example, when asynchronous operations complete and a return is hit, when an error is thrown, etc.

    ```
    ```swift
    func makeSandwich() async throws -> Sandwich {
    startSpinner()
    defer stopSpinner() // will be called when error is thrown or when all operations complete and a result is returned.
    @@ -322,7 +322,7 @@ Returning multiple values

    Completion handler APIs may have multiple result arguments (not counting an error argument). These are naturally represented by tuple results in `async` functions:

    ```
    ```swift
    // Before
    func makeClubSandwich(completionHandler: (part1: Sandwich?, part2: Sandwich?, error: Error?) -> Void)

    @@ -337,7 +337,7 @@ Because the coroutine transformation described in this design lives at the level

    Asynchronous semantics are implemented with [futures](https://en.wikipedia.org/wiki/Futures_and_promises) — types that represent a future value which may not be resolved yet. Extending the standard library with the following declarations allow programmers to reason about this and work with them:

    ```
    ```swift
    class Future<T> {
    func get() async -> T {...}
    }
    @@ -371,7 +371,7 @@ There are multiple possible designs for this with different tradeoffs. The maxi

    The declarations above are imported both in their normal completion handler form, but also in their nicer `async` forms:

    ```
    ```swift
    func makeSandwich() async
    func makeSandwich() async -> Sandwich
    func makeSandwich() async throws -> Sandwich
    @@ -391,7 +391,7 @@ We have already seen how asynchronous functions call synchronous and asynchronou

    Because `async/await` fundamentally transforms the control flow within a function, you're only generally allowed to call an `async` function from within another `async` function. This works well until you get all the way to the top of the stack, e.g. an event handler (like an `IBAction`). Those are `Void`-returning functions which are not themselves `async`, and it is important to be able to perform asynchronous work from within them. For example, if you were using a completion handler, your code might look like this:

    ```
    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    makeSandwich(completionHandler: {(sandwich) in
    @@ -406,7 +406,7 @@ This is an essential pattern, but is itself sort of odd: an `async` operation is

    To address this, there should be a function that takes an async closure, starts it, but ignores the result. The design outlined above is sufficient, just use the existing `Future` type for this:

    ```
    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    _ = future {
    @@ -420,7 +420,7 @@ To address this, there should be a function that takes an async closure, starts

    However, it may make sense to add something more explicit, such as a new standard library function:

    ```
    ```swift
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    detachedFuture {
    @@ -438,7 +438,7 @@ The exact naming of this operation is sure to require extensive bikeshedding. F

    With something like `detachedFuture`, it is easy to wrap a result of asynchronous call in a classic completion handler API.

    ```
    ```swift
    func makeSandwichOldWay(completionHandler: (Sandwich) -> Void) {
    detachedFuture {
    let sandwich = await makeSandwich()
    @@ -467,7 +467,7 @@ Cancellation Example

    As in Swift 4, this proposal includes no built-in language support for scheduling tasks or cancellation. Cancellation should continued to be implemented by the programmer where needed, and there are many different patterns that can be applied. However, coupled with improved error handling, implementing common cancellation patterns becomes elegant:

    ```
    ```swift
    @IBAction func makeSandwich(sender: AnyObject) {
    detachedFuture {
    do {
    @@ -504,7 +504,7 @@ Instead of spelling the function type modifier as `async`, it could be spelled a

    It would be a great simplification of the language model to make the `async` modifier on a function imply that the function is `throw`ing, instead of making them orthogonal modifiers. From an intuitive perspective, this makes sense because many of the sorts of operations that are asynchronous (e.g. loading a resource, talking to the network, etc) can also fail. There is also precedent from many other systems that use `async`/`await` for this. Instead of four kinds of function type, we'd only have three:

    ```
    ```swift
    (Int) -> Int // Normal function
    (Int) throws -> Int // Throwing function
    (Int) async -> Int // Asynchronous function, can also throw
    @@ -519,7 +519,7 @@ There are two downsides to doing this: the first of which is that Cocoa has a nu

    The other way to factor the complexity is to make it so that `async` functions default to `throw`ing, but still allow non-`throw`ing `async` functions to be expressed with `nonthrowing` (or some other spelling). This provides this model:

    ```
    ```swift
    (Int) -> Int // Normal function
    (Int) throws -> Int // Throwing function
    (Int) async -> Int // Asynchronous function, can also throw.
    @@ -534,13 +534,13 @@ Source Compatibility

    This is a generally additive feature, but it does turn take `async` and `await` as keywords, so it will break code that uses them as identifiers. This is expected to have very minor impact: the most pervasive use of `async` as an identifier occurs in code that works with dispatch queues, but fortunately keywords are allowed as qualified member names, so code like this doesn't need any change:

    ```
    ```swift
    myQueue.async { ... }
    ```

    That said, there could be obscure cases that break. One example that occurs in the Swift testsuite is of the form:

    ```
    ```swift
    extension DispatchQueue {
    func myThing() {
    async {
    @@ -573,7 +573,7 @@ e.g. "wait on a list of pending operations, continuing when the first is availab

    Standard `map`, `flatMap` and `reduce` methods could be provided on `Future` and `ThrowingFuture`. The runtime could implement parallel execution of all mapping operations on futures, allowing you to write something like this:

    ```
    ```swift
    var urls : [URL]
    // ...
    urls.map(fetch).map(parse).map(analyze).reduce([]){ $0.append($1) }
  24. @lattner lattner revised this gist Aug 9, 2017. 1 changed file with 23 additions and 19 deletions.
    42 changes: 23 additions & 19 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -46,7 +46,7 @@ This "pyramid of doom" makes it difficult to keep track of code that is running,
    Handling errors becomes difficult and very verbose.

    ```
    func makeSandwich2(completionBlock: (result:Sandwich?, error:NSError?) -> Void) {
    func makeSandwich2(completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    cutBread { bread, error in
    guard let bread = bread else {
    completionBlock(nil, error)
    @@ -107,7 +107,7 @@ func makeSandwich5(recipient: Person, completionBlock: (result: Sandwich) -> Voi
    It's easy to bail out by simply returning without calling the appropriate block. When forgotten, the issue is very hard to debug:

    ```
    func makeSandwich3(completionBlock: (result: Sandwich?, error: NSError?) -> Void) {
    func makeSandwich3(completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    cutBread { bread, error in
    guard let bread = bread else {
    return // <- forgot to call the block
    @@ -126,7 +126,7 @@ When you do not forget to call the block, you can still forget to return after t
    Thankfully `guard` syntax protects against that to some degree, but it's not always relevant.

    ```
    func makeSandwich4(recipient:Person, completionBlock: (result: Sandwich?, error: NSError?) -> Void) {
    func makeSandwich4(recipient:Person, completionBlock: (result: Sandwich?, error: Error?) -> Void) {
    if recipient.isVegeterian {
    if let sandwich = cachedVegeterianSandwich {
    completionBlock(cachedVegeterianSandwich) // <- forgot to return after calling the block
    @@ -227,7 +227,7 @@ func makeSandwich2() async -> Sandwich {
    let cheese = future { await cutCheese() }
    let ham = future { await cutHam() }
    let tomato = future { await cutTomato() }
    return Sandwich([await bread, await cheese, await ham, await tomato)
    return Sandwich([await bread.get(), await cheese.get(), await ham.get(), await tomato.get())
    }
    ```

    @@ -252,7 +252,7 @@ Error handling syntax introduced in Swift 2 composes naturally with asynchronous
    func makeSandwich() async throws -> Sandwich
    // Semantically similar to:
    func makeSandwich(completionHandler: (result:Sandwich?, error:NSError?)->Void)
    func makeSandwich(completionHandler: (result: Sandwich?, error: Error?)->Void)
    ```

    @@ -324,7 +324,7 @@ Completion handler APIs may have multiple result arguments (not counting an erro

    ```
    // Before
    func makeClubSandwich(completionHandler: (part1: Sandwich?, part2: Sandwich?, error: NSError?) -> Void)
    func makeClubSandwich(completionHandler: (part1: Sandwich?, part2: Sandwich?, error: Error?) -> Void)
    // After
    func makeSandwich() async throws -> (Sandwich, Sandwich)
    @@ -338,14 +338,18 @@ Because the coroutine transformation described in this design lives at the level
    Asynchronous semantics are implemented with [futures](https://en.wikipedia.org/wiki/Futures_and_promises) — types that represent a future value which may not be resolved yet. Extending the standard library with the following declarations allow programmers to reason about this and work with them:

    ```
    class Future<T> {...}
    class ThrowingFuture<T> : Future<T> { ... }
    class Future<T> {
    func get() async -> T {...}
    }
    class ThrowingFuture<T> {
    func get() async throws -> T {...}
    }
    func future<T>(fn : () async -> T) -> Future<T> { ... }
    func future<T>(fn : () async throws -> T) -> ThrowingFuture<T> { ... }
    ```

    There are two types to represent a value that will be computed in the future: `Future<T>` for non-throwing asynchronous functions and `ThrowingFuture<T>` for throwing ones. It is expected that there is some magic used to allow the `await` keyword to work on a value of the Future types (e.g. an `Awaitable` protocol that both types conform to).
    There are two types to represent a value that will be computed in the future: `Future<T>` for non-throwing asynchronous functions and `ThrowingFuture<T>` for throwing ones.

    `Future` and `ThrowingFuture` are classes because they have reference semantics. Of course, other designs are possible. In many trivial cases, instances of these classes could be completely optimized away, assuming the definition of these types are `fragile` (in the resilience model sense).

    @@ -400,7 +404,7 @@ Because `async/await` fundamentally transforms the control flow within a functio

    This is an essential pattern, but is itself sort of odd: an `async` operation is being fired off immediately (#1), then runs the subsequent code (#3), and the completion handler (#2) runs at some time later -- on some queue (often the main one). This pattern frequently leads to mutation of global state (as in this example) or to making assumptions about which queue the completion handler is run on. While this may not be very safe, it is essential that the model encompasses this pattern, because it is a practical necessity in Cocoa development.

    To address this, there should be a function that takes an async closure, starts it, but ignores the result. With the design outlined above, it should be sufficient to use the existing `Future` type for this:
    To address this, there should be a function that takes an async closure, starts it, but ignores the result. The design outlined above is sufficient, just use the existing `Future` type for this:

    ```
    @IBAction func buttonDidClick(sender:AnyObject) {
    @@ -414,12 +418,12 @@ To address this, there should be a function that takes an async closure, starts
    }
    ```

    However, this is extremely implicit. Instead, it may make sense to add something more explicit, such as a new standard library function to explicitly represent this:
    However, it may make sense to add something more explicit, such as a new standard library function:

    ```
    @IBAction func buttonDidClick(sender:AnyObject) {
    // 1
    detatchedFuture {
    detachedFuture {
    // 2
    let sandwich = await makeSandwich()
    imageView.image = sandwich.preview
    @@ -428,22 +432,22 @@ However, this is extremely implicit. Instead, it may make sense to add somethin
    }
    ```

    The exact naming of this operation is sure to require extensive bikeshedding. For the purpose of this proposal, we simply observe that this is easy to build the standard library if necessary.
    The exact naming of this operation is sure to require extensive bikeshedding. For the purpose of this proposal, we simply observe that this is easy to build the standard library if desirable.

    #### Wrapping async function in a block-based function

    With something like `detatchedFuture`, it is easy to wrap a result of asynchronous call in a classic completion handler API.
    With something like `detachedFuture`, it is easy to wrap a result of asynchronous call in a classic completion handler API.

    ```
    func makeSandwichOldWay(completionHandler: (Sandwich) -> Void) {
    detatchedFuture {
    detachedFuture {
    let sandwich = await makeSandwich()
    completionHandler(sandwich)
    }
    }
    func makeSandwichWithErrorOldWay(completionHandler: (Sandwich?, NSError?) -> Void) {
    detatchedFuture {
    func makeSandwichWithErrorOldWay(completionHandler: (Sandwich?, Error?) -> Void) {
    detachedFuture {
    do {
    let sandwich = try await makeSandwichWithError()
    completionHandler(sandwich, nil)
    @@ -465,14 +469,14 @@ As in Swift 4, this proposal includes no built-in language support for schedulin

    ```
    @IBAction func makeSandwich(sender: AnyObject) {
    detatchedFuture {
    detachedFuture {
    do {
    let bread = try await kitchen.cutBread()
    let cheese = try await kitchen.cutCheese()
    let ham = try await kitchen.cutHam()
    let tomato = try await kitchen.cutTomato()
    eat(Sandwich(bread, cheese, ham, tomato]))
    } catch CocoaErrorDomain.UserCancelled {
    } catch CocoaError.userCancelled {
    // Ignore, user quit the kitchen.
    } catch {
    // Some really interesting error happened
  25. @lattner lattner revised this gist Aug 8, 2017. 1 changed file with 33 additions and 10 deletions.
    43 changes: 33 additions & 10 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -200,8 +200,7 @@ func makeSandwich() async -> Sandwich { ... }
    func makeSandwich(completionHandler: (result: Sandwich) -> Void) { ... }
    ```

    `async` functions can use the new `await` keywords to invoke other asynchronous functions
    in a simple imperative style. Now the first example (from the pyramid of doom explanation, above) can be rewritten in a more natural way:
    Calls to `async` functions can implicitly suspend the current coroutine. To make this apparent to maintainers of code, you are required to "mark" expressions that call `async` functions with the new `await` keyword (exactly analogously to how `try` is used to mark subexpressions that contain throwing calls). Putting these pieces together, the first example (from the pyramid of doom explanation, above) can be rewritten in a more natural way:

    ```
    func cutBread() async -> Bread
    @@ -224,10 +223,10 @@ To allow parallel execution, move `await` from the call to the result when it is

    ```
    func makeSandwich2() async -> Sandwich {
    let bread = future { cutBread() }
    let cheese = future { cutCheese() }
    let ham = future { cutHam() }
    let tomato = future { cutTomato() }
    let bread = future { await cutBread() }
    let cheese = future { await cutCheese() }
    let ham = future { await cutHam() }
    let tomato = future { await cutTomato() }
    return Sandwich([await bread, await cheese, await ham, await tomato)
    }
    ```
    @@ -280,10 +279,10 @@ To enable `throw`ing `async` calls execute in parallel, `try await` must be used

    ```
    func makeSandwich4() async throws -> Sandwich {
    let breadFuture = future { cutBread() }
    let cheeseFuture = future { cutCheese() }
    let hamFuture = future { cutHam() }
    let tomatoFuture = future { cutTomato() }
    let breadFuture = future { await cutBread() }
    let cheeseFuture = future { await cutCheese() }
    let hamFuture = future { await cutHam() }
    let tomatoFuture = future { await cutTomato() }
    let bread = try await breadFuture
    let cheese = try await cheeseFuture
    @@ -526,6 +525,30 @@ The other way to factor the complexity is to make it so that `async` functions d
    This model provides a ton of advantages: it is arguably the right defaults for the vast majority of clients (reducing boilerplate and syntactic noise), provides the ability for the importer and experts to get what they want. The only downside of is that it is a less obvious design than have two orthogonal axes, but in the opinion of the proposal author, this is probably the right set of tradeoffs.


    Source Compatibility
    --------------------

    This is a generally additive feature, but it does turn take `async` and `await` as keywords, so it will break code that uses them as identifiers. This is expected to have very minor impact: the most pervasive use of `async` as an identifier occurs in code that works with dispatch queues, but fortunately keywords are allowed as qualified member names, so code like this doesn't need any change:

    ```
    myQueue.async { ... }
    ```

    That said, there could be obscure cases that break. One example that occurs in the Swift testsuite is of the form:

    ```
    extension DispatchQueue {
    func myThing() {
    async {
    ...
    }
    }
    }
    ```

    This can be addressed by changing the code to use `self.async` or backticks. The compiler should be able to detect a large number of these cases and produce a fixit.


    Potential Future Directions
    ---------------------------

  26. @lattner lattner revised this gist Aug 4, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -523,7 +523,7 @@ The other way to factor the complexity is to make it so that `async` functions d
    (Int) async(nonthrowing) -> Int // Asynchronous function, doesn't throw.
    ```

    This provides arguably the right defaults and low boilerplate, provides the ability for the importer and experts to get what they want, but is a bit strange.
    This model provides a ton of advantages: it is arguably the right defaults for the vast majority of clients (reducing boilerplate and syntactic noise), provides the ability for the importer and experts to get what they want. The only downside of is that it is a less obvious design than have two orthogonal axes, but in the opinion of the proposal author, this is probably the right set of tradeoffs.


    Potential Future Directions
  27. @lattner lattner revised this gist Aug 4, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -189,7 +189,7 @@ Today, function types can be normal or `throw`ing. This proposal extends them t
    (Int) async throws -> Int // #4: Asynchronous function, can also throw.
    ```

    The function types have the obvious subtype relationship between them: #1 can implicitly convert to #2 (as it does today) as well as #3/#4. #2 and #3 can convert to #4.
    Just as a normal function (#1) will implicitly convert to a throwing function (#2), an async function (#3) implicitly converts to a throwing async function (#4).

    On the function declaration side of the things, you can declare a function as being asynchronous just as you declare it to be throwing, but use the `async` keyword:

  28. @lattner lattner revised this gist Aug 3, 2017. 1 changed file with 26 additions and 36 deletions.
    62 changes: 26 additions & 36 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -22,7 +22,7 @@ To provide motivation for why it is important to do something here, lets look at
    Sequence of simple operations is unnaturally composed in the nested blocks.

    ```
    func makeSandwich1(completionBlock: (result:Sandwich)->Void) {
    func makeSandwich1(completionBlock: (result: Sandwich) -> Void) {
    cutBread { bread in
    cutCheese { cheese in
    cutHam { ham in
    @@ -46,7 +46,7 @@ This "pyramid of doom" makes it difficult to keep track of code that is running,
    Handling errors becomes difficult and very verbose.

    ```
    func makeSandwich2(completionBlock: (result:Sandwich?, error:NSError?)->Void) {
    func makeSandwich2(completionBlock: (result:Sandwich?, error:NSError?) -> Void) {
    cutBread { bread, error in
    guard let bread = bread else {
    completionBlock(nil, error)
    @@ -150,22 +150,9 @@ for i in 1...10 {
    }
    ```

    However, if you want to write this as a Swift sequence, you have to define this as something that incrementally produces values, you have to write this in a much more awkward way, for example something like:
    However, if you want to write this as a Swift sequence, you have to define this as something that incrementally produces values. There are multiple ways to do this (e.g. using `AnyIterator`, or the `sequence(state:,next:)` functions), but none of them approach the clarity and obviousness of the imperative form.

    ```
    func getSequence() -> AnySequence<Int> {
    var i = 1
    let seq = sequence(first: 1) { (last : Int) -> Int? in
    i += 1
    return i < 11 ? i*i : nil
    }
    return AnySequence(seq)
    }
    ```

    In languages that have generators, you can write something more close to this, which is much easier to reason about:
    In contrast, languages that have generators allow you to write something more close to this:

    ```
    func getSequence() -> AnySequence<Int> {
    @@ -178,6 +165,9 @@ func getSequence() -> AnySequence<Int> {
    }
    ```

    It is the responsibility of the compiler to transform the function into a form that incrementally produces values, by producing a state machine.



    The Coroutine transformation
    ----------------------------
    @@ -190,16 +180,18 @@ This proposal adds general coroutine support to Swift, biasing the nomenclature
    Async semantics
    ---------------

    Today, function types can be normal or `throw`ing. This proposal extends them to also be allowed to be `async`. These are all valid function types, with the obvious subtype relations between them:
    Today, function types can be normal or `throw`ing. This proposal extends them to also be allowed to be `async`. These are all valid function types:

    ```
    (Int) -> Int // Normal function
    (Int) throws -> Int // Throwing function
    (Int) async -> Int // Asynchronous function
    (Int) async throws -> Int // Asynchronous function, can also throw.
    (Int) -> Int // #1: Normal function
    (Int) throws -> Int // #2: Throwing function
    (Int) async -> Int // #3: Asynchronous function
    (Int) async throws -> Int // #4: Asynchronous function, can also throw.
    ```

    Just as you can define a function as being `throw`ing, you can also opt it into coroutine semantics by using the `async` keyword and replacing closure argument with the usual return type:
    The function types have the obvious subtype relationship between them: #1 can implicitly convert to #2 (as it does today) as well as #3/#4. #2 and #3 can convert to #4.

    On the function declaration side of the things, you can declare a function as being asynchronous just as you declare it to be throwing, but use the `async` keyword:

    ```
    func makeSandwich() async -> Sandwich { ... }
    @@ -246,11 +238,9 @@ In the above example all four operations will start one after another, and the u
    func future<T>(fn : () async -> T) -> Future<T> { ... }
    ```

    ... the function will wait for completion of every one of them before returning a sandwich. In other words, `await` indicates that the execution of the current flow may be interupted to wait for a computed value. Note In order to reduce mistakes and to help maintainers of code, the compiler requires calls to asynchronous functions to be "marked" with the `await` keyword (which marks a point which will implicitly wait) even though that is the default behavior. This serves as a marker for readers of the code (similar to the extant `try` keyword in Swift) to allow maintainers to more easily reason about control flow in the function.

    Note that `await` does not block the thread, it only tells compiler to organize the remaining calls as a continuation of the awaited operation. There is no risk of deadlock when using `await` and it is not possible to use it to make the current thread wait for the result.
    ... the function will wait for completion of every one of them before returning a sandwich. Note that `await` does not block flow of execution: if the value is not yet ready, execution of the current `async` function is suspended, and control flow passes to something higher up in the stack.

    Finally, you are only allowed to invoke an `async` function from within another `async` function or closure. By analogy to Swift 2 error handling, you cannot call a throwing function unless you're in a throwing function or inside of a `do/catch` block.
    Finally, you are only allowed to invoke an `async` function from within another `async` function or closure. This follows the model of Swift 2 error handling, where you cannot call a throwing function unless you're in a throwing function or inside of a `do/catch` block.


    Error handling
    @@ -280,7 +270,7 @@ func makeSandwich3() async throws -> Sandwich {
    let cheese = try await cutCheese()
    let ham = try await cutHam()
    let tomato = try await cutTomato()
    return Sandwich(bread + cheese + ham + tomato)
    return Sandwich([bread, cheese, ham, tomato])
    }
    ```

    @@ -300,7 +290,7 @@ func makeSandwich4() async throws -> Sandwich {
    let ham = try await hamFuture
    let tomato = try await tomatoFuture
    return Sandwich(bread + cheese + ham + tomato)
    return Sandwich([bread, cheese, ham, tomato])
    }
    ```
    The above example the unevaluated computation are wrapped into a "future" value through the following standard library function overload:
    @@ -324,7 +314,7 @@ func makeSandwich() async throws -> Sandwich {
    let cheese = try await cutCheese()
    let ham = try await cutHam()
    let tomato = try await cutTomato()
    return Sandwich(bread + cheese + ham + tomato)
    return Sandwich([bread, cheese, ham, tomato])
    }
    ```

    @@ -335,7 +325,7 @@ Completion handler APIs may have multiple result arguments (not counting an erro

    ```
    // Before
    func makeClubSandwich(completionHandler: (part1:Sandwich?, part2:Sandwich?, error:NSError?)->Void)
    func makeClubSandwich(completionHandler: (part1: Sandwich?, part2: Sandwich?, error: NSError?) -> Void)
    // After
    func makeSandwich() async throws -> (Sandwich, Sandwich)
    @@ -469,10 +459,10 @@ func makeSandwichWithErrorOldWay(completionHandler: (Sandwich?, NSError?) -> Voi

    Most Objective-C APIs should be automatically imported as `async` by the compiler (described below), however some APIs may not be automatically importable. If so, it would make sense to add an API (possibly in the GCD layer) to enable this. This is a framework level change though, so it is beyond the scope of this proposal.

    Cancellation
    ------------
    Cancellation Example
    --------------------

    There is no built-in language support for scheduling tasks and cancellation. Cancellation must be implemented by the programmer where needed. However, coupled with improved error handling, implementing cancellation becomes elegant:
    As in Swift 4, this proposal includes no built-in language support for scheduling tasks or cancellation. Cancellation should continued to be implemented by the programmer where needed, and there are many different patterns that can be applied. However, coupled with improved error handling, implementing common cancellation patterns becomes elegant:

    ```
    @IBAction func makeSandwich(sender: AnyObject) {
    @@ -482,7 +472,7 @@ There is no built-in language support for scheduling tasks and cancellation. Can
    let cheese = try await kitchen.cutCheese()
    let ham = try await kitchen.cutHam()
    let tomato = try await kitchen.cutTomato()
    eat(Sandwich(bread + cheese + ham + tomato))
    eat(Sandwich(bread, cheese, ham, tomato]))
    } catch CocoaErrorDomain.UserCancelled {
    // Ignore, user quit the kitchen.
    } catch {
    @@ -497,7 +487,7 @@ There is no built-in language support for scheduling tasks and cancellation. Can
    }
    ```

    Internally, `kitchen` may use `NSOperations` or custom `cancelled` flag.
    Internally, `kitchen` may use `NSOperations` or custom `cancelled` flag. The intent of this section is to give a single example of how to approach this, not to define a normative or all-encompassing approach that should be used in all cases.


    Alternate Design Options
  29. @lattner lattner revised this gist Aug 3, 2017. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -10,7 +10,7 @@ This paper introduces a first class [Coroutine model](https://en.wikipedia.org/w

    It is important to understand that this is proposing compiler support that is completely concurrency runtime-agnostic. This proposal does not include a new runtime model (like "actors") - it works just as well with GCD as with pthreads or another API. The only runtime support required is the definition of a couple of types and compiler support logic for manipulating the implicitly generated closures.

    This proposal is derived from a similar proposal written by [Oleg Andreev](https://github.com/oleganza), available [here](https://gist.github.com/oleganza/7342ed829bddd86f740a). It has been significantly rewritten by [Chris Lattner](https://github.com/lattner), with input from [Joe Groff](https://github.com/jckarter).
    This is derived from an earlier proposal written by [Oleg Andreev](https://github.com/oleganza), available [here](https://gist.github.com/oleganza/7342ed829bddd86f740a). It has been significantly rewritten by [Chris Lattner](https://github.com/lattner), with input from [Joe Groff](https://github.com/jckarter).

    Common problems with completion handlers
    ----------------------------------------
  30. @lattner lattner revised this gist Aug 3, 2017. 1 changed file with 83 additions and 77 deletions.
    160 changes: 83 additions & 77 deletions async_swift_proposal.md
    View file
    Original file line number Diff line number Diff line change
    @@ -4,28 +4,30 @@ Async/Await for Swift
    * Proposal: SE-XXXX
    * Author: [Chris Lattner](https://github.com/lattner)

    Modern Cocoa development involves a lot of asynchronous programming using blocks and completion handlers, but block-based APIs are hard to use when number of operations grows and dependencies between them become more complicated. This proposal describes a language extension to make this a lot more natural and less error prone.
    Modern Cocoa development involves a lot of asynchronous programming using closures and completion handlers, but completion handlers APIs are hard to use when number of operations grows and dependencies between them become more complicated. This proposal describes a language extension to make this a lot more natural and less error prone.

    This paper introduces first class *asynchronous semantics* and *Futures* to Swift, which are concepts [available in many other languages now](https://en.wikipedia.org/wiki/Await) (including C#, Javascript, Python, and others). Functions can opted into to being *async*, allowing the programmer can compose complex logic involving asynchronous operations - while the compiler produces necessary closures and state machines to implement that logic.
    This paper introduces a first class [Coroutine model](https://en.wikipedia.org/wiki/Coroutine) to Swift. Further, it provides standard library affordances to expose these as *asynchronous semantics* and *Futures* to Swift, which are concepts [available in many other languages now](https://en.wikipedia.org/wiki/Await) (including C#, Javascript, Python, and others). Functions can opted into to being *async*, allowing the programmer can compose complex logic involving asynchronous operations - while the compiler produces necessary closures and state machines to implement that logic.

    This proposal does not propose new runtime model, nor "actors" or "coroutines". It is important to understand that this is proposing compiler support that is completely runtime-agnostic: it works just as well with GCD as with pthreads or some other API. The only runtime support required is the definition of a couple of types and compiler support logic for manipulating the implicitly generated closures.
    It is important to understand that this is proposing compiler support that is completely concurrency runtime-agnostic. This proposal does not include a new runtime model (like "actors") - it works just as well with GCD as with pthreads or another API. The only runtime support required is the definition of a couple of types and compiler support logic for manipulating the implicitly generated closures.

    An initial version of this document was written by [Oleg Andreev](https://github.com/oleganza), available [here](https://gist.github.com/oleganza/7342ed829bddd86f740a). It has been significantly rewritten by [Chris Lattner](https://github.com/lattner).
    This proposal is derived from a similar proposal written by [Oleg Andreev](https://github.com/oleganza), available [here](https://gist.github.com/oleganza/7342ed829bddd86f740a). It has been significantly rewritten by [Chris Lattner](https://github.com/lattner), with input from [Joe Groff](https://github.com/jckarter).

    Common problems with completion handlers
    ----------------------------------------

    To provide motivation for why it is important to do something here, lets look at a number of gross problems that Cocoa (and server/cloud) programmers frequently face.

    #### Problem 1: Pyramid of doom

    Sequence of simple operations is unnaturally composed in the nested blocks.

    ```
    func makeSandwich1(completionBlock: (result:Sandwich)->Void) {
    cutBread { buns in
    cutCheese { cheeseSlice in
    cutHam { hamSlice in
    cutTomato { tomatoSlice in
    completionBlock(Sandwich([buns, cheeseSlice, hamSlice, tomatoSlice]))
    cutBread { bread in
    cutCheese { cheese in
    cutHam { ham in
    cutTomato { tomato in
    completionBlock(Sandwich([bread, cheese, ham, tomato]))
    }
    }
    }
    @@ -37,14 +39,16 @@ makeSandwich1 { sandwich in
    }
    ```

    This "pyramid of doom" makes it difficult to keep track of code that is running, and the stack of closures leads to many second order effects.

    #### Problem 2: Verbosity and old-style error handling

    Handling errors becomes difficult and very verbose.

    ```
    func makeSandwich2(completionBlock: (result:Sandwich?, error:NSError?)->Void) {
    cutBread { buns, error in
    guard let buns = buns else {
    cutBread { bread, error in
    guard let bread = bread else {
    completionBlock(nil, error)
    return
    }
    @@ -63,7 +67,7 @@ func makeSandwich2(completionBlock: (result:Sandwich?, error:NSError?)->Void) {
    completionBlock(nil, error)
    return
    }
    completionBlock(Sandwich([buns, cheeseSlice, hamSlice, tomatoSlice]), nil)
    completionBlock(Sandwich([bread, cheeseSlice, hamSlice, tomatoSlice]), nil)
    }
    }
    }
    @@ -81,15 +85,15 @@ makeSandwich2 { sandwich, error in

    #### Problem 3: Conditional execution is hard and error-prone

    When trying to conditionally execute some asynchronous function, it becomes a huge pain. Half of the code must be wrapped in a helper "continuation" closure that is conditionally executed.
    Conditionally executing an asynchronous function is a huge pain. Perhaps the best approach is to write half of the code in a helper "continuation" closure that is conditionally executed, like this:

    ```
    func makeSandwich5(recipient:Person, completionBlock: (result:Sandwich)->Void) {
    let continuation:(contents:SandwichContents)->Void = {
    func makeSandwich5(recipient: Person, completionBlock: (result: Sandwich) -> Void) {
    let continuation: (contents: SandwichContents) -> Void = {
    // ... continue and call completionBlock eventually
    }
    if !recipient.isVegeterian {
    continuation(mashroom)
    continuation(mushroom)
    } else {
    cutHam { hamSlice in
    continuation(hamSlice)
    @@ -98,19 +102,18 @@ func makeSandwich5(recipient:Person, completionBlock: (result:Sandwich)->Void) {
    }
    ```

    #### Problem 4: Many mistakes are easy to make

    #### Problem 4: Forget to call a completion handler

    It's easy to bail out by simply returning without calling the appropriate block. When forgotten, the issue is very hard to debug.
    It's easy to bail out by simply returning without calling the appropriate block. When forgotten, the issue is very hard to debug:

    ```
    func makeSandwich3(completionBlock: (result:Sandwich?, error:NSError?)->Void) {
    cutBread { buns, error in
    guard let buns = buns else {
    func makeSandwich3(completionBlock: (result: Sandwich?, error: NSError?) -> Void) {
    cutBread { bread, error in
    guard let bread = bread else {
    return // <- forgot to call the block
    }
    cutCheese { cheeseSlice, error in
    guard let cheeseSlice = cheeseSlice else {
    cutCheese { cheese, error in
    guard let cheese = cheese else {
    return // <- forgot to call the block
    }
    ...
    @@ -119,14 +122,11 @@ func makeSandwich3(completionBlock: (result:Sandwich?, error:NSError?)->Void) {
    }
    ```


    #### Problem 5: Forget to return after calling a completion handler

    When you do not forget to call the block, you can still forget to return after that.
    Thankfully `guard` syntax protects against that to some degree, but it's not always relevant.

    ```
    func makeSandwich4(recipient:Person, completionBlock: (result:Sandwich?, error:NSError?)->Void) {
    func makeSandwich4(recipient:Person, completionBlock: (result: Sandwich?, error: NSError?) -> Void) {
    if recipient.isVegeterian {
    if let sandwich = cachedVegeterianSandwich {
    completionBlock(cachedVegeterianSandwich) // <- forgot to return after calling the block
    @@ -136,11 +136,11 @@ func makeSandwich4(recipient:Person, completionBlock: (result:Sandwich?, error:N
    }
    ```

    #### Problem 6: Because Completion handlers are awkward, too many APIs are defined synchronously
    #### Problem 5: Because Completion handlers are awkward, too many APIs are defined synchronously

    This is hard to quantify, but the author believes that the awkwardness of defining and using asynchronous APIs (using completion handlers) has led to many APIs being defined with apparently synchronous behavior, even when they can block. This can lead to problematic performance and responsiveness problems in UI applications - e.g. spinning cursor. It can also lead to the definition of APIs that cannot be used when asynchrony is critical to achieve scale, e.g. on the server.

    #### Problem 7: Other "resumable" computations are awkward to define
    #### Problem 6: Other "resumable" computations are awkward to define

    The problems described above are on specific case of a general class of problems involving "resumable" computations. For example, if you want to write code that produces a list of squares of numbers, you might write something like this:

    @@ -182,24 +182,24 @@ func getSequence() -> AnySequence<Int> {
    The Coroutine transformation
    ----------------------------

    These problem have been faced in many systems and many languages, and the abstraction of [Coroutines](https://en.wikipedia.org/wiki/Coroutine) is a standard way to address them. Without delving too much into theory, Coroutines are an extension of basic functions that allow a function to return a value *or be suspended*. They can be used to implement generators, asynchronous models, and other capabilities, and there is a large body of work on the theory, implementation, and optimization of them.
    These problem have been faced in many systems and many languages, and the abstraction of [Coroutines](https://en.wikipedia.org/wiki/Coroutine) is a standard way to address them. Without delving too much into theory, Coroutines are an extension of basic functions that allow a function to return a value *or be suspended*. They can be used to implement generators, asynchronous models, and other capabilities - there is a large body of work on the theory, implementation, and optimization of them.

    This proposal adds general coroutine support to Swift, biasing the nomenclature and terminology towards the most common use-case: defining and using asynchronous APIs, eliminating many of the problems working with completion handlers. The choice of terminology (`async` vs `yields`) is a bikeshed topic which needs to be addressed, but isn't pertinent to the core semantics of the model.
    This proposal adds general coroutine support to Swift, biasing the nomenclature and terminology towards the most common use-case: defining and using asynchronous APIs, eliminating many of the problems working with completion handlers. The choice of terminology (`async` vs `yields`) is a bikeshed topic which needs to be addressed, but isn't pertinent to the core semantics of the model. See [Alternate Design Options][#alternate-design-options] at the end for an exploration of syntactic options in this space.


    Async semantics
    ---------------

    Today, function types can be normal, `throw`ing, and this proposal extends them to also be allowed to be `async`. These are all valid function types:
    Today, function types can be normal or `throw`ing. This proposal extends them to also be allowed to be `async`. These are all valid function types, with the obvious subtype relations between them:

    ```
    (Int) -> Int // Normal function
    (Int) throws -> Int // Throwing function
    (Int) async -> Int // Asynchronous function
    (Int) async throws -> Int // Asynchronous function, can also throw.
    (Int) -> Int // Normal function
    (Int) throws -> Int // Throwing function
    (Int) async -> Int // Asynchronous function
    (Int) async throws -> Int // Asynchronous function, can also throw.
    ```

    Just as you can define a function as being throwing, you can also opt it into *async semantics* by using the `async` keyword and replacing closure argument with the usual return type:
    Just as you can define a function as being `throw`ing, you can also opt it into coroutine semantics by using the `async` keyword and replacing closure argument with the usual return type:

    ```
    func makeSandwich() async -> Sandwich { ... }
    @@ -209,7 +209,7 @@ func makeSandwich(completionHandler: (result: Sandwich) -> Void) { ... }
    ```

    `async` functions can use the new `await` keywords to invoke other asynchronous functions
    in a simple imperative style. Now the first example can be rewritten in a more natural way:
    in a simple imperative style. Now the first example (from the pyramid of doom explanation, above) can be rewritten in a more natural way:

    ```
    func cutBread() async -> Bread
    @@ -226,12 +226,12 @@ func makeSandwich1() async -> Sandwich {
    }
    ```

    Under the hood, the compiler rewrites this code using nested closures like in example `makeSandwich1`. Note that every operation starts only after the previous one has completed.
    Under the hood, the compiler rewrites this code using nested closures like in example `makeSandwich1` above. Note that every operation starts only after the previous one has completed.

    To allow parallel execution, move `await` from the call to the result when it is needed, and wrap the asynchronous calls in a `Future`, which can itself be `await`d on later.
    To allow parallel execution, move `await` from the call to the result when it is needed, and wrap the asynchronous calls in a "future", which can itself be `await`d on later.

    ```
    async func makeSandwich2() -> Sandwich {
    func makeSandwich2() async -> Sandwich {
    let bread = future { cutBread() }
    let cheese = future { cutCheese() }
    let ham = future { cutHam() }
    @@ -246,7 +246,7 @@ In the above example all four operations will start one after another, and the u
    func future<T>(fn : () async -> T) -> Future<T> { ... }
    ```

    and then the function will wait for completion of every one of them before returning a sandwich. In other words, `await` indicates that the execution of the current flow may be interupted to wait for a computed value. Note In order to reduce mistakes and to help maintainers of code, the compiler requires calls to asynchronous functions to be "marked" with the `await` keyword (which marks a point which will implicitly wait) even though that is the default behavior. This serves as a marker for readers of the code (similar to the extant `try` keyword in Swift) to allow maintainers to more easily reason about control flow in the function.
    ... the function will wait for completion of every one of them before returning a sandwich. In other words, `await` indicates that the execution of the current flow may be interupted to wait for a computed value. Note In order to reduce mistakes and to help maintainers of code, the compiler requires calls to asynchronous functions to be "marked" with the `await` keyword (which marks a point which will implicitly wait) even though that is the default behavior. This serves as a marker for readers of the code (similar to the extant `try` keyword in Swift) to allow maintainers to more easily reason about control flow in the function.

    Note that `await` does not block the thread, it only tells compiler to organize the remaining calls as a continuation of the awaited operation. There is no risk of deadlock when using `await` and it is not possible to use it to make the current thread wait for the result.

    @@ -270,23 +270,23 @@ func makeSandwich(completionHandler: (result:Sandwich?, error:NSError?)->Void)
    Our example thus becomes (compare with the example `makeSandwich1`):

    ```
    async func cutBread() throws -> Bread
    async func cutCheese() throws -> Cheese
    async func cutHam() throws -> Ham
    async func cutTomato() throws -> Vegetable
    func cutBread() async throws -> Bread
    func cutCheese() async throws -> Cheese
    func cutHam() async throws -> Ham
    func cutTomato() async throws -> Vegetable
    func makeSandwich3() async throws -> Sandwich {
    let bread = await cutBread()
    let cheese = await cutCheese()
    let ham = await cutHam()
    let tomato = await cutTomato()
    let bread = try await cutBread()
    let cheese = try await cutCheese()
    let ham = try await cutHam()
    let tomato = try await cutTomato()
    return Sandwich(bread + cheese + ham + tomato)
    }
    ```

    Internally, compiler rewrites the code using closures and exits early if any intermediate call throws an error. Every `try await` will serialize execution, so `cutCheese` will only begin after `cutBread` completes.

    To enable throwing async calls execute in parallel, `try await` must be used after all necessary operations has begun.
    To enable `throw`ing `async` calls execute in parallel, `try await` must be used after all necessary operations has begun.

    ```
    func makeSandwich4() async throws -> Sandwich {
    @@ -303,7 +303,7 @@ func makeSandwich4() async throws -> Sandwich {
    return Sandwich(bread + cheese + ham + tomato)
    }
    ```
    The above example the unevaluated computation are wrapped into a `Future` value through the following standard library function overload:
    The above example the unevaluated computation are wrapped into a "future" value through the following standard library function overload:

    ```
    func future<T>(fn : () async throws -> T) -> ThrowingFuture<T> { ... }
    @@ -314,7 +314,7 @@ In this example we begin all operations at once and then first wait for completi
    Behavior of `defer`
    -------------------

    `defer` continues to be bound by the programmer visible syntactic scope. The deferred statement is executed when the the current scope is exited. For example, when asynchronous operations complete and a return is hit, when an error is thrown, etc.
    There is no change to the semantics of `defer`, it continues to be bound by the programmer visible syntactic scope. The deferred statement is executed when the the current scope is exited. For example, when asynchronous operations complete and a return is hit, when an error is thrown, etc.

    ```
    func makeSandwich() async throws -> Sandwich {
    @@ -344,7 +344,7 @@ func makeSandwich() async throws -> (Sandwich, Sandwich)
    Future types
    ------------

    Because the coroutine transformation described in this design lives at the level of function types, it can be applied to many different applications that benefit from the inversion of control that the coroutine transformation applies. That said, the most useful application is to provide `async`/`await` as a better way to work with asynchronous functions specifically. Similar APIs could be used for other applications (e.g. using generators to define more convenient iterator patterns), but those are beyond the scope of this proposal.
    Because the coroutine transformation described in this design lives at the level of function types, it can be applied to many different applications that benefit from the inversion of control that the coroutine transformation provides (e.g. generators). That said, the most important first application is to provide `async`/`await` as a better way to work with asynchronous functions. Similar APIs could be used for other applications, but those are beyond the scope of this proposal.

    Asynchronous semantics are implemented with [futures](https://en.wikipedia.org/wiki/Futures_and_promises) — types that represent a future value which may not be resolved yet. Extending the standard library with the following declarations allow programmers to reason about this and work with them:

    @@ -358,10 +358,12 @@ func future<T>(fn : () async throws -> T) -> ThrowingFuture<T> { ... }

    There are two types to represent a value that will be computed in the future: `Future<T>` for non-throwing asynchronous functions and `ThrowingFuture<T>` for throwing ones. It is expected that there is some magic used to allow the `await` keyword to work on a value of the Future types (e.g. an `Awaitable` protocol that both types conform to).

    `Future` and `ThrowingFuture` are classes because they have reference semantics. Of course, other designs are possible. In many trivial cases, instances of these classes could be completely optimized away, assuming the definition of these types are `fragile` (in the resilience model sense).

    Conversion of Objective-C APIs
    ------------------------------

    Details are beyond the scope of this proposal, but it is important to enhance the Objective-C importer to project completion-handler based APIs into `async` forms. This is a transformation comparable to how `NSError**` functions are imported as `throws` functions. Having the importer do this means that Cocoa APIs will be modernized en-mass.
    Full details are beyond the scope of this proposal, but it is important to enhance the importer to project Objective-C completion-handler based APIs into `async` forms. This is a transformation comparable to how `NSError**` functions are imported as `throws` functions. Having the importer do this means that many Cocoa APIs will be modernized en-mass.

    There are multiple possible designs for this with different tradeoffs. The maximally source compatible way to do this is to import completion handler-based APIs in two forms: both the completion handler and the `async` form. For example, given:

    @@ -390,11 +392,11 @@ There are many details that should be defined as part of this importing process
    Sync and async functions interoperability
    -----------------------------------------

    We have already seen how asynchronous functions call synchronous and asynchronous ones, but we haven't explored how synchronous ones (e.g. event handlers) call async ones and how async ones can wrap old-fashioned functions still written with completion handlers.
    We have already seen how asynchronous functions call synchronous and asynchronous ones, but we haven't explored how synchronous ones (e.g. event handlers) call `async` ones and how `async` ones can wrap old-fashioned functions still written with completion handlers.

    #### Calling an async function from a non-async function

    Because `async/await` fundamentally transforms the control flow within a function, you're only generally allowed to call an `async` function from within another `async` function. This works well until you get all the way to the top of the stack, e.g. an event handler (like an `IBAction`). Those are void-returning functions which are not themselves `async`, and it is important to be able to perform asynchronous work from within them. For example, if you were using a completion handler, your code might look like this:
    Because `async/await` fundamentally transforms the control flow within a function, you're only generally allowed to call an `async` function from within another `async` function. This works well until you get all the way to the top of the stack, e.g. an event handler (like an `IBAction`). Those are `Void`-returning functions which are not themselves `async`, and it is important to be able to perform asynchronous work from within them. For example, if you were using a completion handler, your code might look like this:

    ```
    @IBAction func buttonDidClick(sender:AnyObject) {
    @@ -407,9 +409,9 @@ Because `async/await` fundamentally transforms the control flow within a functio
    }
    ```

    This is an essential pattern, but is itself sort of odd: an `async` operation is being fired off immediately (#1), then runs the subsequent code (#3), and the completion handler (#2) runs at some time later -- on some queue (often the main one). This pattern frequently requires mutating global state or making assumptions about which queue the completion handler is run on. While this may not be very safe, it is essential that the model encompasses this pattern, because it is a practical necessity.
    This is an essential pattern, but is itself sort of odd: an `async` operation is being fired off immediately (#1), then runs the subsequent code (#3), and the completion handler (#2) runs at some time later -- on some queue (often the main one). This pattern frequently leads to mutation of global state (as in this example) or to making assumptions about which queue the completion handler is run on. While this may not be very safe, it is essential that the model encompasses this pattern, because it is a practical necessity in Cocoa development.

    To address this, there should be a function that takes an async closure, starts it, but ignores the result. It is sufficient to use the existing Future type for this:
    To address this, there should be a function that takes an async closure, starts it, but ignores the result. With the design outlined above, it should be sufficient to use the existing `Future` type for this:

    ```
    @IBAction func buttonDidClick(sender:AnyObject) {
    @@ -437,7 +439,7 @@ However, this is extremely implicit. Instead, it may make sense to add somethin
    }
    ```

    The exact naming of this operation is sure to require extensive bikeshedding. For the purpose of this proposal, we simply observe that this is easy to build in the standard library if necessary.
    The exact naming of this operation is sure to require extensive bikeshedding. For the purpose of this proposal, we simply observe that this is easy to build the standard library if necessary.

    #### Wrapping async function in a block-based function

    @@ -463,14 +465,14 @@ func makeSandwichWithErrorOldWay(completionHandler: (Sandwich?, NSError?) -> Voi
    }
    ```

    #### Wrapping completion handler functions in async function
    #### Wrapping completion handler functions in `async` function

    Most Objective-C APIs should be automatically imported as `async` by the compiler (described below), however there are likely to be part of the long tail of APIs that cannot be automatically imported. It makes sense to add an API to GCD to enable this. This is a framework level change though, so it is beyond the scope of this proposal.
    Most Objective-C APIs should be automatically imported as `async` by the compiler (described below), however some APIs may not be automatically importable. If so, it would make sense to add an API (possibly in the GCD layer) to enable this. This is a framework level change though, so it is beyond the scope of this proposal.

    Cancellation
    ------------

    There is no built-in language support for scheduling tasks and cancellation. Cancellation must be implemented by the programmer where needed. However, coupled with improved error handling, implementing cancellation becomes easy:
    There is no built-in language support for scheduling tasks and cancellation. Cancellation must be implemented by the programmer where needed. However, coupled with improved error handling, implementing cancellation becomes elegant:

    ```
    @IBAction func makeSandwich(sender: AnyObject) {
    @@ -490,7 +492,7 @@ There is no built-in language support for scheduling tasks and cancellation. Can
    }
    }
    @IBAction func exitKitchen(sender:AnyObject) {
    @IBAction func exitKitchen(sender: AnyObject) {
    kitchen.cancel()
    }
    ```
    @@ -503,32 +505,32 @@ Alternate Design Options

    #### Spelling of `async` keyword

    Instead of spelling the function type modifier as `async`, it could be spelled as `yields`, since the functionality really is about coroutines, not about asynchrony by itself. The recommendation to use `async/await` is to make sure that the most common use case (asynchrony) uses industry standard terms. The other coroutine use cases would be much less common, at least according to the unscientific opinion of the proposal author.
    Instead of spelling the function type modifier as `async`, it could be spelled as `yields`, since the functionality really is about coroutines, not about asynchrony by itself. The recommendation to use `async/await` biases towards making sure that the most common use case (asynchrony) uses industry standard terms. The other coroutine use cases would be much less common, at least according to the unscientific opinion of the proposal author.

    #### Make `async` be a subtype of `throws` instead of orthogonal to it

    It would be a great simplification of the language model to make the `async` modifier on a function imply that the function is `throw`ing, instead of making them orthogonal modifiers. From an intuitive perspective, this makes sense because many of the sorts of operations that are asynchronous (e.g. loading a resource, talking to the network, etc) can also fail. Instead of four kinds of function type, we'd only have three:
    It would be a great simplification of the language model to make the `async` modifier on a function imply that the function is `throw`ing, instead of making them orthogonal modifiers. From an intuitive perspective, this makes sense because many of the sorts of operations that are asynchronous (e.g. loading a resource, talking to the network, etc) can also fail. There is also precedent from many other systems that use `async`/`await` for this. Instead of four kinds of function type, we'd only have three:

    ```
    (Int) -> Int // Normal function
    (Int) throws -> Int // Throwing function
    (Int) async -> Int // Asynchronous function, can also throw
    (Int) -> Int // Normal function
    (Int) throws -> Int // Throwing function
    (Int) async -> Int // Asynchronous function, can also throw
    ```


    Going this direction would simplify a lot of things: `ThrowingFuture` would go away, as would the overload of `future`. Arguably the `try` marker could be dropped from `try await`, because all `await`s would be known to throw. For user code, it would reduce the number of times you'd see the ugly `async throws` modifier stack.
    Going this direction would simplify a lot of things: `ThrowingFuture` would go away, as would the overload of `future`. Arguably the `try` marker could be dropped from `try await`, because all `await`s would be known to throw. For user code, it would directly reduce the number of times you'd see the ugly `async throws` modifier stack.

    There are two downsides to doing this: the first of which is that Cocoa has a number of completion handler APIs that cannot throw, and not having the ability to express that would make the importer story more complex. The second is that uses of these APIs would now cause pointless `do`/`catch` blocks which would be dead.

    #### Make `async` default to `throws`

    The other way to factor the complexity is to make it so that `async` functions default to throwing, but that it can be disabled with `nonthrowing` (or some other spelling). This provides this model:
    The other way to factor the complexity is to make it so that `async` functions default to `throw`ing, but still allow non-`throw`ing `async` functions to be expressed with `nonthrowing` (or some other spelling). This provides this model:

    ```
    (Int) -> Int // Normal function
    (Int) throws -> Int // Throwing function
    (Int) async -> Int // Asynchronous function, can also throw.
    (Int) async(nonthrowing) -> Int // Asynchronous function
    (Int) -> Int // Normal function
    (Int) throws -> Int // Throwing function
    (Int) async -> Int // Asynchronous function, can also throw.
    (Int) async(nonthrowing) -> Int // Asynchronous function, doesn't throw.
    ```

    This provides arguably the right defaults and low boilerplate, provides the ability for the importer and experts to get what they want, but is a bit strange.
    @@ -543,6 +545,10 @@ This proposal has been kept intentionally minimal, but there are many possible w

    Given the availability of convenient asynchrony in Swift, it would make sense to introduce new APIs to take advantage of it. Filesystem APIs are one example that would be great to see. The Swift on Server working group would also widely adopt these features.

    #### `rethrows` could be generalized to support potentially `async` operations

    The `rethrows` modifier exists in Swift to allow limited abstraction over function types by higher order functions. It would be possible to define a similar mechanism to allow abstraction over `async` operations as well.

    #### Array form of `await` and other operations

    There are many interesting APIs that could be added to support different sort of patterns,
    @@ -569,4 +575,4 @@ Thanks

    @oleganza thanked @groue and @pierlo for their feedback on his original proposal. Thanks to @oleganza for the original draft which influenced this!

    Thanks to Joe Groff for his extensive feedback which helped reshape this proposal into something that is a lot more general than it started as.
    Thanks to [Joe Groff](https://github.com/jckarter) for his extensive feedback which helped reshape this proposal into something that is a lot more general than it started as.