Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save ivanxpetrov/a4ed9b4ac1c63b4552c9a8acfc423854 to your computer and use it in GitHub Desktop.
Save ivanxpetrov/a4ed9b4ac1c63b4552c9a8acfc423854 to your computer and use it in GitHub Desktop.
Memory usage guidelines

Memory<T> usage guidelines

This document describes the relationship between Memory<T> and its related classes (MemoryPool<T>, IMemoryOwner<T>, etc.). It also describes best practices when accepting Memory<T> instances in public API surface. Following these guidelines will help developers write clear, bug-free code.

First, a tour of the basic exchange types

  • Span<T> is the basic exchange type that represents contiguous buffers. These buffers may be backed by managed memory (such as T[] or System.String). They may also be backed by unmanaged memory (such as via stackalloc or a raw void*). The Span<T> type is not heapable, meaning that it cannot appear as a field in classes, and it cannot be used across yield or await boundaries.

  • Memory<T> is a wrapper around an object that can generate a Span<T>. For instance, Memory<T> instances can be backed by T[], System.String (readonly), and even SafeHandle instances. Memory<T> cannot be backed by "transient" unmanaged memory; e.g., it is forbidden to back a Memory<T> with stackalloc. The Memory<T> type is heapable, meaning that it can appear as a field in a class, and it can be used across yield and await boundaries.

There are also ReadOnlySpan<T> and ReadOnlyMemory<T> types that correspond to read-only versions of Span<T> and Memory<T>, respectively.

Owners, consumers, and lifetime management

Let's stick a pin in Memory<T> for now and speak about buffers in more general terms. Since buffers can be passed around between APIs, and since buffers can sometimes be accessed from multiple threads, we need to introduce lifetime semantics. There are two core concepts.

The first concept is ownership. The owner of a buffer instance is responsible for lifetime management, including destroying the buffer when it is no longer in use. All buffers have a single owner. Generally the owner is the component which created the buffer or which received the buffer from a factory. Ownership can also be transferred; Component A can relinquish control of the buffer to Component B, at which point Component A may no longer use the buffer, and Component B becomes responsible for destroying the buffer when it is no longer in use.

The second concept is consumption. The consumer of a buffer instance is allowed to use the buffer instance, perhaps writing to or reading from it. Buffers have one consumer at a time unless some external synchronization mechanism is provided.

Importantly, the active consumer of a buffer is not necessarily the buffer's owner. Consider the following pseudocode, where the Buffer type is a stand-in for an arbitrary buffer type.

// Writes 'value' as a human-readable string to the output buffer.
void WriteInt32ToBuffer(int value, Buffer buffer);

// Prints the contents of the buffer to the console.
void PrintBufferToConsole(Buffer buffer);

// Application code
void Main()
{
    var buffer = CreateBuffer();
    try {
        int value = Int32.Parse(Console.ReadLine());
        WriteInt32ToBuffer(value, buffer);
        PrintBufferToConsole(buffer);
    } finally {
        buffer.Destroy();
    }
}

In this pseudocode, the Main method creates the buffer so becomes its owner, and Main is thus responsible for destroying the buffer when it's no longer in use. The buffer only ever has one consumer at a time (first WriteInt32ToBuffer, then PrintBufferToConsole), and neither of the consumers owns the buffer.

Memory<T> and lifetime management

At this point, let's reintroduce Memory<T> into the picture, along with one more type: IMemoryOwner<T>.

The type IMemoryOwner<T> is, as its name suggests, the unit of ownership of the associated Memory<T> instance. If a component has an IMemoryOwner<T> reference, then that component owns the buffer.

Memory<T> is itself the unit of consumption. If a component has a Memory<T> reference, then that component consumes the buffer.

To clarify this point, consider once again the earlier pseudocode, but let's now introduce real types into the system.

// Writes 'value' as a human-readable string to the output buffer.
void WriteInt32ToBuffer(int value, Memory<char> buffer);

// Prints the contents of the buffer to the console.
void PrintBufferToConsole(Memory<char> buffer);

// Application code
void Main()
{
    IMemoryOwner<char> owner = MemoryPool<char>.Shared.Rent();
    try {
        int value = Int32.Parse(Console.ReadLine());
        WriteInt32ToBuffer(value, owner.Memory);
        PrintBufferToConsole(owner.Memory);
    } finally {
        owner.Dispose();
    }

    // Alternatively, with 'using' syntax instead of 'try / finally'

    using (var owner = MemoryPool<char>.Shared.Rent())
    {
        int value = Int32.Parse(Console.ReadLine());
        WriteInt32ToBuffer(value, owner.Memory);
        PrintBufferToConsole(owner.Memory);
    }
}

Again, in this code, the Main method holds the reference to the IMemoryOwner<char> instance, so the Main method is the owner of the buffer. The WriteInt32ToBuffer and PrintBufferToConsole methods accept Memory<T> as a public API, therefore they consume the buffer. (And they only consume it one-at-a-time.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment