Skip to content

Instantly share code, notes, and snippets.

@KyleAMathews
Created November 6, 2025 18:03
Show Gist options
  • Save KyleAMathews/31d27c86ade8856e38f18c0cf24ac6da to your computer and use it in GitHub Desktop.
Save KyleAMathews/31d27c86ade8856e38f18c0cf24ac6da to your computer and use it in GitHub Desktop.
Electric E2E Test Infrastructure Documentation - Complete Reference for TanStack DB Implementation

Electric E2E Test Infrastructure Analysis - Complete Documentation

Summary

You now have comprehensive documentation on Electric's e2e test setup, derived from examining their TypeScript client test infrastructure. The documentation is ready to be used as a blueprint for implementing similar patterns in TanStack DB.

What You'll Find Here

Four complementary documents totaling over 2400 lines and 60KB:

  1. ELECTRIC_E2E_PATTERNS.md - Deep dive into architecture, patterns, and design decisions
  2. QUICK_REFERENCE.md - Quick lookup guide with copy-paste templates
  3. ACTUAL_CODE_EXCERPTS.md - Real working code from Electric's test suite
  4. ELECTRIC_E2E_INDEX.md - Navigation guide and quick reference index

Quick Start

If you want to understand the overall architecture:

Start with ELECTRIC_E2E_PATTERNS.md sections 1-3

If you want to copy code and get going:

Start with QUICK_REFERENCE.md and ACTUAL_CODE_EXCERPTS.md

If you want both understanding and practical code:

  1. Read ELECTRIC_E2E_PATTERNS.md Section 8 (key architectural patterns)
  2. Copy from ACTUAL_CODE_EXCERPTS.md
  3. Reference QUICK_REFERENCE.md for any questions

Key Insights from Electric's Approach

Docker Orchestration

  • Postgres runs on port 54321, Electric server on 3000
  • tmpfs used for Postgres data directory (significant speed improvement)
  • Health check waits for server startup (10-second timeout)
  • Services orchestrated with depends_on for proper startup order

Database Isolation

  • Uses shared database with per-test schema isolation (electric_test)
  • Each test gets a unique table name: "table name for {taskId}_{randomSuffix}"
  • Unique names aid debugging (shows which test created the table)
  • Single schema approach beats separate databases per test

Test Lifecycle Management

Four-level lifecycle:

  1. Global Setup (once per test run) - health check, schema creation
  2. Per-File Setup (vitest setup files)
  3. Per-Test Fixtures (setup/teardown for each test)
  4. Cleanup Functions (automatic via fixture teardown)

Fixture Composition

  • Uses Vitest's test.extend() for composable fixtures
  • Fixtures build on each other: testWithDb → testWithIssuesTable → custom extensions
  • Each fixture level adds new functionality while inheriting parent fixtures
  • Clear dependency chains make debugging easier

Parameterized Testing

  • Uses it.for() and describe.for() for systematic multi-configuration testing
  • Avoids code duplication by testing multiple modes (fetch vs SSE, etc.)
  • Template string interpolation in test names shows parameters

Serial Execution

  • fileParallelism: false prevents concurrent test execution
  • Essential for shared database safety
  • Makes debugging deterministic and easier

Most Important Design Decisions

  1. Schema-based isolation (not database-per-test) - more efficient
  2. Unique table names with task.id - prevents collisions and aids debugging
  3. Global setup with health check - ensures server readiness
  4. Fixture composition - enables reusable, composable test infrastructure
  5. Serial execution - critical for shared database
  6. Non-throwing cleanup - allows subsequent cleanup steps even if one fails
  7. Inline SQL for tables - tests are self-contained, no migration complexity

Configuration Values to Remember

Host: localhost
Port: 54321 (Postgres)
User: postgres
Password: password
Database: electric
Test Schema: electric_test
Server URL: http://localhost:3000
Health Check Endpoint: http://localhost:3000/v1/health
Timeout: 10 seconds
Execution: Serial (fileParallelism: false)

Copy-Paste Ready

All major patterns are provided in copy-paste form:

  • Docker Compose configuration
  • Global setup template
  • Fixture templates
  • Parameterized test examples
  • Health check implementation
  • Cleanup patterns

For TanStack DB Implementation

The documents include a specific replication checklist (ELECTRIC_E2E_PATTERNS.md Section 10):

  1. Create Docker Compose with Postgres + TanStack server
  2. Implement global-setup.ts with health check
  3. Create test context fixtures with test.extend()
  4. Set fileParallelism: false in vitest.config.ts
  5. Create unique table names per test with task.id
  6. Add parameterized tests with it.for() if needed
  7. Use AbortController for stream cleanup
  8. Document connection defaults

File Structure for TanStack DB (Based on Electric)

Recommended structure:

packages/your-package/
├── vitest.config.ts
├── test/
│   ├── support/
│   │   ├── global-setup.ts        # Health check + context
│   │   ├── test-context.ts        # Fixtures
│   │   └── test-helpers.ts        # Utilities
│   └── *.test.ts                  # Your tests

Key Files from Electric (for reference)

  • Docker: ~/.support/docker-compose.yml
  • Vitest: packages/typescript-client/vitest.config.ts
  • Global Setup: packages/typescript-client/test/support/global-setup.ts
  • Fixtures: packages/typescript-client/test/support/test-context.ts
  • Helpers: packages/typescript-client/test/support/test-helpers.ts
  • Tests: packages/typescript-client/test/*.test.ts

Document Statistics

Document Lines Size Purpose
ELECTRIC_E2E_PATTERNS.md 1028 29KB Comprehensive guide
ACTUAL_CODE_EXCERPTS.md 787 22KB Real code examples
QUICK_REFERENCE.md 334 7.7KB Quick lookup
ELECTRIC_E2E_INDEX.md 261 8.8KB Navigation guide
Total 2410 67.5KB Complete reference

Technologies Referenced

  • Vitest 3.0+ (test framework with fixture system)
  • Node 'pg' library (PostgreSQL client)
  • Docker (container orchestration)
  • TypeScript (type safety)
  • Vitest context injection (provide/inject)

How to Navigate

  1. New to e2e testing? Start with ELECTRIC_E2E_PATTERNS.md Section 1-3
  2. Want specific patterns? Use QUICK_REFERENCE.md or ELECTRIC_E2E_INDEX.md
  3. Need working code? Go to ACTUAL_CODE_EXCERPTS.md
  4. Lost? Check ELECTRIC_E2E_INDEX.md for topic mapping

Next Steps

  1. Review ELECTRIC_E2E_PATTERNS.md to understand the approach
  2. Copy relevant Docker configuration
  3. Create global-setup.ts using templates
  4. Implement test fixtures using the examples
  5. Write your first parameterized test
  6. Reference QUICK_REFERENCE.md for any questions

All code examples are actual implementations from Electric's test suite. Generated from: ~/programs/electric/packages/typescript-client/test Ready for TanStack DB implementation.

Actual Code Excerpts from Electric TypeScript Client Tests

All examples are taken directly from: ~/programs/electric/packages/typescript-client/test

1. Vitest Configuration (vitest.config.ts)

import { defineConfig } from 'vitest/config'

export default defineConfig({
  test: {
    globalSetup: `test/support/global-setup.ts`,
    setupFiles: [`vitest-localstorage-mock`],
    typecheck: { enabled: true },
    fileParallelism: false,
    coverage: {
      provider: `istanbul`,
      reporter: [`text`, `json`, `html`, `lcov`],
      include: [`**/src/**`],
    },
    reporters: [`default`, `junit`],
    outputFile: `./junit/test-report.junit.xml`,
    environment: `jsdom`,
  },
})

Key insights:

  • fileParallelism: false is essential for serial execution with shared database
  • Coverage reporters: istanbul with multiple output formats
  • junit output for CI/CD integration
  • TypeScript type checking enabled

2. Global Setup (test/support/global-setup.ts)

import type { GlobalSetupContext } from 'vitest/node'
import { makePgClient } from './test-helpers'

const url = process.env.ELECTRIC_URL ?? `http://localhost:3000`
const proxyUrl = process.env.ELECTRIC_PROXY_CACHE_URL ?? `http://localhost:3002`

// name of proxy cache container to execute commands against,
// see docker-compose.yml that spins it up for details
const proxyCacheContainerName = `electric_dev-nginx-1`
// path pattern for cache files inside proxy cache to clear
const proxyCachePath = `/var/cache/nginx/*`

// eslint-disable-next-line quotes -- eslint is acting dumb with enforce backtick quotes mode, and is trying to use it here where it's not allowed.
declare module 'vitest' {
  export interface ProvidedContext {
    baseUrl: string
    proxyCacheBaseUrl: string
    testPgSchema: string
    proxyCacheContainerName: string
    proxyCachePath: string
  }
}

function waitForElectric(url: string): Promise<void> {
  return new Promise<void>((resolve, reject) => {
    const timeout = setTimeout(
      () => reject(`Timed out waiting for Electric to be active`),
      10000
    )

    const tryHealth = async () =>
      fetch(`${url}/v1/health`)
        .then(async (res): Promise<void> => {
          if (!res.ok) return tryHealth()
          const { status } = (await res.json()) as { status: string }
          if (status !== `active`) return tryHealth()
          clearTimeout(timeout)
          resolve()
        })
        .catch((err) => {
          clearTimeout(timeout)
          reject(err)
        })

    return tryHealth()
  })
}

/**
 * Global setup for the test suite. Validates that our server is running, and creates and tears down a
 * special schema in Postgres to ensure clean slate between runs.
 */
export default async function ({ provide }: GlobalSetupContext) {
  await waitForElectric(url)

  const client = makePgClient()

  await client.connect()
  await client.query(`CREATE SCHEMA IF NOT EXISTS electric_test`)

  provide(`baseUrl`, url)
  provide(`testPgSchema`, `electric_test`)
  provide(`proxyCacheBaseUrl`, proxyUrl)
  provide(`proxyCacheContainerName`, proxyCacheContainerName)
  provide(`proxyCachePath`, proxyCachePath)

  return async () => {
    await client.query(`DROP SCHEMA electric_test CASCADE`)
    await client.end()
  }
}

Key insights:

  • Health check recursively polls /v1/health endpoint
  • 10-second timeout for server startup
  • Multiple environment variable overrides supported
  • Module augmentation for type-safe context injection
  • Cleanup function returns from main function

3. Test Context Fixtures (test/support/test-context.ts)

Base Database Client Fixture

export const testWithDbClient = test.extend<{
  dbClient: Client
  aborter: AbortController
  baseUrl: string
  pgSchema: string
  clearShape: ClearShapeFn
}>({
  dbClient: async ({}, use) => {
    const searchOption = `-csearch_path=${inject(`testPgSchema`)}`
    const client = makePgClient({ options: searchOption })
    await client.connect()
    await use(client)
    await client.end()
  },
  aborter: async ({}, use) => {
    const controller = new AbortController()
    await use(controller)
    controller.abort(`Test complete`)
  },
  baseUrl: async ({}, use) => use(inject(`baseUrl`)),
  pgSchema: async ({}, use) => use(inject(`testPgSchema`)),
  clearShape: async ({}, use) => {
    await use(
      async (
        table: string,
        options: {
          handle?: string
        } = {}
      ) => {
        const baseUrl = inject(`baseUrl`)
        const url = new URL(`${baseUrl}/v1/shape`)
        url.searchParams.set(`table`, table)

        if (options.handle) {
          url.searchParams.set(SHAPE_HANDLE_QUERY_PARAM, options.handle)
        }

        const resp = await fetch(url.toString(), { method: `DELETE` })

        if (!resp.ok) {
          // if we've been passed a shape handle then we should expect this delete call to succeed.
          if (resp.status === 404) {
            // the shape wasn't found, so maybe it wasn't created in the first place
          } else {
            console.error(
              await FetchError.fromResponse(resp, `DELETE ${url.toString()}`)
            )
            throw new Error(
              `Could not delete shape ${table} with ID ${options.handle}`
            )
          }
        }
      }
    )
  },
})

Issues Table Fixture (Extends testWithDbClient)

export const testWithIssuesTable = testWithDbClient.extend<{
  issuesTableSql: string
  issuesTableUrl: string
  issuesTableKey: string
  updateIssue: UpdateIssueFn
  deleteIssue: DeleteIssueFn
  insertIssues: InsertIssuesFn
  clearIssuesShape: ClearIssuesShapeFn
  waitForIssues: WaitForIssuesFn
}>({
  issuesTableSql: async ({ dbClient, task }, use) => {
    const tableName = `"issues for ${task.id}_${Math.random().toString(16).replace(`.`, `_`)}"`
    await dbClient.query(`
    DROP TABLE IF EXISTS ${tableName};
    CREATE TABLE ${tableName} (
      id UUID PRIMARY KEY,
      title TEXT NOT NULL,
      priority INTEGER NOT NULL
    );
    COMMENT ON TABLE ${tableName} IS 'Created for ${task.file?.name.replace(/'/g, `\``) ?? `unknown`} - ${task.name.replace(`'`, `\``)}';
  `)
    await use(tableName)
    await dbClient.query(`DROP TABLE ${tableName}`)
  },
  issuesTableUrl: async ({ issuesTableSql, pgSchema, clearShape }, use) => {
    const urlAppropriateTable = pgSchema + `.` + issuesTableSql
    await use(urlAppropriateTable)
    try {
      await clearShape(urlAppropriateTable)
    } catch (_) {
      // ignore - clearShape has its own logging
      // we don't want to interrupt cleanup
    }
  },
  issuesTableKey: ({ issuesTableSql, pgSchema }, use) =>
    use(`"${pgSchema}".${issuesTableSql}`),
  updateIssue: ({ issuesTableSql, dbClient }, use) =>
    use(({ id, title }) =>
      dbClient.query(`UPDATE ${issuesTableSql} SET title = $2 WHERE id = $1`, [
        id,
        title,
      ])
    ),
  deleteIssue: ({ issuesTableSql, dbClient }, use) =>
    use(({ id }) =>
      dbClient.query(`DELETE FROM ${issuesTableSql} WHERE id = $1`, [id])
    ),
  insertIssues: ({ issuesTableSql, dbClient }, use) =>
    use(async (...rows) => {
      const placeholders = rows.map(
        (_, i) => `($${i * 3 + 1}, $${i * 3 + 2}, $${i * 3 + 3})`
      )
      const { rows: rows_1 } = await dbClient.query(
        `INSERT INTO ${issuesTableSql} (id, title, priority) VALUES ${placeholders} RETURNING id`,
        rows.flatMap((x) => [x.id ?? uuidv4(), x.title, 10])
      )
      return rows_1.map((x) => x.id)
    }),

  clearIssuesShape: async ({ clearShape, issuesTableUrl }, use) => {
    use((handle?: string) => clearShape(issuesTableUrl, { handle }))
  },

  waitForIssues: ({ issuesTableUrl, baseUrl, aborter }, use) =>
    use(
      ({
        numChangesExpected,
        shapeStreamOptions,
      }: {
        numChangesExpected?: number
        shapeStreamOptions?: Partial<ShapeStreamOptions>
      }) =>
        waitForTransaction({
          baseUrl,
          table: issuesTableUrl,
          shapeStreamOptions,
          numChangesExpected,
          aborter,
        })
    ),
})

Key insights:

  • Tables named with task.id + random suffix for uniqueness
  • SQL comments include file name and test name for debugging
  • Fixtures can depend on parent fixtures
  • Helper functions wrap common operations
  • Error handling in cleanup doesn't throw

4. Test Helpers (test/support/test-helpers.ts)

import {
  ShapeStream,
  ShapeStreamInterface,
  ShapeStreamOptions,
} from '../../src/client'
import { Client, ClientConfig } from 'pg'
import { Message, Row } from '../../src/types'
import { isChangeMessage } from '../..//src'
import { isUpToDateMessage } from '../../src/helpers'

export function makePgClient(overrides: ClientConfig = {}) {
  return new Client({
    host: `localhost`,
    port: 54321,
    password: `password`,
    user: `postgres`,
    database: `electric`,
    options: `-csearch_path=electric_test`,
    ...overrides,
  })
}

export function forEachMessage<T extends Row<unknown>>(
  stream: ShapeStreamInterface<T>,
  controller: AbortController,
  handler: (
    resolve: () => void,
    message: Message<T>,
    nthDataMessage: number
  ) => Promise<void> | void
) {
  let unsub = () => {}
  return new Promise<void>((resolve, reject) => {
    let messageIdx = 0

    unsub = stream.subscribe(async (messages) => {
      for (const message of messages) {
        try {
          await handler(
            () => {
              controller.abort()
              return resolve()
            },
            message as Message<T>,
            messageIdx
          )
          if (isChangeMessage(message)) messageIdx++
        } catch (e) {
          controller.abort()
          return reject(e)
        }
      }
    }, reject)
  }).finally(unsub)
}

export async function waitForTransaction({
  baseUrl,
  table,
  numChangesExpected,
  shapeStreamOptions,
  aborter,
}: {
  baseUrl: string
  table: string
  numChangesExpected?: number
  shapeStreamOptions?: Partial<ShapeStreamOptions>
  aborter?: AbortController
}): Promise<Pick<ShapeStreamOptions, `offset` | `handle`>> {
  const waitAborter = new AbortController()
  if (aborter?.signal.aborted) waitAborter.abort()
  else aborter?.signal.addEventListener(`abort`, () => waitAborter.abort())
  const issueStream = new ShapeStream({
    ...(shapeStreamOptions ?? {}),
    url: `${baseUrl}/v1/shape`,
    params: {
      ...(shapeStreamOptions?.params ?? {}),
      table,
    },
    signal: waitAborter.signal,
    subscribe: true,
  })

  numChangesExpected ??= 1
  let numChangesSeen = 0
  await forEachMessage(issueStream, waitAborter, (res, msg) => {
    if (isChangeMessage(msg)) {
      numChangesSeen++
    }

    if (numChangesSeen >= numChangesExpected && isUpToDateMessage(msg)) {
      res()
    }
  })
  return {
    offset: issueStream.lastOffset,
    handle: issueStream.shapeHandle,
  }
}

5. Parameterized Tests (test/integration.test.ts)

import { describe, expect, inject, vi } from 'vitest'
import { v4 as uuidv4 } from 'uuid'
import { 
  testWithIssuesTable as it,
  testWithMultitypeTable as mit,
} from './support/test-context'

const BASE_URL = inject(`baseUrl`)

const fetchAndSse = [{ liveSse: false }, { liveSse: true }]

it(`sanity check`, async ({ dbClient, issuesTableSql }) => {
  const result = await dbClient.query(`SELECT * FROM ${issuesTableSql}`)

  expect(result.rows).toEqual([])
})

describe(`HTTP Sync`, () => {
  it.for(fetchAndSse)(
    `should work with empty shape/table (liveSSE=$liveSse)`,
    async ({ liveSse }, { issuesTableUrl, aborter }) => {
      // Get initial data
      const shapeData = new Map()
      const issueStream = new ShapeStream({
        url: `${BASE_URL}/v1/shape`,
        params: {
          table: issuesTableUrl,
        },
        subscribe: false,
        signal: aborter.signal,
        liveSse,
      })

      await new Promise<void>((resolve, reject) => {
        issueStream.subscribe((messages) => {
          messages.forEach((message) => {
            if (isChangeMessage(message)) {
              shapeData.set(message.key, message.value)
            }
            if (isUpToDateMessage(message)) {
              aborter.abort()
              return resolve()
            }
          })
        }, reject)
      })
      const values = [...shapeData.values()]

      expect(values).toHaveLength(0)
    }
  )

  it.for(fetchAndSse)(
    `should get initial data (liveSSE=$liveSse)`,
    async ({ liveSse }, { insertIssues, issuesTableUrl, aborter }) => {
      // Add an initial row.
      const uuid = uuidv4()
      await insertIssues({ id: uuid, title: `foo + ${uuid}` })

      // Get initial data
      const shapeData = new Map()
      const issueStream = new ShapeStream({
        url: `${BASE_URL}/v1/shape`,
        params: {
          table: issuesTableUrl,
        },
        signal: aborter.signal,
        liveSse,
      })

      await new Promise<void>((resolve) => {
        issueStream.subscribe((messages) => {
          messages.forEach((message) => {
            if (isChangeMessage(message)) {
              shapeData.set(message.key, message.value)
            }
            if (isUpToDateMessage(message)) {
              aborter.abort()
              return resolve()
            }
          })
        })
      })
      const values = [...shapeData.values()]

      expect(values).toMatchObject([{ title: `foo + ${uuid}` }])
    }
  )

  mit.for(fetchAndSse)(
    `should parse incoming data (liveSSE=$liveSse)`,
    async ({ liveSse }, { dbClient, aborter, tableSql, tableUrl }) => {
      // Create a table with data we want to be parsed
      await dbClient.query(
        `
      INSERT INTO ${tableSql} (txt, i2, i4, i8, f8, b, json, jsonb, ints, ints2, int4s, bools, moods, moods2, complexes, posints, jsons, txts, value, doubles)
      VALUES (
        'test',
        1,
        2147483647,
        9223372036854775807,
        4.5,
        TRUE,
        '{"foo": "bar"}',
        '{"foo": "bar"}',
        '{1,2,3}',
        $1,
        $2,
        $3,
        $4,
        $5,
        $6,
        $7,
        $8,
        $9,
        $10,
        $11
      )
    `,
        [
          [[1, 2, 3], [4, 5, 6]],
          [1, 2, 3],
          [true, false, true],
          [`sad`, `ok`, `happy`],
          [
            [`sad`, `ok`],
            [`ok`, `happy`],
          ],
          [`(1.1, 2.2)`, `(3.3, 4.4)`],
          [5, 9, 2],
          [
            [`foo`, `bar`],
            [`baz`, `qux`],
          ],
          { something: `else` },
          [1.1, 2.2, 3.3],
        ]
      )
    }
  )
})

6. Parameterized Describe Blocks (test/client.test.ts)

import { describe, expect, inject } from 'vitest'
import { testWithIssuesTable as it } from './support/test-context'
import { ShapeStream, Shape } from '../src'

const BASE_URL = inject(`baseUrl`)

const fetchAndSse = [{ liveSse: false }, { liveSse: true }]

describe.for(fetchAndSse)(`Shape (liveSSE=$liveSse)`, ({ liveSse }) => {
  it(`should sync an empty shape`, async ({ issuesTableUrl, aborter }) => {
    const start = Date.now()
    const shapeStream = new ShapeStream({
      url: `${BASE_URL}/v1/shape`,
      params: {
        table: issuesTableUrl,
      },
      signal: aborter.signal,
      liveSse,
    })
    const shape = new Shape(shapeStream)

    expect(await shape.value).toEqual(new Map())
    expect(await shape.rows).toEqual([])
    expect(shape.lastSyncedAt()).toBeGreaterThanOrEqual(start)
    expect(shape.lastSyncedAt()).toBeLessThanOrEqual(Date.now())
    expect(shape.lastSynced()).toBeLessThanOrEqual(Date.now() - start)
  })

  it(`should throw on a reserved parameter`, async ({ aborter }) => {
    expect(() => {
      const shapeStream = new ShapeStream({
        url: `${BASE_URL}/v1/shape`,
        params: {
          table: `foo`,
          // @ts-expect-error should not allow reserved parameters
          live: `false`,
        },
        liveSse,
        signal: aborter.signal,
      })
      new Shape(shapeStream)
    }).toThrowErrorMatchingSnapshot()
  })

  it(`should notify with the initial value`, async ({
    issuesTableUrl,
    insertIssues,
    aborter,
  }) => {
    const [id] = await insertIssues({ title: `test title` })

    const start = Date.now()
    const shapeStream = new ShapeStream({
      url: `${BASE_URL}/v1/shape`,
      params: {
        table: issuesTableUrl,
      },
      signal: aborter.signal,
      liveSse,
    })
    const shape = new Shape(shapeStream)

    const rows = await new Promise((resolve) => {
      shape.subscribe(({ rows }) => resolve(rows))
    })

    expect(rows).toEqual([{ id: id, title: `test title`, priority: 10 }])
    expect(shape.lastSyncedAt()).toBeGreaterThanOrEqual(start)
    expect(shape.lastSyncedAt()).toBeLessThanOrEqual(Date.now())
    expect(shape.lastSynced()).toBeLessThanOrEqual(Date.now() - start)
  })
})

7. Non-Database Tests with beforeEach/afterEach (test/expired-shapes-cache.test.ts)

import { beforeEach, afterEach, describe, expect, it, vi } from 'vitest'
import { ShapeStream } from '../src'
import {
  ExpiredShapesCache,
  expiredShapesCache,
} from '../src/expired-shapes-cache'
import { EXPIRED_HANDLE_QUERY_PARAM } from '../src/constants'

describe(`ExpiredShapesCache`, () => {
  let cache: ExpiredShapesCache
  const shapeUrl = `https://example.com/v1/shape`
  let aborter: AbortController
  let fetchMock: ReturnType<typeof vi.fn>

  beforeEach(() => {
    localStorage.clear()
    cache = new ExpiredShapesCache()
    expiredShapesCache.clear()
    aborter = new AbortController()
    fetchMock = vi.fn()
    vi.clearAllMocks()
  })

  afterEach(() => aborter.abort())

  it(`should mark shapes as expired and check expiration status`, () => {
    const shapeUrl1 = `https://example.com/v1/shape?table=test1`
    const shapeUrl2 = `https://example.com/v1/shape?table=test2`
    const handle1 = `handle-123`

    // Initially, shape should not have expired handle
    expect(cache.getExpiredHandle(shapeUrl1)).toBe(null)

    // Mark shape as expired
    cache.markExpired(shapeUrl1, handle1)

    // Now shape should return expired handle
    expect(cache.getExpiredHandle(shapeUrl1)).toBe(handle1)

    // Different shape should not have expired handle
    expect(cache.getExpiredHandle(shapeUrl2)).toBe(null)
  })

  it(`should persist expired shapes to localStorage`, () => {
    const shapeUrl = `https://example.com/v1/shape?table=test`
    const handle = `test-handle`

    // Mark shape as expired
    cache.markExpired(shapeUrl, handle)

    // Check that localStorage was updated
    const storedData = JSON.parse(
      localStorage.getItem(`electric_expired_shapes`) || `{}`
    )
    expect(storedData[shapeUrl]).toEqual({
      expiredHandle: handle,
      lastUsed: expect.any(Number),
    })
  })
})

8. Cache Testing with Docker Container Access (test/cache.test.ts)

import { describe, expect, assert, inject } from 'vitest'
import { exec } from 'child_process'
import { setTimeout as sleep } from 'node:timers/promises'
import { testWithIssuesTable } from './support/test-context'

const maxAge = 1 // seconds
const staleAge = 3 // seconds

enum CacheStatus {
  MISS = `MISS`,
  EXPIRED = `EXPIRED`,
  STALE = `STALE`,
  HIT = `HIT`,
}

function getCacheStatus(res: Response): CacheStatus {
  return res.headers.get(`X-Proxy-Cache`) as CacheStatus
}

export async function clearProxyCache({
  proxyCacheContainerName,
  proxyCachePath,
}: {
  proxyCacheContainerName: string
  proxyCachePath: string
}): Promise<void> {
  return new Promise((res) =>
    exec(
      `docker exec ${proxyCacheContainerName} sh -c 'rm -rf ${proxyCachePath}'`,
      (_) => res()
    )
  )
}

const it = testWithIssuesTable.extend<{
  proxyCacheBaseUrl: string
  clearCache: () => Promise<void>
}>({
  proxyCacheBaseUrl: async ({ clearCache }, use) => {
    await clearCache()
    use(inject(`proxyCacheBaseUrl`))
  },
  clearCache: async ({}, use) => {
    use(
      async () =>
        await clearProxyCache({
          proxyCacheContainerName: inject(`proxyCacheContainerName`),
          proxyCachePath: inject(`proxyCachePath`),
        })
    )
  },
})

describe(`HTTP Proxy Cache`, () => {
  it(`should get a short max-age cache-control header in live mode`, async ({
    insertIssues,
    proxyCacheBaseUrl,
    issuesTableUrl,
  }) => {
    // First request get initial request
    const initialRes = await fetch(
      `${proxyCacheBaseUrl}/v1/shape?table=${issuesTableUrl}&offset=-1`,
      {}
    )

    expect(initialRes.status).toBe(200)
    expect(getCacheStatus(initialRes)).toBe(CacheStatus.MISS)

    // add some data and follow with live request
    await insertIssues({ title: `foo` })
    const searchParams = new URLSearchParams({
      table: issuesTableUrl,
      handle: initialRes.headers.get(`electric-handle`)!,
      offset: initialRes.headers.get(`electric-offset`)!,
      live: `true`,
    })

    const liveRes = await fetch(
      `${proxyCacheBaseUrl}/v1/shape?${searchParams.toString()}`,
      {}
    )
    expect(liveRes.status).toBe(200)
    expect(getCacheStatus(liveRes)).toBe(CacheStatus.MISS)

    // Second request gets a cached response
    const cachedRes = await fetch(
      `${proxyCacheBaseUrl}/v1/shape?${searchParams.toString()}`,
      {}
    )
    expect(cachedRes.status).toBe(200)

    expect(getCacheStatus(cachedRes)).toBe(CacheStatus.HIT)
  })
})

Summary of Patterns Used

  1. Global Setup: Health check + schema creation + context injection
  2. Fixtures: Chained .extend() for composable test setup
  3. Parameterization: it.for() and describe.for() for configuration testing
  4. Isolation: Unique table names with task ID + random suffix
  5. Cleanup: Per-fixture teardown with error handling
  6. Helpers: Utility functions for common operations
  7. Typing: Module augmentation for context injection type safety
  8. Serial Execution: fileParallelism: false for shared database safety

Electric E2E Test Setup - Complete Documentation Index

This directory contains comprehensive documentation about how Electric's TypeScript client implements e2e testing with Docker, Postgres, and Vitest. These patterns are ready to be replicated for TanStack DB.

Documents in This Set

1. ELECTRIC_E2E_PATTERNS.md (Main Reference - 1028 lines)

Comprehensive guide with detailed explanations and architectural patterns

Contents:

  • Docker orchestration setup and configuration
  • Database isolation strategies (schema-based)
  • Three/four-level lifecycle management (global, per-file, per-test, per-fixture)
  • Setup/teardown patterns and cleanup strategies
  • Migration handling (inline SQL approach)
  • Test configuration and utilities
  • Parameterized testing patterns (it.for(), describe.for())
  • Real-world usage examples with complete code
  • Key architectural patterns and design decisions
  • Best practices from Electric
  • Replication checklist for TanStack DB
  • Environment configuration reference

Best for: Understanding the "why" and "how" behind decisions, architectural overview


2. QUICK_REFERENCE.md (Cheat Sheet - 200+ lines)

Quick lookup guide with copy-paste templates

Contents:

  • Key files to reference
  • Docker Compose templates (minimal, ready to copy)
  • Global setup pattern (code template)
  • Test context fixtures (code template)
  • Parameterized test template
  • Configuration values table
  • Test isolation strategy diagram
  • Fixture inheritance chain
  • Common patterns (insert, wait, cleanup)
  • Health check pattern
  • Environment variables reference
  • Critical settings checklist
  • Debugging tips
  • Performance optimization tips

Best for: Quick lookup, finding specific patterns, copy-paste templates


3. ACTUAL_CODE_EXCERPTS.md (Real Code - 400+ lines)

Actual code directly from Electric's test suite

Contents:

  • Vitest configuration (with explanations)
  • Global setup code (test/support/global-setup.ts)
  • Test context fixtures (testWithDbClient, testWithIssuesTable)
  • Test helpers (test/support/test-helpers.ts)
  • Parameterized tests (it.for examples)
  • Parameterized describe blocks (describe.for examples)
  • Non-database tests (beforeEach/afterEach pattern)
  • Cache testing with Docker container access
  • Summary of patterns used

Best for: Understanding exact implementation, copy-paste working code, seeing real patterns in action


How to Use These Documents

Scenario 1: Getting Started with E2E Testing

  1. Start with ELECTRIC_E2E_PATTERNS.md - Section 1-3 for Docker and database isolation
  2. Look at QUICK_REFERENCE.md - Copy Docker Compose template
  3. Review ACTUAL_CODE_EXCERPTS.md - See actual implementations

Scenario 2: Implementing Fixtures

  1. Read ELECTRIC_E2E_PATTERNS.md - Section 3 (lifecycle management)
  2. Check QUICK_REFERENCE.md - Fixture inheritance chain
  3. Copy code from ACTUAL_CODE_EXCERPTS.md - Section 3 (test context fixtures)

Scenario 3: Setting Up Parameterized Tests

  1. Check QUICK_REFERENCE.md - Parameterized test section
  2. Read ELECTRIC_E2E_PATTERNS.md - Section 6 (parameterized testing)
  3. Copy examples from ACTUAL_CODE_EXCERPTS.md - Sections 5-6

Scenario 4: Debugging Test Issues

  1. Refer to QUICK_REFERENCE.md - Debugging tips section
  2. Check ELECTRIC_E2E_PATTERNS.md - Section 9 (best practices)
  3. Look at ACTUAL_CODE_EXCERPTS.md - Error handling patterns

Key Patterns at a Glance

1. Docker Composition

Postgres (port 54321) + Server (port 3000)
Uses tmpfs for speed, depends_on for ordering

2. Database Isolation

Electric DB (shared)
  -> electric_test schema (created once)
    -> Unique tables per test (task.id + random suffix)

3. Test Lifecycle

Global Setup (once per run)
  ├─ Health check
  ├─ Create test schema
  └─ Provide context
  
Per-Test Fixtures (for each test)
  ├─ Create DB connection
  ├─ Create table
  ├─ Run test
  └─ Cleanup (drop table, close connection)

4. Fixture Composition

testWithDb
  extends testWithDb
    extends testWithDb
      (each level adds more fixtures)

5. Parameterization

const configs = [{ mode: 'a' }, { mode: 'b' }]
it.for(configs)('test', ({ mode }) => ...)
// Runs twice, once per config

Electric File References

If you need to look at the actual Electric codebase:

  • Docker setup: ~/programs/electric/.support/docker-compose.yml
  • Vitest config: ~/programs/electric/packages/typescript-client/vitest.config.ts
  • Global setup: ~/programs/electric/packages/typescript-client/test/support/global-setup.ts
  • Fixtures: ~/programs/electric/packages/typescript-client/test/support/test-context.ts
  • Helpers: ~/programs/electric/packages/typescript-client/test/support/test-helpers.ts
  • Tests: ~/programs/electric/packages/typescript-client/test/*.test.ts

Core Concepts Explained

Schema-Based Isolation (Not Database-Based)

Why: Reduces connection overhead, simplifies cleanup, allows serial execution with shared database

Unique Table Names with Task ID

Why: Prevents test collisions, aids debugging, makes it clear which test created the table

Fixture Composition (test.extend())

Why: Reusable, composable, isolated concerns, clear dependency chains

Global Setup with Health Check

Why: Ensures server is ready before tests run, provides context to all tests, handles one-time setup

Parameterized Tests with it.for() / describe.for()

Why: Tests multiple configurations systematically, reduces code duplication, clear test matrix

Serial Execution (fileParallelism: false)

Why: Prevents concurrency issues with shared database, simplifies debugging


Configuration Defaults

Setting Value Can Override
Postgres Host localhost -
Postgres Port 54321 -
Postgres User postgres -
Postgres Password password -
Postgres Database electric -
Server URL http://localhost:3000 SERVER_URL env
Test Schema electric_test hardcoded
Health Check Timeout 10 seconds in code
Parallel Execution false (serial) vitest.config.ts

Critical Settings for Success

  1. Vitest: fileParallelism: false (MUST be disabled for shared DB)
  2. Docker: tmpfs for Postgres data directory (speed)
  3. Docker: depends_on: postgres for backend service (ordering)
  4. Global Setup: Health check before proceeding (reliability)
  5. Fixtures: Unique table names (isolation)
  6. Cleanup: Don't throw in cleanup code (allows subsequent cleanup)

Replication Steps for TanStack DB

  1. Create Docker Compose with Postgres + TanStack server
  2. Implement global-setup.ts with health check
  3. Create test context fixtures with test.extend()
  4. Set fileParallelism: false in vitest.config.ts
  5. Create unique table names per test with task.id
  6. Add parameterized tests with it.for() if needed
  7. Use AbortController for stream cleanup
  8. Document connection defaults

Additional Resources

These documents reference:

  • Vitest 3.0+ (fixture system)
  • Node 'pg' library (PostgreSQL client)
  • Docker (container orchestration)
  • TypeScript (type safety)

Quick Lookup by Topic

Docker Setup: See QUICK_REFERENCE.md (Docker Compose section) Database Isolation: See ELECTRIC_E2E_PATTERNS.md Section 2 Fixtures: See ACTUAL_CODE_EXCERPTS.md Section 3 or ELECTRIC_E2E_PATTERNS.md Section 3 Parameterization: See QUICK_REFERENCE.md (Parameterized Test section) Health Check: See QUICK_REFERENCE.md (Health Check Pattern section) Debugging: See QUICK_REFERENCE.md (Debugging Tips section) Real Examples: See ACTUAL_CODE_EXCERPTS.md (all sections) Architectural Overview: See ELECTRIC_E2E_PATTERNS.md Section 8


File Summary

File Size Purpose Best For
ELECTRIC_E2E_PATTERNS.md 29KB Comprehensive reference Understanding architecture
QUICK_REFERENCE.md 7.7KB Quick lookup Finding specific patterns
ACTUAL_CODE_EXCERPTS.md 22KB Real working code Copy-paste implementations
ELECTRIC_E2E_INDEX.md This file Navigation guide Finding what you need

Total Documentation Size

Approximately 60KB of comprehensive documentation covering:

  • 1000+ lines of detailed explanations
  • 400+ lines of real working code
  • Copy-paste templates for immediate use
  • Architecture diagrams and flow charts
  • Best practices and design patterns
  • Debugging tips and troubleshooting guides
  • Complete configuration reference

Generated from exploration of: ~/programs/electric/packages/typescript-client/test All code excerpts are actual implementations from Electric's test suite.

Electric TypeScript Client E2E Test Infrastructure & Patterns

Overview

Electric's TypeScript client uses Vitest as the test framework with comprehensive e2e testing against Docker-containerized Postgres and Electric server. The approach provides excellent patterns for test isolation, parameterized testing, and database management.


1. DOCKER ORCHESTRATION & SETUP

Docker Compose Configuration

Location: ~/.support/docker-compose.yml

The setup uses two main services:

version: '3.3'
name: 'electric_example-${PROJECT_NAME:-default}'

services:
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: electric
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
    ports:
      - 54321:5432
    volumes:
      - ./postgres.conf:/etc/postgresql/postgresql.conf:ro
    tmpfs:
      - /var/lib/postgresql/data
      - /tmp
    command:
      - postgres
      - -c
      - config_file=/etc/postgresql/postgresql.conf

  backend:
    image: electricsql/electric:canary
    environment:
      DATABASE_URL: postgresql://postgres:password@postgres:5432/electric?sslmode=disable
      ELECTRIC_INSECURE: true
    ports:
      - 3000:3000
    build:
      context: ../packages/sync-service/
    depends_on:
      - postgres

Key Design Decisions:

  • Postgres uses tmpfs for /var/lib/postgresql/data for fast test execution
  • Postgres config file loaded from host for custom settings
  • Electric backend depends_on postgres for startup ordering
  • Insecure mode only for dev/test (NOT production)
  • Default credentials hardcoded: postgres:password
  • Default port mapping: Postgres 54321, Electric 3000

Connection Details for Tests

Tests connect via:

  • Host: localhost
  • Port: 54321 (Postgres)
  • User: postgres
  • Password: password
  • Database: electric

2. DATABASE ISOLATION: UNIQUE DATABASES PER TEST

Schema-Based Isolation Strategy

Rather than creating separate databases, Electric uses schema isolation:

Global Setup (test/support/global-setup.ts):

/**
 * Global setup for the test suite. Validates that our server is running, and creates and tears down a
 * special schema in Postgres to ensure clean slate between runs.
 */
export default async function ({ provide }: GlobalSetupContext) {
  await waitForElectric(url)

  const client = makePgClient()

  await client.connect()
  await client.query(`CREATE SCHEMA IF NOT EXISTS electric_test`)

  provide(`baseUrl`, url)
  provide(`testPgSchema`, `electric_test`)
  provide(`proxyCacheBaseUrl`, proxyUrl)
  provide(`proxyCacheContainerName`, proxyCacheContainerName)
  provide(`proxyCachePath`, proxyCachePath)

  return async () => {
    await client.query(`DROP SCHEMA electric_test CASCADE`)
    await client.end()
  }
}

Per-Test Fixture Level Isolation (test/support/test-context.ts):

Each test gets a unique table name that includes task ID and random suffix:

issuesTableSql: async ({ dbClient, task }, use) => {
  // Creates unique table name like: "issues for ABC123_a1b2c3"
  const tableName = `"issues for ${task.id}_${Math.random()
    .toString(16)
    .replace(`.`, `_`)}"`
  
  // Setup table for test
  await dbClient.query(`
    DROP TABLE IF EXISTS ${tableName};
    CREATE TABLE ${tableName} (
      id UUID PRIMARY KEY,
      title TEXT NOT NULL,
      priority INTEGER NOT NULL
    );
    COMMENT ON TABLE ${tableName} IS 'Created for ${
      task.file?.name.replace(/'/g, `\``) ?? `unknown`
    } - ${task.name.replace(`'`, `\``)}';
  `)
  
  await use(tableName)
  
  // Cleanup table after test
  await dbClient.query(`DROP TABLE ${tableName}`)
},

Multi-Type Table Isolation - Same pattern for complex types:

tableSql: async ({ dbClient, task }, use) => {
  const tableName = `"multitype table for ${task.id}_${Math.random()
    .toString(16)
    .replace(`.`, `_`)}"`

  await dbClient.query(`
    DROP TABLE IF EXISTS ${tableName};
    DROP TYPE IF EXISTS mood;
    CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
    CREATE TABLE ${tableName} (
      txt VARCHAR,
      i2 INT2 PRIMARY KEY,
      ...
    )`)

  await use(tableName)

  // Full cleanup including custom types
  await dbClient.query(`
    DROP TABLE ${tableName};
    DROP TYPE IF EXISTS mood;
  `)
},

Benefits of This Approach:

  • Single database connection simplifies setup/teardown
  • Tests can run serially without interference
  • Clear table names aid in debugging (include test file + test name + random suffix)
  • Comment on table shows which test created it
  • Schema-based rather than database-based reduces connection overhead
  • Full cleanup including custom types and domains

3. SETUP/TEARDOWN PATTERNS & LIFECYCLE MANAGEMENT

Three-Level Lifecycle Management

Level 1: Global Setup (One-time per test run)

File: vitest.config.ts

export default defineConfig({
  test: {
    globalSetup: `test/support/global-setup.ts`,
    setupFiles: [`vitest-localstorage-mock`],
    fileParallelism: false,  // Critical: prevents parallel file execution
    coverage: { ... },
    reporters: [`default`, `junit`],
    outputFile: `./junit/test-report.junit.xml`,
    environment: `jsdom`,
  },
})

Key settings:

  • fileParallelism: false - Tests run serially (important for shared database)
  • globalSetup runs once before all tests
  • setupFiles runs before each test file

Level 2: Global Setup/Teardown

File: test/support/global-setup.ts

export default async function ({ provide }: GlobalSetupContext) {
  // SETUP
  await waitForElectric(url)
  const client = makePgClient()
  await client.connect()
  await client.query(`CREATE SCHEMA IF NOT EXISTS electric_test`)
  
  provide(`baseUrl`, url)
  provide(`testPgSchema`, `electric_test`)
  
  // Return cleanup function (runs once at end of all tests)
  return async () => {
    await client.query(`DROP SCHEMA electric_test CASCADE`)
    await client.end()
  }
}

Level 3: Fixture-Level Lifecycle (Per test)

File: test/support/test-context.ts

export const testWithDbClient = test.extend<{
  dbClient: Client
  aborter: AbortController
  baseUrl: string
  pgSchema: string
  clearShape: ClearShapeFn
}>({
  // Setup: Create connection, inject context
  dbClient: async ({}, use) => {
    const searchOption = `-csearch_path=${inject(`testPgSchema`)}`
    const client = makePgClient({ options: searchOption })
    await client.connect()
    
    // Pass client to test
    await use(client)
    
    // Cleanup: Close connection
    await client.end()
  },
  
  // Setup: Create abort controller for cancellation
  aborter: async ({}, use) => {
    const controller = new AbortController()
    await use(controller)
    
    // Cleanup: Abort any pending operations
    controller.abort(`Test complete`)
  },
  
  // Inject provided values
  baseUrl: async ({}, use) => use(inject(`baseUrl`)),
  pgSchema: async ({}, use) => use(inject(`testPgSchema`)),
  
  // Custom utility: Clear shape caches
  clearShape: async ({}, use) => {
    await use(async (table: string, options = {}) => {
      const baseUrl = inject(`baseUrl`)
      const url = new URL(`${baseUrl}/v1/shape`)
      url.searchParams.set(`table`, table)
      
      if (options.handle) {
        url.searchParams.set(SHAPE_HANDLE_QUERY_PARAM, options.handle)
      }
      
      const resp = await fetch(url.toString(), { method: `DELETE` })
      if (!resp.ok && resp.status !== 404) {
        throw new Error(`Could not delete shape`)
      }
    })
  },
})

Level 4: Table Fixtures (Extends dbClient)

export const testWithIssuesTable = testWithDbClient.extend<{
  issuesTableSql: string
  issuesTableUrl: string
  issuesTableKey: string
  updateIssue: UpdateIssueFn
  deleteIssue: DeleteIssueFn
  insertIssues: InsertIssuesFn
  clearIssuesShape: ClearIssuesShapeFn
  waitForIssues: WaitForIssuesFn
}>({
  issuesTableSql: async ({ dbClient, task }, use) => {
    const tableName = `"issues for ${task.id}_${...}"`
    await dbClient.query(`CREATE TABLE ${tableName} (...)`)
    await use(tableName)
    await dbClient.query(`DROP TABLE ${tableName}`)
  },
  
  issuesTableUrl: async ({ issuesTableSql, pgSchema, clearShape }, use) => {
    const urlAppropriateTable = pgSchema + `.` + issuesTableSql
    await use(urlAppropriateTable)
    try {
      await clearShape(urlAppropriateTable)
    } catch (_) {
      // ignore - clearShape has its own logging
    }
  },
  
  // Insert helper
  insertIssues: ({ issuesTableSql, dbClient }, use) =>
    use(async (...rows) => {
      const placeholders = rows.map(
        (_, i) => `($${i * 3 + 1}, $${i * 3 + 2}, $${i * 3 + 3})`
      )
      const { rows: rows_1 } = await dbClient.query(
        `INSERT INTO ${issuesTableSql} (id, title, priority) VALUES ${placeholders} RETURNING id`,
        rows.flatMap((x) => [x.id ?? uuidv4(), x.title, 10])
      )
      return rows_1.map((x) => x.id)
    }),
})

Explicit beforeEach/afterEach Patterns (For non-database tests)

For tests that don't need database access:

import { beforeEach, afterEach, describe, expect, it, vi } from 'vitest'

describe(`ExpiredShapesCache`, () => {
  let cache: ExpiredShapesCache
  let aborter: AbortController

  beforeEach(() => {
    localStorage.clear()
    cache = new ExpiredShapesCache()
    aborter = new AbortController()
    vi.clearAllMocks()
  })

  afterEach(() => aborter.abort())

  it(`should mark shapes as expired`, () => {
    // test code
  })
})

4. MIGRATION HANDLING

Electric doesn't use traditional migrations in the test setup. Instead:

  1. Global setup creates schema: CREATE SCHEMA IF NOT EXISTS electric_test
  2. Per-test fixture creates tables: Each test creates its own tables with exact schema
  3. Table structure is inline SQL: No migration files needed for tests
  4. Custom types created per-test: Types like mood enum are created per test if needed

Example: Multi-type table setup with custom types

await dbClient.query(`
  DROP TABLE IF EXISTS ${tableName};
  DROP TYPE IF EXISTS mood;
  DROP TYPE IF EXISTS complex;
  DROP DOMAIN IF EXISTS posint;
  
  CREATE TYPE mood AS ENUM ('sad', 'ok', 'happy');
  CREATE TYPE complex AS (r double precision, i double precision);
  CREATE DOMAIN posint AS integer CHECK (VALUE > 0);
  
  CREATE TABLE ${tableName} (
    txt VARCHAR,
    i2 INT2 PRIMARY KEY,
    i4 INT4,
    i8 INT8,
    f8 FLOAT8,
    b BOOLEAN,
    json JSON,
    jsonb JSONB,
    ints INT8[],
    ...
  )
`)

// Cleanup includes all types and domains
await dbClient.query(`
  DROP TABLE ${tableName};
  DROP TYPE IF EXISTS mood;
  DROP TYPE IF EXISTS complex;
  DROP DOMAIN IF EXISTS posint;
`)

Why this approach:

  • Tests are self-contained and don't depend on external migration state
  • Schema is visible where it's used
  • Easy to understand what data each test expects
  • No migration versioning complexity for tests
  • Clear cleanup in same file as setup

5. TEST CONFIGURATION & UTILITIES

Vitest Configuration

File: vitest.config.ts

import { defineConfig } from 'vitest/config'

export default defineConfig({
  test: {
    globalSetup: `test/support/global-setup.ts`,
    setupFiles: [`vitest-localstorage-mock`],
    typecheck: { enabled: true },
    fileParallelism: false,
    coverage: {
      provider: `istanbul`,
      reporter: [`text`, `json`, `html`, `lcov`],
      include: [`**/src/**`],
    },
    reporters: [`default`, `junit`],
    outputFile: `./junit/test-report.junit.xml`,
    environment: `jsdom`,
  },
})

Health Check Pattern

File: test/support/global-setup.ts

function waitForElectric(url: string): Promise<void> {
  return new Promise<void>((resolve, reject) => {
    const timeout = setTimeout(
      () => reject(`Timed out waiting for Electric to be active`),
      10000
    )

    const tryHealth = async () =>
      fetch(`${url}/v1/health`)
        .then(async (res): Promise<void> => {
          if (!res.ok) return tryHealth()
          const { status } = (await res.json()) as { status: string }
          if (status !== `active`) return tryHealth()
          clearTimeout(timeout)
          resolve()
        })
        .catch((err) => {
          clearTimeout(timeout)
          reject(err)
        })

    return tryHealth()
  })
}

Key utilities:

  • Polls health endpoint until status is "active"
  • 10 second timeout before failure
  • Recursive polling (retry until success or timeout)

Database Client Helper

File: test/support/test-helpers.ts

export function makePgClient(overrides: ClientConfig = {}) {
  return new Client({
    host: `localhost`,
    port: 54321,
    password: `password`,
    user: `postgres`,
    database: `electric`,
    options: `-csearch_path=electric_test`,
    ...overrides,
  })
}

Design:

  • Encapsulates connection defaults
  • Allows overrides for specific tests (e.g., search_path for schemas)
  • Uses "pg" library (PostGres native client)

Context Providers (Vitest 3.0+)

declare module 'vitest' {
  export interface ProvidedContext {
    baseUrl: string
    proxyCacheBaseUrl: string
    testPgSchema: string
    proxyCacheContainerName: string
    proxyCachePath: string
  }
}

Values are injected in tests via inject():

const BASE_URL = inject(`baseUrl`)

Message Waiting Helper

File: test/support/test-helpers.ts

export async function waitForTransaction({
  baseUrl,
  table,
  numChangesExpected,
  shapeStreamOptions,
  aborter,
}: {
  baseUrl: string
  table: string
  numChangesExpected?: number
  shapeStreamOptions?: Partial<ShapeStreamOptions>
  aborter?: AbortController
}): Promise<Pick<ShapeStreamOptions, `offset` | `handle`>> {
  const waitAborter = new AbortController()
  if (aborter?.signal.aborted) waitAborter.abort()
  else aborter?.signal.addEventListener(`abort`, () => waitAborter.abort())
  
  const issueStream = new ShapeStream({
    ...(shapeStreamOptions ?? {}),
    url: `${baseUrl}/v1/shape`,
    params: {
      ...(shapeStreamOptions?.params ?? {}),
      table,
    },
    signal: waitAborter.signal,
    subscribe: true,
  })

  numChangesExpected ??= 1
  let numChangesSeen = 0
  
  await forEachMessage(issueStream, waitAborter, (res, msg) => {
    if (isChangeMessage(msg)) {
      numChangesSeen++
    }

    if (numChangesSeen >= numChangesExpected && isUpToDateMessage(msg)) {
      res()
    }
  })
  
  return {
    offset: issueStream.lastOffset,
    handle: issueStream.shapeHandle,
  }
}

6. PARAMETERIZED TESTING PATTERNS

Simple Parameter List

File: test/integration.test.ts

const fetchAndSse = [{ liveSse: false }, { liveSse: true }]

// Parameterized describe block
describe(`HTTP Sync`, () => {
  // Test runs twice: once with liveSse=false, once with liveSse=true
  it.for(fetchAndSse)(
    `should work with empty shape/table (liveSSE=$liveSse)`,
    async ({ liveSse }, { issuesTableUrl, aborter }) => {
      const issueStream = new ShapeStream({
        url: `${BASE_URL}/v1/shape`,
        params: {
          table: issuesTableUrl,
        },
        subscribe: false,
        signal: aborter.signal,
        liveSse,  // Parameter from the array
      })

      await new Promise<void>((resolve, reject) => {
        issueStream.subscribe((messages) => {
          messages.forEach((message) => {
            if (isChangeMessage(message)) {
              shapeData.set(message.key, message.value)
            }
            if (isUpToDateMessage(message)) {
              aborter.abort()
              return resolve()
            }
          })
        }, reject)
      })

      const values = [...shapeData.values()]
      expect(values).toHaveLength(0)
    }
  )
})

Parameterized Describe Block

File: test/client.test.ts

const fetchAndSse = [{ liveSse: false }, { liveSse: true }]

describe.for(fetchAndSse)(`Shape (liveSSE=$liveSse)`, ({ liveSse }) => {
  // All tests in this describe run twice (once per parameter set)
  
  it(`should sync an empty shape`, async ({ issuesTableUrl, aborter }) => {
    const start = Date.now()
    const shapeStream = new ShapeStream({
      url: `${BASE_URL}/v1/shape`,
      params: {
        table: issuesTableUrl,
      },
      signal: aborter.signal,
      liveSse,  // Available from describe scope
    })
    const shape = new Shape(shapeStream)

    expect(await shape.value).toEqual(new Map())
    expect(await shape.rows).toEqual([])
    expect(shape.lastSyncedAt()).toBeGreaterThanOrEqual(start)
  })

  it(`should notify with the initial value`, async ({
    issuesTableUrl,
    insertIssues,
    aborter,
  }) => {
    const [id] = await insertIssues({ title: `test title` })

    const shapeStream = new ShapeStream({
      url: `${BASE_URL}/v1/shape`,
      params: {
        table: issuesTableUrl,
      },
      signal: aborter.signal,
      liveSse,
    })
    const shape = new Shape(shapeStream)

    const rows = await new Promise((resolve) => {
      shape.subscribe(({ rows }) => resolve(rows))
    })

    expect(rows).toEqual([{ id: id, title: `test title`, priority: 10 }])
  })
})

Parameterized Multi-Type Tests

File: test/integration.test.ts

mit.for(fetchAndSse)(
  `should parse incoming data (liveSSE=$liveSse)`,
  async ({ liveSse }, { dbClient, aborter, tableSql, tableUrl }) => {
    // Create a table with data we want to be parsed
    await dbClient.query(
      `
      INSERT INTO ${tableSql} (txt, i2, i4, i8, f8, b, json, jsonb, ints, ints2, int4s, bools, moods, moods2, complexes, posints, jsons, txts, value, doubles)
      VALUES (
        'test',
        1,
        2147483647,
        9223372036854775807,
        4.5,
        TRUE,
        '{"foo": "bar"}',
        '{"foo": "bar"}',
        '{1,2,3}',
        $1,
        $2,
        $3,
        $4,
        $5,
        $6,
        $7,
        $8,
        $9,
        $10,
        $11
      )
    `,
      [
        [[1, 2, 3], [4, 5, 6]],
        [1, 2, 3],
        [true, false, true],
        [`sad`, `ok`, `happy`],
        [
          [`sad`, `ok`],
          [`ok`, `happy`],
        ],
        [`(1.1, 2.2)`, `(3.3, 4.4)`],
        [5, 9, 2],
        [
          [`foo`, `bar`],
          [`baz`, `qux`],
        ],
        { something: `else` },
        [1.1, 2.2, 3.3],
      ]
    )

    const issueStream = new ShapeStream({
      url: `${BASE_URL}/v1/shape`,
      params: {
        table: tableUrl,
      },
      signal: aborter.signal,
      liveSse,
    })

    // ... rest of test
  }
)

Parameterization Key Points:

  • it.for(params) runs test multiple times with each param set
  • describe.for(params) runs all tests in describe block multiple times
  • Parameters are destructured in callback: async ({ liveSse }, { fixtures })
  • Test name uses template string interpolation: (liveSSE=$liveSse)
  • Works with custom fixtures (like mit for multi-type tables)

7. REAL-WORLD USAGE EXAMPLES

Complete Integration Test Example

import { describe, expect, inject } from 'vitest'
import { testWithIssuesTable as it } from './support/test-context'
import { ShapeStream } from '../src'

const BASE_URL = inject(`baseUrl`)
const fetchAndSse = [{ liveSse: false }, { liveSse: true }]

describe(`HTTP Sync`, () => {
  // Basic test with auto-cleanup via fixtures
  it(`sanity check`, async ({ dbClient, issuesTableSql }) => {
    const result = await dbClient.query(`SELECT * FROM ${issuesTableSql}`)
    expect(result.rows).toEqual([])
  })

  // Parameterized test with initial data
  it.for(fetchAndSse)(
    `should get initial data (liveSSE=$liveSse)`,
    async ({ liveSse }, { insertIssues, issuesTableUrl, aborter }) => {
      // Setup: Insert data
      const uuid = uuidv4()
      await insertIssues({ id: uuid, title: `foo + ${uuid}` })

      // Execute: Create stream and subscribe
      const shapeData = new Map()
      const issueStream = new ShapeStream({
        url: `${BASE_URL}/v1/shape`,
        params: {
          table: issuesTableUrl,
        },
        signal: aborter.signal,
        liveSse,
      })

      // Wait for data
      await new Promise<void>((resolve) => {
        issueStream.subscribe((messages) => {
          messages.forEach((message) => {
            if (isChangeMessage(message)) {
              shapeData.set(message.key, message.value)
            }
            if (isUpToDateMessage(message)) {
              aborter.abort()
              return resolve()
            }
          })
        })
      })

      // Assert
      const values = [...shapeData.values()]
      expect(values).toMatchObject([{ title: `foo + ${uuid}` }])
      
      // Cleanup: Automatic! Fixtures handle:
      // - Table drops (issuesTableSql fixture)
      // - Shape cache clears (issuesTableUrl fixture)
      // - Connection closes (dbClient fixture)
      // - Stream aborts (aborter fixture)
    }
  )
})

Cache Testing with Proxy Container Access

File: test/cache.test.ts

const it = testWithIssuesTable.extend<{
  proxyCacheBaseUrl: string
  clearCache: () => Promise<void>
}>({
  proxyCacheBaseUrl: async ({ clearCache }, use) => {
    await clearCache()
    use(inject(`proxyCacheBaseUrl`))
  },
  clearCache: async ({}, use) => {
    use(
      async () =>
        await clearProxyCache({
          proxyCacheContainerName: inject(`proxyCacheContainerName`),
          proxyCachePath: inject(`proxyCachePath`),
        })
    )
  },
})

export async function clearProxyCache({
  proxyCacheContainerName,
  proxyCachePath,
}: {
  proxyCacheContainerName: string
  proxyCachePath: string
}): Promise<void> {
  return new Promise((res) =>
    exec(
      `docker exec ${proxyCacheContainerName} sh -c 'rm -rf ${proxyCachePath}'`,
      (_) => res()
    )
  )
}

describe(`HTTP Proxy Cache`, () => {
  it(`should get a short max-age cache-control header in live mode`, async ({
    insertIssues,
    proxyCacheBaseUrl,
    issuesTableUrl,
  }) => {
    // First request gets initial request
    const initialRes = await fetch(
      `${proxyCacheBaseUrl}/v1/shape?table=${issuesTableUrl}&offset=-1`,
      {}
    )

    expect(initialRes.status).toBe(200)
    expect(getCacheStatus(initialRes)).toBe(CacheStatus.MISS)

    // Add some data and follow with live request
    await insertIssues({ title: `foo` })
    const searchParams = new URLSearchParams({
      table: issuesTableUrl,
      handle: initialRes.headers.get(`electric-handle`)!,
      offset: initialRes.headers.get(`electric-offset`)!,
      live: `true`,
    })

    const liveRes = await fetch(
      `${proxyCacheBaseUrl}/v1/shape?${searchParams.toString()}`,
      {}
    )
    expect(liveRes.status).toBe(200)
    expect(getCacheStatus(liveRes)).toBe(CacheStatus.MISS)

    // Second request gets cached response
    const cachedRes = await fetch(
      `${proxyCacheBaseUrl}/v1/shape?${searchParams.toString()}`,
      {}
    )
    expect(cachedRes.status).toBe(200)
    expect(getCacheStatus(cachedRes)).toBe(CacheStatus.HIT)
  })
})

8. KEY ARCHITECTURAL PATTERNS

Fixture Composition Pattern

The test fixtures form a chain where each level builds on the previous:

vitest.config.ts (global setup + setup files)
  ↓
test/support/global-setup.ts (health check + schema creation)
  ↓
test/support/test-context.ts - testWithDbClient (DB connection)
  ↓
test/support/test-context.ts - testWithIssuesTable extends testWithDbClient
  ↓
Individual tests using testWithIssuesTable

Each level adds new fixtures while inheriting parent fixtures:

  • testWithDbClient adds: dbClient, aborter, baseUrl, pgSchema, clearShape
  • testWithIssuesTable adds: issuesTableSql, issuesTableUrl, issuesTableKey, insertIssues, deleteIssue, updateIssue, etc.
  • Custom test extends can further extend (like proxyCacheBaseUrl, clearCache)

Lifecycle Management Pattern

SETUP                          USE                        TEARDOWN
├─ Create client ────────────→ Test runs ────────────────→ End connection
├─ Create AbortController ────→ Signal available ────────→ Abort
├─ Create table ──────────────→ Query available ────────→ Drop table
├─ Create helpers ────────────→ Methods available ───────→ (auto-cleanup)
└─ Inject context ────────────→ Injected values ready ──→ Cleanup

Test Isolation Pattern

Global Schema: electric_test (created once, dropped once)
    │
    ├─ Test 1: creates "issues for ABC123_xyz" table
    │   ├─ Test runs
    │   └─ DROP table
    │
    ├─ Test 2: creates "issues for DEF456_abc" table  
    │   ├─ Test runs
    │   └─ DROP table
    │
    └─ Test 3: creates "issues for GHI789_def" table
        ├─ Test runs
        └─ DROP table

Each test has:

  • Unique table names (task.id + random suffix)
  • Isolated data (no cross-test pollution)
  • Full cleanup (drop table + clear shapes + close connections)
  • Comments showing which test created the table (helpful for debugging)

Error Resilience Pattern

try {
  await clearShape(urlAppropriateTable)
} catch (_) {
  // ignore - clearShape has its own logging
  // we don't want to interrupt cleanup
}

Cleanup code doesn't throw, allowing subsequent cleanup steps to run even if one fails.


9. TESTING BEST PRACTICES FROM ELECTRIC

  1. Serial Execution: fileParallelism: false prevents concurrency issues with shared database
  2. Health Checks: Wait for Electric to be "active" before running tests
  3. Unique Table Names: Use task ID + random suffix to prevent collisions
  4. Full Cleanup: Always drop tables, clear caches, close connections
  5. Fixture Composition: Build complex fixtures from simpler ones
  6. Context Injection: Use Vitest's provide()/inject() for test-wide values
  7. Abort Controllers: Use AbortSignal for clean stream shutdown
  8. Parameterized Tests: Use it.for() and describe.for() for testing multiple configurations
  9. Helpful Comments: Add SQL comments showing which test created tables
  10. Graceful Degradation: Catch errors in cleanup code to avoid masking test failures

10. REPLICATION CHECKLIST FOR TANSTACK DB

When implementing similar e2e testing for TanStack DB:

  • Create Docker Compose with Postgres + server service
  • Implement global setup that:
    • Waits for server health endpoint
    • Creates shared test schema/database
    • Provides context values (baseUrl, testDb, etc.)
    • Returns cleanup function
  • Create test fixtures extending from base test:
    • Database client fixture
    • AbortController fixture
    • Table creation fixtures with unique names
    • Helper fixtures (insert, delete, update, query)
  • Set fileParallelism: false in vitest.config.ts
  • Use .extend() to compose fixtures
  • Add parameterized tests with it.for() and describe.for()
  • Use inject() to access provided context
  • Clean up resources in fixture teardown phase
  • Create health check function for server readiness
  • Document connection details and environment setup

File Structure Reference

packages/typescript-client/
├── vitest.config.ts
├── test/
│   ├── support/
│   │   ├── global-setup.ts        # One-time setup for all tests
│   │   ├── test-context.ts         # Fixture definitions
│   │   └── test-helpers.ts         # Utility functions
│   ├── integration.test.ts         # Main e2e tests
│   ├── client.test.ts              # Client functionality tests
│   ├── cache.test.ts               # Cache + proxy tests
│   └── ... other test files
└── src/
    └── ... source code

.support/
└── docker-compose.yml              # Docker services

Environment Configuration

Database Connection (hardcoded in test-helpers.ts)

Host: localhost
Port: 54321
User: postgres
Password: password
Database: electric
Search path: electric_test (for tests)

Electric Server

URL: http://localhost:3000
Health endpoint: http://localhost:3000/v1/health

Proxy Cache (optional, for cache tests)

Container: electric_dev-nginx-1
URL: http://localhost:3002
Cache path: /var/cache/nginx/*

These can be overridden via environment variables in global-setup.ts:

const url = process.env.ELECTRIC_URL ?? `http://localhost:3000`
const proxyUrl = process.env.ELECTRIC_PROXY_CACHE_URL ?? `http://localhost:3002`

Electric E2E Test Setup - Quick Reference

Key Files to Reference

  1. Docker Setup

    • Location: ~/.support/docker-compose.yml
    • Services: Postgres (54321), Electric (3000)
    • Key: Uses tmpfs for speed, config files for customization
  2. Vitest Config

    • Location: vitest.config.ts
    • Critical setting: fileParallelism: false
    • Global setup: test/support/global-setup.ts
  3. Test Infrastructure

    • Global setup: test/support/global-setup.ts
    • Fixtures: test/support/test-context.ts
    • Helpers: test/support/test-helpers.ts

Copy-Paste Templates

1. Docker Compose (postgres + server)

services:
  postgres:
    image: postgres:16-alpine
    environment:
      POSTGRES_DB: electric
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: password
    ports:
      - 54321:5432
    tmpfs:
      - /var/lib/postgresql/data
      - /tmp
  
  backend:
    image: your-server:latest
    environment:
      DATABASE_URL: postgresql://postgres:password@postgres:5432/electric?sslmode=disable
    ports:
      - 3000:3000
    depends_on:
      - postgres

2. Global Setup Pattern

import type { GlobalSetupContext } from 'vitest/node'
import { makePgClient } from './test-helpers'

export default async function ({ provide }: GlobalSetupContext) {
  // Health check
  await waitForServer(process.env.SERVER_URL ?? `http://localhost:3000`)

  // Setup
  const client = makePgClient()
  await client.connect()
  await client.query(`CREATE SCHEMA IF NOT EXISTS test_schema`)

  provide(`baseUrl`, process.env.SERVER_URL ?? `http://localhost:3000`)
  provide(`testSchema`, `test_schema`)

  // Cleanup function
  return async () => {
    await client.query(`DROP SCHEMA test_schema CASCADE`)
    await client.end()
  }
}

function waitForServer(url: string): Promise<void> {
  return new Promise<void>((resolve, reject) => {
    const timeout = setTimeout(() => reject(`Timeout`), 10000)
    const tryHealth = async () =>
      fetch(`${url}/health`)
        .then(async (res) => {
          if (!res.ok) return tryHealth()
          clearTimeout(timeout)
          resolve()
        })
        .catch(() => tryHealth())
    tryHealth()
  })
}

3. Test Context Fixtures

import { test } from 'vitest'
import { Client } from 'pg'

const testWithDb = test.extend<{
  dbClient: Client
  tableName: string
}>({
  dbClient: async ({}, use) => {
    const client = new Client({
      host: `localhost`,
      port: 54321,
      user: `postgres`,
      password: `password`,
      database: `electric`,
    })
    await client.connect()
    await use(client)
    await client.end()
  },

  tableName: async ({ dbClient, task }, use) => {
    const name = `test_${task.id}_${Math.random().toString(16).slice(2)}`
    await dbClient.query(`
      CREATE TABLE ${name} (
        id UUID PRIMARY KEY,
        data TEXT
      )
    `)
    await use(name)
    await dbClient.query(`DROP TABLE ${name}`)
  },
})

export { testWithDb as it }

4. Parameterized Test

const configs = [
  { mode: 'fetch' },
  { mode: 'stream' },
]

describe.for(configs)(`Data sync (mode=$mode)`, ({ mode }) => {
  it(`should sync data`, async ({ dbClient, tableName }) => {
    // Test code using mode parameter
    expect(mode).toBe('fetch')
  })
})

Configuration Values

Key Default Override
Postgres Host localhost -
Postgres Port 54321 -
Postgres User postgres -
Postgres Password password -
Postgres Database electric -
Server URL http://localhost:3000 SERVER_URL env
Test Schema electric_test hardcoded
Serial Execution true fileParallelism: false

Test Isolation Strategy

Database: electric (shared across all tests)
  ↓
Schema: electric_test (created once, dropped once)
  ↓
Per-test tables:
  - "test_ABC123_xyz" (test 1)
  - "test_DEF456_abc" (test 2)
  - "test_GHI789_def" (test 3)

Each table:

  • Has unique name (task.id + random suffix)
  • Is created before test
  • Is dropped after test
  • May have SQL comment showing origin

Fixture Inheritance Chain

testWithDb
  ├─ dbClient (database connection)
  └─ tableName (unique test table)
     ↓
testWithData extends testWithDb
  ├─ all parent fixtures
  ├─ insertRow (helper function)
  └─ queryRows (helper function)
     ↓
testWithCache extends testWithData
  ├─ all parent fixtures
  ├─ cacheUrl
  └─ clearCache

Common Patterns

Insert Data

await dbClient.query(
  `INSERT INTO ${tableName} (id, data) VALUES ($1, $2)`,
  [uuid(), `test data`]
)

Wait for Changes

await new Promise((resolve) => {
  const unsubscribe = stream.subscribe((messages) => {
    if (messages.some(isUpToDate)) {
      unsubscribe()
      resolve()
    }
  })
})

Cleanup with Error Handling

try {
  await cleanup()
} catch (e) {
  console.error(`Cleanup failed but continuing:`, e)
  // Don't throw - let other cleanup steps run
}

Health Check Pattern

async function waitForServer(url: string): Promise<void> {
  return new Promise((resolve, reject) => {
    const timeout = setTimeout(() => reject(`Timeout`), 10000)
    
    const check = async () => {
      try {
        const res = await fetch(`${url}/health`)
        if (res.ok) {
          clearTimeout(timeout)
          resolve()
        } else {
          setTimeout(check, 100)
        }
      } catch {
        setTimeout(check, 100)
      }
    }
    
    check()
  })
}

Environment Variables

In global-setup.ts:

const SERVER_URL = process.env.SERVER_URL ?? `http://localhost:3000`
const DB_PORT = process.env.DB_PORT ?? 54321
const DB_PASSWORD = process.env.DB_PASSWORD ?? `password`

In test-helpers.ts:

export function makePgClient(overrides = {}) {
  return new Client({
    host: process.env.DB_HOST ?? `localhost`,
    port: parseInt(process.env.DB_PORT ?? `54321`),
    password: process.env.DB_PASSWORD ?? `password`,
    user: process.env.DB_USER ?? `postgres`,
    database: process.env.DB_NAME ?? `electric`,
    ...overrides,
  })
}

Critical Settings

Vitest Config

{
  test: {
    globalSetup: `test/support/global-setup.ts`,
    fileParallelism: false,  // CRITICAL: Serial execution
    environment: `jsdom`,
  }
}

Docker Compose

tmpfs:  # Use tmpfs for speed
  - /var/lib/postgresql/data
  - /tmp
depends_on:  # Ensure ordering
  - postgres

Debugging Tips

  1. Table names in DB: Use comments to track origin

    COMMENT ON TABLE "test_ABC_xyz" IS 'Created for file.test.ts - test name'
  2. Connection issues: Check health endpoint first

    curl http://localhost:3000/health
  3. Flaky tests: Add retry logic to health check

    const MAX_ATTEMPTS = 50
    for (let i = 0; i < MAX_ATTEMPTS; i++) {
      try { return await checkHealth() }
      catch { await sleep(100) }
    }
  4. Test isolation: Verify unique table names

    console.log(`Using table: ${tableName}`)
    // Should show different name for each test
  5. Cleanup verification: Check schema is clean

    docker exec postgres psql -U postgres -c "\dt electric_test.*"

Performance Optimization

  1. Use tmpfs for Postgres data
  2. Keep schema structure simple (only what's needed)
  3. Avoid global state between tests
  4. Use fixtures for resource management
  5. Clean up connections immediately
  6. Run tests serially (fileParallelism: false)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment