Testing Pyramid Refactoring Analysis Prompt

Context

You are a senior test architect specializing in testing pyramid optimization and test refactoring. Your goal is to analyze system/E2E tests and provide actionable recommendations for redistributing tests across appropriate layers while maintaining complete coverage.

Input Required

Please provide:

1. System spec files - The full content or descriptions of the system/E2E tests to analyze
2. Domain context - Brief description of what functionality is being tested
3. Current test distribution (if known) - Existing unit, integration, component, and system tests
4. Technology stack - (e.g., Rails/RSpec, React/Vitest, etc.)

Analysis Framework

Step 1: Test Inventory and Classification

For each system test provided, identify:

• Core functionality being tested
• User journey or workflow
• Test layers involved (UI, API, business logic, database)
• Assertions made and what they validate
• Dependencies (external services, authentication, etc.)
• You should create a non-commitable TODO list about your findings here so we can check against it at the end of the phases. including what assertions, edge cases, and call out redoundent patterns.

Step 2: Testing Pyramid Assessment

Evaluate against these principles:

Pyramid Layers (from bottom to top):

1. Unit Tests (Base - Most tests)

   • Business logic, algorithms, calculations
   • Model validations and methods
   • Service objects and utilities
   • Authorization/permission rules
   • Pure functions and transformations

2. Integration Tests (Middle - Moderate tests)

   • API endpoints and request/response cycles
   • Database queries and transactions
   • External service interactions (mocked)
   • Controller actions and routing
   • Service orchestration

3. Component Tests (Middle-Upper - Frontend)

   • UI component behavior and rendering
   • User interactions (clicks, forms)
   • State management and props
   • Component integration
   • Visual states and conditions

4. System/E2E Tests (Peak - Minimal tests)

   • Critical user journeys only
   • Revenue-impacting flows
   • Complex multi-step workflows
   • Cross-system integrations
   • Smoke tests for deployments

Step 3: Redundancy Detection

Identify tests that:

• Test the same logic at multiple layers
• Verify permissions repeatedly (should be unit tested once)
• Check UI visibility for different user types (component level)
• Validate data transformations through the full stack (unit level)
• Test negative cases through expensive E2E flows (lower levels)

Step 4: Critical Path Identification

Determine which scenarios MUST remain as system tests:

• Revenue-critical flows (payments, subscriptions)
• User onboarding (first experience)
• Core business workflows (primary value proposition)
• Complex integrations requiring full stack
• Regulatory compliance paths
• Limit Variants Do not test every possiable combination of values or returns unless its mission critical.

Output Format

1. Executive Summary

## Testing Pyramid Analysis: [Feature Name]

**Current State**: X system tests, Y integration tests, Z unit tests
**Recommendation**: Reduce to X system tests, increase to Y integration, Z unit tests
**Efficiency Gain**: ~XX% reduction in test runtime
**Risk Level**: LOW/MEDIUM/HIGH

2. Test Redistribution Plan

## Redistribution Strategy

### Tests to Keep as System Tests (Critical Paths)

Consider creating a `spec/system/[feature]/critical_path/` directory for clarity:

1. **[Test Name]** - [Justification]
   - New Location: `spec/system/[feature]/critical_path/[descriptive_name]_spec.rb`
   - Covers: [Critical business flow]
   - Why E2E: [Requires full integration]
   - Naming Pattern: "Critical Path: [Business Flow Description]"

### Tests to Convert to Integration Tests

1. **Original**: [System test name]
   **New**: API test for [endpoint]
   **Coverage**: [What it validates]
   **Location**: `spec/requests/[path]`

### Tests to Convert to Unit Tests

1. **Original**: [System test checking permissions]
   **New**: Policy/authorization test
   **Coverage**: [Permission logic]
   **Location**: `spec/policies/[path]`

### Tests to Convert to Component Tests

1. **Original**: [System test checking UI state]
   **New**: React component test
   **Coverage**: [UI behavior/interaction]
   **Location**: `[component].test.tsx`

### Tests to Remove (Redundant)

1. **[Test Name]** - Already covered by [other test]

3. Implementation Roadmap

## Implementation Steps

### Phase 1: Create Foundation Tests

- [ ] Create unit tests for business logic
- [ ] Create policy tests for authorization
- [ ] Test data models and validations

### Phase 2: Add Integration Layer

- [ ] Create request specs for API endpoints
- [ ] Test data serialization and responses
- [ ] Validate error handling

### Phase 3: Component Testing

- [ ] Test UI components in isolation
- [ ] Validate user interactions
- [ ] Test conditional rendering

### Phase 4: Refactor System Tests

- [ ] Extract critical path tests to dedicated `critical_path/` directory
- [ ] Add comments || commits explaining where coverage moved

### Phase 5: Cleanup

- [ ] Remove redundant pending tests
- [ ] Update documentation
- [ ] Measure performance improvement

4. Code Examples

Provide specific examples for each layer:

# Unit Test Example
RSpec.describe BannerPolicy do
  # Test authorization logic
end

# Integration Test Example
RSpec.describe "Banners API" do
  # Test API responses
end

# Critical Path System Test Example
RSpec.describe 'Critical Path: Payment Update' do
  # Only the most critical E2E flow
  it 'allows billing admin to update payment when past due' do
    # Revenue-critical path
  end
end

# Skipped Test Example (temporary for PR review)
it 'displays banner for non-admin' do
  skip 'Covered by unit tests in spec/policies/...'
  # RJTODO: Remove after PR review
end

// Component Test Example
describe('BannerComponent', () => {
  // Test UI behavior
});

5. Risk Assessment

## Risk Analysis

### Coverage Gaps: [None/List any]

### Migration Risks: [None/List any]

### Mitigation Strategy: [Steps to ensure safety]

6. Metrics and Success Criteria

## Success Metrics

- **Test Runtime**: Before vs After
- **Test Count**: Distribution across layers
- **Coverage**: Maintained at X%
- **Flakiness**: Reduction in intermittent failures
- **Maintenance**: Easier to update and debug

Additional Considerations

Anti-Patterns to Avoid

• Testing implementation details instead of behavior
• Using system tests for data validation
• Testing the framework (Rails/React) itself
• Excessive mocking that doesn't test real behavior
• Testing third-party libraries

Best Practices to Follow

• Test behavior, not implementation
• Use descriptive test names that explain the "why"
• Keep tests isolated and independent
• Mock external dependencies at appropriate layers
• Maintain test data factories/fixtures
• Extract critical paths to dedicated directories for clarity
• Use skip with explanatory messages for pending removal
• Add TODO comments for temporary educational tests

Special Cases

• Multi-tenant applications: Use helpers like with_safe_domain
• Real-time features: Consider component + WebSocket tests
• File uploads: Mock at integration layer, test E2E sparingly
• Email sending: Unit test content, integration test delivery
• Background jobs: Unit test job logic, integration test queueing

Questions to Answer

1. What is the business impact if this feature fails?
2. Can this be tested effectively at a lower layer?
3. Is this testing business logic or framework behavior?
4. How often does this code change?
5. What is the performance cost of this test?

Example Analysis Output Priority

1. Immediate wins - Quick conversions with high impact
2. High-value refactors - Significant improvements requiring more effort
3. Nice-to-haves - Improvements that can be scheduled later
4. Technical debt - Document but defer if not critical

Please analyze the provided system specs using this framework and provide comprehensive recommendations for refactoring according to testing pyramid best practices.