Skip to content

Instantly share code, notes, and snippets.

View nerdalert's full-sized avatar
🐈
🦀 🐿

Brent Salisbury nerdalert

🐈
🦀 🐿
View GitHub Profile
#!/bin/bash
# OpenShift MaaS Platform Deployment Script
# This script automates the complete deployment of the MaaS platform on OpenShift
set -e
# Helper function to wait for CRD to be established
wait_for_crd() {
local crd="$1"

MaaS Introspection Architecture

AuthPolicy Summary

Two AuthPolicies manage authentication:

  • maas-control-plane (deploy/manifests/control-plane-auth-policy.yaml) - JWT auth for admin/management
  • data-plane-auth-gateway (deploy/manifests/data-plane-introspect.yaml) - API key auth for model inference

Data Plane Call Workflow (Model Endpoint + API Key)

$ deployment/scripts/install.sh
[INFO] Detecting available deployments...
[INFO] Found deployments: basic gpu simulator
[INFO] Starting MaaS deployment installation
[INFO] Using deployment: simulator (simulator-deployment)
[INFO] Using existing CLUSTER_DOMAIN: apps.maas.octo-emerging.redhataicoe.com
[INFO] Installing dependencies...
🔧 Installing all MaaS dependencies...
🚀 Installing istio...

MaaS Platform Quickstart

This guide provides the essential commands for end-to-end validation of the MaaS platform.

Prerequisites

Set environment variables:

export CONTROL_BASE="http://maas.apps.maas2.octo-emerging.redhataicoe.com"
  • llm-d-infra/charts/llm-d-infra/templates/gateway-infrastructure/gatewayparameters.yaml
{{ if and .Values.gateway.enabled (eq .Values.gateway.gatewayClassName "kgateway") .Values.gateway.gatewayParameters.enabled }}
apiVersion: gateway.kgateway.dev/v1alpha1
kind: GatewayParameters
metadata:
  name: {{ include "gateway.fullname" . }}
  labels: {{ include "common.labels.standard" . | nindent 4 }}
    app.kubernetes.io/gateway: {{ include "gateway.fullname" . }}
$ kubectl get envoyfilter --all-namespaces -o yaml
apiVersion: v1
items:
- apiVersion: networking.istio.io/v1alpha3
  kind: EnvoyFilter
  metadata:
    creationTimestamp: "2025-07-10T05:32:40Z"
    generation: 1
    labels:

vLLM Inference Simulator

Repo at llm-d/llm-d-inference-sim

1 . Start the sim container

podman || docker run --rm --net host ghcr.io/llm-d/llm-d-inference-sim \
  --port 8000 \
 --model "Qwen/Qwen2.5-1.5B-Instruct" \
2025-06-24T04:04:35.7285268Z Current runner version: '2.325.0'
2025-06-24T04:04:35.7315212Z ##[group]Operating System
2025-06-24T04:04:35.7316005Z Ubuntu
2025-06-24T04:04:35.7316512Z 24.04.2
2025-06-24T04:04:35.7316951Z LTS
2025-06-24T04:04:35.7317533Z ##[endgroup]
2025-06-24T04:04:35.7318061Z ##[group]Runner Image
2025-06-24T04:04:35.7318693Z Image: ubuntu-24.04
2025-06-24T04:04:35.7319275Z Version: 20250615.1.0
2025-06-24T04:04:35.7320471Z Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20250615.1/images/ubuntu/Ubuntu2404-Readme.md
2025-06-24T03:52:30.5394764Z Current runner version: '2.325.0'
2025-06-24T03:52:30.5428738Z ##[group]Operating System
2025-06-24T03:52:30.5429883Z Ubuntu
2025-06-24T03:52:30.5430846Z 24.04.2
2025-06-24T03:52:30.5431596Z LTS
2025-06-24T03:52:30.5432469Z ##[endgroup]
2025-06-24T03:52:30.5433460Z ##[group]Runner Image
2025-06-24T03:52:30.5434453Z Image: ubuntu-24.04
2025-06-24T03:52:30.5435403Z Version: 20250615.1.0
2025-06-24T03:52:30.5437208Z Included Software: https://github.com/actions/runner-images/blob/ubuntu24/20250615.1/images/ubuntu/Ubuntu2404-Readme.md