Created
September 11, 2025 16:55
-
-
Save nerdalert/d993fed38b925269285397f71054c6d6 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ deployment/scripts/install.sh | |
| [INFO] Detecting available deployments... | |
| [INFO] Found deployments: basic gpu simulator | |
| [INFO] Starting MaaS deployment installation | |
| [INFO] Using deployment: simulator (simulator-deployment) | |
| [INFO] Using existing CLUSTER_DOMAIN: apps.maas.octo-emerging.redhataicoe.com | |
| [INFO] Installing dependencies... | |
| 🔧 Installing all MaaS dependencies... | |
| 🚀 Installing istio... | |
| 🚢 Installing Istio for MaaS deployment | |
| Using Istio version: 1.26.2 from gcr.io/istio-release | |
| Checking for Gateway API CRDs... | |
| Gateway API CRDs already exist (managed by OpenShift Ingress Operator) | |
| 🔧 Installing Istio base from OCI registry... | |
| Release "istio-base" does not exist. Installing it now. | |
| Pulled: gcr.io/istio-release/charts/base:1.26.2 | |
| Digest: sha256:cbab61e93e28537580a89a940002b3a80dd95dc2c5870ed541111f7483cc990d | |
| NAME: istio-base | |
| LAST DEPLOYED: Thu Sep 11 16:53:18 2025 | |
| NAMESPACE: istio-system | |
| STATUS: deployed | |
| REVISION: 1 | |
| TEST SUITE: None | |
| NOTES: | |
| Istio base successfully installed! | |
| To learn more about the release, try: | |
| $ helm status istio-base -n istio-system | |
| $ helm get all istio-base -n istio-system | |
| 🔧 Installing Istiod from OCI registry... | |
| Release "istiod" does not exist. Installing it now. | |
| Pulled: gcr.io/istio-release/charts/istiod:1.26.2 | |
| Digest: sha256:355a4a60439eca91a8cfbabcba6c1b58bff07ca9f43061e261f86f64b9f35a48 | |
| NAME: istiod | |
| LAST DEPLOYED: Thu Sep 11 16:53:20 2025 | |
| NAMESPACE: istio-system | |
| STATUS: deployed | |
| REVISION: 1 | |
| TEST SUITE: None | |
| NOTES: | |
| "istiod" successfully installed! | |
| To learn more about the release, try: | |
| $ helm status istiod -n istio-system | |
| $ helm get all istiod -n istio-system | |
| Next steps: | |
| * Deploy a Gateway: https://istio.io/latest/docs/setup/additional-setup/gateway/ | |
| * Try out our tasks to get started on common configurations: | |
| * https://istio.io/latest/docs/tasks/traffic-management | |
| * https://istio.io/latest/docs/tasks/security/ | |
| * https://istio.io/latest/docs/tasks/policy-enforcement/ | |
| * Review the list of actively supported releases, CVE publications and our hardening guide: | |
| * https://istio.io/latest/docs/releases/supported-releases/ | |
| * https://istio.io/latest/news/security/ | |
| * https://istio.io/latest/docs/ops/best-practices/security/ | |
| For further documentation see https://istio.io website | |
| ✅ Successfully installed istio | |
| 🚀 Installing cert-manager... | |
| 🔒 Installing cert-manager for MaaS deployment | |
| Using cert-manager version: v1.18.2 | |
| 🔧 Installing cert-manager... | |
| namespace/cert-manager unchanged | |
| customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io unchanged | |
| customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io unchanged | |
| customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io unchanged | |
| customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io unchanged | |
| customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io unchanged | |
| customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io unchanged | |
| serviceaccount/cert-manager-cainjector unchanged | |
| serviceaccount/cert-manager unchanged | |
| serviceaccount/cert-manager-webhook unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-cluster-view unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-view unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-edit unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests unchanged | |
| clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews unchanged | |
| clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector unchanged | |
| clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers unchanged | |
| clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers unchanged | |
| clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates unchanged | |
| clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders unchanged | |
| clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges unchanged | |
| clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim unchanged | |
| clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-approve:cert-manager-io unchanged | |
| clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificatesigningrequests unchanged | |
| clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:subjectaccessreviews unchanged | |
| role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection unchanged | |
| role.rbac.authorization.k8s.io/cert-manager:leaderelection unchanged | |
| role.rbac.authorization.k8s.io/cert-manager-tokenrequest unchanged | |
| role.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving unchanged | |
| rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection unchanged | |
| rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection unchanged | |
| rolebinding.rbac.authorization.k8s.io/cert-manager-cert-manager-tokenrequest unchanged | |
| rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving unchanged | |
| service/cert-manager-cainjector unchanged | |
| service/cert-manager unchanged | |
| service/cert-manager-webhook unchanged | |
| deployment.apps/cert-manager-cainjector unchanged | |
| deployment.apps/cert-manager unchanged | |
| deployment.apps/cert-manager-webhook unchanged | |
| mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook configured | |
| validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook configured | |
| ⏳ Waiting for cert-manager to be ready... | |
| deployment.apps/cert-manager condition met | |
| deployment.apps/cert-manager-cainjector condition met | |
| deployment.apps/cert-manager-webhook condition met | |
| ✅ Successfully installed cert-manager | |
| 🚀 Installing kserve... | |
| 🤖 Installing model serving platform for MaaS deployment | |
| Using vanilla KServe version: v0.15.2 | |
| 🔧 Installing KServe... | |
| namespace/kserve serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/clusterservingruntimes.serving.kserve.io serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/clusterstoragecontainers.serving.kserve.io serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/inferencegraphs.serving.kserve.io serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/inferenceservices.serving.kserve.io serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/localmodelcaches.serving.kserve.io serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/localmodelnodegroups.serving.kserve.io serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/localmodelnodes.serving.kserve.io serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/servingruntimes.serving.kserve.io serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/trainedmodels.serving.kserve.io serverside-applied | |
| serviceaccount/kserve-controller-manager serverside-applied | |
| serviceaccount/kserve-localmodel-controller-manager serverside-applied | |
| serviceaccount/kserve-localmodelnode-agent serverside-applied | |
| role.rbac.authorization.k8s.io/kserve-leader-election-role serverside-applied | |
| clusterrole.rbac.authorization.k8s.io/kserve-localmodel-manager-role serverside-applied | |
| clusterrole.rbac.authorization.k8s.io/kserve-localmodelnode-agent-role serverside-applied | |
| clusterrole.rbac.authorization.k8s.io/kserve-manager-role serverside-applied | |
| clusterrole.rbac.authorization.k8s.io/kserve-proxy-role serverside-applied | |
| rolebinding.rbac.authorization.k8s.io/kserve-leader-election-rolebinding serverside-applied | |
| clusterrolebinding.rbac.authorization.k8s.io/kserve-localmodel-manager-rolebinding serverside-applied | |
| clusterrolebinding.rbac.authorization.k8s.io/kserve-localmodelnode-agent-rolebinding serverside-applied | |
| clusterrolebinding.rbac.authorization.k8s.io/kserve-manager-rolebinding serverside-applied | |
| clusterrolebinding.rbac.authorization.k8s.io/kserve-proxy-rolebinding serverside-applied | |
| configmap/inferenceservice-config serverside-applied | |
| secret/kserve-webhook-server-secret serverside-applied | |
| service/kserve-controller-manager-metrics-service serverside-applied | |
| service/kserve-controller-manager-service serverside-applied | |
| service/kserve-webhook-server-service serverside-applied | |
| deployment.apps/kserve-controller-manager serverside-applied | |
| deployment.apps/kserve-localmodel-controller-manager serverside-applied | |
| Warning: would violate PodSecurity "restricted:latest": restricted volume types (volume "models" uses restricted volume type "hostPath"), seccompProfile (pod or container "manager" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost") | |
| daemonset.apps/kserve-localmodelnode-agent serverside-applied | |
| certificate.cert-manager.io/serving-cert serverside-applied | |
| issuer.cert-manager.io/selfsigned-issuer serverside-applied | |
| mutatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied | |
| validatingwebhookconfiguration.admissionregistration.k8s.io/clusterservingruntime.serving.kserve.io serverside-applied | |
| validatingwebhookconfiguration.admissionregistration.k8s.io/inferencegraph.serving.kserve.io serverside-applied | |
| validatingwebhookconfiguration.admissionregistration.k8s.io/inferenceservice.serving.kserve.io serverside-applied | |
| validatingwebhookconfiguration.admissionregistration.k8s.io/localmodelcache.serving.kserve.io serverside-applied | |
| validatingwebhookconfiguration.admissionregistration.k8s.io/servingruntime.serving.kserve.io serverside-applied | |
| validatingwebhookconfiguration.admissionregistration.k8s.io/trainedmodel.serving.kserve.io serverside-applied | |
| ⏳ Waiting for KServe controller... | |
| deployment.apps/kserve-controller-manager condition met | |
| ✅ Successfully installed kserve | |
| 🚀 Installing prometheus... | |
| 📊 Installing Prometheus Operator for MaaS observability | |
| 🔧 Installing Prometheus Operator... | |
| customresourcedefinition.apiextensions.k8s.io/alertmanagerconfigs.monitoring.coreos.com serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/alertmanagers.monitoring.coreos.com serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/podmonitors.monitoring.coreos.com serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/probes.monitoring.coreos.com serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/prometheusagents.monitoring.coreos.com serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/prometheuses.monitoring.coreos.com serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/prometheusrules.monitoring.coreos.com serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/scrapeconfigs.monitoring.coreos.com serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/servicemonitors.monitoring.coreos.com serverside-applied | |
| customresourcedefinition.apiextensions.k8s.io/thanosrulers.monitoring.coreos.com serverside-applied | |
| clusterrolebinding.rbac.authorization.k8s.io/prometheus-operator serverside-applied | |
| clusterrole.rbac.authorization.k8s.io/prometheus-operator serverside-applied | |
| deployment.apps/prometheus-operator serverside-applied | |
| serviceaccount/prometheus-operator serverside-applied | |
| service/prometheus-operator serverside-applied | |
| ⏳ Waiting for Prometheus Operator to be ready... | |
| deployment.apps/prometheus-operator condition met | |
| 📊 Access Prometheus metrics (in separate terminals): | |
| kubectl port-forward -n llm-observability svc/models-aas-observability 9090:9090 | |
| kubectl port-forward -n kuadrant-system svc/limitador-limitador 8080:8080 | |
| Then visit: http://localhost:9090 (Prometheus) and http://localhost:8080/metrics (Limitador) | |
| ✅ Successfully installed prometheus | |
| 🚀 Installing kuadrant (via Helm)... | |
| 🔄 Updating Helm repo kuadrant... | |
| Hang tight while we grab the latest from your chart repositories... | |
| ...Successfully got an update from the "llm-d-modelservice" chart repository | |
| ...Successfully got an update from the "kuadrant" chart repository | |
| ...Successfully got an update from the "prometheus-community" chart repository | |
| ...Successfully got an update from the "bitnami" chart repository | |
| Update Complete. ⎈Happy Helming!⎈ | |
| 📦 Installing kuadrant-operator chart (1.3.0-alpha2) | |
| Release "kuadrant-operator" does not exist. Installing it now. | |
| NAME: kuadrant-operator | |
| LAST DEPLOYED: Thu Sep 11 16:53:35 2025 | |
| NAMESPACE: kuadrant-system | |
| STATUS: deployed | |
| REVISION: 1 | |
| TEST SUITE: None | |
| ⏳ Waiting for operators to be ready... | |
| deployment.apps/kuadrant-operator-controller-manager condition met | |
| deployment.apps/limitador-operator-controller-manager condition met | |
| Error from server (NotFound): deployments.apps "authorino-operator-controller-manager" not found | |
| deployment.apps/authorino-operator condition met | |
| ✅ Successfully installed kuadrant | |
| 🎉 All components installed successfully! | |
| [SUCCESS] Dependencies installed | |
| [INFO] Cleaning up conflicting operators... | |
| [INFO] Configuring Kuadrant CRs... | |
| kuadrant.kuadrant.io/kuadrant created | |
| limitador.limitador.kuadrant.io/limitador created | |
| authorino.operator.authorino.kuadrant.io/authorino created | |
| istio.sailoperator.io/default unchanged | |
| [SUCCESS] Kuadrant configured | |
| [INFO] Deploying simulator with external access... | |
| namespace/llm created | |
| namespace/llm-observability created | |
| namespace/platform-services unchanged | |
| serviceaccount/kserve-service-account created | |
| serviceaccount/key-manager unchanged | |
| role.rbac.authorization.k8s.io/key-manager-policies created | |
| role.rbac.authorization.k8s.io/key-manager-secrets created | |
| clusterrole.rbac.authorization.k8s.io/key-manager-inference-services unchanged | |
| clusterrole.rbac.authorization.k8s.io/key-manager-kuadrant-restart created | |
| rolebinding.rbac.authorization.k8s.io/key-manager-policies created | |
| rolebinding.rbac.authorization.k8s.io/key-manager-secrets created | |
| clusterrolebinding.rbac.authorization.k8s.io/key-manager-inference-services unchanged | |
| clusterrolebinding.rbac.authorization.k8s.io/key-manager-kuadrant-restart created | |
| configmap/inferenceservice-config configured | |
| secret/freeuser1-apikey created | |
| secret/freeuser2-apikey created | |
| secret/premiumuser1-apikey created | |
| secret/premiumuser2-apikey created | |
| secret/key-manager-admin configured | |
| service/inference-gateway-envoy-metrics created | |
| service/key-manager unchanged | |
| service/key-manager-external unchanged | |
| deployment.apps/key-manager unchanged | |
| gateway.gateway.networking.k8s.io/inference-gateway created | |
| httproute.gateway.networking.k8s.io/key-manager-domain-route created | |
| httproute.gateway.networking.k8s.io/qwen3-domain-route created | |
| httproute.gateway.networking.k8s.io/simulator-domain-route created | |
| referencegrant.gateway.networking.k8s.io/key-manager-access unchanged | |
| authpolicy.kuadrant.io/gateway-auth-policy created | |
| authpolicy.kuadrant.io/key-manager-auth-override created | |
| tokenratelimitpolicy.kuadrant.io/gateway-token-rate-limits created | |
| kuadrant.kuadrant.io/kuadrant unchanged | |
| limitador.limitador.kuadrant.io/limitador unchanged | |
| servicemonitor.monitoring.coreos.com/authorino-runtime created | |
| servicemonitor.monitoring.coreos.com/limitador-runtime created | |
| servicemonitor.monitoring.coreos.com/inference-gateway-envoy-metrics created | |
| ingress.networking.k8s.io/key-manager-ingress unchanged | |
| authorino.operator.authorino.kuadrant.io/authorino unchanged | |
| route.route.openshift.io/key-manager-route created | |
| route.route.openshift.io/qwen3-route created | |
| route.route.openshift.io/simulator-route created | |
| route.route.openshift.io/key-manager-route unchanged | |
| istio.sailoperator.io/default unchanged | |
| securitycontextconstraints.security.openshift.io/kserve-scc configured | |
| servingruntime.serving.kserve.io/vllm-latest created | |
| inferenceservice.serving.kserve.io/qwen3-0-6b-instruct created | |
| inferenceservice.serving.kserve.io/vllm-simulator created | |
| [SUCCESS] Deployment completed successfully! | |
| [SUCCESS] === Deployment Complete === | |
| [INFO] Cluster Domain: apps.maas.octo-emerging.redhataicoe.com | |
| [INFO] External Routes: | |
| - Simulator: simulator-llm.apps.maas.octo-emerging.redhataicoe.com | |
| - Qwen3: qwen3-llm.apps.maas.octo-emerging.redhataicoe.com | |
| - Key Manager: key-manager.apps.maas.octo-emerging.redhataicoe.com | |
| [INFO] Check deployment status with: | |
| kubectl get pods -n llm | |
| kubectl get routes -n llm | |
| 🐈 ip-172-31-45-224:~/maas-billing/v8-helm/maas-billing$ kubectl get pods # --all-namespaces | |
| NAME READY STATUS RESTARTS AGE | |
| inference-gateway-istio-78bcbbdd4c-g7rrv 1/1 Running 0 11s | |
| qwen3-0-6b-instruct-predictor-67d8c9fcc6-z469f 0/1 Pending 0 9s | |
| vllm-simulator-predictor-dcd85657c-n5nwz 0/1 Running 0 8s | |
| 🐈 ip-172-31-45-224:~/maas-billing/v8-helm/maas-billing$ k get routes | |
| NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD | |
| key-manager-route key-manager.apps.summit-gpu.octo-emerging.redhataicoe.com inference-gateway-istio 80 None | |
| qwen3-route qwen3-llm.apps.maas.octo-emerging.redhataicoe.com inference-gateway-istio 80 None | |
| simulator-route simulator-llm.apps.maas.octo-emerging.redhataicoe.com inference-gateway-istio 80 None | |
| 🐈 ip-172-31-45-224:~/maas-billing/v8-helm/maas-billing$ k get inferenceservice -A | |
| NAMESPACE NAME URL READY PREV LATEST PREVROLLEDOUTREVISION LATESTREADYREVISION AGE | |
| llm qwen3-0-6b-instruct 24s | |
| llm vllm-simulator http://vllm-simulator-llm.apps.maas.octo-emerging.redhataicoe.com True 24s | |
| maas-db vllm-simulator False 5d11h | |
| 🐈 ip-172-31-45-224:~/maas-billing/v8-helm/maas-billing$ curl -sS 'http://simulator.db.apps.maas2.octo-emerging.redhataicoe.com/v1/chat/completions' -H 'Authorization: APIKEY 8HuQEpknp-4M-5zzlJ6v169Pe7nZLcO5hnVWD3YPMX9hPl_g' -H 'Content-Type: application/json' -d '{"model":"simulator-model","messages":[{"role":"user","content":"Tell me about MaaS!"}],"max_tokens":100}' | jq | |
| { | |
| "id": "chatcmpl-1757609683", | |
| "object": "chat.completion", | |
| "created": 1757609683, | |
| "model": "simulator-model", | |
| "choices": [ | |
| { | |
| "index": 0, | |
| "message": { | |
| "role": "assistant", | |
| "content": "This is a simulated response to: Tell me about MaaS!" | |
| }, | |
| "finish_reason": "stop" | |
| } | |
| ], | |
| "usage": { | |
| "prompt_tokens": 10, | |
| "completion_tokens": 20, | |
| "total_tokens": 30 | |
| } | |
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment