Skip to content

Instantly share code, notes, and snippets.

@justinlevi

justinlevi/AI Infrastructure and Tools.md

Last active December 12, 2024 13:39
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save justinlevi/a8fe6d7631eb31832bcb6c8499bb18ff to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save justinlevi/a8fe6d7631eb31832bcb6c8499bb18ff to your computer and use it in GitHub Desktop.
Download ZIP

Revisions

  1. justinlevi revised this gist Dec 12, 2024. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions AI Infrastructure and Tools.md
    View file
    Original file line number Diff line number Diff line change
    @@ -65,6 +65,8 @@



    ## References


    - Qdrant: https://qdrant.tech/
    - Pinecone: https://www.pinecone.io/
  2. justinlevi revised this gist Dec 12, 2024. 1 changed file with 27 additions and 27 deletions.
    54 changes: 27 additions & 27 deletions AI Infrastructure and Tools.md
    View file
    Original file line number Diff line number Diff line change
    @@ -66,30 +66,30 @@



    Qdrant: https://qdrant.tech/
    Pinecone: https://www.pinecone.io/
    Weaviate: https://weaviate.io/
    PGVector: https://github.com/pgvector/pgvector
    Supabase: https://supabase.com/
    LLM Serving Frameworks/Tools
    vLLM: https://github.com/vllm-project/vllm
    Ollama: https://ollama.ai/
    LM Studio: https://lmstudio.ai/
    Groq: https://groq.com/
    MLX (Apple ML Ecosystem): https://developer.apple.com/machine-learning/
    KServe: https://kserve.github.io/
    Agent Development Frameworks
    LangChain: https://www.langchain.com/
    LlamaIndex: https://llamaindex.ai/
    AutoGen (Magnetic-One): https://github.com/microsoft/autogen
    CrewAI: https://crew.ai/
    MLOps Pipeline Tools
    Apache Kafka: https://kafka.apache.org/
    Google Cloud MLOps (Vertex AI): https://cloud.google.com/vertex-ai
    Databricks MLflow: https://mlflow.org/
    Kubeflow: https://www.kubeflow.org/
    Delta Lake: https://delta.io/
    Monitoring and Observability
    Arize Phoenix: https://github.com/Arize-ai/phoenix or https://www.arize.com/phoenix
    Evidently: https://evidentlyai.com/
    Seldon: https://www.seldon.io/
    - Qdrant: https://qdrant.tech/
    - Pinecone: https://www.pinecone.io/
    - Weaviate: https://weaviate.io/
    - PGVector: https://github.com/pgvector/pgvector
    - Supabase: https://supabase.com/
    - LLM Serving Frameworks/Tools
    - vLLM: https://github.com/vllm-project/vllm
    - Ollama: https://ollama.ai/
    - LM Studio: https://lmstudio.ai/
    - Groq: https://groq.com/
    - MLX (Apple ML Ecosystem): https://developer.apple.com/machine-learning/
    - KServe: https://kserve.github.io/
    - Agent Development Frameworks
    - LangChain: https://www.langchain.com/
    - LlamaIndex: https://llamaindex.ai/
    - AutoGen (Magnetic-One): https://github.com/microsoft/autogen
    - CrewAI: https://crew.ai/
    - MLOps Pipeline Tools
    - Apache Kafka: https://kafka.apache.org/
    - Google Cloud MLOps (Vertex AI): https://cloud.google.com/vertex-ai
    - Databricks MLflow: https://mlflow.org/
    - Kubeflow: https://www.kubeflow.org/
    - Delta Lake: https://delta.io/
    - Monitoring and Observability
    - Arize Phoenix: https://github.com/Arize-ai/phoenix or https://www.arize.com/phoenix
    - Evidently: https://evidentlyai.com/
    - Seldon: https://www.seldon.io/
  3. justinlevi revised this gist Dec 12, 2024. 1 changed file with 32 additions and 1 deletion.
    33 changes: 32 additions & 1 deletion AI Infrastructure and Tools.md
    View file
    Original file line number Diff line number Diff line change
    @@ -61,4 +61,35 @@

    - **Evidently** - Tool for ML model monitoring and evaluation in production

    - **Seldon** - Enterprise platform for deploying and monitoring ML models at scale
    - **Seldon** - Enterprise platform for deploying and monitoring ML models at scale




    Qdrant: https://qdrant.tech/
    Pinecone: https://www.pinecone.io/
    Weaviate: https://weaviate.io/
    PGVector: https://github.com/pgvector/pgvector
    Supabase: https://supabase.com/
    LLM Serving Frameworks/Tools
    vLLM: https://github.com/vllm-project/vllm
    Ollama: https://ollama.ai/
    LM Studio: https://lmstudio.ai/
    Groq: https://groq.com/
    MLX (Apple ML Ecosystem): https://developer.apple.com/machine-learning/
    KServe: https://kserve.github.io/
    Agent Development Frameworks
    LangChain: https://www.langchain.com/
    LlamaIndex: https://llamaindex.ai/
    AutoGen (Magnetic-One): https://github.com/microsoft/autogen
    CrewAI: https://crew.ai/
    MLOps Pipeline Tools
    Apache Kafka: https://kafka.apache.org/
    Google Cloud MLOps (Vertex AI): https://cloud.google.com/vertex-ai
    Databricks MLflow: https://mlflow.org/
    Kubeflow: https://www.kubeflow.org/
    Delta Lake: https://delta.io/
    Monitoring and Observability
    Arize Phoenix: https://github.com/Arize-ai/phoenix or https://www.arize.com/phoenix
    Evidently: https://evidentlyai.com/
    Seldon: https://www.seldon.io/
  4. justinlevi created this gist Dec 12, 2024.
    64 changes: 64 additions & 0 deletions AI Infrastructure and Tools.md
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,64 @@
    # AI Infrastructure and Tools Overview

    ## Vector Database Storage Frameworks/Services
    *A specialized storage system designed to efficiently handle and query high-dimensional vector data, enabling similarity search and AI applications*

    - **Qdrant** - Open-source vector database written in Rust offering high-performance similarity search with cloud-native scalability

    - **Pinecone** - Serverless vector database platform optimized for machine learning applications with enterprise-grade security

    - **Weaviate** - Fast, flexible AI-native vector database with built-in model serving and multi-tenant capabilities

    - **PGVector** - PostgreSQL extension that enables vector similarity search with ACID compliance and SQL integration

    - **Supabase** - Postgres-based platform that includes vector search capabilities alongside other database features

    ## LLM Serving Frameworks/Tools
    *Frameworks and platforms that enable efficient deployment and serving of large language models, optimizing for performance and scalability in production environments*

    - **vLLM** - High-performance inference engine using PagedAttention for optimal serving throughput

    - **Ollama** - Framework for running and serving large language models locally with cross-platform support

    - **LM Studio** - Local LLM development environment with built-in model management and serving capabilities

    - **Groq** - Cloud platform offering ultra-fast LLM inference with specialized hardware acceleration

    - **MLX** - Apple's machine learning framework optimized for Apple Silicon

    - **KServe** - Kubernetes-based model serving platform supporting multiple frameworks and auto-scaling

    ## Agent Development Frameworks
    *Software infrastructures that support the creation and management of autonomous agents, providing tools for development, deployment, and interaction between AI agents*

    - **LangChain** - Comprehensive framework for building and connecting LLM-powered applications

    - **LangGraph** - Framework specialized in creating stateful, multi-agent workflows

    - **LlamaIndex** - Framework for building RAG applications with data connection capabilities

    - **AutoGen/Magnetic-One** - Microsoft's framework enabling sophisticated multi-agent collaboration patterns

    - **CrewAI** - Platform for creating role-based AI agent teams with specialized tasks and collaboration

    ## MLOps Pipeline Tools
    *Software components that enable continuous integration, delivery, and automation of machine learning workflows, including model training, testing, and deployment*

    - **Apache Kafka** - Distributed streaming platform for building real-time data pipelines

    - **Google Cloud MLOps** - Comprehensive suite of tools for ML model deployment and management

    - **Databricks MLflow** - End-to-end platform for managing the ML lifecycle with experiment tracking

    - **Kubeflow** - Kubernetes-native platform for deploying ML workflows

    - **Delta Lake** - Storage layer that brings ACID transactions to data lakes

    ## Monitoring and Observability
    *Platforms and solutions that provide visibility into AI system performance, helping detect issues, analyze behavior, and ensure reliability of AI deployments*

    - **Arize Phoenix** - Open-source platform specifically designed for LLM observability and evaluation

    - **Evidently** - Tool for ML model monitoring and evaluation in production

    - **Seldon** - Enterprise platform for deploying and monitoring ML models at scale