Lifecycle platformOpen source
MLflow
Эксперименты, registry, модели и GenAI evaluation
- LF AI & Data
- Self-hosted / Cloud
Каталог решений
Инструменты LLMOps и production AI
Inference servers, AI gateways, evaluation, observability, guardrails, registry, orchestration и инфраструктура — в единой системе критериев.
Все 55LLM observability 6Experiment platform 3Inference server 3AI gateway 2AI observability 2AI platform 2Guardrails 2LLM evaluation 2Load testing 2Serving framework 2Serving platform 2Workflow orchestration 2AI monitoring 1AI security 1AI testing 1Data and model versioning 1Data versioning 1Distributed tracing 1Durable workflows 1Evaluation platform 1Feature store 1Inference microservices 1Inference optimization 1Lifecycle platform 1LLM gateway 1LLM serving 1LLM telemetry 1Local runtime 1Managed GenAI 1Metrics 1ML platform 1Model registry 1Observability 1Orchestration 1RAG evaluation 1Telemetry 1Visualization 1
AI gatewayCommercial / open components
Kong AI Gateway
Управление LLM API, политиками и трафиком
- Kong
- Self-hosted / Cloud
AI gatewayOpen source
Envoy AI Gateway
Gateway для маршрутизации LLM-трафика
- Envoy Project
- Kubernetes
LLM gatewayOpen source
LiteLLM
Единый API, routing, budgets и fallback
- BerriAI
- Self-hosted / Cloud
LLM servingOpen source
OpenLLM
Запуск open-source LLM как API
- BentoML
- Self-hosted
Serving frameworkOpen source
Ray Serve
Масштабируемые Python inference-графы
- Anyscale / Ray
- Self-hosted / Cloud
Serving frameworkOpen source
BentoML
Упаковка и развёртывание AI-сервисов
- BentoML
- Self-hosted / Cloud
Serving platformOpen source
Seldon Core
Inference pipelines и оркестрация моделей
- Seldon
- Kubernetes
Serving platformOpen source
KServe
Kubernetes-native serving и autoscaling
- KServe Project
- Kubernetes
Inference serverOpen source
NVIDIA Triton
Multi-framework model serving
- NVIDIA
- Self-hosted
Inference optimizationOpen source
TensorRT-LLM
Оптимизированный inference на NVIDIA GPU
- NVIDIA
- Self-hosted
Inference serverOpen source
Text Generation Inference
Serving генеративных моделей Hugging Face
- Hugging Face
- Self-hosted
Local runtimeOpen source
Ollama
Локальный запуск и разработка LLM-приложений
- Ollama
- Local / Self-hosted
Inference serverOpen source
vLLM
Высокопроизводительный serving LLM на GPU
- vLLM Project
- Self-hosted
Experiment platformCommercial
Neptune
Метаданные экспериментов и model registry
- Neptune Labs
- Cloud / Self-hosted
TelemetryOpen source
OpenTelemetry
Стандарт трасс, метрик и логов
- CNCF
- Self-hosted / Cloud
LLM telemetryOpen source
OpenLLMetry
OpenTelemetry instrumentation для LLM
- Traceloop
- Self-hosted / Cloud
Experiment platformCommercial
Weights & Biases
Эксперименты, models, Weave и evaluation
- Weights & Biases
- Cloud / Dedicated
Data and model versioningOpen source
DVC
Версионирование данных, моделей и экспериментов
- Iterative
- Self-hosted / Cloud
Load testingOpen source
Grafana k6
Нагрузочное и performance-тестирование
- Grafana Labs
- Local / Cloud
AI securityCommercial
Lakera Guard
Защита от prompt injection и угроз GenAI
- Lakera
- Cloud / Private
Feature storeOpen source
Feast
Управление online/offline features
- Feast Project
- Self-hosted / Cloud
GuardrailsOpen source
Guardrails AI
Валидация входов и структурированных выходов
- Guardrails AI
- Self-hosted / Cloud
Load testingOpen source
Locust
Нагрузочное тестирование API
- Locust
- Self-hosted
Distributed tracingOpen source
Jaeger
Распределённые трассы сервисов
- CNCF
- Self-hosted
ML platformOpen source
Kubeflow
ML workflow и платформенные компоненты Kubernetes
- Kubeflow Project
- Kubernetes
GuardrailsOpen source
NeMo Guardrails
Программируемые rails для LLM-приложений
- NVIDIA
- Self-hosted
AI testingOpen source
Giskard
Тестирование, red teaming и quality assurance
- Giskard
- Self-hosted / Cloud
ObservabilityCommercial
Honeycomb
Высококардинальная observability и tracing
- Honeycomb
- Cloud
LLM observabilityCommercial
Datadog LLM Observability
LLM traces в едином observability-контуре
- Datadog
- Cloud
RAG evaluationOpen source
Ragas
Evaluation RAG и генеративных приложений
- Ragas
- Local / Cloud
Managed GenAICommercial
Amazon Bedrock
Managed foundation models и GenAI services
- AWS
- Cloud
LLM evaluationOpen source
DeepEval
Pytest-подобные тесты LLM-приложений
- Confident AI
- Local / Cloud
LLM evaluationOpen source
Promptfoo
Тестирование prompts, моделей и red teaming
- Promptfoo
- Local / Cloud
AI platformCommercial
Vertex AI
ML и GenAI lifecycle в Google Cloud
- Google Cloud
- Cloud
AI observabilityCommercial
Fiddler AI
Model monitoring, explainability и governance
- Fiddler
- Cloud / Self-hosted
AI platformCommercial
Microsoft Foundry
Управление GenAI-приложениями и моделями Azure
- Microsoft
- Cloud
Inference microservicesCommercial
NVIDIA NIM
Оптимизированные inference microservices
- NVIDIA
- Self-hosted / Cloud
AI observabilityCommercial
WhyLabs
Мониторинг данных и моделей
- WhyLabs
- Cloud / Hybrid
AI monitoringOpen source
Evidently
Мониторинг качества, drift и тесты данных
- Evidently AI
- Self-hosted / Cloud
Model registryOpen platform / Commercial
Hugging Face Hub
Модели, datasets, Spaces и registry workflows
- Hugging Face
- Cloud / Enterprise
Data versioningOpen source
lakeFS
Git-подобное версионирование object storage
- Treeverse
- Self-hosted / Cloud
LLM observabilityOpen source
Opik
Open-source evaluation и observability
- Comet
- Self-hosted / Cloud
Evaluation platformCommercial
Braintrust
Evals, datasets, prompts и production logging
- Braintrust
- Cloud
Durable workflowsOpen source
Temporal
Надёжные долгоживущие workflow
- Temporal
- Self-hosted / Cloud
Workflow orchestrationOpen source
Apache Airflow
Планирование и управление batch workflow
- Apache Software Foundation
- Self-hosted / Cloud
LLM observabilityOpen source
Helicone
LLM gateway, logging, cost и latency
- Helicone
- Self-hosted / Cloud
Workflow orchestrationOpen source
Argo Workflows
Kubernetes-native пайплайны
- CNCF
- Kubernetes
LLM observabilityOpen source
Arize Phoenix
Open-source трассировка и evaluation AI-приложений
- Arize AI
- Self-hosted
OrchestrationOpen source
Kubernetes
Оркестрация контейнеров и ресурсов
- CNCF
- Self-hosted / Managed
LLM observabilityCommercial
LangSmith
Трассировка, evaluation и deployment workflows
- LangChain
- Cloud / Hybrid
VisualizationOpen source
Grafana
Дашборды, алерты и корреляция телеметрии
- Grafana Labs
- Self-hosted / Cloud
LLM observabilityOpen source
Langfuse
Трейсы, prompts, datasets, evals и метрики
- Langfuse
- Self-hosted / Cloud
Experiment platformCommercial
Comet
Experiment management и model monitoring
- Comet
- Cloud / Self-hosted
MetricsOpen source
Prometheus
Метрики и alerting инфраструктуры
- CNCF
- Self-hosted