README

Get started: InstallTutorialDocsExamples

Learn more: WebsiteBlogSubscribeTwitterContact

Cortex deploys your machine learning models to your cloud infrastructure. You define your deployment with simple declarative configuration, Cortex containerizes your models, deploys them as autoscaling JSON APIs, and manages their lifecycle in production.

Cortex is actively maintained by Cortex Labs. We're a venture-backed team of infrastructure engineers and we're hiring.

How it works

Define your deployment using declarative configuration:

# cortex.yaml
- kind: api
name: my-api
model: s3://my-bucket/my-model.zip
request_handler: handler.py
compute:
min_replicas: 5
max_replicas: 20

Customize request handling (optional):

# handler.py
def pre_inference(sample, metadata):
# Python code
def post_inference(prediction, metadata):
# Python code

Deploy to your cloud infrastructure:

$ cortex deploy
Deploying ...
Ready! https://amazonaws.com/my-api

Serve real time predictions via scalable JSON APIs:

$ curl -d '{"a": 1, "b": 2, "c": 3}' https://amazonaws.com/my-api
{ prediction: "def" }

Key features

  • Machine learning deployments as code: Cortex deployments are defined using declarative configuration.

  • Autoscaling: Cortex can automatically scale APIs to handle production workloads.

  • Multi framework support: Cortex supports TensorFlow, Keras, PyTorch, Scikit-learn, XGBoost, and more.

  • CPU / GPU support: Cortex can run inference on CPU or GPU infrastructure.

  • Rolling updates: Cortex updates deployed APIs without any downtime.

  • Cloud native: Cortex can be deployed on any AWS account in minutes.