Read Me

Get started: InstallTutorialDemo VideoDocsExamples

Learn more: WebsiteFAQBlogSubscribeTwitterContact

Machine learning pipelines as code

Deploy machine learning applications without worrying about setting up infrastructure, managing dependencies, or orchestrating data pipelines.

How it works

  1. Define your app: define your app using Python, TensorFlow, and PySpark.

  2. $ cortex deploy: deploy end-to-end machine learning pipelines to AWS with one command.

  3. Serve predictions: serve real time predictions via horizontally scalable JSON APIs.

End-to-end machine learning workflow

Data ingestion: connect to your data warehouse and ingest data.

- kind: environment
name: dev
data:
type: csv
path: s3a://my-bucket/data.csv
schema: [@col1, @col2, ...]

Data transformation: use custom Python and PySpark code to transform data.

- kind: transformed_column
name: col1_normalized
transformer_path: normalize.py # Python / PySpark code
input: @col1

Model training: train models with custom TensorFlow code.

- kind: model
name: my_model
estimator_path: dnn.py # TensorFlow code
target_column: @label_col
input: [@col1_normalized, @col2_indexed, ...]
hparams:
hidden_units: [16, 8]
training:
batch_size: 32
num_steps: 10000

Prediction serving: serve real time predictions via JSON APIs.

- kind: api
name: my-api
model: @my_model
compute:
replicas: 3

Deployment: Cortex deploys your pipeline on scalable cloud infrastructure.

$ cortex deploy
Ingesting data ...
Transforming data ...
Training models ...
Deploying API ...
Ready! https://abc.amazonaws.com/my-api

Key features

  • Machine learning pipelines as code: Cortex applications are defined using a simple declarative syntax that enables flexibility and reusability.

  • End-to-end machine learning workflow: Cortex spans the machine learning workflow from feature management to model training to prediction serving.

  • TensorFlow and PySpark support: Cortex supports custom TensorFlow code for model training and custom PySpark code for data processing.

  • Built for the cloud: Cortex can handle production workloads and can be deployed in any AWS account in minutes.