Search…
Cluster configuration
The Cortex cluster may be configured by providing a configuration file to cortex cluster up or cortex cluster configure via the --config flag (e.g. cortex cluster up --config cluster.yaml). Below is the schema for the cluster configuration file, with default values shown (unless otherwise specified):
1
# cluster.yaml
2
3
# EKS cluster name for cortex (default: cortex)
4
cluster_name: cortex
5
6
# AWS region
7
region: us-east-1
8
9
# S3 bucket (default: <cluster_name>-<RANDOM_ID>)
10
# note: your cortex cluster uses this bucket for metadata storage, and it should not be accessed directly (a separate bucket should be used for your models)
11
bucket: # cortex-<RANDOM_ID>
12
13
# list of availability zones for your region (default: 3 random availability zones from the specified region)
14
availability_zones: # e.g. [us-east-1a, us-east-1b, us-east-1c]
15
16
# instance type
17
instance_type: m5.large
18
19
# minimum number of instances (must be >= 0)
20
min_instances: 1
21
22
# maximum number of instances (must be >= 1)
23
max_instances: 5
24
25
# disk storage size per instance (GB) (default: 50)
26
instance_volume_size: 50
27
28
# instance volume type [gp2, io1, st1, sc1] (default: gp2)
29
instance_volume_type: gp2
30
31
# instance volume iops (only applicable to io1 storage type) (default: 3000)
32
# instance_volume_iops: 3000
33
34
# whether the subnets used for EC2 instances should be public or private (default: "public")
35
# if "public", instances will be assigned public IP addresses; if "private", instances won't have public IPs and a NAT gateway will be created to allow outgoing network requests
36
# see https://docs.cortex.dev/v/0.22/miscellaneous/security#private-cluster for more information
37
subnet_visibility: public # must be "public" or "private"
38
39
# whether to include a NAT gateway with the cluster (a NAT gateway is necessary when using private subnets)
40
# default value is "none" if subnet_visibility is set to "public"; "single" if subnet_visibility is "private"
41
nat_gateway: none # must be "none", "single", or "highly_available" (highly_available means one NAT gateway per availability zone)
42
43
# whether the API load balancer should be internet-facing or internal (default: "internet-facing")
44
# note: if using "internal", APIs will still be accessible via the public API Gateway endpoint unless you also disable API Gateway in your API's configuration (if you do that, you must configure VPC Peering to connect to your APIs)
45
# see https://docs.cortex.dev/v/0.22/miscellaneous/security#private-cluster for more information
46
api_load_balancer_scheme: internet-facing # must be "internet-facing" or "internal"
47
48
# whether the operator load balancer should be internet-facing or internal (default: "internet-facing")
49
# note: if using "internal", you must configure VPC Peering to connect your CLI to your cluster operator (https://docs.cortex.dev/v/0.22/guides/vpc-peering)
50
# see https://docs.cortex.dev/v/0.22/miscellaneous/security#private-operator for more information
51
operator_load_balancer_scheme: internet-facing # must be "internet-facing" or "internal"
52
53
# whether to disable API gateway cluster-wide
54
# if set to "public" (the default), each API can specify whether to use API Gateway
55
# if set to "none", no APIs will be allowed to use API Gateway
56
api_gateway: public # must be "public" or "none"
57
58
# additional tags to assign to aws resources for labelling and cost allocation (by default, all resources will be tagged with cortex.dev/cluster-name=<cluster_name>)
59
tags: # <string>: <string> map of key/value pairs
60
61
# whether to use spot instances in the cluster (default: false)
62
# see https://docs.cortex.dev/v/0.22/cluster-management/spot-instances for additional details on spot configuration
63
spot: false
64
65
# see https://docs.cortex.dev/v/0.22/guides/custom-domain for instructions on how to set up a custom domain
66
ssl_certificate_arn:
67
68
# primary CIDR block for the cluster's VPC (default: 192.168.0.0/16)
69
# vpc_cidr: 192.168.0.0/16
Copied!
The default docker images used for your Predictors are listed in the instructions for system packages, and can be overridden in your Realtime API configuration and in your Batch API configuration.
The docker images used by the Cortex cluster can also be overridden, although this is not common. They can be configured by adding any of these keys to your cluster configuration file (default values are shown):
1
# docker image paths
2
image_operator: cortexlabs/operator:0.22.1
3
image_manager: cortexlabs/manager:0.22.1
4
image_downloader: cortexlabs/downloader:0.22.1
5
image_request_monitor: cortexlabs/request-monitor:0.22.1
6
image_cluster_autoscaler: cortexlabs/cluster-autoscaler:0.22.1
7
image_metrics_server: cortexlabs/metrics-server:0.22.1
8
image_inferentia: cortexlabs/inferentia:0.22.1
9
image_neuron_rtd: cortexlabs/neuron-rtd:0.22.1
10
image_nvidia: cortexlabs/nvidia:0.22.1
11
image_fluentd: cortexlabs/fluentd:0.22.1
12
image_statsd: cortexlabs/statsd:0.22.1
13
image_istio_proxy: cortexlabs/istio-proxy:0.22.1
14
image_istio_pilot: cortexlabs/istio-pilot:0.22.1
Copied!
Last modified 10mo ago
Copy link