Search…
Multi-instance
Cortex can be configured to provision different instance types to improve workload performance and reduce cloud infrastructure spend.

Best practices

    1.
    Spot node groups should have a higher priority than on-demand node groups.
    2.
    CPU node groups should have higher priorities than GPU/Inferentia node groups.
    3.
    Node groups with small instance types should have higher priorities than node groups with large instance types.

Examples

CPU spot cluster, with on-demand backup

1
# cluster.yaml
2
3
node_groups:
4
- name: cpu-spot
5
instance_type: m5.large
6
min_instances: 0
7
max_instances: 5
8
priority: 100
9
spot: true
10
spot_config:
11
instance_distribution: [m5a.large, m5d.large, m5n.large, m5ad.large, m5dn.large, m4.large, t3.large, t3a.large, t2.large]
12
- name: cpu-on-demand
13
instance_type: m5.large
14
min_instances: 0
15
max_instances: 5
Copied!

On-demand cluster supporting CPU, GPU, and Inferentia

1
# cluster.yaml
2
3
node_groups:
4
- name: cpu
5
instance_type: m5.large
6
min_instances: 0
7
max_instances: 5
8
priority: 100
9
- name: gpu
10
instance_type: g4dn.xlarge
11
min_instances: 0
12
max_instances: 5
13
- name: inf
14
instance_type: inf.xlarge
15
min_instances: 0
16
max_instances: 5
Copied!

Spot cluster supporting CPU and GPU (with on-demand backup)

1
# cluster.yaml
2
3
node_groups:
4
- name: cpu-spot
5
instance_type: m5.large
6
min_instances: 0
7
max_instances: 5
8
priority: 100
9
spot: true
10
spot_config:
11
instance_distribution: [m5a.large, m5d.large, m5n.large, m5ad.large, m5dn.large, m4.large, t3.large, t3a.large, t2.large]
12
- name: cpu-on-demand
13
instance_type: m5.large
14
min_instances: 0
15
max_instances: 5
16
priority: 50
17
- name: gpu-spot
18
instance_type: g4dn.xlarge
19
min_instances: 0
20
max_instances: 5
21
priority: 20
22
spot: true
23
- name: gpu-on-demand
24
instance_type: g4dn.xlarge
25
min_instances: 0
26
max_instances: 5
Copied!

CPU spot cluster with multiple instance types and on-demand backup

1
# cluster.yaml
2
3
node_groups:
4
- name: cpu-1
5
instance_type: t3.medium
6
min_instances: 0
7
max_instances: 5
8
priority: 100
9
spot: true
10
- name: cpu-2
11
instance_type: m5.2xlarge
12
min_instances: 0
13
max_instances: 5
14
priority: 70
15
spot: true
16
- name: cpu-3
17
instance_type: m5.8xlarge
18
min_instances: 0
19
max_instances: 5
20
priority: 30
21
spot: true
22
- name: cpu-4
23
instance_type: m5.24xlarge
24
min_instances: 0
25
max_instances: 5
Copied!
Last modified 2mo ago