Set up VPC peering

If you are using an internal operator load balancer (i.e. you set operator_load_balancer_scheme: internal in your cluster configuration file before creating your cluster), you can use VPC Peering to enable your Cortex CLI to connect to your cluster operator from another VPC so that you may run cortex commands.

If you are using an internal API load balancer (i.e. you set api_load_balancer_scheme: internal in your cluster configuration file before creating your cluster), you can use VPC Peering to enable prediction requests from another VPC. Note: if you intend to create a public endpoint for your internal API load balancer, see our API Gateway guide.

This guide illustrates how to create a VPC Peering connection between a VPC of your choice and the Cortex load balancers.

Step 1

Navigate to AWS's EC2 Load Balancer dashboard and locate the Cortex operator's load balancer. You can determine which is the operator load balancer by inspecting the kubernetes.io/service-name tag:

step 1a

Click back to the "Description" tab and note the VPC ID of the load balancer and the ID of each of the subnets associated with the load balancer:

step 1b

Navigate to AWS's VPC dashboard and identify the ID and CIDR block of Cortex's VPC:

step 1c

The VPC ID here should match that of the load balancer.

Step 2

Identify the ID and CIDR block of the VPC from which you'd like to connect to the Cortex VPC.

In my case, I have a VPC in the same AWS account and region, and I can locate its ID and CIDR block from AWS's VPC dashboard:

step 2

Step 3

From AWS's VPC dashboard, navigate to the "Peering Connections" page, and click "Create Peering Connection":

step 3a

Name your new VPC Peering Connection (I used "cortex-operator", but "cortex" or "cortex-api" may make more sense depending on your use case). Then configure the connection such that the "Requester" is the VPC from which you'll connect to the Cortex VPC, and the "Accepter" is Cortex's VPC (from step 1).

step 3b

Click "Create Peering Connection", navigate back to the Peering Connections dashboard, select the newly created peering connection, and click "Actions" > "Accept Request":

step 3c
step 3d

Step 4

Navigate to the VPC Route Tables page. Select the route table for the VPC from which you'd like to connect to the Cortex cluster (in my case, I just have one route table for this VPC). Select the "Routes" tab, and click "Edit routes":

step 4a

Add a route where the "Destination" is the CIDR block for Cortex's VPC (identified in Step 1), and the "Target" is the newly-created Peering Connection:

step 4b

Do not create new route tables or change subnet associations.

Step 5

Navigate back to the VPC Route Tables page. There will be a route table for each of the subnets associated with the Cortex operator load balancer (identified in Step 1):

step 5a

For each of these route tables, click "Edit routes" and add a new route where the "Destination" is the CIDR block for the VPC from which you will be connecting to the Cortex cluster (identified in Step 2), and the "Target" is the newly-created Peering Connection:

step 5b

Repeat adding this route for each route table associated with the Cortex operator's subnets; in my case there were three. Do not create new route tables or change subnet associations.

You should now be able to use the Cortex CLI and make prediction requests from your VPC.

Cleanup

Delete the VPC Peering connection before spinning down your Cortex cluster:

cleanup