If you are using an internal operator load balancer (i.e. you set
operator_load_balancer_scheme: internal in your cluster configuration file before creating your cluster), you can use VPC Peering to enable your Cortex CLI to connect to your cluster operator from another VPC so that you may run
cortex commands. Note that because the operator validates that the CLI user is an active IAM user in the same AWS account as the Cortex cluster, it is usually unnecessary to configure the operator's load balancer to be internal.
If you are using an internal API load balancer (i.e. you set
api_load_balancer_scheme: internal in your cluster configuration file before creating your cluster) and you disabled API Gateway for your API (i.e. you set
api_gateway: none in the
networking field of your api configuration), you can use VPC Peering to enable prediction requests from another VPC.
This guide illustrates how to create a VPC Peering connection between a VPC of your choice and the Cortex load balancers.
Navigate to AWS's EC2 Load Balancer dashboard and locate the Cortex operator's load balancer. You can determine which is the operator load balancer by inspecting the
Click back to the "Description" tab and note the VPC ID of the load balancer and the ID of each of the subnets associated with the load balancer:
Navigate to AWS's VPC dashboard and identify the ID and CIDR block of Cortex's VPC:
The VPC ID here should match that of the load balancer.
Identify the ID and CIDR block of the VPC from which you'd like to connect to the Cortex VPC.
In my case, I have a VPC in the same AWS account and region, and I can locate its ID and CIDR block from AWS's VPC dashboard:
From AWS's VPC dashboard, navigate to the "Peering Connections" page, and click "Create Peering Connection":
Name your new VPC Peering Connection (I used "cortex-operator", but "cortex" or "cortex-api" may make more sense depending on your use case). Then configure the connection such that the "Requester" is the VPC from which you'll connect to the Cortex VPC, and the "Accepter" is Cortex's VPC (from step 1).
Click "Create Peering Connection", navigate back to the Peering Connections dashboard, select the newly created peering connection, and click "Actions" > "Accept Request":
Navigate to the VPC Route Tables page. Select the route table for the VPC from which you'd like to connect to the Cortex cluster (in my case, I just have one route table for this VPC). Select the "Routes" tab, and click "Edit routes":
Add a route where the "Destination" is the CIDR block for Cortex's VPC (identified in Step 1), and the "Target" is the newly-created Peering Connection:
Do not create new route tables or change subnet associations.
Navigate back to the VPC Route Tables page. There will be a route table for each of the subnets associated with the Cortex operator load balancer (identified in Step 1):
For each of these route tables, click "Edit routes" and add a new route where the "Destination" is the CIDR block for the VPC from which you will be connecting to the Cortex cluster (identified in Step 2), and the "Target" is the newly-created Peering Connection:
Repeat adding this route for each route table associated with the Cortex operator's subnets; in my case there were three. Do not create new route tables or change subnet associations.
You should now be able to use the Cortex CLI and make prediction requests from your VPC.
Delete the VPC Peering connection before spinning down your Cortex cluster: