404/503 API responses

When making prediction requests to your API, it's possible to get a {"message":"Not Found"} error message (with HTTP status code 404), or a no healthy upstream error message (with HTTP status code 503). This means that there are currently no live replicas running for your API. This could happen for a few reasons:

  1. It's possible that your API is simply not ready yet. You can check the status of your API with cortex get API_NAME, and stream the logs with cortex logs API_NAME.

  2. Your API may have errored during initialization or while responding to a previous request. cortex get API_NAME will show the status of your API, and you can view the logs with cortex logs API_NAME.

  3. If cortex get API_NAME shows your API's status as "updating" for a while and if cortex logs API_NAME doesn't shed any light onto what may be wrong, please see the API is stuck updating troubleshooting guide.

It is also possible to receive a {"message":"Service Unavailable"} error message (with HTTP status code 503) if you are using an API Gateway endpoint for your API and if your request exceeds API Gateway's 29 second timeout. If you don't know whether you are using API Gateway, you can run cortex get <api_name> and check if networking.api_gateway is not set to none in the api configuration. If the request is exceeding the API Gateway timeout, your client should receive the {"message":"Service Unavailable"} response ~29 seconds after making the request. To confirm that this is the issue, you can modify your predict() function to immediately return a response (e.g. return "ok"), re-deploy your API, wait for the update to complete, and try making a request. If your client successfully receives the "ok" response, it is likely that the API Gateway timeout is occurring. You can either modify your predict() implementation to take less time, run on faster hardware (e.g. GPUs), or disable API Gateway for this API by setting api_gateway: none in the networking field of the api configuration (see networking for more details).