Metrics

The cortex get and cortex get API_NAME commands display the request time (averaged over the past 2 weeks) and response code counts (summed over the past 2 weeks) for your APIs:

$ cortex get

env   api                         status   up-to-date   requested   last update   avg request   2XX
aws   iris-classifier             live     1            1           17m           24ms          1223
aws   text-generator              live     1            1           8m            180ms         433
aws   image-classifier-resnet50   live     2            2           1h            32ms          1121126

The cortex get API_NAME command also provides a link to a Grafana dashboard:

dashboard

Metrics in the dashboard

Panel	Description	Note
Request Rate	Request rate, computed over every minute, of an API
In Flight Request	Active in-flight requests for an API.	In-flight requests are recorded every 10 seconds, which will correspond to the minimum resolution.
Active Replicas	Active replicas for an API
2XX Responses	Request rate, computed over a minute, for responses with status code 2XX of an API
4XX Responses	Request rate, computed over a minute, for responses with status code 4XX of an API
5XX Responses	Request rate, computed over a minute, for responses with status code 5XX of an API
p99 Latency	99th percentile latency, computed over a minute, for an API	Value might not be accurate because the histogram buckets are not dynamically set.
p90 Latency	90th percentile latency, computed over a minute, for an API	Value might not be accurate because the histogram buckets are not dynamically set.
p50 Latency	50th percentile latency, computed over a minute, for an API	Value might not be accurate because the histogram buckets are not dynamically set.
Average Latency	Average latency, computed over a minute, for an API

Accessing the dashboard

The dashboard URL is displayed once you run a cortex get <api_name> command.

Alternatively, you can access it on http://<operator_url>/dashboard. Run the following command to get the operator URL:

cortex env list

Default credentials

The dashboard is protected with username / password authentication, which by default are:

• Username: admin

• Password: admin

You will be prompted to change the admin user password in the first time you log in.

Grafana allows managing the access of several users and managing teams. For more information on this topic check the grafana documentation.

Selecting an API

You can select one or more APIs to visualize in the top left corner of the dashboard.

Selecting a time range

Grafana allows you to select a time range on which the metrics will be visualized. You can do so in the top right corner of the dashboard.

Note: Cortex only retains a maximum of 2 weeks worth of data at any moment in time

Available dashboards

There are more than one dashboard available by default. You can view the available dashboards by accessing the Grafana menu: Dashboards -> Manage -> Cortex folder.

Exposed metrics

Cortex exposes more metrics with Prometheus, that can be potentially useful. To check the available metrics, access the Explore menu in grafana and press the Metrics button.

You can use any of these metrics to set up your own dashboards.