# Configuration ```yaml - name: kind: AsyncAPI predictor: # detailed configuration below compute: # detailed configuration below autoscaling: # detailed configuration below update_strategy: # detailed configuration below networking: # detailed configuration below ``` ## Predictor ### Python Predictor ```yaml predictor: type: python path: # path to a python file with a PythonPredictor class definition, relative to the Cortex root (required) dependencies: # (optional) pip: # relative path to requirements.txt (default: requirements.txt) conda: # relative path to conda-packages.txt (default: conda-packages.txt) shell: # relative path to a shell script for system package installation (default: dependencies.sh) config: # arbitrary dictionary passed to the constructor of the Predictor (optional) python_path: # path to the root of your Python folder that will be appended to PYTHONPATH (default: folder containing cortex.yaml) image: # docker image to use for the Predictor (default: quay.io/cortexlabs/python-predictor-cpu:0.33.0, quay.io/cortexlabs/python-predictor-gpu:0.33.0-cuda10.2-cudnn8, or quay.io/cortexlabs/python-predictor-inf:0.33.0 based on compute) env: # dictionary of environment variables log_level: # log level that can be "debug", "info", "warning" or "error" (default: "info") shm_size: # size of shared memory (/dev/shm) for sharing data between multiple processes, e.g. 64Mi or 1Gi (default: Null) ``` ### Tensorflow Predictor ```yaml predictor: type: tensorflow path: # path to a python file with a TensorFlowPredictor class definition, relative to the Cortex root (required) dependencies: # (optional) pip: # relative path to requirements.txt (default: requirements.txt) conda: # relative path to conda-packages.txt (default: conda-packages.txt) shell: # relative path to a shell script for system package installation (default: dependencies.sh) models: # (required) path: # S3 path to an exported SavedModel directory (e.g. s3://my-bucket/exported_model/) (either this, 'dir', or 'paths' must be provided) signature_key: # name of the signature def to use for prediction (required if your model has more than one signature def) config: # arbitrary dictionary passed to the constructor of the Predictor (optional) python_path: # path to the root of your Python folder that will be appended to PYTHONPATH (default: folder containing cortex.yaml) image: # docker image to use for the Predictor (default: quay.io/cortexlabs/tensorflow-predictor:0.33.0) tensorflow_serving_image: # docker image to use for the TensorFlow Serving container (default: quay.io/cortexlabs/tensorflow-serving-cpu:0.33.0, quay.io/cortexlabs/tensorflow-serving-gpu:0.33.0, or quay.io/cortexlabs/tensorflow-serving-inf:0.33.0 based on compute) env: # dictionary of environment variables log_level: # log level that can be "debug", "info", "warning" or "error" (default: "info") shm_size: # size of shared memory (/dev/shm) for sharing data between multiple processes, e.g. 64Mi or 1Gi (default: Null) ``` ## Compute ```yaml compute: cpu: # CPU request per replica. One unit of CPU corresponds to one virtual CPU; fractional requests are allowed, and can be specified as a floating point number or via the "m" suffix (default: 200m) gpu: # GPU request per replica. One unit of GPU corresponds to one virtual GPU (default: 0) mem: # memory request per replica. One unit of memory is one byte and can be expressed as an integer or by using one of these suffixes: K, M, G, T (or their power-of two counterparts: Ki, Mi, Gi, Ti) (default: Null) ``` ## Autoscaling ```yaml autoscaling: min_replicas: # minimum number of replicas (default: 1) max_replicas: # maximum number of replicas (default: 100) init_replicas: # initial number of replicas (default: ) max_replica_concurrency: # the maximum number of in-flight requests per replica before requests are rejected with error code 503 (default: 1024) target_replica_concurrency: # the desired number of in-flight requests per replica, which the autoscaler tries to maintain (default: processes_per_replica * threads_per_process) window: # the time over which to average the API's concurrency (default: 60s) downscale_stabilization_period: # the API will not scale below the highest recommendation made during this period (default: 5m) upscale_stabilization_period: # the API will not scale above the lowest recommendation made during this period (default: 1m) max_downscale_factor: # the maximum factor by which to scale down the API on a single scaling event (default: 0.75) max_upscale_factor: # the maximum factor by which to scale up the API on a single scaling event (default: 1.5) downscale_tolerance: # any recommendation falling within this factor below the current number of replicas will not trigger a scale down event (default: 0.05) upscale_tolerance: # any recommendation falling within this factor above the current number of replicas will not trigger a scale up event (default: 0.05) ``` ## Update strategy ```yaml update_strategy: max_surge: # maximum number of replicas that can be scheduled above the desired number of replicas during an update; can be an absolute number, e.g. 5, or a percentage of desired replicas, e.g. 10% (default: 25%) (set to 0 to disable rolling updates) max_unavailable: # maximum number of replicas that can be unavailable during an update; can be an absolute number, e.g. 5, or a percentage of desired replicas, e.g. 10% (default: 25%) ``` ## Networking ```yaml networking: endpoint: # the endpoint for the API (default: ) ``` --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.cortexlabs.com/0.33/workloads/async-apis/configuration.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.