# Configuration ```yaml - name: kind: BatchAPI predictor: # detailed configuration below compute: # detailed configuration below networking: # detailed configuration below ``` ## Predictor ### Python Predictor ```yaml predictor: type: python path: # path to a python file with a PythonPredictor class definition, relative to the Cortex root (required) config: # arbitrary dictionary passed to the constructor of the Predictor (can be overridden by config passed in job submission) (optional) python_path: # path to the root of your Python folder that will be appended to PYTHONPATH (default: folder containing cortex.yaml) image: # docker image to use for the Predictor (default: quay.io/cortexlabs/python-predictor-cpu:0.28.0 or quay.io/cortexlabs/python-predictor-gpu:0.28.0 based on compute) env: # dictionary of environment variables log_level: # log level that can be "debug", "info", "warning" or "error" (default: "info") shm_size: # size of shared memory (/dev/shm) for sharing data between multiple processes, e.g. 64Mi or 1Gi (default: Null) ``` ### TensorFlow Predictor ```yaml predictor: type: tensorflow path: # path to a python file with a TensorFlowPredictor class definition, relative to the Cortex root (required) models: # use this to serve a single model or multiple ones path: # S3 path to an exported model (e.g. s3://my-bucket/exported_model) (either this or 'paths' field must be provided) paths: # (either this or 'path' must be provided) - name: # unique name for the model (e.g. text-generator) (required) path: # S3 path to an exported model (e.g. s3://my-bucket/exported_model) (required) signature_key: # name of the signature def to use for prediction (required if your model has more than one signature def) ... signature_key: # name of the signature def to use for prediction (required if your model has more than one signature def) server_side_batching: # (optional) max_batch_size: # the maximum number of requests to aggregate before running inference batch_interval: # the maximum amount of time to spend waiting for additional requests before running inference on the batch of requests config: # arbitrary dictionary passed to the constructor of the Predictor (can be overridden by config passed in job submission) (optional) python_path: # path to the root of your Python folder that will be appended to PYTHONPATH (default: folder containing cortex.yaml) image: # docker image to use for the Predictor (default: quay.io/cortexlabs/tensorflow-predictor:0.28.0) tensorflow_serving_image: # docker image to use for the TensorFlow Serving container (default: quay.io/cortexlabs/tensorflow-serving-cpu:0.28.0 or quay.io/cortexlabs/tensorflow-serving-gpu:0.28.0 based on compute) env: # dictionary of environment variables log_level: # log level that can be "debug", "info", "warning" or "error" (default: "info") shm_size: # size of shared memory (/dev/shm) for sharing data between multiple processes, e.g. 64Mi or 1Gi (default: Null) ``` ### ONNX Predictor ```yaml predictor: type: onnx path: # path to a python file with an ONNXPredictor class definition, relative to the Cortex root (required) models: # use this to serve a single model or multiple ones path: # S3 path to an exported model (e.g. s3://my-bucket/exported_model) (either this or 'paths' must be provided) paths: # (either this or 'path' must be provided) - name: # unique name for the model (e.g. text-generator) (required) path: # S3 path to an exported model (e.g. s3://my-bucket/exported_model.onnx) (required) ... config: # arbitrary dictionary passed to the constructor of the Predictor (can be overridden by config passed in job submission) (optional) python_path: # path to the root of your Python folder that will be appended to PYTHONPATH (default: folder containing cortex.yaml) image: # docker image to use for the Predictor (default: quay.io/cortexlabs/onnx-predictor-cpu:0.28.0 or quay.io/cortexlabs/onnx-predictor-gpu:0.28.0 based on compute) env: # dictionary of environment variables log_level: # log level that can be "debug", "info", "warning" or "error" (default: "info") shm_size: # size of shared memory (/dev/shm) for sharing data between multiple processes, e.g. 64Mi or 1Gi (default: Null) ``` ## Compute ```yaml compute: cpu: # CPU request per worker. One unit of CPU corresponds to one virtual CPU; fractional requests are allowed, and can be specified as a floating point number or via the "m" suffix (default: 200m) gpu: # GPU request per worker. One unit of GPU corresponds to one virtual GPU (default: 0) mem: # memory request per worker. One unit of memory is one byte and can be expressed as an integer or by using one of these suffixes: K, M, G, T (or their power-of two counterparts: Ki, Mi, Gi, Ti) (default: Null) ``` ## Networking ```yaml networking: endpoint: # the endpoint for the API (default: ) ``` --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.cortexlabs.com/0.28/workloads/batch-apis/configuration.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.