> For the complete documentation index, see [llms.txt](https://doc.ozforensics.com/oz-knowledge/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://doc.ozforensics.com/oz-knowledge/guides/administrator-guide/installation/deployment-architecture.md).

# Deployment Architecture

This page describes the deployment options for Oz API and helps you choose the one that matches your expected workload. It covers the system components, the requirements for each deployment scenario, and the deployment schemes.

## Terms and definitions

| Term | Description                                                                                                                                                                                                                                                                                                                                                                                                 |
| ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| APM  | <p>Analyses per minute. Please note:</p><ul><li>Analysis is a request for <a href="/pages/Brh1hMR704eb7JZXXM1m">Quality (Liveness) or Biometry</a> analysis using a single media.</li><li>A single analysis with multiple media counts as separate analyses in terms of APM.</li><li>Multiple analysis types on single media (two media for Biometry) count as separate analyses in terms of APM.</li></ul> |
| PoC  | Proof of Concept                                                                                                                                                                                                                                                                                                                                                                                            |
| Node | A Node is a worker machine. Can be either a virtual or a physical machine.                                                                                                                                                                                                                                                                                                                                  |
| HA   | High availability                                                                                                                                                                                                                                                                                                                                                                                           |
| K8s  | Kubernetes                                                                                                                                                                                                                                                                                                                                                                                                  |
| SC   | StorageClass                                                                                                                                                                                                                                                                                                                                                                                                |
| RWX  | ReadWriteMany                                                                                                                                                                                                                                                                                                                                                                                               |

## Component descriptions

Oz API components:

* **APP** is the API front app that receives REST requests, performs preprocessing, and creates tasks for other API components.
* **Celery** is the asynchronous task queue. <mark style="background-color:$warning;">Please note: we stopped supporting Celery in 6.5.0.</mark>

<details>

<summary>API 6.5.0 and below celery queues</summary>

* **Celery-default** processes system-wide tasks.
* **Celery-maintenance** processes maintenance tasks.
* **Celery-tfss** processes analysis tasks.
* **Celery-resolution** checks for completion of all nested analyses within a folder and changes folder status.
* **Celery-preview\_convert** creates a video preview for media.
* **Celery-beat** is a CronJob for managing maintenance celery tasks.
* **Celery-Flower** is a Celery metrics collector.
* **Celery-regula** (optional) processes document analysis tasks.

</details>

* **Redis** is a message broker and result backend for Celery.
* **RabbitMQ** (optional) can be used as a message broker for Celery instead of Redis.
* **nginx** serves static media files for external HTTP(s) requests.
* **1:N** (optional) processes the [Collection](/oz-knowledge/guides/developer-guide/api/oz-api/basic-scenarios/collection.md) analysis.
* **Statistic** (optional) provides [statistics](/oz-knowledge/guides/user-guide/oz-webui/webui-statistics.md)' collection for Web UI.
* **Web UI** provides the [web interface](/oz-knowledge/guides/user-guide/oz-webui.md).

**BIO-Updater** checks for models updates and downloads new models.

**Oz BIO (TFSS)** runs TensorFlow with AI models and makes decisions for incoming media.

{% hint style="danger" %}
The **BIO-Updater** and **BIO** components require access to the following external resources:

* <https://api.cryptlex.com/>
* https\://\*.infra.ozforensics.ai/
* https\://\*.s3.amazonaws.com/
* https\://\*.s3-accelerate.amazonaws.com/

**Oz API** also requires access to <https://api.cryptlex.com/>. Endpoints:

* `POST /v3/activations`
* `PATCH /v3/activations/{id}`
* `POST /v3/activations/{id}/deactivate`
  {% endhint %}

## Deployment scenarios

The deployment scenario depends on the workload you expect.

<table data-full-width="true"><thead><tr><th></th><th>Small business or PoC</th><th>Medium load</th><th>High Loal</th></tr></thead><tbody><tr><td>Use cases</td><td><ul><li>Testing/Development purposes</li><li>Small installations with low number of APM</li></ul></td><td><ul><li>Typical usage with moderate load</li></ul></td><td><ul><li>High load with HA and autoscaling</li><li>Usage with cloud provider</li></ul></td></tr><tr><td><a data-footnote-ref href="#user-content-fn-1">Expected workload</a></td><td><ul><li>~<a data-footnote-ref href="#user-content-fn-2">6-8</a> APM</li><li>~<a data-footnote-ref href="#user-content-fn-2">250 000</a> analyses per month</li></ul></td><td><ul><li>~<a data-footnote-ref href="#user-content-fn-3">40-60</a> APM</li><li><a data-footnote-ref href="#user-content-fn-5">500 000 to 1 000 000</a> analyses per month</li></ul></td><td><ul><li><a data-footnote-ref href="#user-content-fn-4">60+</a> APM</li><li><a data-footnote-ref href="#user-content-fn-4">1 000 000+</a> analyses per month</li></ul></td></tr><tr><td>Environment</td><td>Docker</td><td>Docker</td><td>Kubernetes</td></tr><tr><td>HA</td><td>No</td><td>Partially</td><td>Yes</td></tr><tr><td>Pros</td><td><ul><li>Requires a minimal amount of computing resources</li><li>Low complexity, so no high-qualified engineers are needed on-site</li><li>Easy to manage and support</li></ul></td><td><ul><li>Partially supports HA</li><li>Can be scaled up to support higher workload</li></ul></td><td><ul><li>HA and autoscaling</li><li>Observability and manageability</li><li>Allows high workload and can be scaled up</li></ul></td></tr><tr><td>Cons</td><td><ul><li>Suitable only for low loads, no high APM</li><li>No scaling and high-availability</li></ul></td><td><ul><li>API HA requires precise balancing</li><li>Higher staff qualification requirements</li></ul></td><td><ul><li>High staff qualification requirements</li><li>Additional infrastructure requirements</li></ul></td></tr><tr><td>External resource requirements</td><td><a data-footnote-ref href="#user-content-fn-6">PostgreSQL</a></td><td><ul><li>PostgreSQL</li><li><a data-footnote-ref href="#user-content-fn-7">Shared volume (R/W)</a></li></ul></td><td><ul><li><p>For Kubernetes deployments:</p><ul><li>K8s v1.23+</li><li>ingress-nginx</li><li>clusterIssuer</li><li>kube-metrics</li><li>Prometheus</li><li>clusterAutoscaler</li></ul></li><li>PostgreSQL</li><li><a data-footnote-ref href="#user-content-fn-7">Shared volume (R/W)</a></li><li><a data-footnote-ref href="#user-content-fn-8">Redis</a></li></ul></td></tr></tbody></table>

Autoscaling is implemented on the basis of ClusterAutoscaler and must be supported by your infrastructure.

### Small business or PoC <a href="#small-business-poc" id="small-business-poc"></a>

* Type of containerization: Docker (Podman is supported for API 5 only),
* Type of installation: Docker compose,
* Autoscaling/HA: none.

#### Requirements

**Software**

* Docker Engine 20.10+
* Docker Compose 2.20.3+
* Python 3.4+.

**Storage**

* Depends on media quality, the type and number of analyses, and the required archive depth.
* May be calculated as: \[average media size] \* 2 \* \[analyses per day] \* \[archive depth in days]. Please refer to [this article](/oz-knowledge/other/media-file-size-overview.md) for media size reference.
* Each analysis request performs read and write operations on the storage. Any additional latency in these operations will impact the analysis time.

**Staff qualification**

* Basic knowledge of Linux and Docker.

#### Deployment

**Two nodes:**

<figure><img src="/files/6Rw4vHQ1ARDEp1oX0wZb" alt="Architecture diagram of a two-node Docker deployment. Node 1 hosts the WebSDK, WebUI, NGINX proxy, API, PostgreSQL database, and a local volume. Node 2 hosts the TFSS server and its own local volume. A client reaches the WebSDK through an application; an operator reaches the WebUI through a browser. Both go through the NGINX proxy to the API, which connects to the TFSS server on node 2."><figcaption></figcaption></figure>

Resources:

* 2 nodes,
* 16 CPU/32 RAM for the first node; 8 CPU/16 RAM for the second node.

### Medium load

* Type of containerization: Docker (Podman is supported for API 5 only),
* Type of installation: Docker compose,
* Autoscaling/HA: manual scaling; HA is partially supported.

#### Requirements

**Computational resources**

Depending on load, you can change the number of nodes. However, for 5+ nodes, we recommend that you proceed to the High Load section.

* From 2 to 4 Docker nodes (see [schemes](#deployment-1)):
  * 2 Nodes:
    * 24 CPU/32 RAM per node.
  * 3 Nodes:
    * 16 CPU/24 RAM per node.
  * 4 Nodes:
    * 8 CPU/16 RAM for two nodes (each),
    * 16 CPU/24 RAM for two nodes (each).

We recommend using external self-managed PostgreSQL database and NFS share.

**Software**

* Docker Engine 20.10+
* Docker Compose 2.20.3+
* Python 3.4+.

**Storage**

* Depends on media quality, the type and number of analyses, and the required archive depth.
* May be calculated as: \[average media size] \* 2 \* \[analyses per day] \* \[archive depth in days]. Please refer to [this article](/oz-knowledge/other/media-file-size-overview.md) for media size reference.
* Each analysis request performs read and write operations on the storage. Any additional latency in these operations will impact the analysis time.

**Staff qualification**:

* Advanced knowledge of Linux, Docker, and Postgres.

#### Deployment

**Two nodes:**

<figure><img src="/files/3lLJJfXFdDTFO4IiqR5O" alt="> Two-node Docker deployment with external resources. Node 1 hosts the WebSDK, WebUI, NGINX proxy, and API; node 2 hosts the TFSS server. A client reaches the WebSDK through an application, an operator reaches the WebUI through a browser, and both pass through the NGINX proxy to the API. External resources: a PostgreSQL master database with an optional replica, and an NFS share used by the API and the TFSS server."><figcaption></figcaption></figure>

**Three nodes:**

<figure><img src="/files/6w1cfYM2ilu0WHtEFjbI" alt="Three-node Docker deployment with external resources. Node 1 hosts the WebSDK, WebUI, NGINX proxy, and API; nodes 2 and 3 each host a TFSS server. A client reaches the WebSDK through an application, an operator reaches the WebUI through a browser, and both pass through the NGINX proxy to the API. A second NGINX proxy balances requests between the two TFSS servers. External resources: a PostgreSQL master database with an optional replica, and an NFS share used by the API and both TFSS servers."><figcaption></figcaption></figure>

**Four nodes:**

<figure><img src="/files/P33pX2aZcZcSrmMnxIRa" alt="Four-node high-availability deployment. A border balancer routes client and operator traffic to an NGINX balancer, which distributes it across nodes 1 and 2. Each of these nodes hosts its own NGINX balancer, WebSDK, WebUI, and API. Nodes 3 and 4 each host a TFSS server. External resources: an NFS share and a PostgreSQL master database with a replica."><figcaption></figcaption></figure>

### High load

Please find the installation guide here: [Kubernetes](/oz-knowledge/guides/administrator-guide/installation/installation-in-kubernetes.md).

* Type of containerization: Docker containers with Kubernetes orchestration,
* Type of installation: [Helm charts](/oz-knowledge/guides/administrator-guide/installation/installation-in-kubernetes/helm-charts.md),
* Autoscaling/HA: supports autoscaling; HA for most components.

#### Requirements

**Computational resources**

3-4 nodes. Depending on load, you can change the number of nodes.

* 16 CPU/32 RAM Nodes for the BIO pods,
* 8+ CPU/16+ RAM Nodes for all other workload.

We recommend using external self-managed PostgreSQL database.

{% hint style="danger" %}
Requires RWX (ReadWriteMany) StorageClass or NFS share.
{% endhint %}

**Software**

* Docker Engine 20.10+
* Python 3.4+.

**Storage**

* Depends on media quality, the type and number of analyses, and the required archive depth.
* May be calculated as: \[average media size] \* 2 \* \[analyses per day] \* \[archive depth in days]. Please refer to [this article](/oz-knowledge/other/media-file-size-overview.md) for media size reference.
* Each analysis request performs read and write operations on the storage. Any additional latency in these operations will impact the analysis time.

**Staff qualification**:

* Advanced knowledge of Linux, Docker, Kubernetes, and Postgres.

#### Deployment scheme

<figure><img src="/files/gdUB5bS7a0CvLGMpN1K4" alt="Oz deployment for Kubernetes. An ingress load balancer routes traffic to four components: App, Oz WebUI, Nginx, and Oz WebAdapter, each deployed with a deployment, service, pods, and most with an HPA and a Prometheus exporter. Behind them are Redis, Celery beat, Celery Flower, Celery workers, O2N (Collection), Statistics, Bio, and BIO-Updater. Persistent volumes hold static data, and an external PostgreSQL cluster provides a master database, an optional read-only replica, and a pgvector instance. A separate block represents external resources for license checks and update feeds. The legend distinguishes HTTP(S), FTP, database connections over TCP, optional database connections, data read/write to local filesystem or NFS, and Redis traffic; external PostgreSQL is recommended."><figcaption></figcaption></figure>

[^1]: Values are calculated for specific amount of Nodes, recommended for each deployment type.

[^2]: For a single Node. Results are calculated using [`shots_set`](https://doc.ozforensics.com/oz-knowledge/guides/administrator-guide/installation/pages/u43USb2xDyOYuCaeCPjs#h.85wv26msdlji) with additional frame, and may vary depending on incoming mime type (image, video, zip), media quality, length, size, and image contents

[^3]: For 4 nodes. Results are calculated using [`shots_set`](https://doc.ozforensics.com/oz-knowledge/guides/administrator-guide/installation/pages/u43USb2xDyOYuCaeCPjs#h.85wv26msdlji) with additional frame, and may vary depending on incoming mime type (image, video, zip), media quality, length, size, and image contents

[^4]: Results are calculated using [`shots_set`](https://doc.ozforensics.com/oz-knowledge/guides/administrator-guide/installation/pages/u43USb2xDyOYuCaeCPjs#h.85wv26msdlji) with additional frame, and may vary depending on incoming mime type (image, video, zip), media quality, length, size, and image contents

[^5]: 1 000 000 is for 4 nodes. Results are calculated using [`shots_set`](https://doc.ozforensics.com/oz-knowledge/guides/administrator-guide/installation/pages/u43USb2xDyOYuCaeCPjs#h.85wv26msdlji) with lossless frame, and may vary depending on incoming mime type (image, video, zip), media quality, length, size, and image contents

[^6]: External PostgreSQL is not required for PoC or test deployments

[^7]: Required in case of 2+ nodes with API installed (4+ nodes type of deployment) or in case of deployment in Kubernetes. Can be NFS, EFS, CephFS, Longhorn, or any type of FS with RWX support between many nodes

[^8]: Optional. Only standalone Redis is supported


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://doc.ozforensics.com/oz-knowledge/guides/administrator-guide/installation/deployment-architecture.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.