Hardware and Software Requirements

To launch the services, you'll require:

• CPU: 16 cores,

• RAM: 32 GB,

• Disk: 100 GB, SSD,

• Linux-compatible OS,

• Docker 19.03+ (or Podman 4.4+),

• Docker Compose 1.27+ (or podman-compose 1.2.0+, if you use Podman).

For Docker installations with multiple API servers, you'll also require shared volume or NFS.

Distribution Package Contents

The package you get consists of the following directories and files:

// scripts for preliminary checks of hosts to ensure compliance 
// with software and hardware requirements
|-[checkers]
|   |--pre-checker-all.sh
|   |--pre-checker-api.sh
|   |--pre-checker-bio.sh
// subdirectories with configuration files for the services in use
|-[configs]
|   |--[api]
|   |--[init]
|   |--[nginx]
|   |--[o2n]
|   |--[pg-o2n]
|   |--[postgres]
|   |--[redis]
|   |--[statistic]
|   |--[tfss]
|   |--[webui]
|   |--config.env
// service data and the TFSS models
|-[data]
|   |--api
|   |--pg-o2n
|   |--postgres
|   |--redis
|   |--tfss
// manifest for running all services on a single host
|-docker-compose-all.yml
// manifest for services related only to the API
|-docker-compose-api.yml
// manifest for the TFSS service
|-docker-compose-bio.yml

Installation

Installing TFSS and the API on the Same Host

1. Put the license file in ./configs/tfss/license.key.

2. Unzip the file that contains models into the ./data/tfss/models directory.

3. Before starting system configuration, we recommend running the host readiness check scripts. Navigate to the checkers directory and run the pre-checker-all.sh script.

docker compose --env-file configs/config.env -f docker-compose-all.yml 

1. Set the initial passwords and values:

1. For this configuration, run all services on a single host:

docker compose --env-file configs/config.env -f docker-compose-all.yml up -d

Installing TFSS and the API on Separate Hosts

TFSS Host

1. Create a directory and unzip the distribution package into it. The package contains Docker Compose manifests and directories with the configuration files required for operation.

2. Put the license file in ./configs/tfss/license.key.

3. Unzip the file that contains models into the ./data/tfss/models directory.

4. Before starting system configuration, we recommend running the host readiness check scripts. Navigate to the checkers directory and run the pre-checker-all.sh script.

   Copy

   ./pre-checker-bio.sh

5. For this configuration, run TFSS service on a separate host:

docker compose --env-file configs/config.env -f docker-compose-bio.yml up -d

API Host

1. Create a directory and unzip the distribution package into it. The package contains Docker Compose manifests and directories with the configuration files required for operation.

2. Before starting system configuration, we recommend running the host readiness check scripts. Navigate to the checkers directory and run the pre-checker-all.sh script.

./pre-checker-bio.sh

1. Set the initial passwords and values as described in step 4 of the same host installation.

2. For this configuration, run all services on a single host:

docker compose --env-file configs/config.env -f docker-compose-api.yml up -d

The Oz system consists of many interconnected components.

Most of the components require scaling to ensure functionality during an increase in APM. The scheme below represents how the system processes an analysis using supportive software.

This guide explains how to scale components and improve performance for both Docker and Kubernetes deployments.

Scaling Oz BIO and Celery-TFSS

BIO is the most resource-consuming component as it performs media analysis processing.

Node requirements

• CPU must support avx, avx2, fma, sse4.1, and sse4.2.

• If CPU support avx512f instructions: to slightly increase performance, you can use a specific BIO image.

• We recommend using Intel CPU. AMD CPUs are also supported, but require additional configuration.

• For better performance, each BIO should reside on a dedicated node with reserved resources.

Scaling

Each BIO instance can handle an unlimited number of simultaneous requests. However, as the number increases, the execution time for each request grows.

The assignment of analyses to BIO instances is managed by celery-tfss workers. Each celery-tfss instance is configured to handle 2 analyses simultaneously, as this provides optimal performance and analysis time.

For Kubernetes installations, this behavior can be adjusted using the concurrency parameter in values.yaml. In the default configuration, the number of celery-tfss instances should match the number of BIOs.

The required number of BIOs can be calculated using the formula below, based on the average analysis time and the expected number of analyses:

Here,

• N(BIO) is the required number of BIO nodes

• n(APM) is the expected number of APM,

• t(analysis) is the measured average duration of a single analysis (3 seconds by default),

• C is concurrency (2 by default).

For a large number of requests involving .zip archives (e.g., if you use Web SDK), you might require additional scaling of the celery-preview_convert service.

Database

Oz API is not intended to be a long-term analysis storage system, so you might encounter longer API response time after 10-30 million folders stored in database (depending on the database performance). Thus, we recommend archiving or deleting data from API after one year of storing it.

For the self-managed database, you'll require the following resources:

API

• CPU: 4 cores (up to 8),

• RAM: 8 GB (up to 16),

• SSD storage.

O2N

• CPU: 4 cores (up to 8),

• RAM: equal to the database size,

• SSD storage.

To increase API performance, consider using this list of indexes:

For high-load environments, achieving the best performance requires precise database tuning. Please contact our support for assistance.

Gesture Performance

This table represents analysis duration for different gestures.

API Methods Performance

Below is a list of recommended and non-recommended practices for using Oz API methods.

/authorize

Do

• Refresh expired token when possible.

• Monitor token expiration.

• Use service token only in when appropriate.

Avoid

• Making requests with an expired token.

• Creating a new token instead of refreshing an expiring one.

• Requesting a new token on each API call.

/api/companies

Do

• Create a new named company for your deployment.

Avoid

• Using the default system_company.

/api/users

Do

• Create named accounts for users.

• Create a separate named service account for each business process.

Avoid

• Sharing accounts between users.

• Creating a new user for each analysis request.

/api/healthcheck

Avoid

• Using healthcheck too frequently as it may overwhelm system with internal checks.

GET /api/folders

Do

• Always use the time-limiting query parameters: time.created.min and time.created.max.

• Use the total_omit=true query parameter.

• Use the folder_id query parameter when you know the folder ID.

• Use the technical_meta_data query parameter in case you have meta_data set.

• Use the limit query parameter when possible.

Avoid

• Using the with_analyzes=true query parameter when it is unnecessary, in requests involving long time periods or unspecified durations.

• Using the technical_meta_data query parameter with unspecified additional parameters or time period.

POST /api/folders

Avoid

• Placing too many data in the payload meta_data.

By default, all API methods are published without restrictions, that may possess security threats. For accessing API methods from Internet, we recommend enabling limitations on WAF, border L7 balancer, etc.

API

The information below is relevant for Oz API 5.2.

For Oz API with Mobile SDK, make sure these methods are accessible from the Internet:

You may need to extend this list depending on how Oz API has been integrated into your infrastructure.

Web SDK

For Web SDK, make sure these methods are accessible from the Internet. Your Web SDK URL is the Web Adapter URL you have received from us.

Oz Forensics provides flexible licensing rules for all products. To support licensing for servers without an Internet connection, you need to install an offline license server.

How to Install an Offline Licensing Server

1. Unpack the Software

2. Install the License Service

The installation is complete. For the next steps, use the Web interface on port 8090.

3. Activate the License

Open the default page in your Web browser http://server:8090 and authorize with your login and password.

Go to theSettings section.

Online activation

The easiest way to activate the license server is via the Internet. Enter the license key and click ACTIVATE.

Offline activation

If your server isn't connected to the Internet, use the offline activation option. Click the SWITCH TO OFFLINE ACTIVATION link.

Enter the license key and hit NEXT.

ClickDOWNLOAD REQUEST FILE.

Enter the activation code in the offline activation response window and hit ACTIVATE.

Activate modules

Oz Api Lite

Add an extra command line parameter with the LICENSE_KEY environment variable with license server host address and port.

Oz Bio

Create the license.key file with the content of license host and IP and bind this file to your container:

For monitoring, we use Prometheus and Grafana.

Step 1. Install and Configure Prometheus

Configure Prometheus

Our charts already contain custom resources for Prometheus: serviceMonitor. By default, serviceMonitor is disabled, to enable it, set enable: true in prometheus_exporters → serviceMonitor.

To ensure that Prometheus Operator spots the parameters, add the corresponding Namespace or serviceMonitor itself to the spec section as shown below:

If you don't specify the parameters, all Namespace will be added:

Verify the Settings

If everything is correct, and serviceMonitor has been added to the cluster, you'll see the corresponding custom resource in Custom Resources → monitoring.coreos.com → Service Monitor.

Make sure that Service Monitor contains the resources listed in the screen below:

You can also check these resources in Prometheus itself. Proceed to Status and select Targets in the drop-down menu.

Here is what should be seen:

Step 2. Configure Dashboard

1. Proceed to our repository and find the branch that matches your chart version.

1. Download Oz_dashboard_client.json.

2. Open Grafana and, in the Home menu, select Dashboards.

1. Click New and choose Import from the drop-down menu.

1. Select Upload dashboard JSON file and locate the Oz_dashboard_client.json file you've downloaded. Change filename or directory if neededm but this is optional.

2. Add the prometheus data source to obtain metrics.

3. Click Import and save the dashboard.

Variables

• namespace is a label of the namespace from :tensorflow:cc:saved_model:load_latency{clustername="$cluster"},

• quantile is a quantile value for tables that require it. Possible values: 0.95, 0.5, 0.90, 0.99, 1.

Step 3. Start Monitoring

Below, you'll find the description of component metrics and tables, along with pre-configured alerts.

Celery

Metrics Overview

Grafana Tables

Illustrative screenshots:

Grafana Alerts

Redis

Metrics Overview

Grafana Tables

Illustrative screenshots:

Grafana Alerts

TFSS

Metrics Overview

Grafana Tables

Illustrative screenshots:

Grafana Alerts

Nginx

Metrics Overview

Grafana Tables

Illustrative screenshots:

Grafana Alerts

API

Metrics Overview

Grafana Alerts

Installation

Our engineers will help you to install Oz Web SDK using the standalone installer (requires your technical personnel to take part) or manually (everything is done by us). Once installed, the adapter part generates the plugin files: file with styles (ozliveness.css) and the primary script of the plugin (plugin_liveness.php).

Licensing

To generate the license, we need the domain name of the website where you are going to use Oz Forensics Web SDK, for instance, your-website.com. You can also define subdomains.

1. Unzip the file received:

unzip license.0000aaaa-00aa-00aa-00aa-00000aaaaa.WebSDK_your_website.2022-10-11.json.zip

2. Copy the JSON license file to the host where you’ve deployed the container from the ozforensics/oz-webliveness-dev:latest image.

Example

scp -i ~/ozforensics/keys/id_rsa-test-hostname-vm license.0000aaaa-00aa-00aa-00aa-00000aaaaa.WebSDK_your_website.2022-10-11.json user@hostname:/opt/oz/web-sdk

• -i ~/ozforensics/keys/id_rsa-test-hostname-vm is the path to the public ssh key of your host license.

• 0000aaaa-00aa-00aa-00aa-00000aaaaa.WebSDK_your_website.2022-10-11.json is the JSON license file.

• user is the username on host.

• hostname is the host alias.

• /opt/oz/web-sdk is the directory where you’ve deployed the Web SDK container.

3. Replace the license file.

Example

ssh user@hostname -i ~/ozforensics/keys/id_rsa-test-hostname-vm sudo cp -v /opt/oz/web-sdk/license.license.0000aaaa-00aa-00aa-00aa-00000aaaaa.WebSDK_your_website.2022-10-11.json /opt/oz/web-sdk/license.json

4. Restart the Web SDK container for the new license to be applied.

Example

ssh user@hostname -i ~/ozforensics/keys/id_rsa-test-hostname-vm sudo docker restart web-sdk

• web-sdk is the name of the container you’ve deployed from the ozforensics/oz-webliveness-dev:latest image.

Once the license is added, the system will check its validity on launch.

License Errors

• WA_CORS_ORIGINS defines what sources are allowed to make requests. The's no default value. Please bear in mind that if you don't set this value, the CORS headers will be switched off and no such headers will be added within the Web SDK container.

• WA_CORS_METHODS(optional) – HTTP methods allowed to use. If the variable is not set, it gets the default value, which is 'GET, POST, OPTIONS'. If the variable is not used, any method is accepted.

• WA_CORS_HEADERS(optional) – HTTP headers allowed to use. If the variable is not set, it gets the default value which is 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type'. If the variable is not used, any header is accepted.

An example of using environment variables for server configuration:

docker run -d -p 80:80 \
-e WA_CORS_ORIGINS='*' \
-e WA_CORS_METHODS='GET, POST, OPTIONS' \
-e WA_CORS_HEADERS='DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type' \
ozforensics/oz-webliveness:{tag}

This part covers the containing of the /core/app_config.json file.

• use_for_liveness – the option is used when bank managers are taking clients' videos. If the option is set to true, mobile devices use the back camera by default, and on desktop, flip and oval circling are off.

• preinit – this optional parameter switches on the preliminary loading of scripts and face detection initialization. This is needed to reduce the plugin loading time. The default value is off that means all the scripts are loaded after calling OzLiveness.open(). The script value means that the scripts will be loaded before the plugin launches. The full value enables preliminary loading of scripts and face detection initialization.

• architecture – this optional parameter is used to choose the architecture for Web SDK. The default value is normal.

  • normal is used for the full Oz API version;

  • lite is used for API Lite;

  • capture is used for the mode that aims to work directly with client backend. In this mode, Oz Liveness Web Plugin can be distributed via static files only (without the adapter part; implemented in 0.7.4);

• api_url – Oz API server address, a text parameter;

• api_token – Oz API access token, a text parameter;

• api_use_token – a parameter to specify the source of the Oz API access token for the system. Possible values = config, client.

  • If the parameter value is client, then an Oz API access token is expected to be derived from the JS plug-in;

  • If you specify config in this parameter, a token will be retrieved from the api_token parameter of the Oz Liveness Web Adapter configuration file.

• video_actions_list – block of video file tags used in the system, a text array. Current tag list.

• photo_actions_list – block of photo file tags used in the system, a text array. Current tag list.

• actions_default_importance – this parameter specifies whether override with an actions array from a web plug-in is allowed at launching an analysis. Possible values = true, false;

  • If true, the Adapter will use an actions array from the configuration file.

  • If false, the Adapter will use an actions array forwarded from your browser with the use of the open(options) method.

• actions_default – the actions array. Options include:

  • video_count – the number of transmitted video files, a numeric parameter;

  • photo_front – whether there is a front-side page in the document. Possible values = true, false;

  • photo_back – whether there is a back-side page in the document. Possible values = true, false.

• analyses – a block for configuring the launch of analyses. Options include:

  • quality – launch of Oz Liveness analysis. Possible values = true, false;

  • biometry – launch of Oz Biometry analysis. Possible values = true, false;

  • documents – launch of Oz Text analysis. Possible values = true, false;

  • collection_ids (since 1.4.0) – an array of the identifiers of collections for the Black List analysis.

• extract_best_shot – a parameter that specifies if a direct link to the best shot (bestshot) extracted from the video should be appended to the analysis result. Possible values = true, false. If true, the response will contain a link to the extracted image inoutput_images (results_media);

• result_mode – a parameter that specifies the contents of the server response with verification results. Possible values:

  • safe– only the state of analyses is returned (completed or not yet);

  • status– results of completed analyses are returned;

  • folder – same as status but with folder identifier added;

  • full– full Oz API response on the analyses is returned in the JSON format.

• result_codes– a block of response codes with annotations.

• delete_old_sessions: true, false– whether you want to delete old sessions

• delete_old_sessions_offset_minutes: ***– old sessions deletion time offset (in minutes)

• video_required_actions_list– the array of required actions

"video_required_actions_list":
[
  "video_selfie_smile"
],

• save_lossless_frame: true– saving the original frame without compression

• video_file_format – optional; here you can choose the video file format. This video file is passed to the api. Possible values: zip (recommended) and mov (less secure). If you need to retrieve your captured video in the MP4 format, please find the instructions here.

• debug (since 1.1.0) – if true, enables access to the /debug.php page, which contains information about the current configuration and the current license.

• load_3d_mask (since 1.2.1) – if true, loads the model to process the video taken with the 3D mask functionality. The default value is false which means that the model is not used, and the 3D mask is unavailable (the enable_3d_mask parameter is being ignored).

• enable_3d_mask (since 1.2.1) – enables the 3D mask as the default face capture behavior. This parameter works only if load_3d_mask is set to true; the default value is false.

• master_license_signature (since 1.3.1) – the parameter for the master license signature; the default value is null.

• results_polling_interval (since 1.4.0) – the interval for polling for the analyses’ results in ms; the default value is 1000.

• get_user_media_timeout (since 1.5.0) – it defines camera access timeout in sec; after this timeout it displays a hint on how to solve the problem; default_default for all browsers and android_facebook for Facebook

"get_user_media_timeout": {
        "default_default": 40,
        "android_facebook": 5
    }

• disable_adapter_errors_on_screen (since 1.5.0) – if true, disables the display of errors in modal windows, allowing you to view them solely using the on_error callback. The default value is False.

To install Oz product via Kubernetes, consider using Helm charts.

• Oz API and related components: Helm chart.

  • API 5.2: version 0.11.x,

  • API 5.3 (regulatory update for Kazakhstan): 0.12.x.

• Web SDK: Helm chart. Please note: the version of the chart is not tied to the Web SDK version.

Database Creation

For testing purposes, the database installed and created automatically by the chart is sufficient. However, for production, we strongly recommend using a separate, self-managed database.

Recommended PostgreSQL version: 15.5.

Create a database using the script(s) below.

    CREATE ROLE <<USERNAME>> WITH LOGIN PASSWORD '<<PASSWORD>>'
      INHERIT
      CONNECTION LIMIT -1
      NOSUPERUSER
      NOCREATEDB
      NOCREATEROLE
      NOREPLICATION ;
    CREATE DATABASE <<DB_NAME>>
      WITH OWNER = <<USERNAME>>
          ENCODING = 'UTF8'
          TABLESPACE = pg_default
          CONNECTION LIMIT = -1
          LC_COLLATE='en_US.UTF-8'
          LC_CTYPE='en_US.UTF-8'
          TEMPLATE template0;
    \c <<DB_NAME>>
    CREATE EXTENSION IF NOT EXISTS pgcrypto;

    CREATE ROLE <<O2N_USERNAME>> WITH LOGIN PASSWORD '<<O2N_PASSWORD>>'
      INHERIT
      CONNECTION LIMIT -1
      NOSUPERUSER
      NOCREATEDB
      NOCREATEROLE
      NOREPLICATION ;
    CREATE DATABASE <<O2N_DB_NAME>>
      WITH OWNER = <<O2N_USERNAME>>
          ENCODING = 'UTF8'
          TABLESPACE = pg_default
          CONNECTION LIMIT = -1
          LC_COLLATE='en_US.UTF-8'
          LC_CTYPE='en_US.UTF-8'
          TEMPLATE template0;
    \c <<O2N_DB_NAME>>
    CREATE EXTENSION IF NOT EXISTS pgcrypto;
    CREATE EXTENSION IF NOT EXISTS vector;

To increase performance, consider using this list of indexes:

CREATE INDEX CONCURRENTLY gw_analyse_abstract_company_id_idx ON public.gw_analyse_abstract USING btree (company_id);
CREATE INDEX CONCURRENTLY gw_analyse_abstract_folder_id ON public.gw_analyse_abstract USING btree (folder_id);
CREATE INDEX CONCURRENTLY gw_analyse_abstract_time_updated_state_idx ON public.gw_analyse_abstract USING btree (time_updated) WHERE (state = 'PROCESSING'::analyse_state);
CREATE INDEX CONCURRENTLY idx_gw_analyse_abstract_state_time_updated ON public.gw_analyse_abstract USING btree (state, time_updated);
CREATE INDEX CONCURRENTLY gw_analyse_collection_collection_id_idx ON public.gw_analyse_collection USING btree (collection_id);
CREATE INDEX CONCURRENTLY gw_analyse_collection_person_media_association_person_id_idx ON public.gw_analyse_collection_person_media_association USING btree (person_id);
CREATE INDEX CONCURRENTLY gw_analyse_cpma_media_association_id_idx ON public.gw_analyse_collection_person_media_association USING btree (media_association_id);
CREATE INDEX CONCURRENTLY gw_analyse_result_group_image_analyse_id_idx ON public.gw_analyse_result_group_image USING btree (analyse_id);
CREATE INDEX CONCURRENTLY gw_analyse_result_group_image_image_id_idx ON public.gw_analyse_result_group_image USING btree (image_id);
CREATE INDEX CONCURRENTLY gw_analyse_result_image_media_association_id_idx ON public.gw_analyse_result_image USING btree (media_association_id);   
CREATE INDEX CONCURRENTLY gw_analyse_source_media_association_analyse_id_idx ON public.gw_analyse_source_media_association USING btree (analyse_id);
CREATE INDEX CONCURRENTLY gw_analyse_source_media_association_source_image_id_idx ON public.gw_analyse_source_media_association USING btree (source_image_id);   
CREATE INDEX CONCURRENTLY gw_analyse_source_media_association_source_shots_set_id_idx ON public.gw_analyse_source_media_association USING btree (source_shots_set_id);
CREATE INDEX CONCURRENTLY gw_analyse_source_media_association_source_video_id_idx ON public.gw_analyse_source_media_association USING btree (source_video_id);
CREATE INDEX CONCURRENTLY gw_collection_company_id_idx ON public.gw_collection USING btree (company_id);
CREATE INDEX CONCURRENTLY gw_collection_creator_id_idx ON public.gw_collection USING btree (creator_id);
CREATE INDEX CONCURRENTLY gw_collection_person_collection_id_idx ON public.gw_collection_person USING btree (collection_id);
CREATE INDEX CONCURRENTLY gw_collection_person_image_person_id_idx ON public.gw_collection_person_image USING btree (person_id);
CREATE INDEX CONCURRENTLY gw_folder_company_id_idx ON public.gw_folder USING btree (company_id) WHERE (NOT is_archive);
CREATE INDEX CONCURRENTLY gw_folder_event_session_id_idx ON public.gw_folder USING btree (((meta_data ->> 'event_session_id'::text))) WHERE (NOT is_archive);
CREATE INDEX CONCURRENTLY gw_folder_resolution_author_id_idx ON public.gw_folder USING btree (resolution_author_id);
CREATE INDEX CONCURRENTLY gw_folder_resolution_comment_fts_idx ON public.gw_folder USING gin (to_tsvector('english'::regconfig, (resolution_comment)::text));
CREATE INDEX CONCURRENTLY gw_folder_time_created_folder_id_idx ON public.gw_folder USING btree (time_created DESC, folder_id DESC);
CREATE INDEX CONCURRENTLY gw_folder_time_created_folder_id_nonarchive_idx ON public.gw_folder USING btree (time_created DESC, folder_id DESC) WHERE (is_archive = false);
CREATE INDEX CONCURRENTLY gw_folder_time_created_idx ON public.gw_folder USING btree (time_created DESC);
CREATE INDEX CONCURRENTLY gw_folder_time_created_nonarchive_idx ON public.gw_folder USING btree (time_created DESC) WHERE (is_archive = false);
CREATE INDEX CONCURRENTLY gw_folder_user_id_idx ON public.gw_folder USING btree (user_id);
CREATE INDEX CONCURRENTLY idx_gw_folder_transaction_id ON public.gw_folder USING btree (company_id, ((meta_data ->> 'transaction_id'::text))) WHERE (NOT is_archive);
CREATE INDEX CONCURRENTLY idx_gw_folder_user_id_time_created ON public.gw_folder USING btree (user_id, time_created);
CREATE INDEX CONCURRENTLY gw_folder_image_folder_id_idx ON public.gw_folder_image USING btree (folder_id);
CREATE INDEX CONCURRENTLY gw_folder_report_author_id_idx ON public.gw_folder_report USING btree (author_id);
CREATE INDEX CONCURRENTLY gw_folder_report_folder_id_idx ON public.gw_folder_report USING btree (folder_id);
CREATE INDEX CONCURRENTLY gw_folder_report_report_template_id_idx ON public.gw_folder_report USING btree (report_template_id);
CREATE INDEX CONCURRENTLY idx_gw_folder_shots_set_folder_id ON public.gw_folder_shots_set USING btree (folder_id);
CREATE INDEX CONCURRENTLY gw_folder_video_folder_id_idx ON public.gw_folder_video USING btree (folder_id);
CREATE INDEX CONCURRENTLY gw_logging_event_record_session_id_idx ON public.gw_logging_event_record USING btree (session_id);
CREATE INDEX CONCURRENTLY gw_logging_event_record_time_created_idx ON public.gw_logging_event_record USING btree (time_created);
CREATE INDEX CONCURRENTLY gw_logging_event_record_timemark_idx ON public.gw_logging_event_record USING btree (timemark);
CREATE INDEX CONCURRENTLY gw_logging_event_session_time_created_idx ON public.gw_logging_event_session USING btree (time_created);
CREATE INDEX CONCURRENTLY gw_logging_event_session_user_id_idx ON public.gw_logging_event_session USING btree (user_id);
CREATE INDEX CONCURRENTLY gw_logging_event_session_time_updated_idx ON gw_logging_event_session(time_updated);
CREATE INDEX CONCURRENTLY gw_report_template_company_id_idx ON public.gw_report_template USING btree (company_id);
CREATE INDEX CONCURRENTLY gw_report_template_name_company_id_idx ON public.gw_report_template USING btree (name, company_id);
CREATE INDEX CONCURRENTLY gw_report_template_attachment_filename_report_template_id_idx ON public.gw_report_template_attachment USING btree (filename, report_template_id);
CREATE INDEX CONCURRENTLY gw_report_template_attachment_report_template_id_idx ON public.gw_report_template_attachment USING btree (report_template_id);
CREATE INDEX CONCURRENTLY gw_role_abstract_user_company_id_idx ON public.gw_role_abstract_user USING btree (company_id);
CREATE INDEX CONCURRENTLY gw_role_email_restore_code_user_id_idx ON public.gw_role_email_restore_code USING btree (user_id);
CREATE INDEX CONCURRENTLY gw_role_session_old_access_token_idx ON public.gw_role_session USING btree (old_access_token);
CREATE INDEX CONCURRENTLY gw_role_session_user_id_idx ON public.gw_role_session USING btree (user_id);
CREATE INDEX CONCURRENTLY gw_role_user_photo_user_id_idx ON public.gw_role_user_photo USING btree (user_id);
CREATE INDEX CONCURRENTLY gw_shots_set_frame_shots_set_id_idx ON public.gw_shots_set_frame USING btree (shots_set_id);
CREATE INDEX CONCURRENTLY gw_utils_audit_actor_id_idx ON public.gw_utils_audit USING btree (actor_id);
CREATE INDEX CONCURRENTLY gw_utils_media_company_id_idx ON public.gw_utils_media USING btree (company_id);

Chart Deployment

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: efs-sc-api
provisioner: efs.csi.aws.com
allowVolumeExpansion: true
#Use Retain if you are going to use PVC for static, or your static will be deleted on chart uninstall
reclaimPolicy: Retain 
mountOptions:
  - tls
  - iam
parameters:
  provisioningMode: efs-ap
  fileSystemId: fs-____________
  directoryPerms: "770"
  uid: "1000"
  gid: "1000"
  basePath: ""

To deploy in Kubernetes, download the chart version you require and adjust the values.yaml file. This file specifies parameters for deployment of Oz products.

API

Adjust the values.yaml file, setting the following mandatory parameters before deployment:

• ozDockerHubCreds: you'll receive them from Oz Engineer.

• UserParams:

  • URLs:

    • apiURL: URL for API. May be internal, if you use Web SDK only. For Mobile SDKs, should be public. Please refer to this article for more information.

  • DB: must be set, if you use an external PostgreSQL server. For details, please check Database Creation.

    • use_chart_postgres: false by default. Enables internal PostgreSQL server (not recommended for production).

    • postgresUser: same as <<USERNAME>>.

    • postgresHost: the hostname of your PostgreSQL server.

    • postgresDB: same as <<DB_NAME>>.

    • postgresUserPassword: same as <<PASSWORD>>.

    • postgresPort: 5432 by default.

    • o2nDB:

      • use_chart_o2nDB: false by default. Enables internal PostgreSQL server (not recommended for production).

      • startinit: true by default. Enables database init scripts. Set to false after chart is deployed.

      • creds:

        • postgresHost: the hostname of your PostgreSQL server with O2N database.

        • postgresPort: 5432 by default.

        • postgresDB: same as <<O2N_DB_NAME>>.

        • postgresUser: same as <<O2N_USERNAME>>.

        • postgresUserPassword: Same as <<O2N_PASSWORD>>.

  • Creds:

    • apiAdminLogin: login for new (default) user for API. Will be created on the first run.

    • apiAdminPass: password for the default user.

    • webUILocalAdminLogin: local Admin for Web UI. Should differ from apiAdminLogin.

    • webUILocalAdminPass: password for webUILocalAdminLogin.

  • BIO:

    • licenseKey: you'll receive it from Oz Engineer / Sales.

    • clientToken: you'll receive it from Oz Engineer.

  • pvc:

    • api:

      • static:

        • storageClassName: RWX StorageClass.

        • size: Expected size for PV.

• Params:

  • Global:

    • startinits: false by default. Set to true on the first run, then, after successful deployment, change back to false.

To adjust API behavior, you might want to change other parameters. Please refer to comments in the values.yaml file.

TFSS (BIO)

BIO is a part of the API chart. The BIO pods require separate nodes for each pod. To ensure BIO resides on dedicated nodes, you can use affinity and tolerations.

The BIO behavior can be customized via Params -> global -> affinity in values.yaml.

The default parameters are listed below:

# We recommend using separate nodes for API and BIO pods,
# and separate node for each BIO pod.
 ## Default affinity is:
 ## Try to start API pods on nodes with "oz:api" label, BIO pods on nodes with "oz:bio" label
affinity: 
  API:
  # You may add additional keys, or comment the whole section to disable affinity
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 1
        preference:
          matchExpressions:
          - key: oz
            operator: In
            values:
            - api
    # nodeName: "myNode" #Node name can be changed. Uncomment to use

    # nodeSelector: ##NodeSelector can be changed. Uncomment to use
    #   oz: api

    # fill tolerations according to node taints, uncomment to use
    #tolerations:
    #- key: nodegroup
    #  operator: Equal
    #  value: api
    #  effect: NoSchedule

  BIO:
  # You may add additional keys, or comment the whole section to disable affinity
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 1
        preference:
          matchExpressions:
          - key: oz
            operator: In
            values:
            - bio

    # nodeName: "myNode" ## Node name can be changed. Uncomment to use.

    # nodeSelector: ## NodeSelector can be changed. Uncomment to use.
    #   oz: bio

    # fill tolerations according to node taints, uncomment to use.
    #tolerations:
    #- key: nodegroup
    #  operator: Equal
    #  value: bio
    #  effect: NoSchedule

The example of chart deployment via Helm:

helm install oz-api https://chartmuseum.infra.ozforensics.ai/charts/oz-k8s-0.11.28.tgz --namespace oz-api --values custom-api-values.yaml

Web SDK

For proper deployment, Web SDK requires an API service account. Pre-create a user for Web SDK with the CLIENT type and is_service flag set. Please refer to User Roles for more details.

Adjust the values.yaml file, setting the following mandatory parameters before deployment:

• ozDockerHubCreds: you'll receive them from Oz Engineer.

• UserParams:

  • URLs:

    • apiURL: API URL. Can be an internal API URL.

    • webSDKURL: WebSDK url that will be used for public access.

  • Creds:

    • AdminLogin: login of the user that should be pre-created in API. Do not use the default admin login.

    • AdminPass: password of the pre-created user.

• PVC:

  • persistentStorage: false be default. Set to true if you use more than 1 Web SDK pod.

  • storageClassName: RWX StorageClass.

• Params:

  • websdk:

    • license: should contain your Web SDK license. You'll receive it from Oz Engineer / Sales.

# Format:
# Remark: Indentations are critical for yaml.
    license: |-
      {
          "payload_b64": "",
          "signature": "",
          "enc_public_key": ""
      }

To adjust API behavior, you might want to change other parameters. Please refer to comments in the values.yaml file.

The example of chart deployment via Helm:

helm install oz-websdk https://chartmuseum.infra.ozforensics.ai/charts/oz-k8s-sdk-1.5.1+onPremise.tgz --namespace oz-websdk --values custom-websdk-values.yaml

Some configuration file parameters can be changed without direct editing of the /core/app_config.json file. You can do it using the environment variables.

• WA_ARCHITECTURE – redefines the architecture parameter

• WA_API_URL – redefines the api_url parameter

Here's an example of using environment variables for the dockered Web SDK in the Lite mode:

docker run -d -p 80:80 \
-e WA_ARCHITECTURE=lite \
-e WA_API_URL='http://127.0.0.1:8000/v1/face/liveness/detect' \
ozforensics/oz-webliveness:{tag}

The configuration settings are contained in theconfig.py file. Its location depends on the installation method:

• host machine or Docker container oz-api: /opt/gateway/configs

Postgres Connection

Path to Media Files

All incoming media files are saved in the local directory mounted to one of possible endpoints depending of the installation method:

• host server or Docker container: /opt/gateway/static

• in case of standalone installer: /var/lib/docker/volumes/api_oz-api-static-vol/_data

• any path specified via configuration

In most of the integration cases, the media files can be accessed in web using the direct links to randomly generated filenames.

To access the media, you need to specify in the configuration file their external host name or IP address, port, and connection protocol.

Connection to Oz Bio / Liveness

Timing and Limits

Oz Liveness WEB Adapter is set up via changes in the configuration file stored at the Oz Liveness WEB Adapter server: /core/app_config.json

For Angular and React, replace https://web-sdk.sandbox.ozforensics.com in index.html.

Terms and Definitions

Components' Description

Oz API components:

• APP is the API front app that receives REST requests, performs preprocessing, and creates tasks for other API components.

• Celery is the asynchronous task queue. API has the following celery queues:

  • Celery-default processes system-wide tasks.

  • Celery-maintenance processes maintenance tasks.

  • Celery-tfss processes analysis tasks.

  • Celery-resolution checks for completion of all nested analyses within a folder and changes folder status.

  • Celery-preview_convert creates a video preview for media.

  • Celery-beat is a CronJob for managing maintenance celery tasks.

  • Celery-Flower is a Celery metrics collector.

  • Celery-regula (optional) processes document analysis tasks.

• Redis is a message broker and result backend for Celery.

• RabbitMQ (optional) can be used as a message broker for Celery instead of Redis.

• Nginx serves static media files for external HTTP(s) requests.

• O2N (optional) processes the Blacklist analysis.

BIO-Updater checks for models updates and downloads new models.

Oz BIO (TFSS) runs TensorFlow with AI models and makes decisions for incoming media.

Deployment Scenarios

The deployment scenario depends on the workload you expect.

Autoscaling is implemented on the basis of ClusterAutoscaler and must be supported by your infrastructure.

Small business or PoC

• Type of containerization: Docker,

• Type of installation: Docker compose,

• Autoscaling/HA: none.

Requirements

Software

• Docker 19.03+,

• Podman 4.4+,

• Python 3.4+.

Storage

• Depends on media quality, the type and number of analyses, and the required archive depth.

• Each analysis request performs read and write operations on the storage. Any additional latency in these operations will impact the analysis time.

Staff qualification:

• Basic knowledge of Linux and Docker.

Deployment

1. Single node.

Resources:

• 1 node,

• 16 CPU/32 RAM.

1. Two nodes.

Resources:

• 2 nodes,

• 16 CPU/32 RAM for the first node; 8 CPU/16 RAM for the second node.

Medium Load

• Type of containerization: Docker/Podman,

• Type of installation: Docker compose,

• Autoscaling/HA: manual scaling; HA is partially supported.

Requirements

Computational resources

Depending on load, you can change the number of nodes. However, for 5+ nodes, we recommend that you proceed to the High Load section.

• • 2 Nodes:

    • 24 CPU/32 RAM per node.

  • 3 Nodes:

    • 16 CPU/24 RAM per node.

  • 4 Nodes:

    • 8 CPU/16 RAM for two nodes (each),

    • 16 CPU/24 RAM for two nodes (each).

We recommend using external self-managed PostgreSQL database and NFS share.

Software

• Docker 19.03+,

• Podman 4.4+,

• Python 3.4+.

Storage

• Depends on media quality, the type and number of analyses, and the required archive depth.

• Each analysis request performs read and write operations on the storage. Any additional latency in these operations will impact the analysis time.

Staff qualification:

• Advanced knowledge of Linux, Docker, and Postgres.

Deployment

2 nodes:

3 nodes:

4 nodes:

High Load

• Type of containerization: Type of containerization: Docker containers with Kubernetes orchestration,

• Type of installation: Helm charts,

• Autoscaling/HA: supports autoscaling; HA for most components.

Requirements

Computational resources

3-4 nodes. Depending on load, you can change the number of nodes.

• 16 CPU/32 RAM Nodes for the BIO pods,

• 8+ CPU/16+ RAM Nodes for all other workload.

We recommend using external self-managed PostgreSQL database.

Software

• Docker 19.03+,

• Python 3.4+.

Storage

• Depends on media quality, the type and number of analyses, and the required archive depth.

• Each analysis request performs read and write operations on the storage. Any additional latency in these operations will impact the analysis time.

Staff qualification:

• Advanced knowledge of Linux, Docker, Kubernetes, and Postgres.

Deployment Scheme