Capturing Video and Description of the on_capture_complete Callback

In this article, you’ll learn how to capture videos and send them through your backend to Oz API.

1. Overview

Here is the data flow for your scenario:

1. Oz Web SDK takes a video and makes it available for the host application as a frame sequence.

2. The host application calls your backend using an archive of these frames.

3. After the necessary preprocessing steps, your backend calls Oz API, which performs all necessary analyses and returns the analyses’ results.

4. Your backend responds back to the host application if needed.

2. Implementation

On the server side, Web SDK must be configured to operate in the Capture mode:

The architecture parameter must be set to capture in the app_config.json file.

In your Web app, add a callback to process captured media when opening the Web SDK plugin:

OZLiveness.open({
  ... // other parameters
  on_capture_complete: function(result) {
         // Your code to process media/send it to your API, this is STEP #2
  }
})

The result object structure depends on whether any virtual camera is detected or not.

No Virtual Camera Detected

{
	"action": <action>,
	"best_frame": <bestframe>,
	"best_frame_png": <bestframe_png>,
	"best_frame_bounding_box": {
		"left": <bestframe_bb_left>,
		"top": <bestframe_bb_top>,
		"right": <bestframe_bb_right>,
		"bottom": <bestframe_bb_bottom>
		},
	"best_frame_landmarks": {
		"left_eye": [bestframe_x_left_eye, bestframe_y_left_eye],
		"right_eye": [bestframe_x_right_eye, bestframe_y_right_eye],
		"nose_base": [bestframe_x_nose_base, bestframe_y_nose_base],
		"mouth_bottom": [bestframe_x_mouth_bottom, bestframe_y_mouth_bottom],
		"left_ear": [bestframe_x_left_ear, bestframe_y_left_ear],
		"right_ear": [bestframe_x_right_ear, bestframe_y_right_ear]
		},
	"frame_list": [<frame1>, <frame2>],
	"frame_bounding_box_list": [
		{
		"left": <frame1_bb_left>,
		"top": <frame1_bb_top>,
		"right": <frame1_bb_right>,
		"bottom": <frame1_bb_bottom>
		},
		{
		"left": <frame2_bb_left>,
		"top": <frame2_bb_top>,
		"right": <frame2_bb_right>,
		"bottom": <frame2_bb_bottom>
		},
	],
	"frame_landmarks": [
		{
		"left_eye": [frame1_x_left_eye, frame1_y_left_eye],
		"right_eye": [frame1_x_right_eye, frame1_y_right_eye],
		"nose_base": [frame1_x_nose_base, frame1_y_nose_base],
		"mouth_bottom": [frame1_x_mouth_bottom, frame1_y_mouth_bottom],
		"left_ear": [frame1_x_left_ear, frame1_y_left_ear],
		"right_ear": [frame1_x_right_ear, frame1_y_right_ear]
		},
		{
		"left_eye": [frame2_x_left_eye, frame2_y_left_eye],
		"right_eye": [frame2_x_right_eye, frame2_y_right_eye],
		"nose_base": [frame2_x_nose_base, frame2_y_nose_base],
		"mouth_bottom": [frame2_x_mouth_bottom, frame2_y_mouth_bottom],
		"left_ear": [frame2_x_left_ear, frame2_y_left_ear],
		"right_ear": [frame2_x_right_ear, frame2_y_right_ear]
		}
	],
"from_virtual_camera": null,
"additional_info": <additional_info>
}

Any Virtual Camera Detected

{
	"action": <action>,
	"best_frame": null,
	"best_frame_png": null,
	"best_frame_bounding_box": null,
	"best_frame_landmarks": null
	"frame_list": null,
	"frame_bounding_box_list": null,
	"frame_landmarks": null,
	"from_virtual_camera": {
	"additional_info": <additional_info>,
		"best_frame": <bestframe>,
		"best_frame_png": <best_frame_png>,
		"best_frame_bounding_box": {
			"left": <bestframe_bb_left>,
			"top": <bestframe_bb_top>,
			"right": <bestframe_bb_right>,
			"bottom": <bestframe_bb_bottom>
			},
		"best_frame_landmarks": {
			"left_eye": [bestframe_x_left_eye, bestframe_y_left_eye],
			"right_eye": [bestframe_x_right_eye, bestframe_y_right_eye],
			"nose_base": [bestframe_x_nose_base, bestframe_y_nose_base],
			"mouth_bottom": [bestframe_x_mouth_bottom, bestframe_y_mouth_bottom],
			"left_ear": [bestframe_x_left_ear, bestframe_y_left_ear],
			"right_ear": [bestframe_x_right_ear, bestframe_y_right_ear]
			},
		"frame_list": [<frame1>, <frame2>],
		"frame_bounding_box_list": [
			{
			"left": <frame1_bb_left>,
			"top": <frame1_bb_top>,
			"right": <frame1_bb_right>,
			"bottom": <frame1_bb_bottom>
			},
			{
			"left": <frame2_bb_left>,
			"top": <frame2_bb_top>,
			"right": <frame2_bb_right>,
			"bottom": <frame2_bb_bottom>
			},
			],
		"frame_landmarks": [
			{
			"left_eye": [frame1_x_left_eye, frame1_y_left_eye],
			"right_eye": [frame1_x_right_eye, frame1_y_right_eye],
			"nose_base": [frame1_x_nose_base, frame1_y_nose_base],
			"mouth_bottom": [frame1_x_mouth_bottom, frame1_y_mouth_bottom],
			"left_ear": [frame1_x_left_ear, frame1_y_left_ear],
			"right_ear": [frame1_x_right_ear, frame1_y_right_ear]
			},
			{
			"left_eye": [frame2_x_left_eye, frame2_y_left_eye],
			"right_eye": [frame2_x_right_eye, frame2_y_right_eye],
			"nose_base": [frame2_x_nose_base, frame2_y_nose_base],
			"mouth_bottom": [frame2_x_mouth_bottom, frame2_y_mouth_bottom],
			"left_ear": [frame2_x_left_ear, frame2_y_left_ear],
			"right_ear": [frame2_x_right_ear, frame2_y_right_ear]
			}
		]
	}
}

Here’s the list of variables with descriptions.

Variable

Type

Description

best_frame

String

The best frame, JPEG in the data URL format

best_frame_png

String

The best frame, PNG in the data URL format, it is required for protection against virtual cameras when video is not used

best_frame_bounding_box

Array[Named_parameter: Int]

The coordinates of the bounding box where the face is located in the best frame

best_frame_landmarks

Array[Named_parameter: Array[Int, Int]]

The coordinates of the face landmarks (left eye, right eye, nose, mouth, left ear, right ear) in the best frame

frame_list

Array[String]

All frames in the data URL format

frame_bounding_box_list

Array[Array[Named_parameter: Int]]

The coordinates of the bounding boxes where the face is located in the corresponding frames

frame_landmarks

Array[Named_parameter: Array[Int, Int]]

The coordinates of the face landmarks (left eye, right eye, nose, mouth, left ear, right ear) in the corresponding frames

action

String

An action code

additional_info

String

Information about client environment

Please note:

  • The video from Oz Web SDK is a frame sequence, so, to send it to Oz API, you’ll need to archive the frames and transmit them as a ZIP file via the POST /api/folders request (check our Postman collections).

  • You can retrieve the MP4 video from a folder using the /api/folders/{{folder_id}} request with this folder's ID. In the JSON that you receive, look for the preview_url in source_media. The preview_url parameter contains the link to the video. From the plugin, MP4 videos are unavailable (only as frame sequences).

  • Also, in the POST {{host}}/api/folders request, you need to add the additional_info field. It is required for the capture architecture mode to gather the necessary information about client environment. Here’s the example of filling in the request’s body:

"VIDEO_FILE_KEY": VIDEO_FILE_ZIP_BINARY
"payload": "{
        "media:meta_data": {
           "VIDEO_FILE_KEY": {
              "additional_info": <additional_info>
              }
           }
}"
  • Oz API accepts data without the base64 encoding.

Last updated