3D Velocity Measurement of Translational Motion using a Stereo Camera

— This research aims to create a 3D velocity measurement system using a stereo camera. The 3D velocity in this paper is the velocity which consists of three velocity components in 3D Cartesian coordinates. Particular attention is focused on translational motion. The system set consists of a stereo camera and a mini-PC with Python 3.7, and OpenCV 4.0 installed. The measurement method begins with the selection of the measured object, object detection using template matching, disparity calculation using the triangulation principle, velocity calculation based on object displacement information and time between frames, and the storage of measurement results. The measurement system's performance was tested by experimenting with measuring conveyor velocity from forward-looking and angle-looking directions. The experimental results show that the 3D trajectory of the object can be displayed, the velocity of each component and the speed as the magnitude of the velocity can be obtained, and so the 3D velocity measurement can be performed. The camera can be positioned forward-looking or at a certain angle without affecting the measurement results. The measurement of the speed of the conveyor is 11.6 cm/s with an accuracy of 0.4 cm/s. The results of this study can be applied in the performance inspection process of conveyors and other industrial equipment that requires speed measurement. In addition, it can also be developed for accident analysis in transportation systems and practical tools for physics learning.


I. INTRODUCTION
Velocity measurements are needed in various fields such as transportation, robotics, satellite navigation, and manufacturing. In transportation, velocity measurement is important for accident analysis [1], [2], odometry [3], [4], and autonomous driving [5]. In robotics, velocity is one of the parameters needed in robot navigation [6], [7]. In satellite navigation, velocity measurement is needed to position the satellite as accurately as possible in its orbit [8], [9]. In manufacturing, velocity measurement is carried out as part of conveyor performance control [10], [11].
The existing velocity measurement methods include Lidar, Radar, and video/image. Video/image measurement methods are being developed because they have the potential to measure speed and identify vehicle numbers [12], [13]. Moreover, video/image speed measurements can be used for autonomous car control systems [14].
Based on the camera used, there are two methods, single cameras [15]- [17] and stereo cameras [18]- [20]. Single cameras require that the object's size must be known to obtain the measurement results when a stereo camera uses the principle of triangulation. In terms of performance, the measurement accuracy of a stereo camera is potentially superior to that of a single camera. Stereo cameras can get more data and produce fewer occlusions than a single camera. Any occlusion caused by surface geometry blocks projected light or the camera's view with a single camera would result in no data. On the other hand, stereo cameras effectively use three views of the part to capture more data. Simultaneously, triangulation of the left camera and triangulation of the right camera take more data than a single camera, and then stereo may be obtained with two cameras working together.
Based on the placement of the detector, there are two methods, namely forward-looking and side-looking [21]. Forward-looking is a measurement where the detector is placed parallel to the direction of motion, while side-looking is a measurement where the detector is placed perpendicular to the direction of motion. Both of these placements have the advantage of reducing the three-speed components into one-speed components, making it simpler. However, the measurement results will be biased if the object's motion does not match the placement of the camera.
The solution to this problem is to use 3D velocity measurement, where all three velocity components are measured simultaneously. Thus, the placement of the detector is more flexible without changing the programming algorithm. The detector can not only be placed forward-looking and sidelooking but can be placed angle-looking where the object and camera form a certain angle.
Research on 3D velocity measurement from angle-looking is still limited because it involves more variables and is more complex. However, the advantage of this approach is that the camera can be placed at various angles so that it is more flexible in use than forward-looking and side looking.
This paper proposed 3D velocity measurement using a stereo camera. The detection of objects uses a template matching algorithm. The measurement process is carried out by forward-looking as is usually done and then by angle looking as another alternative. Angle-looking is needed to show that the 3D velocity has been successfully measured so that the difference in the placement of the detector affects the value of the velocity vector component only, but the magnitude of the velocity is constant. This hypothesis will be tested with an experiment. The motion measured in this study is limited to translational motion. Translational motion is the linear motion of an object accompanied by the displacement of that object. Examples of translational motion include the trajectory of a car on the highway, the rate of a conveyor belt, the trajectory of an airplane, a train, and so on. References to the use of stereo cameras for 3D speed measurement in translational motion are still limited.

A. System Description
The system used consists of a calibrated stereo camera and a mini-PC with the specifications shown in Table 1. The stereo camera is mounted on a tripod connected to the Mini PC using a USB cable. The flowchart of the 3D velocity measurement system is shown in Fig. 1. From the resulting stereo image, the object can be measured manually by hovering the cursor over the object and then pressing the left mouse button. The selected object is automatically used as a template measuring 24x24 pixels. The template represents the object in the real space being measured. Templates can be changed before taking measurements or at the time of taking measurements. Users can freely choose objects in the stereo image to be used as templates.
Next, template matching is applied to the stereo image. The template matching process is carried out using the library contained in OpenCV 4.0. In this paper, the template matching process is carried out by dividing the stereo image into two equal parts, namely the left image and the right image, and then applying template matching to both separately. At this stage, there are two conditions. If the template on the left and right images is detected, then proceed to the next process, but if not, the information "No object detected" is displayed, and the template selection process can be repeated.
After the object is detected, the next step is to calculate the disparity and 3D position based on the parameters that have been obtained during calibration. The disparity is the difference in the position of objects on the left and right images. The disparity is calculated using the pinhole model approach, as shown in equation (1) = − The 3D position of each component is calculated using Next, the 3D velocity component is calculated using Then, the 3D velocity vector can be written in the form While speed as the magnitude of velocity can be calculated using equation (9), Finally, the disparity data, the object position in the image, 3D position, and 3D velocity are displayed on the stereo image frame. If needed, the data can be saved in the form of an 'AVI' format video and 'xlsx' format spreadsheet.

B. Experimental Design
The experiment was carried out on an object placed on a conveyor that was constantly moving away from the stereo camera, as shown in Fig.2(a-b). To determine the effect of angle, the position of the stereo camera is varied starting from being in line with the direction of the object's motion as in Fig.  2a (forward-looking) and forming an angle (angle-looking) as in Fig. 2b. The angle variation used is 30° and 45°. The left camera is selected as the reference point (origin). The X-axis is in the horizontal direction, the Y-axis is in the vertical direction, and the Z-axis is perpendicular to the stereo image plane.
The parameter measured includes the components of velocity, velocity, speed, and speed accuracy. Accuracy was calculated using the equation mean absolute deviation (MAE) [11], as shown in equation (10) /01 = with m as the number of measurements, yi as the i th measurement, and y as the average of the measurement results.

A. Measurement Process
The stereo image display is shown in Fig. 3. Detected objects are marked with red boxes on the left and right images. The image coordinate axis (x, y) is marked with a white line with the center of the coordinates (x=0,y=0) located at the center of the camera. The x-axis of the image is marked with a horizontal white line, while the y-axis of the image is marked with a vertical white line.

B. Object Trajectory
The object's trajectory can be obtained through 3D Graphs of changes in the object's position in the measurement process. The motion of the translation object will show a straight-line pattern. The results of measuring changes in object position in a forward-looking manner are shown in Fig. 4a. Based on the figure, it appears that the object's trajectory forms a straight line, changing significantly on the Z-axis with a range of 35-50 cm, while on the X-axis and Y-axis, it remains the same.
The results of the angle-looking measurement with an angle of 30° and 45° to the direction of the object's motion are shown in Fig. 4a and Fig. 4b. Based on the figure, it appears that the trajectory of the object is in the form of a straight line, as in Fig. 4a. The position component on the X-axis and Zaxis change significantly while the Y position component remains. However, the Y position component at an angle of 45° is greater than that of an angle of 30°.
Based on Fig. 4(a-c), it can be shown that the object's trajectory is in the form of a straight line according to the object's motion on the conveyor. Thus this speed measurement system has succeeded in showing the 3D trajectory of the object.
The trajectory of an object is important for visually demonstrating 3D motion. Fig. 4a The trajectory of the object when the camera is in line with the direction of motion Fig. 4b The trajectory of the object when the camera makes an angle of 30° to the direction of motion Fig. 4c The trajectory of the object when the camera makes an angle of 45° to the direction of motion

C. Velocity Component, Velocity, and Speed Measurement
An example of a position graph as a function of time for three variations of camera position is shown in Fig. 5(a-c). Based on Fig. 5(a-c), it can be seen that the relationship between position and time on all components of velocity is linear so that the 3D velocity measurement results are constant in all variations.
The value of the velocity component on each coordinate axis is equivalent to the gradient in the equation obtained by curve fitting a linear equation. In Fig. 5a it can be concluded that the velocity component on each axis is VX = -0.35 cm/s, VY = 0.18 cm/s and VZ = 11.8 cm/s.

A. Speed Measurement and its Accuracy
Speed accuracy is calculated using equation 10. The measurement is carried out five times. The measurement of the speed for the three variations is shown in Fig. 6. The results for each variation are forward-looking (11.6 ± 0.4) cm/s, angle-looking 30° (11.8 ± 0, 2) cm/s, and the anglelooking 45° is (11.7 ± 0.3 ) cm/s. Based on Fig. 6, it appears that the measurement results along with the angle-looking errors of 30° and 45° are within the range of forward-looking error. Thus, it can be stated that the measurement results of the three variations are consistent with each other. The results of this experiment produce a maximum uncertainty of 0.4 cm/s, which has higher precision than the conveyor speed measurement using a single camera that produces an uncertainty of 1 cm/s [11]. Another advantage of this system is that not only the speed obtained but also 3D velocity data is obtained so that the measurement process is more flexible, which can be carried out from a forwardlooking or angle-looking perspective. In the [11] measurement method, the measurement process is limited to side-looking. Yang et al. [13] also utilize a stereo camera for speed measurement. The system is designed to measure all vehicles' speed in ROI automatically. The difference with the system developed in this study is that the 3D velocity measurement is carried out on the desired object only. The maximum accuracy of speed measurement by Yang et al. [13] is 3.8%, while it was 3.4% in this study. Thus, the proposed method was more accurate.

IV. CONCLUSION
The system developed in this study has successfully measured 3D velocity. The camera position can be placed forward-looking or angle-looking. Based on the specifications of the stereo camera used, this system can measure the speed of 11.6 cm/s with an absolute error of 0.4 cm/s.