基于固定/PTZ摄像机系统的开阔水域小目标船舶主动式跟踪方法

游继安; 胡钊政; 肖汉彪; 孟杰

doi:10.3963/j.jssn.1674-4861.2024.03.006

基于固定/PTZ摄像机系统的开阔水域小目标船舶主动式跟踪方法

doi: 10.3963/j.jssn.1674-4861.2024.03.006

游继安^{1, 2, 3, 4,},
胡钊政^{1, 3, 4, ,},
肖汉彪^{1, 3, 4},
孟杰^{1, 3, 4}

1.
武汉理工大学水路交通控制全国重点实验室武汉 430063
2.
湖北工程学院新技术学院湖北孝感 432000
3.
武汉理工大学交通物联网技术湖北省重点实验室武汉 430063
4.
武汉理工大学智能交通系统研究中心武汉 430063

基金项目:

国家自然科学基金项目 52302503

湖北省重点研发计划项目 2022BAA064

湖北省教育厅科学研究计划指导性项目 B2022510

详细信息

作者简介:
游继安(1986—)，博士研究生. 研究方向：智能交通系统. E-mail: 256741@whut.edu.cn

通讯作者:
胡钊政(1979—)，博士，教授. 研究方向：计算机视觉、主动视觉监控等. E-mail: zzhu@whut.edu.cn

中图分类号: TP311
计量
- 文章访问数: 177
- HTML全文浏览量: 131
- PDF下载量: 13
- 被引次数: 0
出版历程
- 收稿日期: 2023-11-10
- 网络出版日期: 2024-10-21

An Active Tracking Method for Small Ships in Open Water Based on Fixed/PTZ Camera System

YOU Ji'an^{1, 2, 3, 4
,},
HU Zhaozheng^{1, 3, 4
, ,},
XIAO Hanbiao^{1, 3, 4},
MENG Jie^{1, 3, 4}

1.
State Key Laboratory of Maritime Technology and Safety, Wuhan University of Technology, Wuhan 430063, China
2.
College of Technology, Hubei Engineering University, Xiaogan 432000, China
3.
Hubei Key Laboratory of Transportation Internet of Things, Wuhan University of Technology, Wuhan 430063, China
4.
Intelligent Transportation Systems Research Center, Wuhan University of Technology, Wuhan 430063, China

摘要

摘要: 仅依靠当前的闭路电视（closed-circuit television，CCTV）系统，往往难以主动跟踪并拍摄内河船舶清晰的图像。针对上述问题，研究了基于固定/平移-倾斜-对焦（pan-tilt-zoom，PTZ）摄像机系统的开阔水域小目标船舶主动式跟踪方法。采用基于虚拟四边形的三层联合标定模型对固定摄像机和PTZ摄像机进行联合标定，将虚拟四边形内的图像坐标与PTZ摄像机的平移角和倾斜角一一对应；引入虚拟四边形的概念，有效过滤虚拟四边形外目标的干扰，提高小目标船舶的检测准确率；利用透视n点（perspective-n-point, PnP）问题算法和虚拟四边形顶点的图像坐标，得到图像坐标与世界坐标间的映射关系，再利用Pan-Tilt-Height（PTH）模型将虚拟四边形中目标的世界坐标转化为PTH坐标；在小目标跟踪过程中，通过连续检测虚拟四边形中船舶边框质心的图像坐标，即可计算得到PTZ摄像机的平移角与倾斜角，从而实现实时主动跟踪的目的，并尽最大限度的保持船舶目标处于PTZ摄像机图像的中心位置。选取湖北省孝感市春晖湖和武汉市汉江中法桥段这2处真实场景，进行可靠性和有效性验证，实验结果表明：①利用改进的目标检测方法对固定摄像机图像中的船舶进行检测，F1-Score分别为96.82%和97.62%；②利用研究的主动式跟踪方法跟踪运动船舶时，PTZ摄像机的跟踪失败率为4.63%。本文研究的主动式跟踪方法的跟踪速率可以达到18.55 fps。
- 智能交通 /
- 小目标船舶 /
- 主动式跟踪 /
- 联合标定
Abstract: It is difficult to actively track and capture clear images of inland river ships with the current Closed Circuit Television (CCTV) system. To fill the gap, an active tracking method for small ships in open waters based on the fixed/pan-tilt-zoom (PTZ) camera system is proposed. A three-layer joint calibration model based on a virtual quadrilateral (VQ) is introduced to jointly calibrate the fixed camera and the PTZ camera, which matches the image coordinate with the pan and tilt angle of the PTZ camera one by one; The introduced VQ filters out the targets outside the quadrilateral, eliminating inference and improving the accuracy of detection. The mapping relationship between the image coordinates and the world coordinates can be obtained by using the Perspective-n-Point (PnP) algorithm and the vertices of the VQ; Fourthly, the world coordinates of the points in the VQ are transformed into the Pan-Tilt-Hight (PTH) coordinates via PTH model. Then, by calculating the coordinate of the ship (the centroid of the ship) in the VQ, the pan and tilt angle of the PTZ camera can be derived, achieving real-time active tracking and keeping the target at the center of the PTZ camera image. To validate the proposed method, two real scenes are introduced, namely Chunhui Lake in Xiaogan City and the Sino-French Bridge section of the Han River in Wuhan City, Hubei Province. The results indicate that, ① the F1 -Scores of the proposed method on the fixed camera are 96.82% and 97.62%, respectively; ② when the proposed method is applied to the PTZ camera for tracking the moving ships, the failure rate is 4.63%. In summary, the proposed active tracking method performs reasonably in practice with a high tracking rate of 18.55 fps.
- intelligent transportation /
- small target ships /
- active tracking /
- joint calibration

HTML全文

图 1 本文算法结构框架图

Figure 1. Algorithm structure frame work diagram

下载: 全尺寸图片幻灯片

图 2 联合感知示意图

Figure 2. The sketch map of the joint perception method

下载: 全尺寸图片幻灯片

图 3 基于虚拟四边形的三层联合标定模型

Figure 3. Fixed / PTZ cameras-based 3-layer joint calibration model

下载: 全尺寸图片幻灯片

图 4 PTH坐标系

Figure 4. PTH coordinate system

下载: 全尺寸图片幻灯片

图 5 实验设备

Figure 5. Experimental Equipment

下载: 全尺寸图片幻灯片

图 6 实验场景

Figure 6. Experimental Scenarios

下载: 全尺寸图片幻灯片

图 7 固定视角图像中目标的检测结果

Figure 7. Detection results of the objects in the fixed image

下载: 全尺寸图片幻灯片

图 8 不同场景下的跟踪结果

Figure 8. The tracking results in different scenes

下载: 全尺寸图片幻灯片

图 9 目标边框与PTZ图像间的位置关系

Figure 9. The relationship between the bounding box of the ship and the PTZ image

下载: 全尺寸图片幻灯片

图 10 主动式跟踪速率

Figure 10. Active tracking rate

下载: 全尺寸图片幻灯片

图 11 不同方法的目标检测结果

Figure 11. Object detection results of different methods

下载: 全尺寸图片幻灯片

图 12 本文方法中跟踪角度的偏离误差

Figure 12. The deviation error of the tracking angle in the proposed method

下载: 全尺寸图片幻灯片

图 13 PTZ摄像机的观测轨迹

Figure 13. Observation trajectory of PTZ camera

下载: 全尺寸图片幻灯片

表 1 在自建船舶数据集上的检测结果对比

Table 1. Detection results on the self-built ship dataset

实验场景	方法	精确率/%	召回率/%	准确率/%	F1-Score/%
场景1	改进前	96.92	95.10	95.04	96.01
场景1	改进后	97.23	96.41	96.25	96.82
场景2	改进前	97.84	96.30	98.24	97.06
场景2	改进后	98.02	97.23	99.2	97.62

下载: 导出CSV

表 2 不同主动式跟踪方法的跟踪效果对比

Table 2. Comparison of the effects of different active tracking methods

方法	ITV/%	OTE/%	OOV/%	跟踪速率/fps
文献[5]	77.63	4.77	17.60	45.45
文献[16]	70.35	9.15	20.50	1.01
本文方法	92.12	3.25	4.63	18.55

下载: 导出CSV

表 3 不同主动式跟踪方法的性能对比

Table 3. Comparison of performance of different active tracking methods

方法	检测准确率/%	OOV/%	跟踪速率/fps	丢失重找
文献[5]	82.40	17.60	45.45	否
文献[16]	96.64	20.50	1.01	是
本文方法	97.72	4.63	18.55	是

下载: 导出CSV

参考文献(22)

[1]	WANG L, LIU Q, DONG S, et al. Effectiveness assessment of ship navigation safety countermeasures using fuzzy cognitive maps[J]. Safety Science, 2019, 117: 352-364. doi: 10.1016/j.ssci.2019.04.027
[2]	赵婷, 王申涛, 牛林, 等. 合成孔径雷达图像舰船尾迹检测算法[J]. 上海交通大学学报, 2020, 54(12): 1259-1268. ZHAO T, WANG S T, NIU L, et al. Detection algorithm of ship wake in SAR Images[J], Journal of Shanghai Jiao Tong University, 2020, 54(12): 1259-1268.
[3]	QU J, LIU R W, GUO Y, et al. Improving maritime traffic surveillance in inland waterways using the robust fusion of AIS and visual data[J]. Ocean Engineering, 2023, 275: 114198. doi: 10.1016/j.oceaneng.2023.114198
[4]	SANG L Z, WALL A, MAO Z, et al. A novel method for restoring the trajectory of the inland waterway ship by using AIS data[J]. Ocean Engineering. 2015, 110: 183-194. doi: 10.1016/j.oceaneng.2015.10.021
[5]	HU Z Z, YOU J A, YUAN K, et al. Grid-based control of active cameras for waterway ship surveillance[C]. The 5^th Inter-national Conference on Transportation Information and Safety(ICTIS). Liverpool, U. K. : IEEE, 2019.
[6]	GUO Y, LIU R W, QU J, et al. Asynchronous trajectory matching-based multimodal maritime data fusion for vessel traffic surveillance in inland waterways[J]. IEEE Transactions on Intelligent Transportation Systems, 2023. 24 (11): 12779-12792. doi: 10.1109/TITS.2023.3285415
[7]	CHEN Z, CHEN D, ZHANG Y, et al. Deep learning for auton-omous shiporiented small ship detection[J]. Safety Science, 2020, 130: 104812. doi: 10.1016/j.ssci.2020.104812
[8]	LIN T Y, MAIRE M, BELONGIE S, et al. Microsoft COCO: common objects in context[C]. Uropean Conference on Computer Vision(ECCV), Zurich, Switzerland: IEEE, 2014.
[9]	王鹏, 神和龙, 尹勇, 等. 基于深度学习的船舶驾驶员疲劳检测算法[J]. 交通信息与安全, 2022, 40(1): 63-71. doi: 10.3963/j.jssn.1674-4861.2022.01.008 WANG P, SHEN H L, YIN Y, et al. A detection algorithm for the fatigue of ship officers based on deep learning technique[J]. Journal of Transport Information and Safety, 2022, 40(1): 63-71. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2022.01.008
[10]	YONG H, HUANG J, XIANG W, et al. Panoramic background image generation for PTZ cameras[J]. IEEE Transactions on Image Processing, 2019, 28(7): 3162-3176. doi: 10.1109/TIP.2019.2894940
[11]	梁文锋, 项志宇. 鲁棒的PTZ摄像机目标跟踪算法[J]. 浙江大学学报(工学版), 2011, 45(1): 59-63. LIANG W F, XIANG Z Y, et, al. Algorithm of robust object tracking using PTZ camera[J]. Journal of Zhejiang university (Engineering Science), 2011, 45(1): 59-63. (in Chinese)
[12]	XUE K, LIU Y, OGUNMAKIN G, et al. Panoramic gaussian mixture model and large-scale range background substrac-tion method for PTZ camera-based surveillance systems[J]. Machine Vision and Applications. 2013, 24: 477-492. doi: 10.1007/s00138-012-0426-4
[13]	LIU N, WU H F, LIN L. Hierarchical ensemble of background models for PTZ-based video surveillance[J]. IEEE Transactions on Cybernetics. 2014, 45: 89-102.
[14]	LISANTI G, MASI I, PERNICI F, et al. Continuous localization and mapping of a pan-tilt-zoom camera for wide area tracking[J]. Machine Vision and Applications. 2016, 27: 1071-1085. doi: 10.1007/s00138-016-0799-x
[15]	万定锐, 周杰. 双PTZ摄像机系统的标定[J]. 中国图象图形学报, 2008(4): 786-793. WAN D R, ZHOU J. Calibration of dual-PTZ-camera system[J]. Journal of Image and Graphics, 2008(4): 786-793. (in Chinese)
[16]	SERGIO B, LEOPOLDO L R, DANIEL S C, et al. Maritime surveillance by multiple data fusion: an application based on deep learning object detection, AIS data and geofencing[C]. The 18^th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. Lisbon, Portugal: Springer 2023.
[17]	SHAKERI M, ZHANG H. Cooperative targeting: detection and tracking of small objects with a dual camera system[C]. Field and Service Robotics: Results of the 9^th International Conference. Toronto, Canada: FSR, 2015.
[18]	BARIS I, BASTANLAR Y L. Classification and tracking of traffic scene objects with hybrid camera systems[C]. International Conference on Intelligent Transportation Systems (ITSC). Yokohama, Japan: IEEE, 2017.
[19]	石皓, 赖世铭, 刘煜, 等. 一种用于鱼眼PTZ主从监控系统的标定方法[J]. 系统仿真学报, 2013, 25(10): 2412-2417. SHI H, LAI S M, LIU Y, et al. Calibration method based on master-slave surveillance system composed fish-eye camera and PTZ dome camera[J]. Journal of System Simulation. 2013, 25(10): 2412-2417. (in Chinese)
[20]	FU J, DING Y, HUANG T, et al. Hand-eye calibration method based on three-dimensional visual measurement in robotic high-precision machining[J]. The International Journal of Advanced Manufacturing Technology, 2022, 119: 3845-3856.
[21]	JOCHER G. YOLOv5[CP/OL]. (2022-02-09)[2023-6-25]. https://github.com/ultralytics/yolov5.
[22]	WANG J, JIA Z, LAI H, et al. A real time target face tracking algorithm based on saliency detection and camshift[J]. Multimedia Tools and Applications, 2023, 82(28): 43599-43624.