Vehicle Re-Identification with Spatio-Temporal Model Leveraging by Pose View Embedding
, Xian Zhong 2,3,*, Xuemei Jia 4, Wenxuan Liu 2, Meng Feng 2, Zheng Wang 4 and Shin’ichi Satoh 5
Round 1
Reviewer 1 Report
The paper presents a method to tackle the vehicle re-identification (Re-ID) problem in open scenarios by leveraging pose view to improve the discrimination performance of visual features and use keypoints to improve the accuracy of pose identification. This research varies from early-stage studies that concentrated on a specific perspective because it encounters greater obstacles due to view variations, illumination changes and occlusions. For vehicle Re-ID, a two-branch framework was created, consisting of a Keypoint-based Pose Embedding Visual (KPEV) model and a Keypoint-based Pose-Guided Spatio-Temporal (KPGST) model. These models are put into the system, and the KPEV and KPGST findings are fused using a Bayesian network. Extensive tests on the VeRi-776 and VehicleID datasets in relation to functional urban surveillance scenarios show that the suggested technique can potentially solve the problem of vehicle re-identification.
The article has a clean layout and is formatted properly. The information presented is well organised in chapters and sustained by lots of explanatory figures and tables. It is written in a scientific style and includes terminology that can’t be understood by someone without prior knowledge of the subject. I think this paper offers a viable solution for solving the vehicle Re-ID problem in urban surveillance systems because it optimizes the appearance features’ framework with both global and regional features and develops a more accurate spatio-temporal constraint model. There is also a solid number of references that shows the subject was well documented.
There are a lot of explanatory figures and images that simplify the process of understanding the content of the paper but some of them could be a little bigger (Figures 2, 3, 4). Some graphs are too small and hard to understand so I suggest making them a little bigger.
The article presents a new approach regarding the vehicle re-identification, or Re-ID, problem in opening scenarios. In order to enhance the discrimination performance of visual features and improve the accuracy of pose recognition, there were proposed the leveraging of the pose view and the utilisation of keypoints respectively. Unlike early-stage studies that focused on a certain view, this research faces more challenges due to variations, illumination changes, occlusions and more. Considering all this, a two-branch framework was designed for vehicle Re-ID, including a Keypoint-based Pose Embedding Visual (KPEV) model and a Keypoint-based Pose-Guided Spatio-Temporal (KPGST) model, which are integrated in said framework. Additional experiments on VeRi-776 and VehicleID datasets show the potential of this new approach.
The article has a well devised structure, being composed of explicit chapters, each containing several subsections which describe the main idea in more detail. There are many formulas and equations which may not be understood by everybody, but the subsequent explanations, as well as the images and schematics, do a very good job in making these concepts easier to grasp. The references that were used offer sufficient credibility, being up to date with the topics presented in the article.
I recommend checking the article with a grammar app and correcting the minor grammar and spelling mistakes.
- line 1: inconsistent hyphenation, both “”re-identification” and “reidentification” were used
- line 6: incorrect use of the comma
- line 40: missing the article “the”
- line 22: “One of significant functions in..” -> “One of the significant functions of..”
- line 29: “along the” -> “along with the”
- line 66: “not perform” -> “not to perform” / "not performing"
More references regarding related work on time-critical virtualized video should be added, for example:
- Xu, Zhuangdi, Harshil S. Shah, and Umakishore Ramachandran. "Coral-Pie: A Geo-Distributed Edge-compute Solution for Space-Time Vehicle Tracking." Proceedings of the 21st International Middleware Conference. 2020.
- Štefanič, Polona, et al. "SWITCH workbench: A novel approach for the development and deployment of time-critical microservice-based cloud-native applications." Future Generation Computer Systems 99 (2019): 197-212.
- Peri, Neehar, et al. "Towards real-time systems for vehicle re-identification, multi-camera tracking, and anomaly detection." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020.
Author Response
Response to “Electronics-1670973 Vehicle Re-identification with Spatio-Temporal Model Leveraging by Pose View Embedding”
Wenxin Huang, Xian Zhong, Xuemei Jia, Wenxuan Liu, Meng Feng, Zheng Wang, Shin'ichi Satoh
We gratefully thank all reviewers for their positive and constructive feedback. We are glad reviewers appreciate our clean layout, proper formation, viable solution, and novel framework. We have improved the writing according to the suggestions. We think now it is in good shape for publication.
Reviewer 1: AQ
The article has a well devised structure, being composed of explicit chapters, each containing several subsections which describe the main idea in more detail. There are many formulas and equations which may not be understood by everybody, but the subsequent explanations, as well as the images and schematics, do a very good job in making these concepts easier to grasp. The references that were used offer sufficient credibility, being up to date with the topics presented in the article. The article has a clean layout and is formatted properly. The information presented is well organised in chapters and sustained by lots of explanatory figures and tables. It is written in a scientific style and includes terminology that can’t be understood by someone without prior knowledge of the subject. I think this paper offers a viable solution for solving the vehicle Re-ID problem in urban surveillance systems because it optimizes the appearance features’ framework with both global and regional features and develops a more accurate spatio-temporal constraint model. There is also a solid number of references that shows the subject was well documented.
Response: Thank you for your positive feedback on our initial submission. We have carefully proofread the manuscript and prepared the revised manuscript.
Comments:
1). There are a lot of explanatory figures and images that simplify the process of understanding the content of the paper but some of them could be a little bigger (Figures 2, 3, 4). Some graphs are too small and hard to understand so I suggest making them a little bigger.
Response: For better understanding, we have adjusted the figures (Figure 2, 3, 4, 8 and 12) to bigger sizes for more clearly illustration.
2). I recommend checking the article with a grammar app and correcting the minor grammar and spelling mistakes.
Response: Thanks for your careful reading and considerate suggestion,we have revised this kind of typos in the uploaded manuscript.
3). More references regarding related work on time-critical virtualized video should be added.
Response: For better reference with time-critical virtualized videos, we have added some reference in the improved manuscript.
-Xu, Zhuangdi, Harshil S. Shah, and Umakishore Ramachandran. "Coral-Pie: A Geo-Distributed Edge-compute Solution for Space-Time Vehicle Tracking." Proceedings of the 21st International Middleware Conference. 2020.
- Štefanič, Polona, et al. "SWITCH workbench: A novel approach for the development and deployment of time-critical microservice-based cloud-native applications." Future Generation Computer Systems 99 (2019): 197-212.
- Peri, Neehar, et al. "Towards real-time systems for vehicle re-identification, multi-camera tracking, and anomaly detection." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
The manuscript proposes a pose view embedding-based vehicle re-ID network. The authors did this by combining a visual and spatio-temporal models, forming a two-branch framework. Although the network is novel and the current experimental results support the authors' claims, I still have the following minor concerns for publication of this manuscript:
- Lack of more recent competing schemes: more recent competing schemes are needed for the comparison of the keypoint regressor (Table 3), pose classifier (Table 5), and re-ID models (Table 6 and Table 9). At least one more competing scheme that is 2021 or later is needed for each of the above comparison.
Author Response
Response to “Electronics-1670973 Vehicle Re-identification with Spatio-Temporal Model Leveraging by Pose View Embedding”
Wenxin Huang, Xian Zhong, Xuemei Jia, Wenxuan Liu, Meng Feng, Zheng Wang, Shin'ichi Satoh
We gratefully thank all reviewers for their positive and constructive feedback. We are glad reviewers appreciate our clean layout, proper formation, viable solution, and novel framework. We have improved the writing according to the suggestions. We think now it is in good shape for publication.
Reviewer 2: AQ
The manuscript proposes a pose view embedding-based vehicle re-ID network. The authors did this by combining a visual and spatio-temporal models, forming a two-branch framework. Although the network is novel and the current experimental results support the authors' claims, I still have the following minor concerns for publication of this manuscript:
Response: Thank you for your positive feedback on our initial submission. We have carefully proofread the manuscript and prepared the revised manuscript.
Comments:
Lack of more recent competing schemes: more recent competing schemes are needed for the comparison of the keypoint regressor (Table 3), pose classifier (Table 5), and re-ID models (Table 6 and Table 9). At least one more competing scheme that is 2021 or later is needed for each of the above comparison.
Response:
1). In Table 3, we evaluate the performance of the designed keypoint regressor with 20 keypoint categories. So far as we know, OIFE is the only method that this keypoint regressor could compare with. Keypoint-based methods, such as Keypoint-Aligned Embeddings for Image Retrieval and Re-identification (WACV 2021: 676-685), utilize 20 keypoints annotation on Veri776, but they didn’t display the keypoint prediction on Veri776 or release their codes. Therefore, we cannot compare our keypoint regressor with theirs in fair.
2). In Table 5, we evaluate the pose classifier KPC. And there lacks of released codes of related pose classifier for us to reproduce them to for comparison.
3). In Table 6, we have added two approaches MCD and LCSR in 2021 for comparing with recent works.
- For Table 9, we discuss the advantage of the spatial-temporal strategy proposed in this paper, demonstrating that the effectiveness (13.98 % gains on KPEV) of spatial-temporal relationship. The proposed KPGST explores the feasibility to apply spatial-temporal relationship in the open world, where appearance features are not reliable enough for identification. For example, in the complicated scenario, only relying appearance would obtain unsatisfied performance, as shown in paper (Xian Zhong, Shilei Zhao, Xiao Wang, Kui Jiang, Wenxuan Liu, Wenxin Huang, Zheng Wang: Unsupervised Vehicle Search in the Wild: A New Benchmark. ACM Multimedia 2021: 5316-5325). And we will extend the insights in this paper to our future work.
Author Response File: Author Response.pdf
