Shared Knowledge Distillation Network for Object Detection
Round 1
Reviewer 1 Report
Comments and Suggestions for Authors The paper proposes a Shared Knowledge Distillation framework to minimize the shared features between the teacher-student layer and the cross-features within the student, which achieves cross-layer distillation without complex transformations. The research program appears to be feasible overall, but please pay attention to the following problems.1. The abstract does not highlight the innovative contributions in model construction.
2. Please provide a detailed explanation of the unique contribution of this paper in comparison to existing literature in the literature review section.
3. This paper does not include detailed diagrams for each module of the modeling framework, and it should reflect the core model structure design in Figure 1.
4. The experimental session should clarify the parameter settings, such as value of the tunable weighting factor in Equation 8.
5. The contributions should be more clearly explained with more details on how to improve the existing results in experiments section.
6. The performance comparison with SOAT algorithm is insufficient. It should include a more comprehensive comparative analysis of typical object detection and instance segmentation methods in subsections 4.1 and 4.2.
7. Figure 3 does not clearly visualize the qualitative analysis, and it is recommended to provide the ground truth.
8 The references lack uniformity in their overall format. For example, references 5, 6, 32, and 35 are all from CVPR.
9. The authors need a careful check of English, formulas and format/style.
Comments on the Quality of English Language
Moderate editing of English language required.
Author Response
Thank you very much for your guidence and valuable suggestions. Please see the attachment.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
Comments and Suggestions for AuthorsThis paper proposed Shared-KD networks for object detection tasks, which aims to overcome optimization challenges caused by disparities in cross-layer features between teacher and student networks. In the reviewer’s opinion, the following technical issues need to be properly addressed:
1. In the methodology section, it would be beneficial to provide a pseudo code to help explain the proposed Shared-KD technique.
2. In the experimental study section, the reviewer suggests adding a “distillation” column in Tables I and II to explain the different distillation loss strategies adopted by the implemented methods.
3. In the hyperparameter sensitivity section, why do the authors only select 2 and 5 for the weight alpha rather than a wider range of values? Justifications need to be provided.
4. In the ablation study section, the authors should include additional models in the cross comparison, by removing or disabling certain modules or components (e.g., identical layer distillation, cross-layer distillation) in the proposed model. Without this step, it is unclear about the functionality of each component in the proposed Shared-KD technique.
5. In the experimental study, results were not discussed in a thorough and convincing manner. What the authors performed was merely comparing the numbers from different methods and concluded that their method outperformed the others. No in-depth discussions were provided that could help explain how and why the proposed different components in the Shared-KD improved the object detection performance. And from the experimental study, it is still unclear how the proposed technique addresses the “optimization challenges caused by disparities in cross-layer features between teacher-student networks”.
6. The authors claimed that “other cross-layer distillation techniques require complex feature transformations and optimizations, resulting in increased training time and resource requirements. … Shared knowledge distillation techniques improve the efficiency … achieving superior performance and training acceleration”, but there is no quantitative evaluation on the model efficiency in the experimental study section.
7. In Figure 3, visualization results from other benchmark methods, such as FGD, should be included for cross comparison purposes.
Comments on the Quality of English LanguageEnglish language is OK.
Author Response
Thank you for your guidance and valuable suggestions. Please see the attachment.
Author Response File: Author Response.pdf
Reviewer 3 Report
Comments and Suggestions for AuthorsThis paper presents a Shared-KD network, a new framework for addressing the challenges associated with cross-layer feature discrepancies in teacher-student networks. The authors show that Shared-KD outperforms other methods, achieving performance gains across different neural network architectures.
The paper presents a scientific contribution in general terms. My only concern is the focus on 5G/6G networks. I don't understand how efficient object detection benefits network transmission. I understand that network transmission is independent of computation, either on the cloud or edge. Please explain.
Author Response
Thank you for your guidance and valuable suggestions. Please see the attachment.
Author Response File: Author Response.pdf
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsAll the comments I raised have been solved.
Author Response
Dear reviewer, thank you very much for your valuable comments and encouragement.
Reviewer 2 Report
Comments and Suggestions for AuthorsQ1:
The “Algorithm 1” needs to be self-contained to explain key procedures. By examining the pseudo code itself, one can see that the input hyperparameter lambda is not used in the subsequent steps. fea^s and fea^t, the two feature variables, are not mentioned in the subsequent steps, either. The authors should use consistent notations of phi_s and phi_t as they used in the paragraphs, and indicate the losses are functions of these features.
Q2:
The changes claimed by the authors were not reflected in the revised manuscript. It could be beneficial to briefly explain the different distillation loss strategies adopted by the implemented methods.
Q3: OK
Q4: OK
Q5: OK
Q6: The changes claimed by the authors were not reflected in the revised manuscript.
Q7: OK
Comments on the Quality of English Language
OK
Author Response
Dear reviewer, thank you very much for your valuable comments and encouragement. We have modified our manuscript as your suggestions. Please see the attachment.
Author Response File: Author Response.pdf
