Facial Expression Recognition of Nonlinear Facial Variations Using Deep Locality De-Expression Residue Learning in the Wild

Round 1

Reviewer 1 Report

This article presents relevant and interesting research, is generally well presented and written, and has an adequate structure. However, I would recommend incorporating some improvements in order to increase the quality of the work:
- The introductory part is, in my view, too short. It would motivate somewhat more the open problem that the authors intend to solve, indicating with clear references which are the challenges to be tackled.
- In the related work part, I believe that the information is somewhat disjointed and disordered. Perhaps I would structure it following a methodology of systematic review of literature, including information on how this systematic search has been carried out. In this part I have to say that, given the topicality of the subject, there is a lack of more updated references (there are none from this year, next to conclude, just a few from the past).
- In the validation part, it might include some descriptive information on the known data sources used, beyond providing only references to those data sets. This would make it clear that the data are sufficiently heterogeneous and valid to carry out such experimentation. In addition, it would include a small discussion section on threats to the validation carried out, precisely addressing these issues.
- In general, I believe that the conclusions part does not highlight how this type of techniques could be applied in other similar domains, nor how to apply the recognition of facial expressions in novel and beneficial applications for society. And I do not detect this problem only in the conclusions: in general, the proposed idea is too concise and is not explained in sufficient depth and at the same time general enough for a reader to learn useful lessons in other similar domains.

Author Response

Responses to Reviewer Comments:

 

The authors are grateful to the reviewers for his/her thorough review, appreciation and salient observations. Personally I am so much thankful to him/her just because   have seen a massive improvement in my paper now just because of his/her experience. It is our sincere hope that our careful revisions will fully address the learned reviewer’s insightful further suggestions and concerns. We are thankful for the reviewer's encouraging review that help us to improve and resubmit the paper.

Reviewer: 1

Comments to the Author:

This article presents relevant and interesting research, is generally well presented and written, and has an adequate structure. However, I would recommend incorporating some improvements in order to increase the quality of the work:

Comment 1. The introductory part is, in my view, too short. It would motivate somewhat more the open problem that the authors intend to solve, indicating with clear references which are the challenges to be tackled.

Response 1: Thanks for your valuable suggestion. More data regarding motivations have been added in the introduction section. In the last paragraph of introduction section from line 52 onwards further details regarding the problems address in this paper is added. Meanwhile the introduction part have been made long as well.

 

Comment 2: In the related work part, I believe that the information is somewhat disjointed and disordered. Perhaps I would structure it following a methodology of systematic review of literature, including information on how this systematic search has been carried out. In this part I have to say that, given the topicality of the subject, there is a lack of more updated references (there are none from this year, next to conclude, just a few from the past).
Response 2: Thank you for the kind suggestions. I have described the methods as sub sections and I got the point of the flow in the related work section and have now ordered the information. Meanwhile given topicality of the subject I have cited few articles from the near past. It can be seen in the related work section from line 81 to 142.

 

Comment 3. In the validation part, it might include some descriptive information on the known data sources used, beyond providing only references to those data sets. This would make it clear that the data are sufficiently heterogeneous and valid to carry out such experimentation. In addition, it would include a small discussion section on threats to the validation carried out, precisely addressing these issues.

Response 3: Thanks for pointing out this and for the suggestion. In the experimental results section for Cohn Kanade Extensive dataset the information about data set has been added from page 299 and same approach has been used for other datasets too. Meanwhile threats to validity subsection has been from line 331-348.

 

 

Comment 4. In general, I believe that the conclusions part does not highlight how this type of techniques could be applied in other similar domains, nor how to apply the recognition of facial expressions in novel and beneficial applications for society. And I do not detect this problem only in the conclusions: in general, the proposed idea is too concise and is not explained in sufficient depth and at the same time general enough for a reader to learn useful lessons in other similar domains.

Response 4: Thank you for the point been raised. In introduction section we have mentioned the use of facial expression recognition i.e. “Facial expression analysis has quite a massive range of practical and  potential applications, such as social robotics, intelligent tutoring systems, medical treatment, personalized service provision, driver fatigue surveillance and various other augmented and virtual reality systems” but still the point of reviewer was valid so have inserted multi-modality concept and mentioned explicitly the fusion of facial expression recognition with any other model which can be fruitful for the society.

 

At the end once again highly obliged for your special comments regarding improvement of the paper.

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper “Facial Expression Recognition of Non Linear Facial Variations using Deep Locality Preserving De-expression Residue Learning in-the-wild” by Asad Ullah, Jing Wang, M. Shahid Anwar, Uzair Saeed and Usman proposed a method to control the magnitude of each Action Unit (AU) and combine several of the Action Unit combinations to leverage learning from the generative and discriminative representations for automatic FER. The effectiveness of their method (DLP-DeRL) through qualitative and quantitative experimental results were validated. The work is nice and comprehensive, however, the following points should be considered:

1)            Figure 1 is not very clear, and the feature fusion and person normalization is not cleared show.

2)            The reviewer did not the biggest contribution of this paper, and the author must make it clear.

3)            The font size is suggested to be as consistent as possible in the Figures.

4)            Line 229, it should have lower case.

5)            The authors should number and make it clear on figure 2.

Author Response

Responses to Reviewer Comments:

 

The authors are grateful to the reviewers for his/her thorough review, appreciation and salient observations. It is our sincere hope that our careful revisions will fully address the learned reviewer’s insightful further suggestions and concerns. We are thankful for the reviewer's encouraging review that help us to improve and resubmit the paper.

 

Reviewer: 2

Comment 1. Figure 1 is not very clear, and the feature fusion and person normalization is not cleared show.

Response 1: Thanks for your valuable suggestion. Changes have been made in the Figure 1, it was having problem just because of having so much details in a single image but now changed the feature fusion and person normalization meanwhile the font size has been adjusted too. For reference you can have a look at the figure after line 172.

 

Comment 2: The reviewer did not the biggest contribution of this paper, and the author must make it clear.

Response 2: Thank you for raising the point. In abstract section we have mentioned to tackle issues like over-fitting and other non-linear facial variations issues related to facial expressions too but there we have mentioned that “We have also addressed the problem of diversification of expressions from lab controlled to real-world scenarios from our cross-database study and proposed a model for enhancement of the discriminative power of deep features while increasing the interclass scatters and by preserving the locality closeness” pointed at line 10 and onwards. But still in order to avoid ambiguity in contribution we have added points to the introduction part too from line 52.

 

Comment 3. The font size is suggested to be as consistent as possible in the Figures.

Response 3: Thanks for pointing out this and for the suggestion. The font sizes in the figures have been adjusted accordingly.

 

Comment 4. Line 229, it should have lower case.

Response 4: Thank you for mentioning the point.  It has been modified. Line 260 for reference.

 

Comment 5. The authors should number and make it clear on figure 2.

Response 5: Thank you for your kind suggestion. The sub figures have been described explicitly in figure 2. You can have a look after line 237.

At the end thanking you for your valuable suggestions.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

The paper has been approved.