A Non-Iterative Method Combined with Neural Network Embedded in Physical Model to Solve the Imaging of Electromagnetic Inverse Scattering Problem

Round 1

Reviewer 1 Report

The paper proposes a combined algorithm for the electromagnetic inverse scattering problems. Basically, a diagonal matrix inversion method (DMI) is used to calculate an initial guess for the distribution of scatterer contrast  (DSC) before feeding it to a generative adversarial network (GAN) to optimize the DSC further. The technical idea is quite interesting and it should be considered for publishing. However, the paper was not well-written with many English grammatical errors, language use issues, which might make it difficult to understand the paper. I suggest the authors revisit the paper thoroughly to fix these issues. I list below some of them just to name a few. Additionally, I have some comments/concerns that I hope the authors could address in the revised manuscript.

1. In the Introduction section, please provide more details of the neural networks used in [23] and [24] and what achievements in these papers. Please elaborate what new in your proposed method as compared to [23] and [24]. I suggest adding more references to previous work that used neural network for the electromagnetic inverse scattering problems.

2. Please highlight the main contributions of this paper in the Introduction section.

3. The DMI introduces some calculation errors but how much do these errors affect to the contrast estimation? It is clear that the GAN is used to eliminate the error produced by DMI but is there any scenario that these errors are so terrible that the GAN can not work?

4. At the end of Section 3.2, it is not clear what Xe and Xr are.

5. In Section 4.1, please elaborate what the differences between the GAN and the pix2pix.

6. At the end of Section 4, it is said that "In the final step, according to the results of the generator and the discriminator, optimize the discriminator and generator to balance them". When do you decide you need to optimize the generator and discriminator? and how to optimize them?

7. In Section 5.1, you mentioned that the scattered field measurements can be calculated by Equation 7. What is the Ein value used here?

8. Please specify whether you used MNIST ground truths to train the GAN or use the contrast calculated by DMI from the MNIST ground truths to train the GAN?

9. Why was the MNIST dataset chosen for training?

10. The GAN was trained at 4.4 GHz. How well does the GAN work at other frequencies?

11. In Section 5.1, there is a sentence "In this example, the real parts of the contrast are the ground truths and the imaginary parts are all set to 0". This is confusing. Isn't the constrast calculated from DMI?

12. In Section 5.2.4, it would be interesting to see results of objects more complex than square shape. The SRE results of the GAN in this case is 76.97342. Please elaborate what can be done to reduce the SREs for actual experimental measurements. Is MNIST dataset good enough for applications? Any guideline to choose training dataset for a particular application?

13. It would be nice to compare your work with other neural network based algorithms for electromagnetic inverse scattering problems, such as [23] and [24].

14. Some grammatical errors, language use suggestion are listed below:

- In the Abstract, "Compared with the convention tomography technology" should read "Compared with the conventional tomography technology"

- In the Abstract, "more real" should be changed to "in a more realistic manner"

- Also in the Abstract,  in the sentence "However, due to the nonlinearity of ISP. The conventional calculation scheme usually has some problems such as unsatisfactory imaging effect and long calculation cost", a comma, instead of period, should be used after ISP and "long calculation cost" should be "high computational cost".

- In the Abstract, in the sentence "the DSC generated by the generator is not only similar to the original distribution of scatterer contrast in numerical distribution, but also the numerical of each point is approximate to the original.", "not only" should be moved to the beginning of the sentence and what is "the numerical"?

- In the first paragraph of the Introduction section, the sentence "Although it has been proved that these methods can provide satisfactory results for objects of medium size and contrast" is not complete. Does it need to link with the next sentence?

- In the 2nd paragraph of the Introduction section, "... with the development of neural network, it has been one of the most influential methods ...". Does "it" here mean neural network? This sentence is weird to me. It could be changed to " ... neural network has been one of the most influential methods ..."

"neural network has also achieved excellent achievement. Such as magnetic resonance imaging [21] and computational optical imaging [22]" should be "neural network has also achieved excellent achievements, such as magnetic resonance imaging [21] and computational optical imaging [22]"

"...reducing computing cost through experiments..." should read "...reducing computational cost through experiments..."

- At the end of Section 3.2, "the condition number is too large that calculate generalized inverse of Es will be some calculation errors" should read "the condition number is too large that calculating generalized inverse of Es will suffer from some calculation errors."

  In the same paragraph, there is something missing in " ... but the distribution the of the elements ..."

- In Section 4.1.2, "send" in " ... the measured data obtained in the second step is send to the discriminator..." should be "sent"

- In the last sentence of the first paragraph in Section 5, remove one etra "the"

- In Section 5.1, it is said "The learning rate of generator is set to 10^-5 and the discriminator’s is set to 2^-4". Should it be 10^-4 here?

 The word "access" in "... 256 GB of memory access with Intel(R) Xeon(R) CPU E5-2690 v4 ..." is redundant.

- In Section 5.2, " ... which was trained successfule ..." should be "which was trained successfully"

Author Response

Point 1: In the Introduction section, please provide more details of the neural networks used in [23] and [24] and what achievements in these papers. Please elaborate what new in your proposed method as compared to [23] and [24]. I suggest adding more references to previous work that used neural network for the electromagnetic inverse scattering problems.

 

Response 1:

The following will be added to the introduction:

DeepNIS is a neural network composed of multiple complex-valued residual convolutional blocks cascaded[23]. The complex-valued residual convolutional is used to approximately characterize the multi-scattering physical mechanism. And the inputs of DeepNIS come from the back-propagation images. DeepNIS greatly reduces the computational time compared with the traditional iterative method. CVP2P constructs a complex-valued network with reference to the Generative Adversarial Networks[24]. The generator of CVP2P adopts a multilayer complex-valued convolution neural network that can calculate complex-valued convolution. The back-propagation images are also used to be the inputs of CVP2P. This algorithm is mainly used to reconstruct binary images, and the reconstruction time is significantly reduced.

 

Point 2: Please highlight the main contributions of this paper in the Introduction section.

 

Response 2:

We replaced the end of the introduction section with the following:

In this paper, we propose a simple but effective method – the diagonal matrix inversion method (DMI) to get an initial estimate of the distribution of scattering contrast. We also refer to the GAN idea to establish a neural network to decrease the error between the initial estimation and the real contrast. The generator of the GAN is similar to a “U-Net”-based architecture [18], and for the discriminator, a “PatchGAN” like classifier is used for auxiliary discrimination [26]. The generator obtains the bottom features through multiple down-sampling layers and then restores the bottom features to the real contrast distribution through up-sampling. The features obtained from the down-sampling layers are connected with the features restored by the up-sampling layer through skip-connection so that the information lost in the process of down-sampling can be collected and the real contrast can be restored more fully. When training, we make the physical model between the generator and the discriminator give assistance. The theoretical measured value can be calculated by the forward model with the contrast distribution obtained by the generator. And then the discriminator judge whether the theoretical measured value is close to the real measured value. This makes the generator Eliminate errors more accurate. Obtaining the initial contrast through the DMI and using the generator to make the initial contrast close to the real contrast is a non-iterative method for solving ISP. After our experimental verification, the GAN combined with the DMI method is superior to the traditional imaging scheme in both imaging effect and speed.

 

 

 

Point 3: The DMI introduces some calculation errors but how much do these errors affect to the contrast estimation? It is clear that the GAN is used to eliminate the error produced by DMI but is there any scenario that these errors are so terrible that the GAN can not work?

 

Response 3:

The error of DMI is introduced by the inversion process of the measurement data . When the condition number of  is large to a certain extent, some values of the initial contrast calculated by DMI will be much larger than other values. Such initial contrast will cause the GAN to fail to work.

 

Point 4: At the end of Section 3.2, it is not clear what  and   are.

 

Response 4:

At the end of Section 3.2, the  is the contrast calculated by DMI, and the   is the contrast calculated by the real contrast with Equation 11. We have fixed it.

 

Point 5: In Section 4.1, please elaborate what the differences between the GAN and the pix2pix.

 

Response 5:

In terms of the generator, the GAN and pix2pix both adopt “U-Net”-based architecture, but the up-sampling layers and down-sampling layers used by pix2pix are 7 layers and our GAN is 5 layers. For discriminator, since the data of each column in the scattering field measurement data matrix does not affect each other, the filter we used is  while pix2pix select . It is different from pix2pix directly inputting the generated result into the discriminator, we input them into the discriminator after the forward model calculation.

 

Point 6: At the end of Section 4, it is said that "In the final step, according to the results of the generator and the discriminator, optimize the discriminator and generator to balance them". When do you decide you need to optimize the generator and discriminator? and how to optimize them?

 

Response 6:

Firstly, the generator is not trained, only the discriminator is trained. Until the discriminator can distinguish the real contrast image from the contrast image generated by the generator. Then stop training the discriminator and train the generator until the image generated by the generator can invalidate the discriminator. After N cycles, the final judgment probability of the discriminator is close to 0.5 and the training process is completed.

 

Point 7: In Section 5.1, you mentioned that the scattered field measurements can be calculated by Equation 7. What is the value used here?

 

Response 7:

 is the incident field, it is similar to the field generated by a point source. It can be calculated by Green Function.

 

Point 8: Please specify whether you used MNIST ground truths to train the GAN or use the contrast calculated by DMI from the MNIST ground truths to train the GAN?

 

Response 8:

First, the scattered field measurements can be calculated by Equation 7 (forward model) with Ground Truths. Second, the initial contrast images can be calculated by Equation 8 and 9 (DMI) with the scattered field measurements calculated by the first step. we use the initial contrast images in the second step for training the GAN. 

 

Point 9: Why was the MNIST dataset chosen for training?

 

Response 9:

Firstly, the MNIST dataset is used for neural network training very commonly and the previous work had proved the MNIST dataset is effective for neural network training. Secondly, the features of handwritten numerals are similar to those of the object we want to actually imaging.

 

Point 10: The GAN was trained at 4.4 GHz. How well does the GAN work at other frequencies?

 

Response 10:

If the GAN was trained at 4.4GHz, the ability to reduce errors will be weakened to some extent at other frequencies. But it still works.

 

Point 11: In Section 5.1, there is a sentence "In this example, the real parts of the contrast are the ground truths and the imaginary parts are all set to 0". This is confusing. Isn't the constrast calculated from DMI?

 

Response 11:

The contrast   is used for forward model to calculate the measurement data . And Es is used for DMI to estimate the contrast  with error. the real part of   is set to ground truth and the imaginary part is set to 0.   is used to the GAN.

 

Point 12: In Section 5.2.4, it would be interesting to see results of objects more complex than square shape. The SRE results of the GAN in this case is 76.97342. Please elaborate what can be done to reduce the SREs for actual experimental measurements. Is MNIST dataset good enough for applications? Any guideline to choose training dataset for a particular application?

 

Response 12:

Adding the actual experimental data to MNIST for training, the trained GAN can reduce the SREs for actual experimental measurements. MNIST may not be perfect for application. It would be better to if add some actual experimental data for training. For a particular application, it is best to add some datasets with features that are similar to the application to MNIST datasets for training.

 

Point 13: It would be nice to compare your work with other neural network based algorithms for electromagnetic inverse scattering problems, such as [23] and [24].

 

Response 13:

We hold the same opinion.

 

Point 14: Some grammatical errors, language use suggestion are listed below:

 

Response 14:

Thank you for your comments concerning our manuscript. The grammatical errors and the language use suggestion we had received and fixed them.

Reviewer 2 Report

The manuscript is devoted to the application of learning neural networks to help solve the inverse electromagnetic scattering problem. This is a very interesting and actual area of research. However, the text style is very poor and contains many errors, especially in the Abstract and in the Introduction. I pointed out some of such examples of drawbacks. However, the whole text must be carefully revised and edited by an English-native-speaking editor.  

Comments to Authors:

line 1: “imaging of electromagnetic scattering problem”.

Accordingly to my knowledge, it is not a common word combination in literature. You can use separate phrases: “microwave imaging” and “(inverse) electromagnetic scattering problem”.

line 2: convention -> conventional

line 4: “.” -> “,”

line 5: “long calculation cost” -> “long calculation time”

line 6: “get the contrast of scatters better” - not a good phrase

line 8: “which could optimize the DSC obtained by DMI” - a bad phrase that distorts the meaning like “which could improve DMI image with help of the DSC”

line 12-line 13: “the DSC generated by the generator is not only similar to the original distribution of scatterer contrast in numerical distribution, but also the numerical of each point is approximate to the original”  - very bad phrase, the meaning is unclear.

line 16: “is an accurate imaging measurement method” – what means “imaging measurement”?

line 16-line 17: “It determined” -> “It determines”

….

line 37: “The imaging of electromagnetic inverse scattering can also be solved by neural network, and some achievements have been achieved” – poorly constructed sentence.

line 57: “denotes the state matrix and measurement matrix respectively” “state matrix” -> “denote the Green's object and data matrixes respectively” (relating to van den Berg’s terminology [10]).

line 58 – line 59 “When each device acts as a transmitter, the equations becomes the following:” – this phrase does not explain a logical transition from eq. (3), (4) to eq. (5), (6). There is no information that Ein, Ez, and Es are matrixes rather than vectors in (5), (6).

line 64-line 65 “Equation 5 and 6 are nonlinear equations, therefore the equations” – bad style.

Eq. (8) is wrong because there is a dimensional mismatch of matrices in the matrix product.

The correct equation is gam^-1 = Ein Es^Y Gm + Gs.

line 78 “serious diagonal matrix” -> “exact diagonal matrix”.

line 85 “Core Idea” -> “main idea” or “central idea”.

line 114-line 115 “And not only the distribution of contrast, the contrast of each pixel can also be generated by the generator.” – missed “but”.

line 145: “each image will be converted” -> “each image was converted”

line 149: “24 transmitting devices” -> “24 transmitting-receiving antennas”

line 224-line 233 and everywhere, like in “5GHz”, “7.3cm” - numbers and units should be separated with spaces.

Author Response

Point 1: The manuscript is devoted to the application of learning neural networks to help solve the inverse electromagnetic scattering problem. This is a very interesting and actual area of research. However, the text style is very poor and contains many errors, especially in the Abstract and in the Introduction. I pointed out some of such examples of drawbacks. However, the whole text must be carefully revised and edited by an English-native-speaking editor. 

 

Response 1:

Thank you for your comments concerning our manuscrpit. Those comments are valuable. We had read through the comments carefully and had revisited the paper thoroughly to fix these issues. The whole text was also revised and edited again.

Reviewer 3 Report

The authors claim that the method is non-iterative, however it requires iterative process using machine learning. The input from DMI is more like an initial guess. Is it appropriate to say it is non-iterative?

Regarding the training in GAN, what kind of data is used for the training? The training data certainly impacts the quality of the reconstruction. How should we identify data set for training when an unknown problem is provided?

The overall results using the proposed method is comparable or even outperform MR-CSI method. How does GAN compare with MR-CSI in terms of computation?  

Any practical advantages of MR-CSI over GAN?

Other Suggestions

Equation 7 and 8 are not apparent – please provide derivation as these equations are important.

Figure 4, 6,  8, 9 and 12: Scale of the images is between 0 to 1. What is the unit of this scale? What are the corresponding permittivity and conductivity?

Figure 5 and 7: What are the scales for the real and imaginary parts?

 

Author Response

Thank you for your comments concerning our manuscrpit. Those comments are valuable. We had read through the comments carefully and had revisited the manuscript. Our reponses for your comments are in the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Thank you for addressing my comments for the previous version of the paper. However, I still have some comments for the revised manuscript as follows:

- In the second paragraph of the Introduction section, when the Generative Adversarial Networks [24] was first mentioned, its abbreviation GAN should be added here.

- I suggest adding more references for previous work that used neural network for the electromagnetic inverse scattering problems rather than just [23] and [24] in the second paragraph of the Introduction section.

- I suggest adding bullet points to highlight the main contributions of this paper at the end of the Introduction section.

- I appreciate the authors' answers to my questions in the author's reply. However, please integrate your answers into the paper too, such as points 7, 8, 9,10, 11, 12, etc

- Regarding the MNIST datasets, the authors said in the author's reply that "the features of handwritten numerals are similar to those of the object we want to actually imaging"--> Please elaborate what features. It looks to me that there was no discussion about the reason of choosing MNIST for training in the paper.

- Regarding the 4.4GHz, the authors said in the author's reply that "the ability to reduce errors will be weakened to some extent at other frequencies. But it still works."--> Please add results to back this up.

- In my comment on the previous version, I suggested the authors add more results of objects more complex than square shape in Section 5.2.4. Please do that.

- Also in my previous comment, I suggested adding results for comparison between the proposed algorithm with other neural network based algorithms for electromagnetic inverse scattering problems, such as [23] and [24]. Please do that.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 3

Reviewer 1 Report

Thank you for addressing my concerns. I just have one minor comment. Please pay attention to grammatical errors and typos throughout the paper, especially the newly added paragraphs.

Author Response

Point 1: Thank you for addressing my concerns. I just have one minor comment. Please pay attention to grammatical errors and typos throughout the paper, especially the newly added paragraphs.

Response 1: Thank you for your comment about our manuscript. We have rechecked and corrected the grammatical errors and typos in our manuscript.