Round 1

Reviewer 1 Report

Using Machine Learning to Profile Asymmetry Between Spiral Galaxies with Opposite Spin Directions

By Lior Shamir

Referee's report 

Dear Editor:

This paper tackles the problem of possible morphological differences between clockwise and counterclockwise spin galaxies, using machine learning techniques for spin orientation classification. If the spin depends only on the galaxy orientation to the observer, the morphological clockwise and counterclockwise patterns should be symmetric.

This paper deserves publication after considering some issues.

• The author's apparent interest in the possible asymmetry between the observed numbers of galaxies with clockwise and counterclockwise spin is clearly reflected in the introduction and most of the discussion section. It is indeed an exciting issue, but it is not the subject of this paper.
• Line 60: 'evidence of asymmetry between morphological differences' is confusing.
• Lines 101-103: I can't see the point of carefully inspecting the 12 galaxy images with different annotations if they are finally discarded.
• Lines 117-120: Deep neural network results are hardly interpretable, but, at least in this work, the Wndchrm method does not provide any statistically significant insight either (see § 3.3). Therefore, I can't see that the lack of interpretability is a drawback for neural networks in this case.
• Lines 123-125: The statement that 'DNNs might not be a sound approach' is not sufficiently documented. DNNs, particularly neural networks with convolutional layers, can detect morphological differences and asymmetries in a large variety of circumstances. It is conceivable that several neural networks may yield better results than the Wndchrm method. Of course, the researcher should take care of possible biases —all ML procedures are subject to biases— but this does not invalidate the methodology.
• Line 265:  'the change is expected to be symmetric' It is not clear what change the author is referencing.
• § 4 Discussion: As commented above, the possible asymmetry between the number of clockwise and counterclockwise spin galaxies dominates the discussion, though it is not a research subject of this paper. This referee would like to read a deep analysis centered on this work's results, highlighting their relevance and possibly further steps.

Author Response

Dear Editor:

This paper tackles the problem of possible morphological differences between clockwise and counterclockwise spin galaxies, using machine learning techniques for spin orientation classification. If the spin depends only on the galaxy orientation to the observer, the morphological clockwise and counterclockwise patterns should be symmetric.

This paper deserves publication after considering some issues.

--Author response: I thank you for the time you spent on reading and commenting on the manuscript, the insightful and useful comments, and also for the rapid response. The comments are really helpful, and I specifically like the first and last comments, which are closely related, but are aligned with what I had in mind when writing the paper. The replies to the specific comments and the description of the changes made to the manuscript are specified below. The changes made to the manuscript are in bold font.

• The author's apparent interest in the possible asymmetry between the observed numbers of galaxies with clockwise and counterclockwise spin is clearly reflected in the introduction and most of the discussion section. It is indeed an exciting issue, but it is not the subject of this paper.

--Author response: I agree! That is also related to your last comment, and also to the exact same comment made by the other reviewer. I mixed between two things that may or may not be related to each other, but with the current evidence there is no basis to the link between the observation reported in Section 3 and the theories discussed in Section 4. I therefore removed the discussion about these theories, that may be related to other observations I am currently working on, but not the one reported here.

• Line 60: 'evidence of asymmetry between morphological differences'is confusing.

--Author response: The sentence was indeed not written well. It has been completely re-written:

“Here, a machine learning method is applied to test whether galaxies that spin clockwise are morphologically the same as galaxies that spin counterclockwise as observed from Earth. While each galaxy is different, comparison of a large number of galaxies enables a statistical analysis that can allow to identify possible morphological differences between galaxies that spin in opposite directions.”

• Lines 101-103: I can't see the point of carefullyinspecting the 12 galaxy images with different annotations if they are finally discarded.

--Author response: Well, without inspecting the galaxies it would not have been possible to determine whether the spin direction can be identified clearly or not. If the spin direction can be identified, the galaxy can still be used. But if the spin direction cannot be identified the galaxy should be discarded. But without inspecting the image manually it is not possible to determine. The way the sentence was written made it unclear. The wording of the sentence has been changed (in bold font) to make it easier to read. A figure (Figure 3 in the revised manuscript) has also been added to show some examples of galaxies that their spin direction cannot be identified clearly.

• Lines 117-120: Deep neural network results are hardly interpretable, but, at least in this work, the Wndchrm method does not provide any statistically significant insight either (see § 3.3). Therefore, I can't see that the lack of interpretability is a drawback for neural networks in this case.

--Author response: Yes. That is correct, but we can only know that after applying the method. If a single specific descriptor would identify the difference between the galaxies the method would have a reasonable chance to identify it. With DNNs, whether such descriptors exist or not, there is no way to identify specific descriptors because DNNs do not break the analysis into explainable attributes. That discussion has been added to the revised paper (in bold font). The approach also allows to identify the Zernike polynomials, which might not be statistically significant but may give some information. As described in the response to the next comment, an experiment with a DNN has been added.

• Lines 123-125: The statement that 'DNNs might not be a sound approach'is not sufficiently documented. DNNs, particularly neural networks with convolutional layers, can detect morphological differences and asymmetries in a large variety of circumstances. It is conceivable that several neural networks may yield better results than the Wndchrm method. Of course, the researcher should take care of possible biases —all ML procedures are subject to biases— but this does not invalidate the methodology.

--Author response: A new section (Section 3.3) has been added to address that. The new section shows results of using a DNN. I was given just five days to prepare the revisions, so I used a neural network that was used in the past for the same task (Dhar & Shamir, 2022). One of the problems with the DNN is that it is that it is not rotationally invariant, so it provides classification between the classes of the original images that is driven by the spin direction. When mirroring the galaxy images, the classification accuracy expectedly drops sharply, but as before it does not become random. This is summarized in the new sub-section 3.3.

• Line 265:  'the change is expected to be symmetric' It is not clear what change the author is referencing.

--Author response: That sentence was not written well, and it has now been revised. The purpose was that when the redshift changes, the change affects clockwise galaxies in the same manner it affects counterclockwise galaxies. Therefore, if clockwise galaxies are morphologically the same as counterclockwise galaxies, different redshift ranges are not expected to lead to higher classification accuracy if the galaxies are morphologically the same (except for their spin direction). That discussion has been added to the revised manuscript.

• 4 Discussion: As commented above, the possible asymmetry between the number of clockwise and counterclockwise spin galaxies dominates the discussion, though it is not a research subject of this paper. This referee would like to read a deep analysis centered on this work's results, highlighting their relevance and possibly further steps.

--Author response: Yes. The discussion in Section 4 was exactly also the comment of the other reviewer. As I replied to Reviewer #2, I actually questioned the relevance of that discussion to the observation myself as I wrote that section. That discussion was forced on the observation, rather than driven from it. The only assumption that can be made is that the difference in morphology is driven from the different distribution in higher redshift ranges, and that has been added to the revised Section 4.

Future work will have to include analysis of far larger datasets with more galaxies, larger footprints, and especially more galaxies with spectra. That will allow to verify and profile the nature of the asymmetry in different directions of observations and redshift ranges, and identify possible patterns (that, if identified, can lead to the theories discussed in the previous version of the manuscript). That has also been added to the revised Section 4.

Reviewer 2 Report

The essay deals with the use of machine-learning software to determine the rotation sign of samples of spiral galaxies. I think that the author is a bit optimistic when writing that "The results show that the classifier was able to predict the spin direction of the galaxy by its image in accuracy higher than mere chance..." (abstract, line 9-10). The showed results indicate a percent of accuracy of 51-55%, depending on the adopted algorithm. It is also rather surprising that this percentage increases with the redshift, because one expects the opposite. While I can understand the need to publish intermediate steps toward a better software, I would avoid the discussion on cosmological topics, which are not clearly supported by the results of the analysis. Therefore, I have no objection to the publication, but I would limit the essay to the software and analysis part (thus removing the Sect 4). 

A couple of additional notes:

• page 2, line 83-84: The author writes: "Because many spiral galaxies, such as elliptical galaxies, do not have a visually clear spin direction, not all galaxies can be assigned with a spin direction by their visual appearance alone." Elliptical galaxies are not spiral galaxies. 
• page 5, line 172-173: the conversion from FITS to TIFF is questionable, as the FITS format guarantee the best format for astronomical data. 

Author Response

The essay deals with the use of machine-learning software to determine the rotation sign of samples of spiral galaxies. I think that the author is a bit optimistic when writing that "The results show that the classifier was able to predict the spin direction of the galaxy by its image in accuracy higher than mere chance..." (abstract, line 9-10). The showed results indicate a percent of accuracy of 51-55%, depending on the adopted algorithm. It is also rather surprising that this percentage increases with the redshift, because one expects the opposite. While I can understand the need to publish intermediate steps toward a better software, I would avoid the discussion on cosmological topics, which are not clearly supported by the results of the analysis. Therefore, I have no objection to the publication, but I would limit the essay to the software and analysis part (thus removing the Sect 4). 

--Author response: First of all, I would like to thank you for the time you spent on reading and commenting on the manuscript, and for the very helpful comments. I especially like the first comment, which the other reviewer also made, and I have to agree with. I actually had the same thoughts when I prepared Section 4. The cosmological theories mentioned in Section 4 might be interesting (or even true), but there is no apparent link between these theories and the observation described in the paper. Section 4 has changed substantially, and the cosmological theories were removed. The first paragraph is just well-known “textbook” information about the anomalies of the nature of galaxy rotation. The second paragraph is a summary of the findings reported in the paper. The link with the redshift is indeed counterintuitive, and can be explained by the increasing asymmetry between the number of galaxies spinning in opposite directions as the redshift increases. That is also explained in the revised Section 4. So Section 4 was not completely removed, but the entire discussion about the cosmological theories was removed. Changes and new addition to the section were made in bold font.

A couple of additional notes:

• page 2, line 83-84: The author writes: "Because many spiral galaxies, such as elliptical galaxies, do not have a visually clear spin direction, not all galaxies can be assigned with a spin direction by their visual appearance alone." Elliptical galaxies are notspiral galaxies. 

--Author response: That’s obviously correct. The intention was "Because many galaxies, such as elliptical galaxies…”. That has been corrected.

• page 5, line 172-173: the conversion from FITS to TIFF is questionable, as the FITS format guarantee the best format for astronomical data. 

--Author response: The following paragraph has been added to address the comment:

“While the TIFF format is not a common file format in astronomy, it is much more frequent in the field of machine vision. Normally, the TIFF format does not allow to deduce accurate photometric information that is available when using other formats such as FITS. But in the case of this study, the important information is not the photometry, but the morphology of the galaxies, and therefore the ability to deduce accurate photometry is not a primary expectation from the image format. Because the TIFF images contain several color channels in a single image file, they provide more useful information to analyze the shape of the galaxy compared to FITS images, that normally provide a single color band. The TIFF files are not compressed, to avoid possible effect of the compression algorithm.”