Evaluation of MEMS NIR Spectrometers for On-Farm Analysis of Raw Milk Composition

Round 1

Reviewer 1 Report

Dear authors,

In this paper you tested the ability of MEMS NIR spectrometers to predict fat, protein, and lactose content of individual cow milk. The evaluation was carried out considering the MIR-predicted trait as a reference analysis (ISO 9622).

The paper is of interest but often not easy to follow. The explanation of the hardware is quite long, but exhaustive and certainly necessary. What I found very confusing and chaotic was the chemometric part; see comments below.

Having NIR-predicted milk composition traits on a daily basis would be useful compared to data available on monthly basis in herds under the official milk recording/DHI. You should highlight this aspect in the manuscript; potentially, having milk composition data twice a day will be of great interest in the era of PLF. With adoption of specific algorithms, one can successfully and fastly detect variation in some milk traits in real time and potentially identify cows with a specific issue to be monitored. Recently published papers studied ‘patterns’ of certain milk components and propose these as indicators of cow health and resilience Few examples are: 10.3168/jds.2019-17290, 10.3168/jds.2017-13270, 10.3168/jds.2019-17826, etc.

The major problem I find in this paper is that you used repeated obs. per cow (≥2 milk samples), no matter either in the validation or in the calibration set. Stated at L216-218. How can you account for this data structure, i.e. for cow-intrinsic factors, i.e. within-animal variability? See section 2.6 and section 3.4 in: 10.1016/j.compag.2020.105734.

A curiosity: why not evaluating predictive ability on bulk milk in this paper?

 

Specific comments:

• L107-108: this sounds like an ‘aim’, remove from here
• L224: 15 combinations (5 spectrometers x 3 components)
• L226-267: not very clear in the current form, a flow chart would be useful to readers. I understand that you i) firstly transformed (or not) the wavelength variables with log, ii) then at the second step you basically performed (or not) the mathematical treatment on wavelength variables, iii) subsequently at the third step you calculated (or not) the derivative , iv) at the fourth step orthogonal signal correction was done (or not), and finally v) you performed (or not) mean-centering. For clarity I would change the sentence at L231-232, as: ‘Several combinations of methods to pre-process the NIR spectra were tested in this study. Overall, the process consisted in six consecutive steps, namely …’
• L232: ‘(2) a baseline correction …. (No process)’: usually we refer to these as mathematical treatments. You can use this term for clarity
• L281: already stated at L223-225
• L310: are you sure that A, B, and C values (Table 1) are statistically similar? Test it
• Table 2: not very clear. See comment for L226-267. If I understand well, basically in this table you report the best combination of steps observed for each trait within each device.
• From L406 onwards: Table ‘1’ should be Table ‘3’, Table ‘2’ should be Table ‘4’, and Table ‘3’ should be Table ‘5’
• L457-459, L465-466 and L486-487: I would not consider ‘good’ the RMSE of lactose just because under the threshold value proposed by ICAR for laboratory standard (0.1) and at-line (0.20) or in-line (0.25) analysers  (https://www.icar.org/Guidelines/13-On-line-Milk-Analysis.pdf). ICAR consider the same value for fat, protein, and lactose…Considering that this trait has a low phenotypic CV in cow milk: do you think this value (0.10) is too much tolerant? Maybe a more restrictive value would be more appropriate?
• L486-490: yes, but still the best coeff. of determination in external validation observed for lactose was not above 0.9, being just 0.668 (NIRONE 2.5 T). I think you need to discuss a bit more the coeff. of determination in external validation obtained for lactose (Table 4, L462). Both 0.19 (TEC5) and 0.198 (NIRONE 1.4 T) impressed me quite a lot.

Author Response

We thank the reviewer for thorough analysis and excellent suggestions. We believe that the suggested changes have improved our paper. We have indicated the reviewer comment with POINT and modifications and our responses with RESPONSE text.

POINT 1: Having NIR-predicted milk composition traits on a daily basis would be useful compared to data available on monthly basis in herds under the official milk recording/DHI. You should highlight this aspect in the manuscript; potentially, having milk composition data twice a day will be of great interest in the era of PLF. With adoption of specific algorithms, one can successfully and fast detect variation in some milk traits in real time and potentially identify cows with a specific issue to be monitored. Recently published papers studied ‘patterns’ of certain milk components and propose these as indicators of cow health and resilience. Few examples are: 10.3168/jds.2019-17290, 10.3168/jds.2017-13270, 10.3168/jds.2019-17826, etc.

RESPONSE: We thank the reviewer for this suggestion. We emphasized this important aspect in the introduction at lines 59-74 “On-farm PLF tools for milk analysis could provide information on a daily basis and would be a great addition to the official milk recordings. The comparison of the data recorded on-site and analysed in the laboratory could offer new insights into the periodic changes in cow physiology and wellbeing. The standardized milk testing by accredited central laboratories sets a considerable delay between the moments of taking milk samples and receiving the results. Moreover, as the frequency of sampling is relatively low, there can be changes in cow health reflected by the milk composition that currently go unnoticed. Daily milk recordings have been shown to provide data for continuous health analysis, while the analysis of individual lactations can indicate the cow’s resilience and show how cows differ in their ability to cope with environmental disturbances such as pathogens, heat waves, and changes in feed composition and feed quantity. The automatically collected daily data amounts to such levels that it is feasible to use big data analytics to determine indicators to analyse cow behavior and health [6-9]. Nevertheless, although several on-farm spectrometers for measuring the milk composition are commercially available, they are not widely used and their accuracy is often insufficient for monitoring health and welfare and optimizing feed.”

In addition, we included new citations to relevant research:

[6] https://doi.org/10.3168/jds.2017-13270; [7] https://doi.org/10.3168/jds.2019-17290; [8] https://doi.org/10.3389/fgene.2018.00692; [9] https://doi.org/10.3168/jds.2019-17826.

POINT 2: The major problem I find in this paper is that you used repeated obs. per cow (≥2 milk samples), no matter either in the validation or in the calibration set. Stated at L216-218. How can you account for this data structure, i.e. for cow-intrinsic factors, i.e. within-animal variability?

RESPONSE: The repeated observations per cow were of different moments in time (morning and evening of 3 consecutive days) and were thus variable in milk composition. The within-animal variability is not specifically addressed in this study as this was not the scope. The aim was to build a model that is robust and that would work for any (new) cow and any (new) sample. As some milk properties (high fat content, specific fatty acids and amino acids and particle size distribution) can be unique for some cows, we don’t want to train the model to learn these cow-specific features as the relation might be totally different for other (new) cows. To avoid training these cow-specific features, we always put all observations of the same cows in either the calibration or the validation set. Moreover, also in cross-validation, all observations of a cow belonged into either the training (+/- 90%) or the test set (+/- 10%). Accordingly, the trained models do not give favorable results for the test or validation set if cow-specific features are modeled. This was better explained in the text in lines 226 – 235 and lines 256 – 258.  Additionally, we added a reference to a paper clearly illustrating the problem of modeling unwanted sample features and how to avoid this: Kemps B, Saeys W, Mertens K, Darius P, De Baerdemaeker J, De Ketelaere B. The importance of choosing the right validation strategy in inverse modelling. J Near Infrared Spectrosc 2010;18:231–7. https://doi.org/10.1255/jnirs.882.

POINT 3: A curiosity: why not evaluating predictive ability on bulk milk in this paper?

RESPONSE: The variation of the bulk milk composition is very small. Accordingly, evaluation of the sensor and the developed models would not provide clear insights on the real determination coefficient, slope and bias of the prediction model. We agree with the reviewer that this would be a very illustrative experiment. However, as the setup is disassembled it would not be possible anymore to do this. The validation performed in this study is much more challenging than a validation on bulk milk would be. Accordingly, we are confident that an evaluation on bulk milk would not provide any new insights on the performance of the measurement setup.

POINT 4: L107-108: this sounds like an ‘aim’, remove from here

RESPONSE: We removed this first sentence.

POINT 5: L224: 15 combinations (5 spectrometers x 3 components)

RESPONSE: We replaced this.

POINT 6: L226-267: not very clear in the current form, a flow chart would be useful to readers. I understand that you i) firstly transformed (or not) the wavelength variables with log, ii) then at the second step you basically performed (or not) the mathematical treatment on wavelength variables, iii) subsequently at the third step you calculated (or not) the derivative , iv) at the fourth step orthogonal signal correction was done (or not), and finally v) you performed (or not) mean-centering. For clarity I would change the sentence at L231-232, as: ‘Several combinations of methods to pre-process the NIR spectra were tested in this study. Overall, the process consisted in six consecutive steps, namely …’

RESPONSE: The reviewer is correct. We changed this sentence according to his suggestion. We believe that an additional flow chart would distract the reader from the actual scope of this study.

POINT 7: L232: ‘(2) a baseline correction …. (No process)’: usually we refer to these as mathematical treatments. You can use this term for clarity

RESPONSE: We used the suggested term instead.

POINT 8: L281: already stated at L223-225

RESPONSE: We removed this sentence.

POINT 9: L310: are you sure that A, B, and C values (Table 1) are statistically similar? Test it

RESPONSE: The duplex technique is widely used for splitting a data set into 2 sets with similar statistical properties. There is no need for the resulting datasets to have statistical properties that are not significantly different as this depends a lot on the samples in the original dataset. This extensive explanation would go beyond the scope of this manuscript. We adapted the text to make clear that the statistical properties of the different datasets are ”in the same range” instead of ”being similar”.

POINT 10: Table 2: not very clear. See comment for L226-267. If I understand well, basically in this table you report the best combination of steps observed for each trait within each device.

RESPONSE: We implemented the suggestions related to the description of the mathematical treatments according to your earlier suggestions. Indeed, Table 2 presents the best combination of steps observed for each trait within each device. We clarified this in the table caption.

POINT 11: From L406 onwards: Table ‘1’ should be Table ‘3’, Table ‘2’ should be Table ‘4’and Table ‘3’ should be Table ‘5’.

RESPONSE: We checked the references to the tables and corrected them where needed. Additionally, we added a reference to Table 4 in line 470 for clarity.

POINT 12: L457-459, L465-466 and L486-487: I would not consider ‘good’ the RMSE of lactose just because under the threshold value proposed by ICAR for laboratory standard (0.1) and at-line (0.20) or in-line (0.25) analysers  (https://www.icar.org/Guidelines/13-On-line-Milk-Analysis.pdf). ICAR consider the same value for fat, protein, and lactose…Considering that this trait has a low phenotypic CV in cow milk: do you think this value (0.10) is too much tolerant? Maybe a more restrictive value would be more appropriate?

RESPONSE: We agree with the reviewer and we changed the text to indicate that the RMSE values meet the ICAR requirements for at-line standards without indicating that these are ”good”. Personally, we agree that the ICAR threshold values should be more restrictive for lactose for them to be of any use. Although we indicated this issue in the text, it is not the scope of this manuscript to argue the ICAR standards.

POINT 13: L486-490: yes, but still the best coeff. of determination in external validation observed for lactose was not above 0.9, being just 0.668 (NIRONE 2.5 T). I think you need to discuss a bit more the coeff. of determination in external validation obtained for lactose (Table 4, L462). Both 0.19 (TEC5) and 0.198 (NIRONE 1.4 T) impressed me quite a lot.

RESPONSE: The determination coefficients for lactose predictions were discussed in lines 469 – 475. We added an extra line to address the poor determination coefficient of TEC5 and NIRONE 1.4 T.

Reviewer 2 Report

The evaluated manuscript titled “Evaluation of MEMS NIR spectrometers for on-farm analysis of raw milk composition” is quite interesting. It concerns the performance of Micro-Electro-Mechanical-System (MEMS) based near-infrared (NIR) spectrometers as on-farm milk analysers. Generally, the Authors analysed too few samples in this complex experimental setup.

Detailed comments:

Line 37: “sufficient accuracy” should be more precisely explained.

Lines 79-80: No seamless merging of paragraphs.

Lines 203-204: “In total, 304 different raw milk samples were analysed from 71 individual cows.” Versus Lines 294-295 “During the three-day measurement campaign at Maaninka Research farm, a total of 299 milk samples from 71 cows were successfully recorded”. The difference should be explained.

Lines 292-294: The aim should be placed at the end of the Introduction.

Moreover, the Authors did not mention the cost of the determination.

Style and language should definitely be improved.

Generally, the manuscript requires repeat testing.

Author Response

We thank the reviewer for thorough analysis and excellent suggestions. We believe that the suggested changes have improved our paper. We have indicated the reviewer comments with POINT and modifications and our responses with RESPONSE text.

POINT 1: Line 37: “sufficient accuracy” should be more precisely explained.

RESPONSE: We have now explained this more precisely at line 37-38, following your suggestion.

POINT 2: Lines 79-80: No seamless merging of paragraphs.

RESPONSE: We have now modified this at line 87-88.

POINT 3: Lines 203-204: “In total, 304 different raw milk samples were analysed from 71 individual cows.” Versus Lines 294-295 “During the three-day measurement campaign at Maaninka Research farm, a total of 299 milk samples from 71 cows were successfully recorded”. The difference should be explained.

RESPONSE: We thank the reviewer for indicating this ambiguity. We changed this to: “In total, 304 different raw milk samples were collected from 71 individual cows for the measurements. A total of 299 samples were successfully measured without prototype malfunction and had received reference data from the Valio laboratory.” Corrected to text at line 211-213.

POINT 4: Lines 292-294: The aim should be placed at the end of the Introduction.

RESPONSE: We modified this as suggested.

POINT 5: Moreover, the Authors did not mention the cost of the determination.

RESPONSE: We have now added a reference to the price of the spectrometers in line 136. All other components are either customized or standard laboratory supplies, accordingly reporting those prices would be irrelevant.

POINT 6: Style and language should definitely be improved.

RESPONSE: Following your suggestion, we have thoroughly checked the style and spelling of the manuscript and corrected it accordingly.

POINT 7: Generally, the manuscript requires repeat testing:

RESPONSE: The study at the Maaninka Research farm lasted for 3 days to enable the collection of a sufficient amount of samples for statistical performance evaluation (299 successful measurements of different samples during 6 milking sessions). Comparable sample sizes were used in similar studies using benchtop or miniature spectrometer devices for raw milk analysis [references for similar studies in paper no 32-36]. The aim was to evaluate the possible performance level of these spectrometers using a prototype device demanding manual sample processing steps. The aim was not to obtain the overall performance at farm-level as this would require an automated device connected for example to a milking robot. The cows that were included in the independent validation set were not involved in the model building. Accordingly, we feel confident that our results show the level of performance that can potentially be achieved with an on-farm device if it has a suitable design for milk processing in an automated manner. The repeated testing of the sensor over a longer period at different farms will be the subject of a follow-up study.

 

Reviewer 3 Report

The authors compare various spectrometers for on-farm use for easily determining fat, protein, and lactose in milk. These devices would be an advancement over sending out samples to certified labs every few weeks, and therefore a paper evaluating them is important.

Minor comments:

Line 30: Spell out International Committee for Animal Reporting

Line 82: integrated circuit does not have to be abbreviated since it is not mentioned again

 

Author Response

We thank the reviewer for thorough analysis and supporting statement. We believe that the suggested changes have improved our paper. We have indicated the modifications and our responses RESPONSE text.

The authors compare various spectrometers for on-farm use for easily determining fat, protein, and lactose in milk. These devices would be an advancement over sending out samples to certified labs every few weeks, and therefore a paper evaluating them is important.

Minor comments:

Line 30: Spell out International Committee for Animal Recording

RESPONSE: We implemented this.

Line 82: integrated circuit does not have to be abbreviated since it is not mentioned again

RESPONSE: Thank you for this comment, we changed the manuscript accordingly.