Mechanisms Underlying Directional Motion Processing and Form-Motion Integration Assessed with Visual Perceptual Learning

Round 1

Reviewer 1 Report

In this paper, the authors try to prove whether dynamic Glass patterns and random dot kinematograms present the same processing mechanisms depending on the activity of the human motion complex. The study is interesting but It presents some faults in the structure of the manuscript and the data analysis.

 

Major concerns:

• The organization of the manuscript is not traditional for academic articles. There is no statistical methods section explaining the techniques to be used and how and in what context they are to be used. The authors describe the techniques together with the results, which in my opinion makes the reading of the results more complicated. In addition, the statistical techniques used are complex and need more explanation of their use as well as citations to be consulted.
• I have several doubts regarding the statistical methods used. The authors use the gamma function as the link function because of the outliers and skewness of the data. They cite the article by Zuur et al (2010)l. on how to avoid certain common statistical problems where Zuur et al. indeed recommend using the gamma function in the case of outliers. Looking at Figure 2 I think the biggest problem is the skewness of the data rather than the outliers. This asymmetry is especially important in the post-test groups. I would like the authors to explain why they have used the link Gamma function and not other solutions such as the log-Normal distribution or inverse gaussian. It would be interesting to know the advantages of this solution over others for this kind of data. Another question that would be interesting to explain is the meaning of the beta estimators in this model. When the link function is unity and the independent variables are qualitative, the beta coefficient shows the difference between the means of the two categories. If the link function is binomial, the value e raised to the beta coefficient is the odds ratio. Therefore, It would be interesting if the beta coefficients or some transformation of beta coefficients have an experimental interpretation in this study. The differences between the experimental conditions are based on p-values but it would be useful to know how large the differences are to better interpret the results. Remember that p-values depend on sample sizes and so “statistical significance does not mean practical significance”, or “non-statistical significance implies non-practical”. It is desirable to support statistical significance with an effect size. Moreover, when the author interprets the differences between conditions they use the means which is incongruent because in markedly skewed distributions with outliers the mean is not the best measure to represent the observations. Recall that for the reason of skewness and outliers the authors have used the Gamma function.
• Regarding the mixed model, the authors have selected from the 14 possible models based on whether the coefficients are random or not. They have used three measures to compare the fitting of models and have chosen the best one. It would be interesting to have a discussion between this best model based on the data and whether other model presents better experimental meaning. Again the results are based on p-values without knowing if the difference in fit between models is large or small. Finally, the selected model has two random coefficients (time and stimulus). It would be interesting to show the variances component for these coefficients of these factors in order to know the magnitude of the differences in the coefficients for the different individuals.

 

Minor Concerns:

• It would be interesting if the authors could explain the method used for the randomization of participants in the study.
• The fact that two of the participants were authors would not bias their results due to a greater knowledge of the techniques.
• In my opinion, the explanation of the box-plot construction in the figure caption is unnecessary because it is well known. It is sufficient to indicate the software with which they have been made.

Author Response

Reviewer #1

In this paper, the authors try to prove whether dynamic Glass patterns and random dot kinematograms present the same processing mechanisms depending on the activity of the human motion complex. The study is interesting but it presents some faults in the structure of the manuscript and the data analysis.

R: We thank Reviewer 1 for the comments and suggestions that helped us to improve our manuscript. We have dealt with all the points raised by the Reviewer. We hope that our manuscript is now of enough quality to be published in Vision. To help pinpoint the changes, we highlighted in yellow the alterations made to the manuscript in response to Reviewer #1’s suggestions.

 

1. The organization of the manuscript is not traditional for academic articles. There is no statistical methods section explaining the techniques to be used and how and in what context they are to be used. The authors describe the techniques together with the results, which in my opinion makes the reading of the results more complicated. In addition, the statistical techniques used are complex and need more explanation of their use as well as citations to be consulted.

R: Thank you for this suggestion we have now reported the statistics used in a separate paragraph (“Data analysis”) before the Result section. Please, see section 2.4.4. of the manuscript (page 5).

 

2. I have several doubts regarding the statistical methods used. The authors use the gamma function as the link function because of the outliers and skewness of the data. They cite the article by Zuur et al. (2010) on how to avoid certain common statistical problems where Zuur et al. indeed recommend using the gamma function in the case of outliers. Looking at Figure 2 I think the biggest problem is the skewness of the data rather than the outliers. This asymmetry is especially important in the post-test groups. I would like the authors to explain why they have used the link Gamma function and not other solutions such as the log-Normal distribution or inverse gaussian. It would be interesting to know the advantages of this solution over others for this kind of data.

R: Thanks for this comment. The Reviewer is correct. The main problems were both the high positive skewness and the outliers. The Gamma distribution was selected after having fitted a Normal, Log-Normal, Gamma and inverse Gaussian distribution to the data. This was done using the R function ‘fitdist’ which reports empirical and theoretical densities, Q-Q plot, P-P plot, and empirical and theoretical CDFs. Besides, we also plotted the percentage discrimination thresholds as a function of the quantiles modeled by the mentioned distributions, and we noticed that almost all the data points fell in the Gamma quantiles. We did not notice a substantial difference between Gamma and inverse Gaussian distribution, so we decided to use the Gamma distribution. Additionally, it should be noted that inter-group variability (skewness of the data) is common in perceptual learning experiments. We discussed this point in the discussion section. Please, see page 14.

 

3. Another question that would be interesting to explain is the meaning of the beta estimators in this model. When the link function is unity and the independent variables are qualitative, the beta coefficient shows the difference between the means of the two categories. If the link function is binomial, the value e raised to the beta coefficient is the odds ratio. Therefore, it would be interesting if the beta coefficients or some transformation of beta coefficients have an experimental interpretation in this study.

R: Thanks for this comment. In general, we prefer to use an identity link function as back transforming the data after the analysis and interpreting main effects and interactions could be misleading. In fact, back transformation can be unreliable because statistically significant differences on the transformed dependent variable are uninformative as to whether significant differences exist on the original untransformed DV metric (Berry et al., 2010). As outlined by Lo & Andrews (2015), applying a non- linear (e.g., log or inverse) transformation to the DV not only normalizes the residuals, but also distorts the ratio scale properties of the measured variable (Stevens, 1946). Therefore, our goal was to statistically assess the DV in its original metric but also meet the mathematical constraints imposed by data distribution. This solution is handled by GLMMs (Lo & Andrews, 2015). In the Data Analysis section (see page 5) we have better motivated the choice of the Gamma distribution and the identitylink function.

Concerning the interpretation of beta parameters, in our study it is not straightforward as in other contexts. For example, as the Reviewer pointed out, for a count outcome, a log link function, and a Poisson distribution (for positive integers) are used. In this case, a one unit increase in a specific independent variable is associated with a certain decrease or increase in the expected log odds of the DV. In our case a similar reasoning is not straightforward. For instance, the estimated parameter Time (post-test) is -10.697. We could possibly interpret this as a decrease of roughly 11 dots (or dipoles) from the pre-test to the post-test, though across stimuli and learning groups. A similar reasoning could be applied to the stimulus parameter, but the interpretation of the other parameters is less clear and, in our opinion, not informative for the goal of the study. Therefore, we preferred to base our interpretation on the significant fixed effects.

 References:

-Berry, W. D., DeMeritt, J. H. R., & Esarey, J. (2010). Testing for Interaction in Binary Logit and Probit Models: Is a Product Term Essential? American Journal of Political Science, 54(1), 248–266.

-Lo, S., & Andrews, S. (2015). To transform or not to transform: using generalized linear mixed models to analyse reaction time data. Frontiers in psychology, 6, 1171.

doi.org/10.3389/fpsyg.2015.01171

-Stevens, S.S. (1946). On the theory of scales of measurement. Science 103,677–680. doi:10.1126/science.103.2684.677

 

4. The differences between the experimental conditions are based on p-values but it would be useful to know how large the differences are to better interpret the results. Remember that p-values depend on sample sizes and so “statistical significance does not mean practical significance”, or “non-statistical significance implies non-practical”. It is desirable to support statistical significance with an effect size.

R: Thank you for your suggestion. P-values are affected by sample size, so the larger the sample, the more likely we are to have smaller p-values. As the Reviewer noted, this could lead to the possibility of a “non-practical significance”. Measuring effect sizes to account for the sort of link between two experimental variables or conditions is a suggested method, but it is not appropriate in our case. In GLMM, because of the way variance is handled, there is still no consensus on how to determine standard effect sizes (Rights & Sterba, 2019). The major difficulty is that values must be standardized by some form of variability estimate, such as the standard deviation. In the case of GLMM, there are various estimations to examine, and it is unclear which ones to choose.

 Reference:

-Rights, J. D., & Sterba, S. K. (2019). Quantifying explained variance in multilevel models: An integrative framework for defining R-squared measures. Psychological Methods.

 

5. Moreover, when the author interprets the differences between conditions, they use the means which is incongruent because in markedly skewed distributions with outliers the mean is not the best measure to represent the observations. Recall that for the reason of skewness and outliers the authors have used the Gamma function.

R: Thank you for this additional comment on data distribution. It is correct that the means are not the most representative values of the results in the conditions used. However, as far as we know, there are no better options. Perhaps the medians, or the model’s predicted values could be used, but in this case, it would be difficult for the readers to compare the results of our study to those of other studies and perhaps - being in fact rare their use in psychological science - could be a source of confusion and misunderstanding.

 

6. Regarding the mixed model, the authors have selected from the 14 possible models based on whether the coefficients are random or not. They have used three measures to compare the fitting of models and have chosen the best one. It would be interesting to have a discussion between this best model based on the data and whether other model presents better experimental meaning.

R: Thanks for this comment. In general, we agree that data-driven model selection is not always the best choice (and indeed, in some cases, data-driven analysis is prone to artifacts, especially when the data is not free of noise and there is little difference between models). However, in our case, there were no precise theoretical reasons - or related to the experimental design - that could lead us to choose a specific model, thus we used a simple exploratory approach.

 

7. Again, the results are based on p-values without knowing if the difference in fit between models is large or small. Finally, the selected model has two random coefficients (time and stimulus). It would be interesting to show the variances component for these coefficients of these factors in order to know the magnitude of the differences in the coefficients for the different individuals.

R: Thanks for this suggestion. We have now reported the variance of the random coefficients and the residual variance. Please, see Table 2.

 

Minor Concerns:

8. It would be interesting if the authors could explain the method used for the randomization of participants in the study.

R: We have specified what we intend for participants randomization at page 3.

9. The fact that two of the participants were authors would not bias their results due to a greater knowledge of the techniques.

R: Because the staircase tracked the 79% correct discrimination on every block (pre- and post-tests and training), the fact that two of the authors participated in the study should not bias the results. Besides, one of the authors was in the GP training group, and the other in the RDK training group.

 

10. In my opinion, the explanation of the box-plot construction in the figure caption is unnecessary because it is well known. It is sufficient to indicate the software with which they have been made.

R: Thanks for this suggestion. We have now shortened the caption of Figure 2 and reported the software (R, v.4.1.3). Please, see caption of Figure 2.

Author Response File: Author Response.pdf

Reviewer 2 Report

The paper is well written in general. Only some minor points. On Page 4, it's better to put the caption and the corresponding figure in the same page. In their proposed experiments, there are only 31 participants, which might seem a little less. Finally, the authors only show some statistical analysis results. It might be interesting if they can provide a few GPs and RDKs for some typical groups.

Author Response

Reviewer #2

The paper is well written in general. Only some minor points.

R: We thank Reviewer 2 for the comments and suggestions that helped us to improve our manuscript. We have dealt with all the points raised by the Reviewer.

 

On Page 4, it's better to put the caption and the corresponding figure in the same page.

R: We have now aligned the figure with its description on the same page. However, these features depend on the final format of the manuscript.

 

In their proposed experiments, there are only 31 participants, which might seem a little less.

R: Psychophysical studies and studies on visual perceptual learning usually use small sample sizes. Based on the previous literature we can state that a sample size of 31 participants is not a small group but enough representative.

References:

-Camilleri, R., Pavan, A., & Campana, G. (2016). The application of online transcranial random noise stimulation and perceptual learning in the improvement of visual functions in mild myopia. Neuropsychologia, 89, 225–231. https://doi.org/10.1016/j.neuropsychologia.2016.06.024

-Jeter, P. E., Dosher, B. A., Petrov, A., & Lu, Z. L. (2009). Task precision at transfer determines specificity of perceptual learning. Journal of vision, 9(3), 1–13. https://doi.org/10.1167/9.3.1

-Maniglia, M., Pavan, A., Cuturi, L. F., Campana, G., Sato, G., & Casco, C. (2011). Reducing crowding by weakening inhibitory lateral interactions in the periphery with perceptual learning. PloS One, 6(10), e25568. https://doi.org/10.1371/journal.pone.0025568

-McGovern, D. P., Webb, B. S., & Peirce, J. W. (2012). Transfer of perceptual learning between different visual tasks. Journal of vision, 12(11), 4. https://doi.org/10.1167/12.11.4

-Xiao, L. Q., Zhang, J. Y., Wang, R., Klein, S. A., Levi, D. M., & Yu, C. (2008). Complete transfer of perceptual learning across retinal locations enabled by double training. Current Biology, 18(24), 1922–1926. https://doi.org/10.1016/j.cub.2008.10.030

 

Finally, the authors only show some statistical analysis results. It might be interesting if they can provide a few GPs and RDKs for some typical groups.

R: Thank you for your comment. We have now updated the results section. Please see pages 5-12. The paragraphs are highlighted in yellow in response to Reviewer #1’s comments.

 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

The authors have included changes that have improved the manuscript. The comments made have been correctly discussed although I do not agree with some of the statements made. Despite this I think the study presents enough quality to be published in Vision