The Outperformance Probability of Mutual Funds
Round 1
Reviewer 1 Report
This is an interesting paper that could be potentially publishable subject to some revisions that are discussed in more detail below.
Research design/Methods: I believe that the authors should discuss further on the choice of index ETFs to compare the selected mutual funds providing citations from the literature.
Implications: Discuss on both methodological and practical implications of the study.
Generalization of the results: Discuss on generalization of the results of the study.
Limitations: Discuss on the limitations of the study, if there are any
Author Response
Please see the attached letter of response.
Author Response File: 
Author Response.pdf
Reviewer 2 Report
I liked the paper. However, although I think it is quite significant I would like to request from the authors to relate results to the standard CAPM, in terms of Jensen's alpha.
Author Response
Please see the attached letter of response.
Author Response File: Author Response.pdf
Reviewer 3 Report
This paper, in my view, requires a minor revision:
Authors should more clearly emphasize the contribution of this work in relation to the existing
solutions (methods: perfomance measures) in the literature. Your literature review must be revised and updated, also with a view to demonstrating what's new in your own work.
Author Response
Please see the attached letter of response.
Author Response File: Author Response.pdf
Reviewer 4 Report
The paper develops an outperformance probability measure as a new performance measure for the mutual funds industry. As a general remark, the topic is quite interesting and certainly have promising implications for both advances in theory and practitioners. Nevertheless, it suffers of a couple of weaknesses that need to be adressed in order for the paper being suitable for publication.
First of all the motivations of the study are quite weak and, overall, they are not convincing. Seemingly the authors report the simplicity of their OP measure compared to typical "return-to-risk performance measures" and its alleged practicity (need of one measure only instead of comparing two different opportunities) as the main motivation for the study. Nevertheless, a more robust motivation requires making reference to efficiency considerations, i.e. the merits of an OP measure compared to typical measures in finding out the funds that outperform the benchmark (and, in fact, the empirical investigation goes in that direction). In doing that, authors should refer their study to the established theory of performance measure overcoming another limitation of the work which is related to the lack of a robust theoretical undeprinning-that would provide a more robust scientific soundness to the work which otherwise is just a matter of modeling. Finally, authors should provide better clarification to the intended contribution of the paper. Is their aim seeking advance in return-to-risk performance measures under a theoretical perspective? In that case authors should discuss theoretical implications of theri work. Otherwise, do authors aim at defining a more efficient performance measurement framework for the asset management industry? In that case, I suggest providing a thorough discussion on how the paper might help bridging theory and practice.
Author Response
Please see the attached letter of response.
Author Response File: Author Response.pdf
Round 2
Reviewer 2 Report
1) Define all symbols in Table 1 right before its presentation as well as part of table descriptions.
2) Table 3 – you definitely need standard errors.
3) Table 4 : All ICV’s are insignificant, right? So I don’t see why “Hence, using the ICV instead of comparing two Sharpe ratios with one another in order to test whether a strategy outperforms its benchmark leads to more significant results.”
4) From the remaining tables only LIBOR and Cash works but not SPY, yet most managers focus on SPY.
Author Response
Dear referee,
Hereby, we provide our responses to your second valuable report:
1) We added all symbols to the caption of Table 1. The placement of tables is very much driven by LaTeX. However, we try now to force LaTeX to put the tables right after its description in the text.
2) All standard errors equal, approximately, 1/\sqrt{m} (see p. 14). (In our case, the approximation is really good: The exact numbers differ only in the fourth decimal from the approximated results.) Hence, it makes no sense to write them down under each number. We clarify this point now in the caption of Table 3, where you can find also the standard error (= 0.2502).
3) You are right. Our statement is somewhat misleading. We speak about economic but not statistical significance. Thus, to clarify this point, we now write: "Hence, using the ICV instead of comparing two Sharpe ratios with one another in order to test whether a strategy outperforms its benchmark leads to results that are more significant in an economic sense. To be more precise, in the majority of cases, the estimated ICVs are positive and their p-values are low enough in order to conclude that most fund managers are able to beat their benchmarks. However, the given results are still insignificant in a statistical sense."
4) We do not feel that only LIBOR and Cash works. In fact, the tables reveal that almost all (estimated) ICVs are positive. This holds true also for SPY (except for PINDX, which is always the worst performer). See, e.g., Table 5 to Table 8, in which the OPs with respect to the SPY are almost always greater than 50%. Thus, at least from an ex-post point of view, almost all mutual funds have outperformed the S&P 500. We emphasize in our work that it is very hard to obtain statistically significant results, i.e., power is a serious problem, in the context of performance measurement. However, it is clear that a null hypothesis that cannot be rejected cannot be considered true, since hypothesis tests are asymmetric by their very nature.
Sincerely,
Gabriel Frahm
Round 3
Reviewer 2 Report
The revised version is much better. Congratulations!
