Two Designs of Automatic Embedded System Energy Consumption Measuring Platforms Using GPIO

Round 1

Reviewer 1 Report

There are no comments

Author Response

Dear reviewer:

Thank you for your valuable comments. We have revised the manuscript (Manuscript ID: applsci-826693) and sent the revised manuscript for your consideration to be published in Applied Sciences. Based on the comments from you, we have made changes of the manuscript, which are detailed below.

 

1. English language and style have been edited;
2. Abstract and section 1 Introduction have been edited, more detailed summary of our work is added;
3. Section 2 and Section 3 are reworked, and more details about our methodology are introduced;
4. In section 5, a new part of test result is added to verify our solution proposed in section 3.1.

Author Response File: Author Response.docx

Reviewer 2 Report

1. I believe that the title should be changed as it does not put forward the work done in this article.
2. The abstract should be reworked to indicate the novelty and the contribution of this work.
3. Introduction:
   1. "Deploying these devices is costly on both money and time, not only the deployment work can be difficult, users also expect the devices can last long and require little maintenance."
      
      This sentence does not make sense as it is right now. Do you mean "users expect the device to last longer"?
      
      
   2. I think the introduction should be reworked entirely:
      1. The writing should be edited as it is barely comprehensible as it is now.
      2. The authors do not state - even briefly - how the existing methods are not adequate.
      3. The authors do not indicate the problematic (and thus the exact relevance) they are trying to solve, and they do not state how this work would do so.
      4. There is no summary of the work in the introduction (This should always be the last paragraph).
   3. Related Works and Motivation
      1. It should be "'motivations"
      2. When citing the authors' name, the citation number should come directly after the name in Mesa-Martinez F J. Also there is a group of authors so, there should be et al. added after the first author.
      3. "Nevertheless, it is inaccuracy,"
         You mean it is inaccurate.
         This paper cannot be published without proper English editing. I cannot continue to highlight language issues further.
      4. There should be no subsection if the authors are citing one to two works per subsection.
      5. The literature overview is, for the lack of any better word, poor. The field of power measuring, profiling, and monitoring in embedded and mobile systems is more than rich and abundant with studies. I do want to discourage the writer but they have not explored the background works, nor the alternative software methods. they should cite at least - as a start - these works, and look in them for other relevant work :
         • (Review) Oussama Djedidi, Mohand A. Djeziri, Power profiling and monitoring in embedded systems: A comparative study and a novel methodology based on NARX neural networks, Journal of Systems Architecture,
           https://doi.org/10.1016/j.sysarc.2020.101805.
         • (Model) Oussama Djedidi, Mohand A. Djeziri, Incremental Modelling and Monitoring of Embedded CPU-GPU Chips, Processes
         • (Model) M.J. Walker, S. Diestelhorst, A. Hansson, A.K. Das, S. Yang, B.M. Al-Hashimi, G.V. Merrett, Accurate and stable run-Time power modeling for mobile and  embedded CPUs, IEEE Trans. Comput. Aided Des.  Integr. Circuits Syst. 36 (1) (2017) 106–119, doi:10.1109/TCAD.2016.2562920
         • (Review) M.A. Hoque, M. Siekkinen, K.N. Khan, Y. Xiao, S. Tarkoma, Modeling, profiling, and debugging the energy consumption of mobile devices, ACM Comput. Surv. 48 (3) (2015) 1–40, doi:10.1145/2840723.
         • (Review) R.W. Ahmad, A. Gani, S.H.A. Hamid, F. Xia, M. Shiraz, A review on mobile application energy profiling: taxonomy, state-of-the-art, and open research issues, J. Network Comput. Appl. 58 (2015) 42–59, doi:10.1016/j.jnca.2015.09.002.
      6. When proper citations are made. The authors should explain how their work differs from software-based works and how it improves upon it.
      7. The Voltage curve is one of the tools used to measure/estimate power consumption but not the only one. Why rely on this one? How does the battery age affect the curve and precision of the measurements?
      8. What the author state in section 2.2.2 is neither clear nor convincing. It is more of a technicality rather than a research issue.
   4. Our Idea and Design
      1. Title of this section should be changed
      2. There are multiple ways to measure and estimate power consumption, as there are also multiple reasons to do so. What is the purpose of your power measurement? Online monitoring? Code optimization? Power management?
      3. I do not understand the purpose of this section. It more technical rather than a research or a scientific contribution
   5. Implementation 
      1. The case study devices should be in a section of their own, and the authors should explain their choice
      2. Figures 1 and 2 have text overflowing and should be redone.
      3. Figure 4 should be made into two figures for clarity. The same for figure 5.
      4. In the results section, no comparisons are made to currently available methods, neither software profilers nor hardware ones.
      5. There is no significant difference between the two proposed measurement schemes. Could the authors explain why there are two?

Author Response

Dear reviewer:

Thank you for your valuable comments. We have revised the manuscript (Manuscript ID: applsci-826693) and sent the revised manuscript for your consideration to be published in Applied Sciences. Based on the comments from you, we have made changes of the manuscript, which are detailed below.

1. I believe that the title should be changed as it does not put forward the work done in this article.
   Response: title now changed to “Two Designs of Automatic Embedded System Energy Consumption Measuring Platform Using GPIO”.

2. The abstract should be reworked to indicate the novelty and the contribution of this work.
   Response: abstract is partially reworked, and more about our work is added.

3. Introduction:

1. "Deploying these devices is costly on both money and time, not only the deployment work can be difficult, users also expect the devices can last long and require little maintenance."
   This sentence does not make sense as it is right now. Do you mean "users expect the device to last longer"?
   Response: it has been rewritten.
   
   
2. I think the introduction should be reworked entirely:
1. The writing should be edited as it is barely comprehensible as it is now.
   Response: English language and style has been edited.
2. The authors do not state - even briefly - how the existing methods are not adequate.
   Response: ‘time’ should be considered is pointed out now, and details are discussed in 2.1 and 2.2.
3. The authors do not indicate the problematic (and thus the exact relevance) they are trying to solve, and they do not state how this work would do so.
   Response: a brief introduction of how our work designs work is added.
4. There is no summary of the work in the introduction (This should always be the last paragraph).
   Response: summary of our work is placed in last paragraph now (line 36-39).
3. Related Works and Motivation
   • (Review) Oussama Djedidi, Mohand A. Djeziri, Power profiling and monitoring in embedded systems: A comparative study and a novel methodology based on NARX neural networks, Journal of Systems Architecture,
     https://doi.org/10.1016/j.sysarc.2020.101805.
   • (Model) Oussama Djedidi, Mohand A. Djeziri, Incremental Modelling and Monitoring of Embedded CPU-GPU Chips, Processes
   • (Model) M.J. Walker, S. Diestelhorst, A. Hansson, A.K. Das, S. Yang, B.M. Al-Hashimi, G.V. Merrett, Accurate and stable run-Time power modeling for mobile and  embedded CPUs, IEEE Trans. Comput. Aided Des.  Integr. Circuits Syst. 36 (1) (2017) 106–119, doi:10.1109/TCAD.2016.2562920
   • (Review) M.A. Hoque, M. Siekkinen, K.N. Khan, Y. Xiao, S. Tarkoma, Modeling, profiling, and debugging the energy consumption of mobile devices, ACM Comput. Surv. 48 (3) (2015) 1–40, doi:10.1145/2840723.
   • (Review) R.W. Ahmad, A. Gani, S.H.A. Hamid, F. Xia, M. Shiraz, A review on mobile application energy profiling: taxonomy, state-of-the-art, and open research issues, J. Network Comput. Appl. 58 (2015) 42–59, doi:10.1016/j.jnca.2015.09.002.
     Response: section2.1 is reworked and more details about latest measuring and modeling methods are talked as well as their pros and cons.
1. It should be "'motivations"
   Response: fixed.
2. When citing the authors' name, the citation number should come directly after the name in Mesa-Martinez F J. Also there is a group of authors so, there should be et al. added after the first author.
   Response: fixed.
3. "Nevertheless, it is inaccuracy,"
   You mean it is inaccurate. 
   This paper cannot be published without proper English editing. I cannot continue to highlight language issues further.
   Response: English language and style has been edited.
4. There should be no subsection if the authors are citing one to two works per subsection.
   Response: fixed.
5. The literature overview is, for the lack of any better word, poor. The field of power measuring, profiling, and monitoring in embedded and mobile systems is more than rich and abundant with studies. I do want to discourage the writer but they have not explored the background works, nor the alternative software methods. they should cite at least - as a start - these works, and look in them for other relevant work :
6. When proper citations are made. The authors should explain how their work differs from software-based works and how it improves upon it.
   Response: 2.2 problems is reworked, and we raised the problems not properly solved in existing methods.
7. The Voltage curve is one of the tools used to measure/estimate power consumption but not the only one. Why rely on this one? How does the battery age affect the curve and precision of the measurements?
   Response: in citation [3], battery capacity degradation has been proven to have a significant impact on accuracy (line 54).
8. What the author state in section 2.2.2 is neither clear nor convincing. It is more of a technicality rather than a research issue.
   Response: section 2.2 is now reworked, and we raised the problems not properly solved in existing methods.
4. Our Idea and Design
1. Title of this section should be changed
   Response: it’s been changed to ‘Methodology and Data Acquisition’.
2. There are multiple ways to measure and estimate power consumption, as there are also multiple reasons to do so. What is the purpose of your power measurement? Online monitoring? Code optimization? Power management?
   Response: section 3 has been reworked, and we discuss how we try to solve the problems proposed in section 2.2.
3. I do not understand the purpose of this section. It more technical rather than a research or a scientific contribution
   Response: same as above.
5. Implementation 
1. The case study devices should be in a section of their own, and the authors should explain their choice
   Response: some detailed information about the device used is added, but we think the ‘device’ is not our focus, so we don’t make it in a separate subsection. And detailed information about the device is introduced in line 177-183.
2. Figures 1 and 2 have text overflowing and should be redone.
   Response: the figures are redone.
3. Figure 4 should be made into two figures for clarity. The same for figure 5.
   Response: the figure are redone.
4. In the results section, no comparisons are made to currently available methods, neither software profilers nor hardware ones.
   Response: we compared it with the method proposed in [10] and the comparison is in 5.3.
5. There is no significant difference between the two proposed measurement schemes. Could the authors explain why there are two?
   Response: limitations of auxiliary design is added in section 4 (line 279-283), and we added 5.4, a unique test to our standalone design.

 

Author Response File: Author Response.docx

Reviewer 3 Report

 

The paper topic is very interesting, and the proposed automation for diversified use cases is valuable. However, the paper lacks detail, clarity regarding the obtained results and quantified assessment of the proposed method against alternative measurement approaches.

Section 2 should be improved; 2.1 should be updated with recent advances and 2.2 should define and quantify performance parameters (such as time/current/ voltage resolution and precision requirements).

Specific comments:

Line41: "Existing methods": Report recent advances in micropower measuring techniques (i.e. such as: https://ieeexplore.ieee.org/document/9080939)

Line48: report actual resolution.

Line 50: Modern implementations and associated performance should be referenced (i.e. https://software-dl.ti.com/ccs/esd/documents/xdsdebugprobes/emu_energytrace.html)

Lines 119-123: The GPIO method is well known, however, it DOES introduce measurement burden, which in micropower/microsecond level is NOT negligible.

 

Section 4 has appropriate length, but lacks detail and clarity.

Figure 2: Does not show the use case download/programming path from the intermediate device to the target device

 

Section 4 and 5: The selection of a Digital power supply/multimeter should be justified (i.e. against a resolution range oscilloscope (such as http://www.farnell.com/datasheets/2265784.pdf))

 

Line47, typo:  "inaccuracy"-> inaccurate

Author Response

Dear reviewer:

Thank you for your valuable comments. We have revised the manuscript (Manuscript ID: applsci-826693) and sent the revised manuscript for your consideration to be published in Applied Sciences. Based on the comments from you, we have made changes of the manuscript, which are detailed below.

The paper topic is very interesting, and the proposed automation for diversified use cases is valuable. However, the paper lacks detail, clarity regarding the obtained results and quantified assessment of the proposed method against alternative measurement approaches.
Response: Section 3 is reworked with details about our methodology, and we have compared our designs with existing method (in [10]) in section 5.3. Test results unique to the standalone design is added in section 5.4.

Section 2 should be improved; 2.1 should be updated with recent advances and 2.2 should define and quantify performance parameters (such as time/current/ voltage resolution and precision requirements).
Response: Section 2 is totally reworked. References are supplemented in section 2.1. We proposed our time resolution requirement in section 2.2. However, we don’t intend to design an instrument that directly measures the physical quantity of voltage and current, what we designed is a platform that helps with issues not properly solved (proposed in section 2.2).

Specific comments:

Line41: "Existing methods": Report recent advances in micropower measuring techniques (i.e. such as: https://ieeexplore.ieee.org/document/9080939)
Response: We don’t intend to design an instrument that directly measures the physical quantity of voltage and current, so we didn’t focus on micropower measuring techniques.

Line48: report actual resolution.
Response: Section 2.1 is totally reworked, and the resolution of battery monitoring unit is 0.1 mAh (line 52).

Line 50: Modern implementations and associated performance should be referenced (i.e. https://software-dl.ti.com/ccs/esd/documents/xdsdebugprobes/emu_energytrace.html)
Response: Section 2.1 is totally reworked, we also referenced similar software from Qualcomm and STMicroelectronics (line 82-90).

Lines 119-123: The GPIO method is well known, however, it DOES introduce measurement burden, which in micropower/microsecond level is NOT negligible.
Response: We measured working current with and without using GPIO, and we found the difference is about 20-30μA. Take the STM32F407 core board we used in this paper for an example, the working current of the board @ 16MHz is about 8.3mA, and about 49.2mA @ 168MHz. The error is about 0.36% for 16MHz, and 0.06% for 168MHz. We think the burden of GPIO can be ignored.

Section 4 has appropriate length, but lacks detail and clarity.
Response: some details about our methodology are added in section 3.

Figure 2: Does not show the use case download/programming path from the intermediate device to the target device
Response: figure has been redone.

Section 4 and 5: The selection of a Digital power supply/multimeter should be justified (i.e. against a resolution range oscilloscope (such as http://www.farnell.com/datasheets/2265784.pdf))
Response: brief introduction of the device and instrument used has been added at the beginning of section 4 (line 177-180).

Line47, typo:  "inaccuracy"-> inaccurate
Response: fixed

Author Response File: Author Response.docx

Reviewer 4 Report

 

In this paper the authors present a novel platform for automated energy consumption measurements that well suits embedded application.  Different designs for the power measurements have been tested using the proposed approach, highlighting the main benefits with respect the traditional techniques.

The study is well conducted, and it is potentially applicable for a wide range of situations where a reliable and rapid method for power measurements is needed.

The paper in general could be improved in the writing style, some sentences are difficult to be interpreted and maybe also a general review of the English language could help in making the paper more fluent.

Anyway, some aspects need to be clarify as reported in the following:

 

2.Related works and motivation

In the 2.1 “existing methods” among energy consumption methods, are reported the instrument used for performing this operation, but especially in 2.1.1 and 2.1.2 the “methods” itself is not well described. The operation to be performed to obtain the energy consumption can help the reader to better understand the framework of this work.

In 2.2.1 the title says “The connection between the power curve and the code running on the board”, probably referring to embedded situation. I think that  a short introduction to this should be given as a general hat to all this paragraph (2.2), explaining the context (i.e. application for embedded system), what do you mean with “board” and “code running”. However, the first two sentences of 2.2.1 are not so clear (i.e. from row70 to 73), please try to reformulate better.

3. our idea and design and 4 implementation

I think that these could be name of sub-paragraph under the same paragraph “3: materials and methods”, in a more traditional way.

At 3.1 “different operating condition” are mentioned: what do you mean? Please explain better.

At 3.2 the GPIO is introduced without a clear explanation of what it is and why is used in this system, please try to reformulate better.

At 4, the use of a keithley power supply is declared; since this is a crucial instruments in this work, can you please reported a summary of the features that are interesting for your work? (e.g. resolution etc…)

In 4.1 the scheme in figure 1 is mentioned but a detailed description of the algorithm is not given, please provide. In the same paragraph, the figure 1 is not completely clear: the purple block  on the left is not well readable and the outputs of the purple block on the right are not so clear.

4.1.2 at line 166 please replace “does exist” with “exists” or “is present”

At 4.2.1 a stm32f7 is used as intermediary device: can you briefly describe this system and its main features?

For both figure 2 and 3 a more detailed description in the text can help the reader to understand those better. Moreover pin names of the words “intermediary device” and “target board” in figure 2 and “usb-ttl” and “target board” in figure 3 are not readable (blue on blue), as well as the pins name (too small, you could make bigger the necessary ones and remove the others).

5 evaluation and results

In table 1 the accuracy of reported measurements could help the reader in understanding significative differences among reported values.

Author Response

Dear reviewer:

Thank you for your valuable comments. We have revised the manuscript (Manuscript ID: applsci-826693) and sent the revised manuscript for your consideration to be published in Applied Sciences. Based on the comments from you, we have made changes of the manuscript, which are detailed below.

 

The paper in general could be improved in the writing style, some sentences are difficult to be interpreted and maybe also a general review of the English language could help in making the paper more fluent.

Response: the English language and style have been edited;

 

Anyway, some aspects need to be clarify as reported in the following:

2.Related works and motivation

In the 2.1 “existing methods” among energy consumption methods, are reported the instrument used for performing this operation, but especially in 2.1.1 and 2.1.2 the “methods” itself is not well described. The operation to be performed to obtain the energy consumption can help the reader to better understand the framework of this work.

Response: Section 2.1 is totally reworked, more details about many latest measuring and modeling method are added, as well as their pros and cons.

 

In 2.2.1 the title says “The connection between the power curve and the code running on the board”, probably referring to embedded situation. I think that  a short introduction to this should be given as a general hat to all this paragraph (2.2), explaining the context (i.e. application for embedded system), what do you mean with “board” and “code running”. However, the first two sentences of 2.2.1 are not so clear (i.e. from row70 to 73), please try to reformulate better.

Response: Section 2.2 is totally reworked, the ‘problems’ is now talked more detailed.

 

our idea and design and 4 implementation

I think that these could be name of sub-paragraph under the same paragraph “3: materials and methods”, in a more traditional way.

Response: Section 3 is now renamed as ‘Methodology and Data Acquisition’.

 

At 3.1 “different operating condition” are mentioned: what do you mean? Please explain better.

Response: Section 3 is totally reworked, and our solutions to the ‘2.2 problems’ is proposed with more details.

 

At 3.2 the GPIO is introduced without a clear explanation of what it is and why is used in this system, please try to reformulate better.

Response: Shortcomings of existing methods is talked in section 2.2, and more details about how GPIO helps are added, as well as pictures of time sequence.

 

At 4, the use of a keithley power supply is declared; since this is a crucial instruments in this work, can you please reported a summary of the features that are interesting for your work? (e.g. resolution etc…)

Response: Some specifications about the instruments we used are introduced now.

 

In 4.1 the scheme in figure 1 is mentioned but a detailed description of the algorithm is not given, please provide. In the same paragraph, the figure 1 is not completely clear: the purple block  on the left is not well readable and the outputs of the purple block on the right are not so clear.

Response: Details about methodology and how our platform works have been added in section 3; and figure 1 is redone.

 

4.1.2 at line 166 please replace “does exist” with “exists” or “is present”

Response: It is replaced.

 

At 4.2.1 a stm32f7 is used as intermediary device: can you briefly describe this system and its main features?

Response: Main feature of the intermediary device are now described in section 3.1 and 3.3.

 

For both figure 2 and 3 a more detailed description in the text can help the reader to understand those better. Moreover pin names of the words “intermediary device” and “target board” in figure 2 and “usb-ttl” and “target board” in figure 3 are not readable (blue on blue), as well as the pins name (too small, you could make bigger the necessary ones and remove the others).

Response: Figure 2 and 3 are redone.

 

5 evaluation and results

In table 1 the accuracy of reported measurements could help the reader in understanding significative differences among reported values.

Response: The limitation of the auxiliary design is talked in line 294-298, and we added 5.4, a test unique to the standalone design (this test is unavailable for auxiliary design).

Reviewer 5 Report

In this paper the authors propose two designs of automatic energy consumption measuring platform. I hope the suggestions listed below can help the authors to improve the quality of the article.

• Even though it is an interesting work, only two of the references is published in the last five years. The reference list should be enriched with recent scientific publications. In addition, the presentation of the literature could be improved. A better classification and a comparative analysis between them would be useful.
• The authors should avoid using first-person plural pronouns.
• The authors have checked punctuation and misprints in the manuscript and made appropriate corrections where necessary. The text needs a careful revision to correct language or typing mistakes.
• In section 4.1 (lines 182-183), the authors write ‘For the test software on the host computer, although there are differences in the specific implementation of different hardware arrangement, these two designs still have many similarities’. The differences and similarities should be clearly clarified and analyzed.
• The authors should refer in more detail and in clearly way the advantages of the two designs of our automatic energy consumption measuring platform and summarize possible limitations.
• It is mentioned in section 5.3 (lines 315-317) that 'We’ve designed more than 1,200 test cases and 40 different hardware environment configurations. A total of over 8,000 tests were conducted, and it took about a whole week to finish them all.' More justification should be furnished on this issue.

Author Response

Dear reviewer:

Thank you for your valuable comments. We have revised the manuscript (Manuscript ID: applsci-826693) and sent the revised manuscript for your consideration to be published in Applied Sciences. Based on the comments from you, we have made changes of the manuscript, which are detailed below.

• Even though it is an interesting work, only two of the references is published in the last five years. The reference list should be enriched with recent scientific publications. In addition, the presentation of the literature could be improved. A better classification and a comparative analysis between them would be useful.

Response: Section 2.2 is reworked; we supplemented the references and in 2.2 we proposed some problems that are not properly solved by existing measurement methods.

• The authors should avoid using first-person plural pronouns.

Response: the English language and style is edited.

• The authors have checked punctuation and misprints in the manuscript and made appropriate corrections where necessary. The text needs a careful revision to correct language or typing mistakes.

Response: same as above.

• In section 4.1 (lines 182-183), the authors write ‘For the test software on the host computer, although there are differences in the specific implementation of different hardware arrangement, these two designs still have many similarities’. The differences and similarities should be clearly clarified and analyzed.

Response: details about this issued are added in line 279-283.

• The authors should refer in more detail and in clearly way the advantages of the two designs of our automatic energy consumption measuring platform and summarize possible limitations.

Response: the limitations of the auxiliary design is added in line 279-283, and we added a new section 5.4, it contains test only available for the standalone design.

• It is mentioned in section 5.3 (lines 315-317) that 'We’ve designed more than 1,200 test cases and 40 different hardware environment configurations. A total of over 8,000 tests were conducted, and it took about a whole week to finish them all.' More justification should be furnished on this issue.

Response: automated and unattended measurement is part of our target. We designed large number of testcases to verify this capability. The paragraph shows the results.

Author Response File: Author Response.docx

Round 2

Reviewer 3 Report

The Authors have addressed many of the comments in the previous review and have allocated resources to alleviate the identified shortcomings.

Both the cited material and the proposed examples indicate that the proposed apparatus is suitable for measuring energy (and power) in specific (non-micro) ranges (i.e. for systems with significant duty cycle and runtime consumption in the range of at least tenths of mWs).

In their responses, the authors accept this statement, however, this is not evident, not quantified in the paper. A suitable statement could be added at the “conclusions” section.

There is a typo @ line 363 ("with out"-> "with our")