A Proof-of-Concept FPGA-Based Clock Signal Phase Alignment System

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper presents a practical hardware implementation of the phase alignment system described theoretically in the authors' earlier works [17] and [34]. The contribution of the work should be clearly stated.

Some sentences or even a paragraph are copied directly from other authors' works, which is not acceptable, e.g. [34]:
"The stream of XOR-ed differences is filtered using a moving average filter (MA filter).
The moving average filter is a common smoothing method; it is a time-domain finite
impulse response filter. It is quite simple in design as well as in implementation. The most
significant parameter that needs to be determined when the MA filter is used is the filter
length. Compared to other types of filters, it has the sharpest step response [21]."

The advantage of this paper is that it contains many practical measurements.
The subject is interesting, but raises many questions.
According to the five-stage calibration procedure, isn't possible to assume that the clock period is greater than the other delays, then the second frequency is not needed?

Is the calibration and timing procedure done only once in the built system and then all the measurements shown have been conducted?
It would be interesting to know how stable is the phase alignment over longer time period, temperature variations or for larger numbers of nodes.

In line 176 the authors state "mathematical simulations have been conducted", what tools were used, self-written programs? Why digital simulation is not sufficient (it was described in the authors' work [17])?

Section 5.1 is unclear. How are the signals generated? Is the 100kHz signal generated based on 20MHz CLK or 100kHz modulates width of 20MHz signal?  Waveforms could be added to show the modulating signal shape.

Line 460 states that "any available communication method could be used for this task by reserving several additional locations in the address space". It is unclear what address space.

The authors only compare their solution with the White Rabbit system, are there really no others to compare with?

The advantage of the article is that it provides a comprehensive review of the bibliography.

Editorial note:
In Figure 3, the symbols and subscripts delta_xx are too small and illegible.
In Figure 4, the font for the description text is too small.

Author Response

Dear Sir or Madam,
I would like to thank you for your comments as well as insightful review. Please find the second revision of the manuscript uploaded with changes and improvements recommended by the reviewers. According to your comments and suggestions several changes and improvements has been introduced. Please find the replies to your comments below:

1. The contribution of the work has been added in Section 2.

2. The cited paragraph has been rewritten.

3. The second clock signal frequency (2 times lower frequency) is needed regardless of the relation between clock signal period and internal delays. Its purpose is to workaround the mathematical issue inherent to modular multiplicative inverse, explained in section 2.5. Due to the nature of modular arithmetic, there are two mathematically correct results – as explained in Section 2.5. Looking at the Equations (8) and multiplying both sides in both equations by 2 – both results are D_1+D_2=2D_target. Therefore both Equations (8) are valid results. However, only one of these results physically equals to the target value. 
On the other hand, assuming the internal delays are negligible, the internal connections delays constraint given in Equation (2) can be omitted.

4. The calibration procedure is done only once prior to the start of the measurements. This information has been added to the Section 4.

5. The verification of stability of the phase alignment over longer time period, temperature variations and larger numbers of nodes is an interesting topic for future research. The future research plans description has been updated in Section 6.1. Thank you for your suggestion.

6. The simulations were custom written in GNU Octave and Verilog HDL simulation. The former was used for mathematical verification of the equations and workaround explained in Section 2 and point 3 above. The latter included behavioral models to verify if the concept is functionally correct. The simulations haven't took into account more complicated issues, such as process and temperature variations, jitter, noise etc. Therefore hardware prototype was needed to verify the concept in practice.

7. The 100kHz signal was generated based on 20MHz clock. The Section 5.1 has been updated to describe this more clearly.

8. The paragraph describing the possibility of using any available communication method has been updated to avoid ambiguity.

9. Unfortunately, no other solutions has been found, apart from the described White Rabbit protocol, common received synchronization pulse and cited patents. This was one of the reasons to develop the described system, because it can be thought of as a solution that falls in between the uncomplicated and very limited in precision synchronization pulse method, and very complicated and very precise White Rabbit.

10. Figures 3 and 4 have been updated with increased text size.

Yours faithfully,
Andrzej Wojciechowski

Reviewer 2 Report

Comments and Suggestions for Authors

Hi Authors. Kindly refer to attached file for my comments. Thanks.

Comments for author File: Comments.pdf

Author Response

Dear Sir or Madam,
I would like to thank you for your comments as well as insightful and positive review. Please find the second revision of the manuscript uploaded with changes and improvements recommended by the reviewers. According to your comments and suggestions several changes and improvements has been introduced. Please find the replies to your comments below:

1. The primary application of the presented system is to enable the development of generators and sequencer-based systems, that can be split into multiple separate blocks and cooperate with relatively high accuracy and precision. The Section 6 has been updated to describe the potential applications in more detail.

2. The existing low-frequency solutions (common received synchronization pulse method) are fundamentally limited by a system clock period. Typically these operate in range up to hundreds of kHz frequency.
The existing high-complexity solutions (the White Rabbit protocol) typically operates with a 62.5 MHz and 125 MHz clock signal frequency, but probably can be reconfigured for different values.
The presented solution does not impose any frequency range limitations. The implemented proof-of-concept design frequency range is limited by the internal FPGA components supported frequency range and PCB signal integrity. One of the goals of the presented solution was to avoid limiting the useful frequency range if possible.

3. The main benefits of an FPGA-based implementation are its reconfigurability, which is crucial in the development and debugging phase, as well as the fact that FPGA already contains required components, such as a PLL block with phase shifting capability (the MMCM block). Additionally, FPGA devices are widely used in generators and sequencer-based systems, which can simplify the implementation of the presented solution in such a system, as no or minimal additional components could be required.
On the other hand, ASIC implementation would improve the performance and simplify the fulfillment of the internal connections delays constraint given in Equation (2). In comparison, the implementation in every new FPGA target device would require a careful verification of the aforementioned constraint fulfillment.

4. The developed mathematical model includes delays for all components and traces on synchronization clock paths of the introduced design. The schematic diagram containing the delays is presented in Figure 3. The model assumes that the delays are isotropic, deterministic and invariable during the calibration process. The additional assumptions are Phase Shifter block's uniform phase shift steps which affect the minimal resource utilization. However, non-uniform phase shifts steps (e.g. each step shifting the signal by a different amount, compared to the previous step) can also be used successfully (assuming the phase shifting function is known), but with greater resource utilization.

5. The internal connections delays constraint given in Equation (2) enabled simple calibration calculations described in Section 2. Without this constraint, the calibration procedure and calculations would be more complex, likely similar to the White Rabbit synchronization procedure. Additionally, as previously mentioned, further resource reduction has been achieved due to the linearity of the Phase Shifter block's phase shift function.

6. The preliminary verification results have been listed in Table 3 and section 5.1. At the current proof-of-concept stage of development, no comparison with high-complexity, high-accuracy solutions in real-world scenarios has been done yet. However, this is an interesting topic for further research. Thank you for your suggestion.

7. The presented intermediate solution covers only clock signal phase alignment capability, with no time synchronization, compared to the White Rabbit protocol. Additionally, the initial results have shown slightly lower phase-alignment precision compared to the White Rabbit protocol results.
In comparison to a common received synchronization pulse method, the presented solution is more complex and requires noticeably more effort during implementation, i.e. due to the internal connections delays constraint given in Equation (2).
The Section 5 has been updated to describe these comparisons.

8. The performance verification procedure has been described in detail and the results have been posted in Sections 4 and 5 of the paper.

9. The future research plans for this solutions include different hardware setup for performance verification in wider range of clock signal frequencies, verification of stability of the phase alignment in different conditions, such as over longer time period, with temperature variations or larger numbers of nodes. Additionally, there are plans to implement and verify a support for tree topology, compared to the current daisy-chain topology. The future research plans description has been updated in Section 6.1.

10. Figure 4 font for the description text has been resized and the diagrams have been explained in Section 2.7.

Yours faithfully,
Andrzej Wojciechowski

Reviewer 3 Report

Comments and Suggestions for Authors

• What is the main objective of the system described in the paper?
• How does the system use an FPGA to align clock signal phases?
• What are the key steps or phases involved in the proof-of-concept demonstration?
• What are the potential advantages of using an FPGA for this type of system?
• What are the practical applications of such a system in the electronics industry?
• What are the main conclusions or findings from this proof-of-concept study?
• What are the practical implications of this research for the information and communication technology industry?
• What are the next steps for research or development of this system?

Author Response

Dear Sir or Madam,
I would like to thank you for your comments as well as insightful and positive review. Please find the second revision of the manuscript uploaded with changes and improvements recommended by the reviewers. According to your comments and suggestions several changes and improvements has been introduced. Please find the replies to your comments below:

1. The main objective is to develop an alternative to existing phase alignment solutions. In contrast to existing methods, it supports high frequencies and relatively high accuracy and precision with relatively low complexity. Additionally, the presented solution does not impose any frequency range limitations. The implemented proof-of-concept design frequency range is limited by the internal FPGA components supported frequency range and PCB signal integrity. One of the goals of the presented solution was to limit the useful frequency range as minimally as possible. The presented general concept is also applicable to both FPGA devices, as well as custom ASICs.

2. The presented FPGA implementation utilizes hardware MMCM component in FPGA device with a novel calibration algorithm implemented in a digital controller block. The details of the calibration procedure and block diagram of the phase alignment system implemented using FPGA resources are described in Sections 2 and 3, as well as Figure 3 and 7.

3. The presented phase alignment system requires a five-stage calibration procedure. The first stage involves finding a phase shift value which results in minimal (close to zero) phase shift in a phase comparator in one of the nodes. The second stage is similar to the previous one, but executed using clock signal with 2 times lower frequency. The third stage involves finding a phase shift value which results in minimal (close to zero) phase shift in a phase comparator in the other node (compared to the stage 1). The fourth stage is similar to the previous one, but executed using clock signal with 2 times lower frequency. The last stage calculates the target phase shift value using the results obtained during previous stages. The detailed mathematical description is explained in Section 2.

4. The main benefits of an FPGA-based implementation are its reconfigurability, which is crucial in the development and debugging phase, as well as the fact that FPGA already contains required components, such as a PLL block with phase shifting capability (the MMCM block). Additionally, FPGA devices are widely used in generators and sequencer-based systems (which are main fields of potential application), which can simplify the implementation of the presented solution in such a system, as no or minimal additional components would be required.
On the other hand, ASIC implementation would improve the performance and simplify the fulfillment of the internal connections delays constraint given in Equation (2). In comparison, the implementation in every new FPGA target device would require a careful verification of the aforementioned constraint fulfillment.

5. The primary application of the presented system is to enable the development of generators and sequencer-based systems, that can be split into multiple separate blocks and cooperate with relatively high accuracy and precision. The Section 6 has been updated to describe the potential applications in more detail.

6. The main conclusions are that automatic precise and accurate clock signal phase alignment is possible using only FPGA resources (with no additional active components). Additionally, the obtained results have proven that FPGA  chips can successfully be used in applications requiring precise timing relations between individual components interconnects.
The achieved preliminary results are comparable (in terms of precision and accuracy) to the significantly more sophisticated solution, such as White Rabbit protocol. In comparison to a common received synchronization pulse method, the presented solution significantly improves both precision and accuracy of the phase alignment, while utilizing relatively minimal resources. However, the presented solution requires noticeably more effort during implementation compared to the synchronization pulse method. The detailed results and conclusions are described in Sections 5 and 6.

7. The future research plans for this solutions include different hardware setup for performance verification in wider range of clock signal frequencies, verification of stability of the phase alignment in different conditions, such as over longer time period, with temperature variations or larger numbers of nodes. Additionally, there are plans to implement and verify a support for tree topology, compared to the current daisy-chain topology.
The future research plans description has been updated in Section 6.1.

Yours faithfully,
Andrzej Wojciechowski

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

All comments have been accepted.