**Abbreviations**

The following abbreviations are used in this manuscript:



#### **Appendix A. Supplementary Case Data from the Guizhou Grid, China**

Table A1 shows the BESS properties of the 14 microgrids in the Guizhou Grid, including capacity, initial SOC, charge and discharge restriction and charge and discharge efficiency.


**Table A1.** Battery energy storage system properties of 14 microgrids in the Guizhou Grid.

The peak/flat/valley electricity price formulated by Guizhou Grid, China is presented in Table A2, which divides a day into three types of time internals.

**Table A2.** Peak/flat/valley electricity price formulated by the Guizhou Grid.


The learning rate *α*, discount factor *γ* and greedy degree *ε* parameters of the 14 microgrids are given in Table A3.

**Table A3.** Q-Learning Parameters of 14 Microgrids.


The values of hyper parameters that appear in this paper are given in Table A4.

**Table A4.** Hyper parameters settings for the proposed Q-learning algorithm.

