**1. Introduction**

Many information objects have a hierarchical or recursive structure. In this case, a tree structure is a convenient form of representing such information objects. This allows us to describe an information object by a combinatorial set and apply combinatorial generation algorithms for it. A combinatorial set is a finite set whose elements have some structure and there is an algorithm for constructing the elements of this set. The elements of combinatorial sets (combinatorial objects) like combinations, permutations, partitions, compositions, paths, graphs, trees, etc. play an important role in mathematics and computer science.

Knuth [1] gives a detailed overview of the formation and development of the direction related to designing combinatorial algorithms. In this overview, special attention is paid to the procedure for traversing all possible elements of a given combinatorial set. This problem can be studied as enumerating, listing, and generating elements of a given combinatorial set. On the other hand, Ruskey [2] introduces the concept of combinatorial generation and distinguishes the following four tasks in this area:

1. Listing: generating elements of a given combinatorial set sequentially;


Before applying combinatorial generation algorithms, it is necessary to develop them. General methods for developing combinatorial generation algorithms were studied by such researches as E.M. Reingold [3], D.L. Kreher [4], E. Barcucci [5,6], S. Bacchelli [7,8], A. Del Lungo [9,10], V. Vajnovszki [11–13], P. Flajolet [14,15], C. Martinez and X. Molinero [16–20], B.Y. Ryabko and Y.S. Medvedeva [21–23], and V.V. Kruchinin [24].

There are several basic general methods for developing combinatorial generation algorithms:


Each of these methods claims the universality of its application in the development of new combinatorial generation algorithms. The study of these methods have shown the following results connected with their limitations and requirements [25]:


Also, there are many combinatorial generation algorithms that are based on features of the applied combinatorial set or that are based on simple counting techniques (for example, see [26–28]). Therefore, the methods used for developing such algorithms cannot be universal (they cannot be applied to develop new combinatorial generation algorithms for other combinatorial sets).

Thus, there is no universal general method that can be applied for developing new combinatorial generation algorithms. The main purpose of our research is to derive and improve general methods for developing combinatorial generation algorithms. In this paper, we consider and extend Kruchinin's method for developing combinatorial generation algorithms, which is based on the use of AND/OR trees. This method:


However, to apply this method, it is necessary to know an expression of the cardinality function of a combinatorial set that must satisfy the following conditions:


If an expression of the cardinality function *f* of a combinatorial set *A* satisfies the conditions presented above, then we will say that *<sup>f</sup>* belongs to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*}. The requirement of the cardinality function that belongs to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*} is the main restriction of Kruchinin's method. If the required form of the cardinality function is unknown for a given combinatorial set, then this method cannot be applied to develop combinatorial generation algorithms.

The organization of this paper is as follows. Section 2 of this paper is devoted to a brief description of the main theoretical points of the used method for developing combinatorial generation algorithms. In Section 3, our modification of the original method is presented. The modification is based on applying the method of compositae from the theory of generating functions. To confirm the effectiveness of using the proposed modification of the original method, we develop new ranking and unranking algorithms for the following combinatorial sets: permutations, permutations with ascents, combinations, Dyck paths with return steps, labeled Dyck paths with ascents on return steps. The obtained results are shown in Section 4.

#### **2. Method for Developing Combinatorial Generation Algorithms Based on AND/OR Trees**

Kruchinin [24] introduces a method for developing combinatorial generation algorithms, which is based on the use of AND/OR trees. This method is based on representing a combinatorial set in the form of an AND/OR tree structure for which the total number of its variants is equal to the value of the cardinality function of the combinatorial set. Using an AND/OR tree structure, it is possible to develop listing, ranking, and unranking algorithms for a given combinatorial set. The effectiveness of this method is shown in the development of combinatorial generation algorithms for a large number of combinatorial sets (for example, permutations, combinations, partitions, compositions, the Fibonacci numbers, the Catalan numbers, the Stirling numbers, tree structures, and formal languages).

An AND/OR tree is a tree structure that contains nodes of two types: AND nodes and OR nodes. Figure 1 shows a way for representing nodes in an AND/OR tree.

**Figure 1.** The representation of nodes in an AND/OR tree.

A variant of an AND/OR tree is a tree structure obtained by removing all edges except one for each OR node. Figure 2 shows an example of an AND/OR tree and all its variants.

**Figure 2.** An AND/OR tree and all its variants.

If we know the cardinality function *f* of a combinatorial set *A* that belongs to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*}, then we can construct an AND/OR tree structure for which the total number of its variants is equal to the value of the cardinality function (Theorems 1–3 and Corollaries 1–2 in [24]. To do this, it is necessary to perform the following steps for the cardinality function *f* :


Thus, the method for developing combinatorial generation algorithms based on AND/OR trees can be written in the following form:

**Input:** The cardinality function *<sup>f</sup>* of a combinatorial set *<sup>A</sup>* that belongs to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*}. **Output:** The combinatorial generation algorithms

$$\text{RankVariant}(v) : \mathcal{W}(D) \to \mathbb{N}\_{|\mathcal{W}(D)|}$$

and

$$\mathsf{UnrrankVar}\mathsf{ant}(r) : \mathbb{N}\_{|\mathcal{W}(D)|} \to \mathcal{W}(D)\_{\prime\prime}$$

where each variant *v* of an AND/OR tree *D* constructed for the combinatorial set *A* must correspond to a specific combinatorial object *a* ∈ *A*. That is, the bijection *A* ↔ *W*(*D*) must be defined, where *W*(*D*) is the set of all the variants *v* of the AND/OR tree *D*. The combination of this bijection with the algorithms RankVariant(*v*) and UnrankVariant(*r*) represents the desired combinatorial generation algorithms Rank(*a*) : *<sup>A</sup>* <sup>→</sup> <sup>N</sup> and Unrank(*r*) : <sup>N</sup> <sup>→</sup> *<sup>A</sup>* (for ranking and unranking elements of the combinatorial set *A*).

The ranking algorithm RankVariant(*v*) allows us to associate each variant *v* ∈ *W*(*D*) of the AND/OR tree *<sup>D</sup>* with a unique number *<sup>r</sup>* <sup>∈</sup> <sup>N</sup>|*W*(*D*)<sup>|</sup> <sup>=</sup> {0, 1, ... , <sup>|</sup>*W*(*D*)| − <sup>1</sup>} called the rank. The rank *r* of a combinatorial object *a* represented as a variant *v* of the corresponding AND/OR tree *D* is determined by the value of *l*(*z*) for the node *z* of the variant *v* that is the root of the AND/OR tree *D*. The value of *l*(*z*) corresponds to a number for the node *z* that satisfies the condition

$$0 \le l(z) < w(z)\_r$$

where *w*(*z*) shows the number of variants in the subtree of the node *z*.

The value of *l*(*z*) is calculated according to the following rules:

1. If a node *z* of the variant *v* is a leaf of the AND/OR tree *D*, then

$$l(z) = 0\text{-}$$

2. If a node *z* of the variant *v* is an AND node of the AND/OR tree *D*, then

$$l(z) = l(s\_1^{(z)}) + w(s\_1^{(z)}) \left( l(s\_2^{(z)}) + w(s\_2^{(z)}) \left( \dots \left( l(s\_{n-1}^{(z)}) + w(s\_{n-1}^{(z)}) l(s\_n^{(z)}) \right) \dots \right) \right),$$

where *n* is equal to the number of sons for the node *z* and *s* (*z*) *<sup>i</sup>* is the *i*-th son of the node *z*;

3. If a node *z* of the variant *v* is an OR node of the AND/OR tree *D*, then

$$l(z) = l(s\_k^{(z)}) + \sum\_{i=1}^{k-1} w(s\_i^{(z)})\_i$$

where *k* is the position of the son *s* (*z*) *<sup>k</sup>* of the node *z* chosen in the variant *v* among all its sons in the AND/OR tree *D*.

For the general case, the rules for ranking the variants of an AND/OR tree was formalized and presented as Algorithm 1. To run this algorithm, it is necessary to start from the command RankVariant(*root*, *v*, *D*), where *root* is the root of an AND/OR tree *D*.

#### **Algorithm 1:** A general algorithm for ranking the variants of an AND/OR tree.

**<sup>1</sup>** RankVariant (*z*, *v*, *D*) **<sup>2</sup> begin <sup>3</sup> if** *z* = *a leaf of the AND/OR tree D* **then** *l* := 0 **<sup>4</sup> if** *z* = *an AND node of the AND/OR tree D* **then <sup>5</sup>** *n* := the number of sons for the node *z* **<sup>6</sup>** *l* := RankVariant (*s* (*z*) *<sup>n</sup>* , *v*, *D*) **<sup>7</sup> for** *i* := *n* − 1 **to** 1 **do** *l* := RankVariant (*s* (*z*) *<sup>i</sup>* , *v*, *D*) + *w*(*s* (*z*) *<sup>i</sup>* ) · *l* **<sup>8</sup> end <sup>9</sup> if** *z* = *an OR node of the AND/OR tree D* **then <sup>10</sup>** *k* := the position of the son of the node *z* chosen in the variant *v* among all its sons **<sup>11</sup>** *l* := RankVariant (*s* (*z*) *<sup>k</sup>* , *v*, *D*) **<sup>12</sup> for** *i* := 1 **to** *k* − 1 **do** *l* := *l* + *w*(*s* (*z*) *<sup>i</sup>* ) **<sup>13</sup> end <sup>14</sup> return** *l* **<sup>15</sup> end**

The unranking algorithm UnrankVariant(*r*) performs the inverse operation to the algorithm RankVariant(*v*). That is, the algorithm UnrankVariant(*r*) allows us to associate each rank *<sup>r</sup>* <sup>∈</sup> <sup>N</sup>|*W*(*D*)<sup>|</sup> with a unique variant *v* of the AND/OR tree *D*. The algorithm UnrankVariant(*r*) is based on the inverse actions to the calculations used in the algorithm RankVariant(*v*). For the general case, the rules for unranking the variants of an AND/OR tree were formalized and presented as Algorithm 2.

**Algorithm 2:** A general algorithm for unranking the variants of an AND/OR tree.


#### **3. Modification of the Method for Developing Combinatorial Generation Algorithms**

The use of the method for developing combinatorial generation algorithms based on AND/OR trees has the following two restrictions:

Firstly, if we do not know the cardinality function of a combinatorial set that belongs to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*}, then we cannot construct the corresponding AND/OR tree for the combinatorial set. Therefore, this method for developing combinatorial generation algorithms cannot be applied without an AND/OR tree structure.

If an AND/OR tree structure is constructed for a combinatorial set, then there is a new problem. This problem is associated with finding a bijection between the elements of the combinatorial set and the set of variants of the AND/OR tree (that is, each variant *v* of an AND/OR tree *D* constructed for the combinatorial set *A* must correspond to a specific combinatorial object *a* ∈ *A*, and vice versa). A general approach for solving this problem does not exist, since each combinatorial set has its own and completely unique characteristics. We propose to use the following recommendation: it is necessary to consider the changes that occur in the structure of combinatorial objects when moving from one node

of an AND/OR tree to another (considering the type of a node, the label of a node and the selected sons) and show these changes in the bijection.

For solving the first problem, we propose to apply the theory of generating functions, since it is one of the basic approaches in modern combinatorics and generating functions are already known for many combinatorial sets. An ordinary generating function of a sequence (*an*)*n*≥<sup>0</sup> is the following formal power series [29]:

$$A(t) = a\_0 + a\_1t + a\_2t^2 + \dots = \sum\_{n \ge 0} a\_n t^n.$$

For the coefficients of the powers of generating functions, the notion of the compositae of a generating function was introduced in [30]. The composita of an ordinary generating function

$$G(t) = \sum\_{n>0} \mathcal{g} n^{t^n}$$

is the following function with two variables *G*Δ(*n*, *k*), which is a coefficients function of the *k*-th power of the generating function *G*(*t*):

$$\left(G(t)\right)^k = \sum\_{n \ge k} G^\Lambda(n,k)t^n.$$

This mathematical apparatus provides such operations on compositae as shift, addition, multiplication, composition, reciprocation, and compositional inversion of generating functions. Such operations on compositae allow us to obtain explicit expressions for the coefficients of generating functions. To obtain an explicit expression for the coefficients of a generating function using the method of compositae, it is necessary to decompose the given generating function into functions for that the compositae are known and apply the corresponding operations to them. More detailed information about the compositae can be found in [30–35].

If for a given combinatorial set *A* we consider its subset *An* ⊂ *A*, which contains only the combinatorial objects of size *n*, then the cardinality function *f*(*n*) = |*An*| of this combinatorial set *An* can be described by a generating function

$$F(t) = \sum\_{n\geq 0} f\_n t^n = \sum\_{n\geq 0} f(n)t^n = \sum\_{n\geq 0} |A\_n| t^n.$$

Hence, to obtain an expression for the cardinality function *f*(*n*) of a combinatorial set for which a generating function is known, we can apply the method of compositae for obtaining an explicit expression of the coefficients *fn* of the generating function [25]. Moreover, the operations on compositae can be extended to the case of multivariate generating functions. This makes it possible to obtain expressions for the cardinality functions of combinatorial sets that are described by more than one parameter.

The obtained method for developing combinatorial generation algorithms with its modification is presented as a sequence of the following steps:


The combination of the algorithms defined in the last two steps of the modified method forms a bijection *<sup>A</sup>* <sup>↔</sup> <sup>N</sup> and represents the combinatorial generation algorithms Rank(*a*) : *<sup>A</sup>* <sup>→</sup> <sup>N</sup> for ranking and Unrank(*r*) : <sup>N</sup> <sup>→</sup> *<sup>A</sup>* for unranking elements of the combinatorial set *<sup>A</sup>*.

#### **4. Application of the Modification of the Method for Developing Combinatorial Generation Algorithms**

Next, we consider the process of developing ranking and unranking algorithms using the obtained method for developing combinatorial generation algorithms. We describe a combinatorial set, construct an AND/OR tree, find a bijection between the elements of the combinatorial set and the set of variants of the AND/OR tree, and develop algorithms for ranking and unranking the variants.

#### *4.1. Combinatorial Set*

Let us consider the following combinatorial object: a labeled Dyck *n*-path with *m* ascents on return steps. A Dyck *n*-path is a lattice path in the plane which begins at (0, 0), ends at (2*n*, 0), and consists of steps (1, 1) called rises or up-steps and (1, −1) called falls or down-steps [36]. A return step is a down-step at level 1 (a return to the ground level 0) [37]. In a labeled Dyck *n*-path with *m* ascents on return steps, each down-step has its own label (a unique value from 1 to *n*). If we consider the sequence of labels of down-steps of a Dyck *n*-path starting from (0, 0), then it has exactly *m* ascents.

Figure 3 shows all possible variants of the considered labeled Dyck paths for *n* = 3 and *m* = 1.

**Figure 3.** All labeled Dyck paths of size 3 with 1 ascent on return steps.

*Mathematics* **2020**, *8*, 962

The total number of labeled Dyck *n*-paths with *m* ascents on return steps is defined by the elements of the Euler–Catalan number triangle (the sequence *A*316773 in [38]). That is, the cardinality function of this combinatorial set is equal to the Euler–Catalan numbers, which are denoted by *EC<sup>m</sup> <sup>n</sup>* [39].

Applying the method of compositae for obtaining explicit expressions for the coefficients of generating functions, we have found a generating function and the following explicit formula [39]:

$$\mathrm{EC}\_{n}^{m} = \begin{cases} 1, & \text{for } n = m = 0; \\ \sum\_{k=m+1}^{n} \frac{n!}{k!} \mathrm{CT}\_{n}^{k} \mathrm{E}\_{k}^{m}, & \text{otherwise,} \end{cases} \tag{1} \\ \begin{cases} 1, & \text{for } n = m = 0; \\ \sum\_{k=m+1}^{n} \mathrm{CT}\_{n}^{k} \mathrm{C}\_{n}^{k} \mathrm{E}\_{k}^{m} \mathrm{P}\_{n-k}, & \text{otherwise,} \end{cases} \tag{1}$$

where *CT<sup>m</sup> <sup>n</sup>* is the transposed Catalan triangle, *C<sup>m</sup> <sup>n</sup>* is the number of *m*-combinations of *n* elements, *E<sup>m</sup> <sup>n</sup>* is the Euler triangle, and *Pn* is the number of permutations of *n* elements.

### *4.2. AND/OR Tree*

Equation (1) belongs to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*}, and there are the well-known formulas for the components of Equation (1), which also belong to the required algebra:

• the elements of the transposed Catalan triangle (the sequence *A*033184 in [38]) show the number of Dyck *n*-paths with *m* return steps and can be calculated using the following recurrence [37]:

$$\mathcal{C}T\_n^{\mathfrak{m}} = \mathcal{C}T\_{n-1}^{\mathfrak{m}-1} + \mathcal{C}T\_n^{\mathfrak{m}+1}, \quad \mathcal{C}T\_n^0 = 0, \quad \mathcal{C}T\_n^{\mathfrak{m}} = 1; \tag{2}$$

• the number of *m*-combinations of *n* elements (the sequence *A*007318 in [38]) can be calculated using the following recurrence [40]:

$$\mathbb{C}\_{n}^{\rm un} = \mathbb{C}\_{n-1}^{\rm un} + \mathbb{C}\_{n-1}^{\rm un-1}, \quad \mathbb{C}\_{n}^{\rm u} = \mathbb{C}\_{n}^{0} = 1; \tag{3}$$

• the elements of the Euler triangle (the sequence *A*173018 in [38]) show the number of permutations of *n* elements with *m* ascents and can be calculated using the following recurrence [41]:

$$E\_n^{\mathfrak{m}} = (m+1)E\_{n-1}^{\mathfrak{m}} + (n-m)E\_{n-1}^{\mathfrak{m}-1}, \quad E\_n^{\mathfrak{n}-1} = E\_n^0 = 1;\tag{4}$$

• the number of permutations of *n* elements (the sequence *A*000142 in [38]) can be calculated using the following recurrence [40]:

$$P\_n = nP\_{n-1} \quad \text{ } P\_0 = 1.\tag{5}$$

Since Equation (1) and all the formulas for its components belong to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*}, we can construct the AND/OR tree structure for *EC<sup>m</sup> <sup>n</sup>* , which is presented in Figure 4. A bijection between the labeled Dyck *n*-paths with *m* ascents on return steps and the variants of the AND/OR tree is defined by the following rules:


**Figure 4.** An AND/OR tree for *EC<sup>m</sup> n* .

Next, we need to construct AND/OR tree structures for all the subtrees of the AND/OR tree for *EC<sup>m</sup> <sup>n</sup>* . We also need to find bijections between the variants of these AND/OR trees and the corresponding combinatorial objects.

Since Equation (2) belongs to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*}, we can construct the AND/OR tree structure for *CT<sup>m</sup> <sup>n</sup>* , which is presented in Figure 5. A bijection between the Dyck *n*-paths with *m* return steps and the variants of the AND/OR tree is defined by the following rules:


**Figure 5.** An AND/OR tree for *CT<sup>m</sup> n* .

For a compact representation, we encode a Dyck *n*-path by a sequence *a* = (*a*1, ... , *a*2*n*), where *ai* ='u' encodes an up-step and *ai* ='d' encodes a down-step. We also encode a variant of an AND/OR tree by a sequence *v* = (*v*1, *v*2, ...) of the selected sons of the OR nodes in this tree (the left son corresponds to *vi* = 0 and the right son corresponds to *vi* = 1). An example of applying the obtained bijection for Dyck paths with return steps is presented in Table A1.

Since Equation (3) belongs to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*}, we can construct the AND/OR tree structure for *C<sup>m</sup> <sup>n</sup>* , which is presented in Figure 6. A bijection between the *m*-combinations of *n* elements and the variants of the AND/OR tree is defined by the following rules:


**Figure 6.** An AND/OR tree for *C<sup>m</sup> n* .

For a compact representation, we encode an *m*-combination of *n* elements by a sequence *a* = (*a*1, ... , *an*), where *ai* = 0 encodes that the *i*-th element is not selected and *ai* = 1 encodes that the *i*-th element is selected. We also encode a variant of an AND/OR tree by a sequence *v* = (*v*1, *v*2, ...) of the selected sons of the OR nodes in this tree (the left son corresponds to *vi* = 0 and the right son corresponds to *vi* = 1). An example of applying the obtained bijection for combinations is presented in Table A2.

Since Equation (4) belongs to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*}, we can construct the AND/OR tree structure for *E<sup>m</sup> <sup>n</sup>* , which is presented in Figure 7. A bijection between the permutations of *n* elements with *m* ascents and the variants of the AND/OR tree is defined by the following rules:


**Figure 7.** An AND/OR tree for *E<sup>m</sup> n* .

For a compact representation, we encode a variant of an AND/OR tree by a sequence *v* = (*v*1, *v*2, ...) of the selected sons of the OR nodes in this tree, where each *vi* is represented as a pair (*vi*,1, *vi*,2). In this pair: *vi*,1 = 0 corresponds to the left son of the OR node labeled *E<sup>m</sup> <sup>n</sup>* and *vi*,2 determines the selected son of the OR node labeled *m* + 1; *vi*,1 = 1 corresponds to the right son of the OR node labeled *E<sup>m</sup> <sup>n</sup>* , *vi*,2 determines the selected son of the OR node labeled *n* − *m*. An example of applying the obtained bijection for permutations with ascents is presented in Table A3.

Since Equation (5) belongs to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*}, we can construct the AND/OR tree structure for *Pn*, which is presented in Figure 8. A bijection between the permutations of *n* elements and the variants of the AND/OR tree is defined by the following rules:


**Figure 8.** An AND/OR tree for *Pn*.

For a compact representation, we encode a variant of an AND/OR tree by a sequence *v* = (*v*1, *v*2, ...) of the selected sons of the OR nodes in this tree. An example of applying the obtained bijection for permutations is presented in Table A4.

We have constructed AND/OR trees for all the combinatorial sets presented in this paper. Hence, we can develop algorithms for ranking and unranking the variants of the AND/OR trees.

#### *4.3. Ranking and Unranking Algorithms*

Based on Algorithms 1 and 2, we can develop algorithms for ranking and unranking the variants of the constructed AND/OR trees for *EC<sup>m</sup> <sup>n</sup>* , *CT<sup>m</sup> <sup>n</sup>* , *C<sup>m</sup> <sup>n</sup>* , *E<sup>m</sup> <sup>n</sup>* , and *Pn*.

For the AND/OR tree for *CT<sup>m</sup> <sup>n</sup>* , which is presented in Figure 5, we develop an algorithm for ranking its variants (Algorithm 3) and an algorithm for unranking its variants (Algorithm 4). Combining the developed algorithms with the derived rules for the bijection, we get algorithms for ranking and unranking the combinatorial set of Dyck *n*-paths with *m* return steps.

**Algorithm 3:** An algorithm for ranking the variants of the AND/OR tree for *CT<sup>m</sup> n* .

**<sup>1</sup>** RankVariant\_CT (*v* = (*v*1, *v*2,...), *n*, *m*) **<sup>2</sup> begin <sup>3</sup> if** *m* = *n* **then** *r* = 0 **<sup>4</sup> else <sup>5</sup> if** *v*<sup>1</sup> = 0 **then** *r* := RankVariant\_CT ((*v*2,...), *n* − 1, *m* − 1) **<sup>6</sup> else** *<sup>r</sup>* :<sup>=</sup> RankVariant\_CT ((*v*2,...), *<sup>n</sup>*, *<sup>m</sup>* <sup>+</sup> 1) <sup>+</sup> *CTm*−<sup>1</sup> *<sup>n</sup>*−<sup>1</sup> **<sup>7</sup> end <sup>8</sup> return** *r* **<sup>9</sup> end**

**Algorithm 4:** An algorithm for unranking the variants of the AND/OR tree for *CT<sup>m</sup> n* .

```
1 UnrankVariant_CT (r, n, m)
2 begin
3 if m = n then v := ()
4 else
5 if r < CTm−1 n−1 then v :=concat ((0), UnrankVariant_CT (r, n − 1, m − 1))
6 else v :=concat ((1), UnrankVariant_CT (r − CTm−1 n−1 , n, m + 1))
7 end
8 return v
9 end
```
In these algorithms, () denotes an empty sequence and a function concat denotes merging sequences. That is, if we have a sequence *a* = (*a*1, ... , *an*) and a sequence *b* = (*b*1, ... , *bm*), then we can get the following sequence:

$$\mathsf{concat}(a,b) = (a\_1, \ldots, a\_n, b\_1, \ldots, b\_m).$$

For the AND/OR tree for *C<sup>m</sup> <sup>n</sup>* , which is presented in Figure 6, we develop an algorithm for ranking its variants (Algorithm 5) and an algorithm for unranking its variants (Algorithm 6). Combining the developed algorithms with the derived rules for the bijection, we get algorithms for ranking and unranking the combinatorial set of *m*-combinations of *n* elements.

**Algorithm 5:** An algorithm for ranking the variants of the AND/OR tree for *C<sup>m</sup> n* .

 RankVariant\_C (*v* = (*v*1, *v*2,...), *n*, *m*) **begin if** *m* = 0 *or m* = *n* **then** *r* = 0 **else if** *v*<sup>1</sup> = 0 **then** *r* := RankVariant\_C ((*v*2,...), *n* − 1, *m*) **else** *<sup>r</sup>* :<sup>=</sup> RankVariant\_C ((*v*2,...), *<sup>n</sup>* <sup>−</sup> 1, *<sup>m</sup>* <sup>−</sup> 1) <sup>+</sup> *<sup>C</sup><sup>m</sup> n*−1 **end return** *r* **end**

**Algorithm 6:** An algorithm for unranking the variants of the AND/OR tree for *C<sup>m</sup> n* .

```
1 UnrankVariant_C (r, n, m)
2 begin
3 if m = 0 or m = n then v := ()
4 else
5 if r < Cm
               n−1 then v :=concat ((0), UnrankVariant_C (r, n − 1, m))
6 else v :=concat ((1), UnrankVariant_C (r − Cm
                                                 n−1, n − 1, m − 1))
7 end
8 return v
9 end
```
For the AND/OR tree for *E<sup>m</sup> <sup>n</sup>* , which is presented in Figure 7, we develop an algorithm for ranking its variants (Algorithm 7) and an algorithm for unranking its variants (Algorithm 8). Combining the developed algorithms with the derived rules for the bijection, we get algorithms for ranking and unranking the combinatorial set of permutations of *n* elements with *m* ascents.

**Algorithm 7:** An algorithm for ranking the variants of the AND/OR tree for *E<sup>m</sup> n* .

```
1 RankVariant_E (v = ((v1,1, v1,2),(v2,1, v2,2),...), n, m)
2 begin
3 if m = 0 or m = n − 1 then r = 0
4 else
5 if v1,1 = 0 then
6 l1 := v1,2 − 1
7 l2 := RankVariant_E (((v2,1, v2,2),...), n − 1, m)
8 r := l1 + (m + 1)l2
9 end
10 else
11 l1 := v1,2 − 1
12 l2 := RankVariant_E (((v2,1, v2,2),...), n − 1, m − 1)
13 r := l1 + (n − m)l2 + (m + 1)Em
                                    n−1
14 end
15 end
16 return r
17 end
```
**Algorithm 8:** An algorithm for unranking the variants of the AND/OR tree for *E<sup>m</sup> n* .

```
1 UnrankVariant_E (r, n, m)
2 begin
3 if m = 0 or m = n then v := ()
4 else
5 if r < (m + 1)Em
                   n−1 then
6 l1 := r mod m + 1
7 l2 :=  r
               m+1

8 v :=concat (((0, l1 + 1)), UnrankVariant_E (l2, n − 1, m))
9 end
10 else
11 r := r − (m + 1)Em
                        n−1
12 l1 := r mod n − m
13 l2 :=  r
               n−m

14 v :=concat (((1, l1 + 1)), UnrankVariant_E (l2, n − 1, m − 1))
15 end
16 end
17 return v
18 end
```
For the AND/OR tree for *Pn*, which is presented in Figure 8, we develop an algorithm for ranking its variants (Algorithm 9) and an algorithm for unranking its variants (Algorithm 10). Combining the developed algorithms with the derived rules for the bijection, we get algorithms for ranking and unranking the combinatorial set of permutations of *n* elements.

**Algorithm 9:** An algorithm for ranking the variants of the AND/OR tree for *Pn*.

```
1 RankVariant_P (v = (v1, v2,...), n)
2 begin
3 if n = 0 then r = 0
4 else
5 l1 := v1 − 1
6 l2 := RankVariant_P ((v2,...), n − 1)
7 r := l1 + nl2
8 end
9 return r
10 end
```
**Algorithm 10:** An algorithm for unranking the variants of the AND/OR tree for *Pn*.

```
1 UnrankVariant_P (r, n)
2 begin
3 if n = 0 then v := ()
4 else
5 l1 := r mod n
6 l2 :=  r
              n

7 v :=concat ((l1 + 1), UnrankVariant_P (l2, n − 1))
8 end
9 return v
10 end
```
For the AND/OR tree for *EC<sup>m</sup> <sup>n</sup>* , which is presented in Figure 4, we develop an algorithm for ranking its variants (Algorithm 11) and an algorithm for unranking its variants (Algorithm 12). Combining the developed algorithms with the derived rules for the bijection, we get algorithms for ranking and unranking the combinatorial set of labeled Dyck *n*-paths with *m* ascents on return steps.

**Algorithm 11:** An algorithm for ranking the variants of the AND/OR tree for *EC<sup>m</sup> n* .

**<sup>1</sup>** RankVariant\_EC (*v* = (*k*, *v*1, *v*2, *v*3, *v*4), *n*, *m*) **<sup>2</sup> begin <sup>3</sup>** *l*<sup>1</sup> := RankVariant\_CT (*v*1, *n*, *k*) **<sup>4</sup>** *l*<sup>2</sup> := RankVariant\_C (*v*2, *n*, *k*) **<sup>5</sup>** *l*<sup>3</sup> := RankVariant\_E (*v*3, *k*, *m*) **<sup>6</sup>** *l*<sup>4</sup> := RankVariant\_P (*v*4, *n* − *k*) **<sup>7</sup>** *r* := *l*<sup>1</sup> + *CT<sup>k</sup> <sup>n</sup>*(*l*<sup>2</sup> + *C<sup>k</sup> <sup>n</sup>*(*l*<sup>3</sup> + *E<sup>m</sup> <sup>k</sup> l*4)) + *k*−1 ∑*i*=*m*+1 *CT<sup>i</sup> nC<sup>i</sup> nE<sup>m</sup> <sup>i</sup> Pn*−*<sup>i</sup>* **<sup>8</sup> return** *r* **<sup>9</sup> end**

**Algorithm 12:** An algorithm for unranking the variants of the AND/OR tree for *EC<sup>m</sup> n* .

```
1 UnrankVariant_EC (r, n, m)
2 begin
3 k := m + 1
4 sum := 0
5 while sum + CTk
                    nCk
                      nEm
                        k Pn−k ≤ r do
6 sum := sum + CTk
                        nCk
                           nEm
                             k Pn−k
7 k := k + 1
8 end
9 r := r − sum
10 l1 := r mod CTk
                    n
11 r :=
          r
           CTk
             n

12 l2 := r mod Ck
                  n
13 r :=
          r
           Ck
            n

14 l3 := r mod Em
                  k
15 l4 :=
           r
           Em
            k

16 v1 := UnrankVariant_CT (l1, n, k)
17 v2 := UnrankVariant_C (l2, n, k)
18 v3 := UnrankVariant_E (l3, k, m)
19 v4 := UnrankVariant_P (l4, n − k)
20 v = (k, v1, v2, v3, v4)
21 return v
22 end
```
In these algorithms, we use all the above mentioned algorithms for ranking and unranking the variants of the AND/OR trees for *CT<sup>m</sup> <sup>n</sup>* , *C<sup>m</sup> <sup>n</sup>* , *E<sup>m</sup> <sup>n</sup>* , and *Pn*. For a compact representation, a variant of the AND/OR tree for *EC<sup>m</sup> <sup>n</sup>* is encoded by a sequence *v* = (*k*, *v*1, *v*2, *v*3, *v*4), where:


#### **5. Conclusions**

In this paper, we study methods for developing combinatorial generation algorithms and present basic general methods for solving this task. We consider one of these methods, which is based on AND/OR trees, and extend it by using the mathematical apparatus of the theory of generating functions.

Using an AND/OR tree structure, it is possible to develop listing, ranking, and unranking algorithms for a given combinatorial set. However, the use of the method for developing combinatorial generation algorithms based on AND/OR trees has the following restriction: the cardinality function of a combinatorial set must belong to the algebra {N, <sup>+</sup>, <sup>×</sup>, *<sup>R</sup>*}. For solving this problem, we propose to apply the method of compositae for obtaining explicit expression of the coefficients of generating functions, since the theory of generating functions is one of the basic approaches in combinatorics. The limitation of this method is that it can be applied only for a combinatorial set for which a generating function is known.

As a result, we formalize the proposed idea in our modification of the original method for developing combinatorial generation algorithms. In addition, one of the main contributions of the paper is the application of this method. To confirm the effectiveness of using the proposed method, we develop new ranking and unranking algorithms for the following combinatorial sets: labeled Dyck *n*-paths with *m* ascents on return steps, Dyck *n*-paths with *m* return steps, *m*-combinations of *n* elements, permutations of *n* elements with *m* ascents, permutations of *n* elements. For each of them, we construct an AND/OR tree, find a bijection between the elements of the combinatorial set and the set of variants of the AND/OR tree, and develop algorithms for ranking and unranking the variants of the AND/OR tree.

All the developed algorithms have been realized in the computer algebra system "Maxima" and validated by exhaustive generation for fixed values of combinatorial set parameters. It also has shown that all the developed algorithms have polynomial time complexity. Several examples of applying the obtained results can be found in Appendix A.

As further research, we will consider the development of new and effective combinatorial generation algorithms in the field of applied mathematics. For example, it can be done for combinatorial sets that represent different types of chemical compounds [42], molecular structures such as RNA and DNA [43–45], etc. We also plan further improvements to the presented method for developing combinatorial generation algorithms, for example, through the usage of other types of trees [46].

**Author Contributions:** Investigation, Y.S., V.K., and D.K.; methodology, V.K.; writing—original draft preparation, Y.S.; writing—review and editing, D.K. All authors have read and agreed to the published version of the manuscript.

**Funding:** The reported study was supported by the Russian Science Foundation (project no. 18-71-00059).

**Acknowledgments:** The authors would like to thank the referees for their helpful comments and suggestions.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A. Examples of Ranking the Elements of Combinatorial Sets**

**Table A1.** Ranking the combinatorial set of Dyck *n*-paths with *m* return steps for *n* = 5 and *m* = 2.



**Table A1.** *Cont.*

**Table A2.** Ranking the combinatorial set of *m*-combinations of *n* elements for *n* = 5 and *m* = 2.


**Table A3.** Ranking the combinatorial set of permutations of *n* elements with *m* ascents for *n* = 4 and *m* = 2.


**Table A4.** Ranking the combinatorial set of permutations of *n* elements for *n* = 4.



**Table A4.** *Cont.*

#### **References**


c 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
