A New String Edit Distance and Applications

Abstract

String edit distances have been used for decades in applications ranging from spelling correction and web search suggestions to DNA analysis. Most string edit distances are variations of the Levenshtein distance and consider only single-character edits. In forensic applications polymorphic genetic markers such as short tandem repeats (STRs) are used. At these repetitive motifs the DNA copying errors consist of more than just single base differences. More often the phenomenon of “stutter” is observed, where the number of repeated units differs (by whole units) from the template. To adapt the Levenshtein distance to be suitable for forensic applications where DNA sequence similarity is of interest, a generalized string edit distance is defined that accommodates the addition or deletion of whole motifs in addition to single-nucleotide edits. A dynamic programming implementation is developed for computing this distance between sequences. The novelty of this algorithm is in handling the complex interactions that arise between multiple- and single-character edits. Forensic examples illustrate the purpose and use of the Restricted Forensic Levenshtein (RFL) distance measure, but applications extend to sequence alignment and string similarity in other biological areas, as well as dynamic programming algorithms more broadly.

1. Introduction

String similarity is an important tool for many applications, including web search completion suggestions [1], spelling correction [2], and ontology alignment [3]. Some of the tools developed in the field of string similarity carry over well to DNA applications, such as DNA-protein matching, sequence assembly, and local similarity searches [4], as well as Longest Common Substring and Longest Common Subsequence algorithms [5]. Determining similarity of DNA strands is important in phylogenetics, and the volume of available data continues to increase over time. Alignment-free methods have been developed, in part to account for issues that arise when traditional sequence alignment is pushed to its limit. These include the use of geometric representations in various dimensions, as well as graph theory [6].

Most string edit distances consider only single character edits and are variations of Levenshtein distance. These standard string edit distances are inadequate for analyzing similarity between DNA sequences in forensic applications where the goal is to identify the individual who is the source of the DNA in a crime sample. To fill this gap, a new string edit distance is defined, referred to as Restricted Forensic Levenshtein (RFL) distance, that accommodates the addition or deletion of one or more motifs (consecutively repeating sequences of nucleotides 2 to 6 base pairs long) as edit types separate from single-nucleotide insertion, deletion, and substitution. This makes the problem of computing the distance much more difficult. To address this, a dynamic programming algorithm for computing the RFL distance between sequences has been developed. Forensic examples are used to illustrate the purpose and use of RFL distance but the applications extend to sequence alignment and string similarity in other biological applications. The algorithm’s novel contribution is in efficiently handling the complex interactions that arise between multiple- and single-character edits.

1.1. Background

DNA-based human identification is an extremely powerful tool with many important applications such as identification of perpetrators of crimes, disaster victim identification, DNA typing of skeletal remains, and relationship testing [7,8,9]. The same methods are also applicable in food authenticity, poaching, and counter-bioterrorism [10]. DNA measurement technologies have made great strides recently, and now even minute quantities of DNA can be reliably measured [10].

In forensic DNA testing, genetic material present at a crime scene is extracted and amplified through the polymerase chain reaction (PCR). In principle, this technology replicates specific regions of the human genome present in the sample by a repeated process of doubling. Errors may get introduced to the products during the PCR process, and these error sequences are called artifacts. Traditionally, analysis of the amplified DNA products is carried out via a length-based measurement using capillary electrophoresis (CE); however, with the introduction of sequencing to the workflow, the amplified products now enter the process of massively parallel sequencing (MPS) providing a sequencing-based measurement for an allele. After the sequencing is complete, the generated data can be used as an input for bioinformatics pipelines. The data analysis provides details about the measured quantity of the expected PCR product sequences (here referred to as parent alleles) and numerous artifact sequences related to the range of genetic markers examined in these assays. In typing a standard single-source sample for a set of autosomal markers, the sequences observed should include one to two different parent alleles (depending on whether the sample was homozygous or heterozygous at the autosomal locus) and numerous artifact sequences that are present because of copying errors in the amplification and sequencing process. Each detected sequence is quantified by the number of sequences observed and is commonly referred to as the depth of coverage (DoC). Along with the parent alleles there are artifacts that are also detected and are often present at a low DoC, and therefore easily distinguishable from the parent alleles. Each of the artifacts is an error sequence of a true parent, and one can generally infer which parent it came from, but occasionally the parent sequences are similar enough that it is difficult to tell which is the actual parent sequence. For single-contributor DNA samples, this is typically not an issue, but crime scene samples motivating this work are rarely from a single individual. Crime scene samples often consist of multiple individuals, contributing different amounts of genetic information. For example, in a complex mixture (e.g., three contributors at 70%–25%–5%) it can be challenging to determine the respective parent allele of each contributor and their corresponding artifact(s).

In this light, quantifying what it means for nucleotide sequences to be similar becomes a crucial task, which has led to the notion of a generalized Levenshtein distance that allows gains and losses of multiple letters at once (here the change of multiple bases at once in the sequence string), and the development of an algorithm to calculate it. A major contribution of the proposed algorithm is in handling the exponential explosion of complexity when multiple-character edits and single-character edits interact, which requires careful techniques to resolve.

1.2. Short Tandem Repeats

The DNA identification community commonly uses short tandem repeats (STRs) as their genetic marker of choice [11,12]. STR typing methods are accurate and sensitive [13]. The work in this paper is designed for the analysis of DNA samples in forensic applications, but the methods presented here can be applied in other areas that use STR analysis. An STR is a region of DNA that contains consecutively repeating motifs. To the left and right of this repeating region is a non-repeating section of DNA called a flanking region. For example:

ACTCC ATG ATG ATG ATG GGTTCTGA

Here the motif ATG is repeated four times, which is represented in a short format as [ATG]4.

As sequencing technology has developed, individual nucleotides within STR regions in the human genome can be measured increasingly economically and accurately, whereas in the past people were limited to measuring the total length of the targeted sequence. The standard length-based CE methods do not allow for the detection of intra-motif variations. For a real-data example, consider a sample with a two-allele locus with repeat regions

$$\mathrm{A}:\left[\mathrm{TCTA}\right]8\left[\mathrm{TCTG}\right]1\left[\mathrm{TCTA}\right]1$$

and

$$\mathrm{B}:\left[\mathrm{TCTA}\right]10.$$

Note A and B are identical by length. When both alleles at a locus share the same length but differ by sequence, they are known as isometric heterozygotes, and CE methods cannot discriminate between them. CE types both of these alleles as 10, an allele with a length of ten repeats, and classifies this sample as homozygous at this locus. By recognizing variation at the sequence level, the power to discriminate between individuals increases [14]. If an artifact sequence C = [TCTA]7 [TCTG]1 [TCTA]1 was detected in the sequencing output, it would be reasonable to infer that there is a higher likelihood of it being an error derived from A rather than B, since fewer changes are required to edit A into C than B into C. Consequently, it would make sense to infer that the artifact is related to A. With CE, A and B are viewed as identical, and no such conclusions about the artifact can be made. As crime scene samples are most often mixtures, the additional resolution of the alleles and artifacts provided by the use of sequencing can be advantageous compared to solely length-based analysis.

In STR regions, motifs often expand and contract as a unit. This phenomenon of slippage of the polymerase on the template is referred to as stutter. The artifacts of in vitro stutter products are consistently present in measurable quantity and therefore are accounted for in routine forensic DNA analysis [15]. Furthermore, stutter occurs in vivo in STR regions even at conservative estimates at least 3–4 orders of magnitude more often than other random point mutations [16]. To reflect this, a generalization of string edit distance was used to capture similarity between sequences in MPS output. The goal was to include the addition or deletion of a motif as an edit type with a cost that can vary independently from single-nucleotide edits, to reflect the fact that these motif losses and expansions (called backward and forward stutter, respectively) occur consistently across various in vitro applications [17]. The proposed algorithm is also applicable to in vivo methods such as rare disease diagnosis and cancer progression tracking that use STR markers including their stutter [16].

It is desirable to develop a distance that gives high-frequency artifacts a lower distance to the true alleles than low-frequency artifacts, so it is important to capture both reverse and forward stutter as low-cost events. Standard string edit distances such as the Levenshtein distance modify one character at a time, so the cost of dropping or adding a multi-character motif is necessarily equal to the sum of the costs of dropping or adding the individual characters. Furthermore, unique motifs can stutter within a single STR region, with each motif having a potentially unique cost. Beyond the issue of edit cost, stutters and single-character edits can interact in complicated ways, and algorithms like the Levenshtein distance and its relatives do not begin to address the issues that arise in that interaction.

String edit distances have been in use for decades, with many papers published on different applications. Review of the literature revealed diverse uses, but multi-character extensions of edit distances were rarer. One paper described a spelling correction algorithm using a generalized metric that allows for generic word-to-word edits [18]. The aim and methods described in the study are different than those presented here; however, if the code and details for implementation were included in [18], direct comparison with the independent method presented here would be of interest. Another paper working with multi-character edits introduces some theoretical ideas relevant to dictionary based searches and generalizations of the Damerau-Levenshtein distance [19], but their searches are approximate and require symmetry, which cannot be used here since insertions and deletions of both characters and motifs happen with different frequencies. There is a variation on the dynamic programming approach that allows for generalized edits from one substring to another with non-symmetric costs, but it also prohibits multi- and single-character edit interactions, and no code or software was available for comparative testing [20]. Generalized edit distances were infrequently mentioned in the literature, and attempts to find comparable algorithms described with code attached were unsuccessful. The work presented in this paper uses an algorithm readily available for public use that also allows for a large set of overlapping edits.

The ideas behind the RFL distance are applicable to any situation requiring a minimal edit distance solved via dynamic programming, notable among which is graph edit distance  [21]. By extending the idea of precomputed motifs to other dynamic programming settings, one could consider a minimal graph distance that allows the addition or loss of an entire subgraph as its own edit to have an individual cost. Therefore, this idea could have implications wherever graph edit distances are used, such as handwriting recognition [22], fingerprint recognition [23], and cheminformatics [24].

The proposed algorithm to compute RFL distance has fully customizable costs, and is therefore capable of reflecting the idea that lower costs correspond to higher probability edits. This connects to the notion that more frequent events would be favored in a maximum likelihood approach over lower probability events. The Python code for the RFL algorithm is available on GitHub [25].

2. Restricted Forensic Levenshtein Distance

The proposed algorithm handles both single- and multiple-character edits, as well as their interactions. The single-character part of the algorithm is built on the original Levenshtein distance, which will be reviewed below. An introduction of the RFL distance follows.

2.1. Levenshtein Distance Overview

String edit distances are based on the minimal number of operations required to transform one string into another. This gives a way to measure dissimilarity of objects in the non-Euclidean space of strings, using the notion that two strings that are similar should differ in only a few characters, while two strings that are dissimilar would require many changes to turn one into the other. The edit distance proposed here could be viewed as a generalization of the Levenshtein distance [26], which counts the minimal number of single-character deletions, insertions, and substitutions required to transform one string into another. For example, the distance from CAT to CGT is one, from CAT to CA is one, and from CAT to CATTG is two. Other edit distances exist, with different edits allowed. Damerau-Levenshtein considers indels, substitutions, and transpositions [27], while the longest common subsequence metric only looks at indels [28]. Rather than having a common cost of 1, substitution, insertion, deletions, etc. can each have their own unique cost [29].

The calculation of minimal edit distance is typically done via dynamic programming [30]. The standard dynamic programming solution to Levenshtein will be briefly discussed below because it is crucial to understanding the novel contributions of the proposed algorithm. Examples will be presented to maximize clarity.

2.2. Forensic Distance Overview

The STR regions of DNA that are currently used for forensic identification were historically selected based on the polymorphic length variations of these markers observed in the population. Most of these are tetranucleotide repeats, but di-, tri-, penta- or hexanucleotide repeats also exist in the tandem oriented arrays. At sequence level a trinucleotide marker can be described as a string such as ACTACTACTACT, or abbreviated, as in [ACT]4. Backward stutter refers to losing a motif, or [ACT]3 in this example. Less-common forward stutter refers to gaining a motif, or [ACT]5 here. There are other possibilities, such as substitution, e.g., [ACT]4 to changing into an [ACT]2 AGT [ACT]1 via a C → G substitution, but the individual single-character edits are significantly rarer than motif stutter, and are already taken into account under the classical Levenshtein distance.

Stutter is more common than insertion or deletion, so it is modeled as a separate edit type. This enables the cost of stutter in the edit distance to be lowered to reflect its higher probability, instead of modeling it as the sum of individual nucleotide changes. In order to accomplish this, the RFL algorithm includes a step of looking at a set of substrings of various lengths with their associated costs from a precomputed dictionary. Whereas the standard dynamic programming solution looks back a single letter, the proposed approach looks back several letters. The particular dictionary of substrings is what gives the RFL algorithm the title “restricted”, since the size of the dictionary limits how far back the algorithm looks.

A naive approach would be to count the difference in the number of motifs and use the classical Levenshtein distance of the flanking regions, but this method misses vast numbers of edit interactions, and further misinterprets single-character edits in the STR region as stutter, e.g., [ACT]4 editing to [ACT]3 AGT would be qualified under a naive approach as back stutter, since there is one fewer ACT motif, but inspection clearly reveals a C→ G substitution. The complexity of the solution developed in this paper comes from the correct distinction of single-character edits and stutter, and the interaction of multi- and single-character edits. For example, the sequence [ACT]4 could change to [ACT]5 via forward stutter, then a C could be deleted, for a final result of [ACT]4 AT. This final string could have been attained by inserting the letters A and T from the original [ACT]4, but another option was the forward stutter and single nucleotide deletion. Different edits contribute different (customizable) costs, so one route may be more optimal than the other depending on the parameters. Extending these interactions across multiple edits becomes exponentially complex, as motifs can stutter forward and then characters can insert into the middle of that motif, for example, or a SNP could occur which turns a substring into a motif that can then stutter back, and neither of these would be detected by a naive approach. For example, if ACT were the motif, detecting the string GACGT as a three-edit insertion (one forward stutter, two single-character insertions) instead of five, or detecting TCT as a two-edit deletion instead of three, is a powerful capability, especially when considering that each edit can have vastly different costs.

The RFL algorithm handles insertions, deletions, and substitutions, the addition and deletion of prespecified character motifs, and the overlapped interactions of the multi- and single-character edit types. A comparison between classical Levenshtein, the naive approach, and the RFL algorithm is shown in

Table 1. Edits Accounted for in String Edit Distances.

Operations Accounted for	Classical Levenshtein	Naive Forensic	RFL
Single-character edits	✓	✓	✓
Pure motif stutter		✓	✓
Distinguishing stutter and single-character edits			✓
Multi- and single-character edit interactions			✓

.

Theoretical complexities arise when considering these edit interactions, as addressed in Example 1, Section 2.3.3, and Appendix A.1. These complexities are unbounded, so it is necessary to restrict the search space. However, the flexible parameter restrictions are required to accurately model the underlying sequence behavior.

A data-driven approach requires knowing which motifs at each locus stutter at significant rates. For the design of the RFL algorithm, sequence analysis data from 22 autosomal STRs was used from a set of 661 samples from four U.S. populations. The sequence data was previously generated using the PowerSeq Auto/Y System kit (Promega Corp., Madison, WI, USA) and was analyzed using STRait Razor v3.0 [31]. The output files were used to identify motifs of length 2–6 that were the basis of observed stutter artifacts at each autosomal locus. This analysis is described in Section 3.1.

2.3. Details of the RFL Algorithm

The foundation of the RFL algorithm is built on the Levenshtein distance. In this section, a perspective on the standard dynamic programming algorithm for classical Levenshtein is discussed, then Weighted Levenshtein is discussed, and lastly the novel modifications are presented. Zero-based indexing for strings will be used, with $s[i:j]$ to represent the substring of s from the i-th to the $(j-1)$th character, inclusive. Similarly, $s[i:]$ will represent the substring composed of the ith character up to and including the last character, and $s[:j]$ will represent the first j characters, up to index $j-1$, inclusive.

2.3.1. A Perspective on Dynamic Programming

From this point on, a working knowledge of the Levenshtein distance is assumed, with the following perspective. To compute the distance between strings a and b with unit edit costs, the base case is that one of the strings is empty. If a is the empty string then insert each character of $b,$ and the number of those insertions is the cost. If b is the empty string, delete each character of $a,$ and the number of those deletions is the cost. If the first characters of a and b are the same, then delete $a\left[0\right]$ and $b\left[0\right]$ for no cost and the overall distance will be the same as the Levenshtein distance between the remainder of the strings $a[1:]$ and $b[1:]$ (denoted as tail(a) and tail(b), respectively).

In case the first characters of a and b do not match, the standard dynamic programming algorithm for Levenshtein distance effectively aligns the leading character of a with the leading character of b, then modifies the leading character of a to match the leading character of b. Once the leading characters of the strings match, the matching leading characters can be deleted for free.

For example, in order to compute the Levenshtein distance from AGTCT to GACT, the A must be deleted and the first T switched to an A, for a cost of two. This eyeball calculation only works for simple examples, so Figure 1 illustrates this perspective on the classical Levenshtein algorithm.

Note that in Figure 1 there could have been an additional T concatenated to the beginning of AGTCT and then immediately deleted. This would have been a valid edit, but not a minimal one. At each step the minimal cost must be taken, and this will be accomplished via dynamic programming. For a $n_a$-character string a and an $n_b$-character string b, the dynamic programming algorithm constructs an $(n_a+1)\times (n_b+1)$ matrix, and overall it is $O\left(n_a{n}_b\right)$ in time complexity.

2.3.2. A Note on Weighted Edit Distance

In a setting where different edits happen with different frequencies, one might want to weight edits differently from each other. This could be because DNA exhibits higher rates of changing a particular character to another, or in spelling correction because certain letters are closer to others on the keyboard. For example, inserting an A could be penalized more than any other letter, and substituting C for the letter G could be incentivized. This could be done by raising the insertion cost of an A to 2 and lowering the substitution cost of G → C to 0.5. The edit distance algorithm remains the same, but whenever those edits are performed, the cost is different. This might change the optimal edit path from one string to another. Normally, converting CAT into TGT would be two substitutions: C → T and A → G. In the case where all substitution costs are equal to 4 and insertion and deletion costs are still equal to 1, the previous edit path involving two substitutions would cost a total of 8, and a new minimal-cost path would be CAT → AT → T → GT → TGT, which would cost 4. This could be made much more complicated by changing every possible edit cost to be different than every other. The dynamic programming algorithm solution is able to take any possible assortment of costs and compute the minimal cost given those parameters.

This can easily make the distance non-commutative, i.e., it is a directed distance. In a case where the substitution cost of C → G is 3, but all other costs remain equal to 1, then the distance from CAT to GAT is 2 (delete C, insert G; this is still less costly than changing C to G directly), but the distance from GAT to CAT is 1 (changing G to C still costs 1). Therefore the distance hereafter is referred to as being from a sequence s to another sequence t, which represents that the edits begin at s and change into $t.$ A library for the weighted Levenshtein, optimized in Cython, is available [32].

2.3.3. Restricted Forensic Levenshtein Distance

Here the full steps of the RFL algorithm are presented. To compute the distance between strings a and $b,$ if a is the empty string, simply insert each character of $b,$ and the number of those insertions is the cost. If b is the empty string, delete each character of $a,$ and the number of those deletions is the cost. If the first characters of a and b are the same, then delete $a\left[0\right]$ and $b\left[0\right]$ for no cost and the overall distance will be the same as the Levenshtein distance between the remainder of the strings $a[1:]$ and $b[1:]$ (denoted as tail(a) and tail(b), respectively).

In case the first characters of a and b do not match, line the two strings on top of each other with $a\left[0\right]$ aligned to $b\left[0\right],$ with all edits being made to the parent string a (this becomes important later, when the distance becomes asymmetric). The goal of the RFL algorithm is to make the leading characters of the strings match, in which case the matching leading characters can be deleted for free. For instance, the distance from CAT to CGT is the same as from AT to GT, since the leading C does not require editing. Returning to the aligned strings a and $b,$ there are three options. The first is that the leading character of a can be changed into the leading character of b, which corresponds to substitution errors in PCR. A new leading character of a can be inserted to match the leading character of b, which corresponds to insertion. Lastly, the leading character of a can be deleted so that the new leading character of a matches the leading character of b, which corresponds to deletion.

The goal is to be able to include the deletion or insertion of multiple consecutive letters as a separate edit type at a reduced cost. The classical Levenshtein solution is capable of gaining or losing multiple letters, but the cost will always be the sum total of the costs of the necessary single-character interim operations.

The classical Levenshtein algorithm looks at a character and determines what operation at that position gives the lowest cost to change a single letter at that spot when changing one string into another. It does this iteratively, breaking the original problem into sub-problems by addressing a single letter at a time. The RFL modification further looks at multiple characters at once. This will correspond to looking back several positions within the dynamic programming matrix. Only particular substrings will be allowed as their own edit types, as opposed to any general substring. The RFL solution is to create a dictionary of multi-character substrings and their associated insertion and deletion costs. When a substring—for example CTG—needs to be inserted or deleted, the relevant cost associated with that substring in the dictionary is retrieved, and that option is included for the algorithm to consider when taking a minimum cost step at that index. This substring CTG corresponds to looking back three steps in the dynamic program.

For the sake of illustration, assume unit costs for all edits, and consider AAAGA, the motif at the Penta D locus. If AAAGA is permitted to be added or deleted all at once, then inserting AAAA will be cheaper if AAAGA is first built and then the G deleted. Inserting AAA would cost the same either way: inserting 3 As, or inserting AAAGA (+1) and then deleting an A (+1) and deleting a G (+1). Inserting AAAGTA would cost 2 (+1 to stutter and +1 to insert), as opposed to costing 6 by inserting one at a time. The RFL algorithm needs to account for this, or else it would give the string AAAGTA a cost of 6 because the motif AAAGA is “broken” by the T.

The question naturally arises: if costs are being precomputed for different substrings, which substrings should be included? Suppose that for a specific motif (e.g., AAAGA), all possible substrings and their associated costs were included. Note that any string with the letters AAAGA (in that order) would be cheaper to insert by first stuttering forward and then adding letters in between, instead of adding them one by one. This is shown in Example 1 to motivate the need for restricting the search space. The resolution of this example after the restrictions are in place will be shown in Example 3.

Example 1.

To build the string

A(TTTT)A(TTTT)A(TTTT)G(TTTT)A

using edit operations starting with an empty string, 21 letters would need to be inserted, for a cost of 21. This would cost 4 less overall by inserting the motif AAAGA first via stutter (cost of 1) and then inserting all the remaining nucleotides in between (cost of 16), for a total cost of 17. This is true for an unboundedly long string, with any number of characters in between the motif characters as long as the letters AAAGA are present in that order.

Even if a string had the letters A…A…A…G…A with 2000 nucleotides in between each letter, by including it in the dictionary, the algorithm would qualify it as a modified stutter motif. Thus, not only is an infinite (or virtually infinite) dictionary impossible, it is undesirable and potentially absurd. In addition, there is no evidence to suggest that PCR results in such drastic numbers of errors. The faithfulness of the polymerase used in PCRs for STR typing has improved enough that these only err sporadically, which is supported by the fact that only small deviations were observed in artifacts in this study. Such assignment of stutter cost over excessive length of the string is undesirable, therefore a limit to the extent of the modification was incorporated.

The appendix contains more detailed discussion regarding the complex cases that arise when considering this additional edit type, but a few examples are included here. As mentioned previously, it is possible to have intra-motif substitutions and indels. The STR [ACT]4 could give rise to [ACT]2 AT [ACT]1. The difference in the number of ATG motifs here is $4-3=1,$ but there was no stutter—just deletion of the letter C. Including the interaction of stutter with other edit types quickly explodes complexity in the minimal cost computation.

Thus, for every possible nucleotide sequence of length $\le 2k-1$, where k is the length of the motif, the classical, weighted Levenshtein distances of both inserting these strings from scratch or deleting them are computed, with the requirement that the edit path of that substring contains exactly one stutter. Specifically, the precomputed insertions contain short sequences of letters with an associated cost, assuming the edit path begins with a motif insertion. The precomputed deletions contain the same strings as the insertion list, but with the cost of deleting, assuming the edit path ends with a motif deletion. These data are stored in a pair of dictionaries. In the GitHub resource, this function is called lsdp, an acronym standing for Levenshtein Stutter Dictionary Pair. Specifics are illustrated in Example 2.

Example 2.

For the motif TCTA, the stutter dictionary d has two keys: ‘insert cost’ and ‘delete cost’. Each of these keys gives a dictionary $d\left[key\right]$ of substrings and their associated costs. A selection is given as follows, where all edit costs equal to one—note that the insert and delete dictionaries will always contain the same entries, but different entries are shown here to demonstrate different costs:

$$\begin{array}{cc}\hfill d=& \{\mathit{insert}\mathit{cost}:\hfill \\ \hfill & \{\mathit{TCTA}:1,\hfill \\ \hfill & \mathit{TCTAT}:2,\hfill \\ \hfill & A:4,\hfill \\ \hfill & \mathit{GGGGGGG}:8,\hfill \\ \hfill & \dots \},\hfill \\ \hfill & \mathit{delete}\mathit{cost}:\hfill \\ \hfill & \{\mathit{TCTATGG}:4,\hfill \\ \hfill & \mathit{TCT}:2,\hfill \\ \hfill & \mathit{TCTA}:1,\hfill \\ \hfill & \dots \}\hfill \\ \hfill & \}\hfill \end{array}$$

Note that strings like ‘GGGGGGG’ and ‘A’ cost more to insert in this dictionary than they do in classical Levenshtein, since the RFL algorithm first inserts a motif for a cost of 1 before modifying it into the desired string. A symmetric fact holds for deletion entries. However, since RFL includes classical Levenshtein operations and takes the lowest-cost path, inserting the string ‘A’ will be considered both as a cost 1 edit (from the classical Levenshtein solution) and a cost 4 edit (from the dictionary), with the lower cost option being taken.

In Example 1, a sequence was shown to demonstrate that an unbounded search space would not be desirable even if it was possible. Example 3 explains how the restriction affects the answer on that use case.

Example 3.

In Example 1, it was illustrated that the string

A(TTTT)A(TTTT)A(TTTT)G(TTTT)A

would cost 21 to insert from a blank string if no motifs were considered, but it would cost 17 if the cost dictionary for the motif AAAGA contained strings of arbitrary length. The length of the motif AAAGA leads to a bound of $2\times 5-1=9$ on the dictionary substrings. The RFL algorithm with this bound in place will see parts of the parent string and insert them from the dictionary when it is cheaper to do so. For example, the edit path from a blank string to the motif AAAGA costs 1 via forward stutter and the edit distance from AAAGA to the first six characters of the string, ATTTTA, costs 4 (three substitutions and one insertion), so to insert the first six characters of the string totals 5, whereas inserting each letter individually costs 6. As the RFL algorithm moves along the string, the next place it becomes cheaper to use the dictionary is when inserting ATTTTGT, which would cost 7 with single-character edits, but instead costs 6 via the dictionary (since AAAGA → ATTTTGT requires three substitutions and an insertion). There are no other places where the RFL algorithm finds a more optimal route. Although an unbounded search space would give a cost of 17 and a classical Levenshtein would give a cost of 21, the RFL algorithm with the $2k-1$ bound gives a cost of 19.

It must be noted that the sequence in Example 3 was given as an extreme example. Motifs stutter in the data both forward or backward with notable frequency, with vastly fewer single-character edits. It would be vanishingly rare to see as many insertions as this extreme example demonstrates. Although it is a single straightforward step to change the dictionary to include more substrings, doing so will not reflect the true relative frequency of stutter versus single-character insertions and deletions. Each dictionary key accounts for exactly one stutter, whether forward or in reverse. Once substrings of length $2k$ or more are being considered, it is much more likely that two stutters are at play instead of one, and the case of two stutters is already well-accounted for in the RFL implementation as it stands, since each motif will get treated separately. Rare, more complex theoretical exceptions are described in Appendix A.1.

In the RFL algorithm, all edits are fully customizable. Every edit from one nucleotide to another can be set with a unique cost, and every relevant motif can stutter forward and backward with unique costs.

The classical Levenshtein distance separately considers the cost of an insertion, deletion, or substitution, and takes the minimum of the three at each step to expand the matrix via dynamic programming. To account for all the substrings accessible via stutter, the RFL algorithm considers the same possibilities as classical Levenshtein, and additionally considers at each step the options of inserting or deleting any l-character portion, where l ranges from 1 to $2k-1,$ and where the respective cost is defined by the pre-built dictionary.

2.3.4. How Restricted to Be?

Stopping at string length $2k-1$ is a parameter easily changed by generating a longer dictionary via the lsdp function. The RFL algorithm intakes the dictionary and can look back as far as the dictionary allows. Previously discussed in Section 2.3.3 was the motivation to consider only motifs that are slightly modified. Insertions were so rare in the data analysed in this study that by the time an additional k individual nucleotides have been inserted into a motif of length k, it is desirable for the proposed algorithm to be looking at a potential pair of motifs instead of one highly mutated motif. If an STR DNA sequence is gaining or losing $2k$ characters at once, it is more likely due to two separate stutters than to a single stutter that was then affected by an additional k inserts or deletes. Furthermore, in this data set, stutter and substitutions are much more common than deletions, and all are more common than insertions, giving more reason to cap the string length.

These lists of strings and their associated costs are computationally intensive to create (although still generally under a minute on a personal laptop), but once one is made (stored as a dictionary in Python for fast lookups) it can be used any number of times, as long as the motif and costs stay the same. Expanding this dictionary for completeness could be another area for further research.

2.3.5. Multiple Motifs and Time Complexity

Some forensic DNA loci have multiple motifs (e.g., [ACT]5 [AGGT]12). A pair of stutter dictionaries is computed for every motif, and at each step in the RFL algorithm the dictionaries of each motif are checked. This allows for any number and combination of preset motifs to be included in the calculation, each with fast dictionary lookups.

The proposed algorithm allows any prespecified set of motifs. For every additional motif, two inner for-loops are added at each step in the dynamic programming matrix to check the cost of inserting and deleting anywhere from 1 to $2k_i-1$ characters from the dictionary associated with that motif of length $k_i$. Because there are two for-loops running through the list of all motifs, and within each of them there is another for-loop running through 1 to $2k_i-1$ characters for motif i, the algorithm grows in complexity by those factors. Where the classical Levenshtein distance is $O\left(n_a{n}_b\right)$ for strings of length $n_a$ and $n_b$, the RFL distance theoretically requires $O\left(n_a{n}_{b}mk\right)$ time, where m is the number of motifs and k is the length of the largest motif $max\left(k_i\right)$, assuming $O\left(1\right)$ dictionary lookup speed. However, in applications with standard autosomal STRs, motifs are all of length at most 5 ($2k_i-1\le 9$), so this algorithm can be considered as $O\left(n_a{n}_{b}m\right),$ growing with the number of motifs considered as well as the lengths of the strings. Additionally, standard autosomal STRs have no more than a handful of motifs to be considered—a maximum of three, in this work—so the asymptotic growth becomes $O\left(n_a{n}_b\right)$ once more.

2.4. Implementation of the RFL Distance

Consider two strings, a and $b,$ where $\left|a\right|=m$ and $\left|b\right|=n,$ with zero-based indexing used throughout. The goal of the RFL algorithm is to construct an $(m+1)\times (n+1)$ matrix d, where each $d[i,j]$ is equal to the distance from the first i characters of a to the first j characters of $b,$ resulting in the output $d[m,n].$ Recall that the distance is not commutative, so “parent” refers to a and “child” refers to b (since the string b originally began at a and went through edits to turn into b).

Throughout this section, recall the mechanics behind Example 1. The first characters of the parent and child are aligned, and the algorithm then modifies the parent. With each edit the parent is modified to make the leading characters of both strings match, which characters can then be removed for free, and thus the modifications eventually lead to a pair of empty strings and the cost of each step tallied.

2.4.1. Building the First Row

The cost of editing an empty string into an empty string is 0, so $d[0,0]$ is always 0. The element $d[0,1]$ is the cost of editing an empty string into the first character of the child string $b.$ A list of cost options at this step will be made and then the minimal cost option taken. Obtaining the first character can either be done by inserting it directly or by stuttering forward a motif and then deleting characters back. For example, if the first character of the string b was G, and the motifs being considered were GGC and ATTC, then the following costs would be appended to the options list:

• The insert cost of the character G
• The cost of stuttering forward GGC plus the cost of the weighted Levenshtein distance from GGC to G.
• The cost of stuttering forward ATTC plus the cost of the weighted Levenshtein distance from ATTC to G.

Each motif can have a different stutter cost associated with it that can vary across loci, and in forensic applications forward stutter will be more expensive than backward since it is a rarer artifact. In this example, GGC might cost 3 to stutter forward and 1 to stutter backward, while ATTC might cost 2 to stutter forward and 0.5 to stutter backward.

The quantity $d[0,1]$ is the minimum of the computed options, and the algorithm moves forward to $d[0,2].$ Now there are several possibilities:

• $d[0,1]$ + the cost of inserting the second character via pure insertion (i.e., the cost of inserting $b\left[1\right]$, using 0-based indexing)
• $d[0,1]$ + the cost of inserting the second character via the stutter dictionary (for each stutter dictionary)
• $d[0,0]$ + the cost of inserting the first two characters via the stutter dictionary (for each stutter dictionary)

Furthermore, in general, for each motif, if the motif lengths are $k_i$, for every length j less than or equal to $2k_i-1$, substrings of characters from j positions back to the current can be checked, and if less costly those characters can be inserted via the stutter dictionary. It is impossible to look back further than the beginning of the string, however.

To construct the first row $d[0,:]$, pseudocode outlined in Listing 1 is followed. Building the first column $d[:,0]$ follows similarly.

Listing 1. Building the first row of the dynamic programming matrix in the RFL algorithm.

2.4.2. Filling Out the Matrix

This step combines the complexity of building the first row and column. At each point $d[i,j]$, the algorithm looks back at the previous entries $d[i-1,j],d[i,j-1],$ and $d[i-1,j-1]$ and adds the costs of deletion, insertion, and substitution, respectively. Additionally, for each motif length, it looks back at $d[i-r,j]$ and $d[i,j-r]$, for reverse and forward stutter, respectively, for r from 1 to $2k_i-1$ for each motif i of length $k_i$. After considering all these costs, the entry $d[i,j]$ is the minimum.

In Listing 2, $d[m-1,n-1]$ is the distance from the first $m-1$ characters of the parent to the first $n-1$ characters of the child. Because of zero-based indexing, parent[$m-1$] is the m-th character of the parent.

Listing 2. Building the dynamic programming matrix in the RFL algorithm.

To demonstrate the RFL algorithm in depth, a fully worked example is given in Appendix A.2.

2.4.3. Practical Implementation

The RFL algorithm is written in Python, compatible with NumPy and therefore Numba, which increases the speed over the same algorithm in pure Python by a factor of about 30 via an @njit wrapper. However, the Numba-optimized version of the algorithm only works if including the possibility of stuttering a single motif, due to limitations in the implementation with Numba’s inherent dictionary types. Based on testing of the weighted-levenshtein package (written in Cython) [32], the RFL algorithm has potential for significant increases in speed. In preliminary testing, modifying the data structures to integer-coded arrays and rearranging the way the cost dictionaries are stored can accelerate the computations by more than an order of magnitude via Numba. Furthermore, preliminary testing of a Numba prototype for the base Levenshtein algorithm suggests that in the case of DNA, Numba can meet or exceed the speed of the generic Cython package.

2.5. Validation of the Algorithm

The ideas in Section 4 of [20] can be adapted to provide theoretical justification for the RFL distance. Ukkonen’s proof is for when all of the string edits can be performed in parallel, i.e., no edit interactions such as a forward stutter followed by a substitution within the stuttered motif (e.g., ‘’ → AATT → AACT). The RFL distance is specifically designed to address edit interactions that cannot be performed in parallel. However, with the lsdp cost dictionaries in place, the larger family of dictionary edits can once again be performed in parallel because the edit interactions are already built in, and Ukkonen’s justification holds, although their implementation is different and no code was available for comparison testing.

Various examples have been computed to verify the RFL distance behaves as it should with different costs when compared to the Cython package [32].

3. Results and Discussion

An overarching goal of this work was to develop a metric that was inversely proportional to sequence relative frequency in MPS data analysis output. A demonstration of the effect of the RFL distance algorithm using the result from

Table 2. High-stutter motifs at CODIS-20 and PENTA loci, based on 661 individuals, using a threshold of 0.167 Allele Coverage Ratio (ACR)—the ratio of the second-highest count to the highest—to determine homo- vs. heterozygosity in identifying true alleles. For each motif, LUS summary statistics are taken across all alleles. Back stutter rate is the average of all single-stutter proportions resulted from amplification and sequencing, averaged across all loci where parents with that motif were present. The last two columns describe how many alleles and unique alleles with the relevant motif at each locus were present in the data. This analysis is described further in Section 3.1.

Locus	Motif	Median LUS	Min LUS	Max LUS	Mean on-LUS Back Stutter Rate	Total Parents at Locus with Motif	Unique Parents at Locus with Motif
CSF1PO	TCTA	11	4	15	0.0219 (163)	1159	17
D10S1248	GGAA	14	8	19	0.0291 (190)	1182	13
D12S391	TAGA	12	7	19	0.0279 (115)	1287	84
D12S391	CAGA	6	3	10	0.0085 (67)	1287	84
D13S317	TATC	12	7	16	0.0183 (105)	1219	30
D13S317	AATC	2	2	3	0.0021 (56)	803	10
D16S539	GATA	11	5	14	0.0214 (123)	1207	17
D18S51	AGAA	15	9	24	0.0249 (125)	1242	27
D19S433	TCCT	13	7	18	0.0218 (133)	1194	22
D1S1656	TATC	13	9	17	0.0332 (141)	1270	30
D1S1656	AC	6	5	6	0.0059 (26)	1270	30
D21S11	TATC	12	7	15	0.0224 (96)	1264	79
D21S11	TGTC	6	4	8	0.0021 (21)	1264	79
D22S1045	ATT	12	5	16	0.0316 (233)	1179	13
D2S1338	GGAA	13	8	17	0.0221 (80)	1269	69
D2S1338	GGCA	7	3	9	0.0033 (34)	1269	69
D2S441	CTAT	11	7	14	0.0188 (114)	1205	24
D3S1358	CTAT	13	5	17	0.0309 (143)	1238	26
D3S1358	CTGT	2	2	4	0.0035 (85)	987	19
D5S818	ATCT	12	7	15	0.0257 (119)	1237	31
D7S820	CTAT	10	6	13	0.0163 (96)	1229	30
D8S1179	CTAT	12	8	15	0.0251 (115)	1237	34
D8S1179	CTGT	2	2	3	0.0013 (51)	20	6
FGA	GAAA	14	9	19	0.0211 (109)	1247	32
Penta D	GAAAA	11	5	17	0.0051 (47)	1236	18
Penta E	TTTTC	12	5	25	0.0114 (89)	1240	25
TH01	AATG	7	5	11	0.008 (63)	1165	12
TPOX	AATG	9	5	13	0.0113 (90)	1140	14
VWA	ATAG	12	3	16	0.0247 (146)	1242	32
VWA	ACAG	4	3	6	0.0023 (50)	1242	32
VWA	GATG	3	3	4	0.0019 (8)	81	2

is shown in Figure 2.

The non-monotonic plot in Figure 2 has a peak in frequency at a distance of 4 from the true parent sequence in the file, corresponding to stutter of an entire tetranucleotide motif being a more common occurrence than deleting 3 or 5 characters individually. The RFL distance counts this stutter with a cost of 1, thus giving monotonicity. In loci with motifs of other lengths, the frequency peak appears at that particular length. In loci exhibiting multiple motifs, there are increases in frequency at both lengths. There are smaller spikes at a distance of twice the motif length (a distance of 8 in Figure 2), corresponding to double stutter, but they are much less noticeable.

This metric also has applications in deconvolution, as shown in the following example of real data in a known mixture of two people at the locus D8S1179. According to Table 2, the RFL algorithm uses the motifs CTAT and CTGT. Two true alleles at this locus are $p_1$ = [CTAT]12 and $p_2$ = [CTAT]2 CTGT [CTAT]10. The artifact a = [CTAT]2 CTGT [CTAT]9 also appears in the results. It is desirable to capture sequence dissimilarity corresponding to decreased likelihood of an artifact being from a given allele. Using the Levenshtein distance with unit costs, the distance from $p_1$ to a is equal to 1, since the only edit required is changing the third A to a G. The Levenshtein distance from $p_2$ to a is equal to 4, since the motif CTAT must be removed letter-by-letter. However, when using the RFL distance, CTAT can be removed for a cost of 1, so then the distances between the artifact and each true allele are both equal to 1, and the deconvolution is less clear. Future work will fit costs to the data that vary by edit type to more confidently infer which parent was more likely the origin for the artifacts in cases such as this.

Another way to compare the two metrics is by visualizing the pairwise distances of output in two dimensions. Uniform Manifold Approximation and Projection (UMAP) is one method of performing this dimension reduction technique [33]. The UMAP algorithm was applied to all pairwise RFL distances and original Levenshtein distances of a 3:1 mixture at locus D1S1656. The locus is typed with three true alleles, with the higher contributor being homozygous and contributing three times as much DNA as the lower contributor, who was heterozygous for this locus. The RFL distance showed more defined clustering than classical Levenshtein across many combinations of hyperparameters, supporting the notion that the RFL distance captures PCR artifact similarity better than classical Levenshtein. Figure 3 shows a plot made with a representative combination of hyperparameters, where each artifact is colored by which parent it is closest to (there were no ties).

3.1. Selecting Motifs for Each Locus

The RFL algorithm requires a motif or set of motifs to include as options to lose or gain. The original motivation for the RFL algorithm was to address the high rates of stutter, particularly back stutter, that appear in sequencing data analysis output. In order to give the algorithm the correct motifs to use, the available true allele sequences were exhaustively searched for any motif of length 2–6 that repeated at least twice consecutively.

For every locus, and for every potential motif identified at that locus, the back stutter rates were computed for all available files by taking the count of the stuttered artifact sequence divided by the sum of all reads for the locus. The results were then filtered to include only loci that showed back stutter at a mean rate of at least 0.001 across samples. Major stutter is the loss of a motif that decreases the longest uninterrupted stretch (LUS), i.e., stutter that results in the maximum number of consecutive motifs being reduced. Similarly, minor stutter is the loss of a motif that does not decrease the LUS. This differentiation was made because stutter in a LUS region was more common than stutter outside of a LUS region by several orders of magnitude in the data. Many places in the data showed stretches of a motif at locations other than the LUS region. For example, in every true allele at the locus CSF1PO, a locus with only one STR motif, there were two other locations of that motif repeating in lower numbers than the LUS in the data.

Motifs were grouped by self-permutations, e.g., if ATAG consecutively repeated at a sequence at least twice, usually TAGA and AGAT would as well. Within each group of self-permutations the motif with the highest mean LUS was chosen, using alphabetization to break ties. The final step was to take the motif at each locus with the largest mean LUS (there were no ties). This final candidate was chosen as the primary motif at the locus—in essence, the motif that had the highest mean stutter rate at that locus.

Next, every primary motif in every sequence in the data was substituted with the non-nucleotide letter Z, and the entire motif-finding process was repeated to look for secondary motifs—the motifs that stutter second-most at each locus. This mechanic of replacing the primary motifs with the letter Z ensured there was no overlap of primary and secondary motifs.

This process was iterated until no motifs with high stutter rates and LUS remained. Only VWA had candidate motifs for the tertiary round, and no loci had candidates for quaternary motifs.

Combining the three rounds of processing gives Table 2. Within each locus, the rows are sorted in order of the rounds of analysis (and thus in order of stutter frequency).

4. Conclusions

A new string dissimilarity measure is proposed with a flexible and publicly available algorithm for implementing it. It has shown promise in ongoing forensic applications thus far, but generalized Levenshtein with multi-character edits could have uses with in vivo sequences as well. Applications in graph edit distance and consequent potential uses in handwriting recognition, fingerprint recognition, and cheminformatics were also discussed. Allowing the interaction of edit types—such as a motif stuttering forward, then undergoing indels or substitutions within, before, or after that motif–is a novel addition that expands the useful applications of this work, and is accomplished via the dictionary structure in a way that is fast, feasible, and open to further speed optimization. It allows for a more accurate picture of the stutter phenomenon and how similar two STR sequences truly are when any stutter is expected, whether from PCR or in vivo polymerase slippage.

In practice, this work and the associated code available can benefit those analyzing STR sequencing data, manufacturers and developers providing software tools to accurately recognize and label sequences from these data types, thus, indirectly contributing to reliable reporting of STR sequence data, both in the forensic genetics and the medical diagnosis fields where accuracy in interpretation of STR sequence data is crucial. To count the number of edit operations to get from one sequence to another, set all edit costs to one (this will be symmetric, since every operation will have an inverse of equal cost). This is a useful first pass on sequence similarity. Optimizing the costs to reflect the frequency of the edits in the data requires a kit-specific understanding of artifact frequencies, since those would be used to determine the relative costs of each operation. Applications such as sequence alignment may use costs motivated by traits other than pure artifact frequency, depending on the information desired.

Further work could be done to penalize the overall number of edit operations in addition to the cost of each individual operation. It would also be a simple addition to consider transpositions as a separate possible edit operation, though for the original application it was unnecessary.

The restrictions in place because of the stutter dictionary have proven helpful for several reasons, but do limit effectiveness in rare cases. A motif inserting in the middle of another motif, for example, would not be detected by the RFL algorithm, and neither would a motif spread apart by too many single-character intra-motif insertions (len(motif)$-1$ insertions, in particular). Although this limit is easily changeable in the code, the way the algorithm is written requires a hard cap. For forensic applications, however, when the strings are only a few edits apart and insertions are rare, the restrictions do not limit effectiveness.

5. Disclaimer

Any commercial equipment, instruments, materials, or software identified in this paper are to foster understanding only. Such identification does not imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the materials, equipment, or software identified are necessarily the best available for the purpose.