Complete Subhedge Projection for Stepwise Hedge Automata ^†

Abstract

We demonstrate how to evaluate stepwise hedge automata (Shas) with subhedge projection while completely projecting irrelevant subhedges. Since this requires passing finite state information top-down, we introduce the notion of downward stepwise hedge automata. We use them to define in-memory and streaming evaluators with complete subhedge projection for Shas. We then tune the evaluators so that they can decide on membership at the earliest time point. We apply our algorithms to the problem of answering regular XPath queries on Xml streams. Our experiments show that complete subhedge projection of Shas can indeed speed up earliest query answering on Xml streams so that it becomes competitive with the best existing streaming tools for XPath queries.

1. Introduction

Hedges are sequences of letters from the alphabet and trees $\langle h\rangle$ where h is again a hedge. Hedges are abstract from the syntactical details of Xml or JSON documents while still being able to represent them. The linearization of a hedge is a nested word that can be written in a file. In this article, we study the problem of hedge pattern matching, i.e., whether a given hedge h belongs to a given regular hedge language L. Regular hedge languages can be defined in XPath or JSONPath in particular.

Projection is necessary for the efficiency of many algorithms on words, trees, hedges, or nested words. Intuitively, an algorithm is projecting if it visits only a fragment of the input structure: in the best case, the part that is relevant to the problem under consideration. The relevance of projection for Xml processing was already noticed by [1,2,3,4]. Saxon’s in-memory evaluator, for instance, projects input Xml documents relative to an Xslt program, which contains a collection of XPath queries to be answered simultaneously [5]. The QuiXPath tool [4] evaluates XPath queries in streaming mode with subtree and descendant projection. Projection during the evaluation of JSONPath queries on JSON documents in streaming mode is called fast-forwarding [6].

Projecting in-memory evaluation assumes that the full graph of the input hedge is constructed beforehand. Nevertheless, projection may still save time if one has to match several patterns on the same input hedge or if the graph was constructed for different reasons anyway. In streaming mode, the situation is similar: the whole input hedge on the stream needs to be parsed, but the evaluators need to inspect only nodes that are not projected away. Given that pure parsing is by two or three orders of magnitude faster than pattern evaluation, the time gain of projection may be considerable.

The starting point of this article is the notion of the subhedge irrelevance of positions of hedge h with respect to a hedge language L that we introduce. These are positions of h where, for all possible continuations, inserting any subhedge does not affect membership to L. We contribute an algorithm for hedge pattern matching with complete subhedge projection. Our algorithm decides hedge pattern matching while completely projecting away the subhedges located at irrelevant subhedge positions. In other words, it decides whether a hedge h matches a pattern L without visiting any subhedge of h that is located at a position that is irrelevant with respect to L.

Regular hedge languages can be represented by nested regular expressions, which can be derived from regular XPath queries [7] or by some kind of hedge automata, which can be compiled from nested regular expressions. We use stepwise hedge automata (Shas) [8], a recent variant of standard hedge automata, which, in turn, date back to the 1960s [9,10]. Shas mix up the bottom-up processing of standard tree automata with the left-to-right processing of finite word automata (Dfas). They neither support top-down processing nor have an explicit stack, unlike nested word automata Nwas [11,12,13,14,15]. Shas have a good notion of bottom-up and left-to-right determinism, avoiding the problematic notion of determinism for standard hedge automata and the too-costly notion of determinism for Nwas. Furthermore, compilers from nested regular expressions to Shas are available [8], and determinization algorithms [16] produce small Shas for all regular XPath queries in the existing practical benchmarks [7,17].

The motivation for the present work is to add subhedge projection to a recent streaming algorithm for deterministic Shas that can evaluate regular XPath queries in the earliest manner. Most alternative streaming approaches avoid earliest query answering by considering sublanguages of regular queries without delays so that node selection depends only on the past of the node but not on the future [18,19] or by admitting late selection [20]. In particular, it was recently demonstrated that the earliest query answering for regular monadic queries defined by deterministic Shas [21] has a lower worst-case complexity than for deterministic Nwas [22]. Thereby, the earliest query answering for regular queries became feasible in practice for the first time. On the other hand side, it is still slower experimentally than the best existing non-earliest approaches for general regular monadic queries on Xml streams (see [23] for a comparison) since the latter supports projection. How projections could be made for Shas has not been studied before the conference version of the present article [24].

Consider the example of the XPath filter $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$ that is satisfied by an Xml document if its root is a list element that has an item child. With the encoding of the Xml document as hedges are chosen in the present article, satisfying hedges have the form $\langle \mathtt{list}·h_1·\langle \mathtt{item}·h_2\rangle ·h_3\rangle ·h_4$, and for some hedges, $h_1,h_2,h_3,h_4$. When evaluating this filter on a hedge, it is sufficient to inspect the first subtree for having the root label $\mathtt{list}$ and then all its children until one with a root label $\mathtt{item}$ is found. The subhedges of these children, i.e., the hedge $h_3$ and the subhedges of the top-level trees in $h_1$ and $h_3$, can be projected away, as well as the hedge $h_4$: they do not have to be inspected for evaluating this filter. However, one must memorize whether the level of the current node is 0, 1, or greater. This level of information can be naturally updated in a top-down manner. The evaluators of Shas, however, operate bottom-up and left-to-right exclusively. Therefore, projecting evaluators for Shas need to be based on more general machines. We propose downward stepwise hedge automata (Sha^↓s), a variant of Shas that supports top-down processing. They are basically Neumann and Seidl’s pushdown forest automata [25], except that they are applied to unlabeled hedges instead of labeled forests. Furthermore, Nwas are known to operate in basically the same manner on nested words [26] while allowing for slightly more general visible pushdowns. We then distinguish subhedge projection states for Sha^↓s, and show how to use them to evaluate Shas with subhedge projection both in memory and in streaming mode.

As a first contribution, we present the safe-no-change compiler from Shas to Sha^↓s that can distinguish appropriate subhedge projection states. The idea is that the safe-no-change Sha^↓ can distinguish contexts in which the states of the Sha will safely not change. For instance, the XPath filter $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$ can be defined by a deterministic Sha operting bottom-up which our compiler maps to the Sha^↓ operating top-down. The context information made explicit by top-down processing is about the levels of the states. This permits us to distinguish projection states starting from level 2, and in which subhedges can be ignored. We prove the soundness of our compiler based on a nontrivial invariant that we establish (we note that the proof required an adaptation of the original compiler from FCT [24]). We also present a counter-example showing that the safe-no-change projection is not complete for subhedge projection. It shows that a subhedge may be irrelevant even though its state may still be changing.

As a second contribution, we propose the congruence projection algorithm. It again compiles Shas to Sha^↓s but relies on the congruence relations of automata states. We then prove that congruence projection yields not only a sound algorithm but that this algorithm is also complete for subhedge projection, i.e., all strongly irrelevant subhedges (see Definition 9) are evaluated into a looping state. Congruence projection starts on the top-level of hedges with the Myhill–Nerode congruence that is well-known from automata minimization.

For languages with words L, this congruence identifies prefixes v that have the same residual language $v^{-1}\left(L\right)=\{w\mid v·w\in L\}$. A prefix v is then an irrelevant suffix if its residual language $v^{-1}\left(L\right)$ is either universal or empty. In the case of hedges, which extend on words by adding hierarchical nesting via a tree constructor, the congruence must be adapted when moving down into subtrees.

We show that both compilers may increase the size of the automata exponentially since they must be able to convert bottom-up deterministic automata on monadic trees into top-down deterministic automata. That is the same as inverting deterministic automata on words, which is well known to raise an exponential blow-up in the worst case (e.g., for the family of languages ${(a+b)}^{\ast }·a·{(a+b)}^n$, where $n\in \mathbb{N}$, the minimal deterministic left-to-left automaton, has $2^{n+1}$ states while the minimal deterministic right-to-left automaton has $n+1$ states).

The exponential explosion can be avoided when interested in the evaluation of hedges by automaton with subhedge projection, by not constructing the complete Sha^↓s statically. Instead, only the required part of the Sha^↓s may be constructed on the fly when evaluating a hedge.

Our third contribution is the refinement of the two previous compilers so that they can also distinguish safe states for selection at the earliest position. For this, we combine these compilers with a variant of the earliest membership tester of [21] that operates in memory by compiling Sha^↓s instead of Nwas. Furthermore, membership failure is detected at the earliest position, too. In this way, we obtain the earliest in-memory membership tester for deterministic Shas.

Our fourth contribution is to show how to run the previous in-memory evaluators in streaming mode while reading the linearization of the hedge into a nested word as an input. Thereby, we can improve on the recent earliest streaming membership tester behind the AStream tool [21] by adding subhedge projection to it.

Our fifth and last contribution is an implementation and experimental evaluation of the streaming earliest query answering algorithm for dShas by introducing subhedge projection into the AStream tool [21]. We implemented both safe-no-change and congruence projection. For this, we lifted our earliest membership tester with subhedge projection to an earliest query answering algorithm for monadic queries. For the experimentation, we started with deterministic Shas constructed with the compiler from [8] for the forward regular XPath queries of the XPathMark benchmark [27] and real-world XPath queries [7]. It turns out that congruence projection projects much more strongly than the safe-no-change projection for at least half of the benchmark queries. It reduces the running time for all regular XPath queries considerably since large parts of the input hedges can be projected away. In our benchmark, the projected percentage ranges from 75.7% to 100% of the input stream. For XPath queries that contain only child axes, the earliest query answering algorithm of AStream with congruence projection becomes competitive in efficiency with the best existing streaming tool called QuiXPath [23], which is not always earliest for some queries. Our current implementation of the earliest congruence projection in AStream is by a factor from 1.3 to 2.9 slower than QuiXPath on the benchmark queries. The improvement is smaller for XPath queries with the descendant axis, where less subhedge projection is possible. Instead, some kind of descendant projection would be needed. Still, even in the worst case in our benchmark, the earliest congruence projection with AStream is currently only by a factor of 13.8 slower than QuiXPath.

• Outline. We start with preliminaries on hedges and nested words in Section 2. In Section 3, we recall the problem of hedge pattern matching and formally define irrelevant subhedges. In Section 4, we recall the definition of Shas, introduce Sha^↓s, and discuss their in-memory evaluators. In Section 5, we introduce the notion of subhedge projection states, define when an Sha^↓ is complete for subhedge projection, and present an in-memory evaluator for Sha^↓s with subhedge projection. In Section 6, we introduce the safe-no-change projection, state its soundness, and show its incompleteness. In Section 7, we introduce congruence projection an prove it to be sound and complete for subhedge projection. The earliest in-memory evaluators for Sha^↓s with subhedge projection follow in Section 8. Streaming variants of our in-memory evaluators are derived systematically in Section 9. Section 10 discusses how to lift our algorithms from membership testing to monadic queries. Section 11 discusses our practical experiments. Section 12 discusses further related work on automata notions related to Shas and Sha^↓s. Appendix A presents the reformulation of the earliest query-answering algorithm from [21] based on Sha^↓s and with schema restrictions. Appendix B provides sound proof for the safe-no-change projection.

• Publication Comments. This original version of the journal article was published at the FCT conference [24]. Compared to the previous publication, the following contributions are new:
  • The definition of subhedge irrelevant prefixes of nested words and the definition of completeness for subhedge projection (Section 3).
  • The addition of schema restrictions throughout the paper and the notion of schema completeness (Section 4.3).
  • The notion of completeness for subhedge projection (Section 5.2).
  • The soundness proof of the safe-no-change projection algorithm in Section 6.2, moved to Appendix B.
  • The congruence projection algorithm and the proof of its soundness and completeness for subhedge projection (Section 7).
  • A systematic method to add subhedge projection to an earliest Sha^↓ (Section 8).
  • A systematic method to reduce the correctness of streaming automata evaluators to the corresponding in-memory evaluators (Section 9).
  • A discussion of how to deal with monadic queries while exploiting the availability of schema restrictions (Section 10).
  • An implementation of congruence projection with experimental results for XPath evaluation on Xml streams (Section 11).
  • A longer discussion on related work (Section 12).
  • In Appendix A, we also show how to obtain the earliest dSha^↓s from dShas by adapting the compiler from [21] mapping dShas to the earliest dNwas.

2. Preliminaries

Let A and B be sets. If A is a subset of B, then the complement of A in B is denoted by $\overline{A}=B\backslash A$ while keeping the set relative to which the complement is taken implicit. The domain of the binary relation $r\subseteq A\times B$ is $\mathit{dom}\left(r\right)=\{a\in A\mid \exists b\in B.(a,b)\in r\}$. A partial function $f:A\hookrightarrow B$ is a binary relation $f\subseteq A\times B$ that is functional. A total function $f:A\to B$ is a partial function $f:A\hookrightarrow B$ with $\mathit{dom}\left(f\right)=A$.

• Words. Let $\mathbb{N}$ be the set of natural numbers, including 0. Let the alphabet $\Sigma$ be a set. The set of words over $\Sigma$ is ${\Sigma }^{\ast }={\cup }_{n\in \mathbb{N}}{\Sigma }^n$. A word is $(a_1,\dots ,a_n)\in {\Sigma }^n$, where $n\in \mathbb{N}$ is written as $a_1\dots a_n$. We denote the empty word of length 0 by $\epsilon \in {\Sigma }^0$, and by $v_1·v_2\in {\Sigma }^{\ast }$ the concatenation of two words $v_1,v_2\in {\Sigma }^{\ast }$.

If $v=u·v^{\prime }·w$ is a word, then we call u a prefix of v and $v^{\prime }$ a factor of v and w a suffix of v. Given any subset $L\subseteq {\Sigma }^{\ast }$, we denote the set of prefixes of words in L by $\mathit{prefs}\left(L\right)$ and the set of suffixes of words in L by $\mathit{suffs}\left(L\right)$.

For any subalphabet ${\Sigma }^{\prime }\subseteq \Sigma$, the projection of a word $v\in {\Sigma }^{\ast }$ to ${\Sigma }^{\prime }$ is the word ${\mathit{proj}}_{{\Sigma }^{\prime }}\left(v\right)$ in ${\left({\Sigma }^{\prime }\right)}^{\ast }$ that is obtained from v by removing all letters from $\Sigma \backslash {\Sigma }^{\prime }$.

• Hedges. Hedges are sequences of letters and trees $\langle h\rangle$ with a hedge h. More formally, a hedge $h\in {\mathcal{H}}_{\Sigma }$ has the following abstract syntax:

  $$\begin{array}{ccc}\hfill \mathit{h},{\mathit{h}}^{\prime }\in {\mathcal{H}}_{\Sigma }& ::=& \epsilon \left|a\right|\langle \mathit{h}\rangle |\mathit{h}·{\mathit{h}}^{\prime }\mathrm{where}a\in \Sigma \hfill \end{array}$$

  We assume $\epsilon ·h=h·\epsilon =h$ and $(h·h_1)·h_2=h·(h_1·h_2)$. Therefore, we consider any word in ${\Sigma }^{\ast }$ as a hedge in ${\mathcal{H}}_{\Sigma }$, i.e., ${\Sigma }^{\ast }\ni aab=a·a·b\in {\mathcal{H}}_{\Sigma }$. Any hedge can be converted or stored as a graph. For the signature $\Sigma =\{list,item,A,\dots ,Z\}$, the graph of an example hedge in ${\mathcal{H}}_{\Sigma }$ is shown in Figure 1. It encodes the sequence of Xml documents in Figure 2:

• Nested Words. A nested word over $\Sigma$ is a word over the alphabet $\widehat{\Sigma }=\Sigma \cup \left\{\langle ,\rangle \right\}$ in which all parentheses are well-nested. Intuitively, this means that for any opening parenthesis, there is a matching closing parenthesis and vice versa.

The set of nested words over $\Sigma$ can be defined as the subsets of words over $\widehat{\Sigma }$ that are a linearization of the hedge, where the linearization function $\mathit{nw}:{\mathcal{H}}_{\Sigma }\to {(\Sigma \cup \left\{\langle ,\rangle \right\})}^{\ast }$ is defined as follows:

$$\begin{array}{cccc}\hfill \mathit{nw}\left(\epsilon \right)=\epsilon ,& \hfill \mathit{nw}(\langle \mathit{h}\rangle )=\langle ·\mathit{nw}\left(\mathit{h}\right)·\rangle ,& \hfill \mathit{nw}\left(a\right)=a,& \mathit{nw}(\mathit{h}·{\mathit{h}}^{\prime })=\mathit{nw}\left(\mathit{h}\right)·\mathit{nw}\left({\mathit{h}}^{\prime }\right).\end{array}$$

So, the set of all nested words over $\Sigma$ is $\mathit{nw}\left({\mathcal{H}}_{\Sigma }\right)$. Let $\mathit{hdg}$ be the inverse of the injective function $\mathit{nw}$ restricted to its image, so the mapping from nested words to hedges with $\mathit{hdg}\left(\mathit{nw}\right(h\left)\right)=h$ for all $h\in {\mathcal{H}}_{\Sigma }$.

• Nested Word Prefixes. Prefixes, suffixes, and factors of nested words may not be nested words themselves. For instance, the hedge $h=a·\langle b·c\rangle$ has the linearization $\mathit{nw}\left(h\right)=a\langle bc\rangle$. Its prefix $a\langle b$ is not well-nested since it has a dangling opening parenthesis. Its suffix $c\rangle$ is also not well-nested since it has a dangling closing parenthesis.

Any algorithm that traverses a hedge h in a top-down and left-to-right manner inspects all the prefixes of $\mathit{nw}\left(h\right)$. Any prefix v of $\mathit{nw}\left(h\right)$ not only distinguishes the position of the hedge h but also specifies the part of the hedge that is located before this position in the linearization $\mathit{nw}\left(h\right)$. An example is illustrated graphically in Figure 3. This particularly holds for streaming algorithms that receive the nested word $\mathit{nw}\left(h\right)$ as an input on a stream and may be inspected only once from the left to the right. But it holds equally for in-memory algorithms that receive the input hedge h as a hierarchical structure whose graph is stored in memory and then traverses the graph top-down and left-to-right. Any $\langle \rangle$ node will then be visited twice: once when reading an opening parenthesis 〈 and going down into a subhedge, and another time when going up to the closing parenthesis 〉 after having processed the subhedge.

3. Problem

We formally define the pattern-matching problem for hedges and the notion of subhedge irrelevance, which allows us to characterize the parts of the input hedge that do not need to be visited during pattern matching.

3.1. Hedge Pattern Matching

We are interested in algorithms that match a hedge pattern against a hedge. We start with algorithms that test whether such matching exists. This is sometimes called filtering or, alternatively, Boolean query answering. In Section 10, we consider the more general problem of returning the set of matchings. This problem is sometimes called monadic query answering. Our experiments in Section 11 apply monadic query answering for regular XPath queries.

We start from hedge patterns that are nested regular expressions with letters in the signature $\Sigma$. For these, we fix a set V of recursion variables and assume that it is disjointed from $\Sigma$.

$$e,e^{\prime }\in {\mathit{nRegExp}}_{\Sigma }::=\epsilon \mid a\mid z\mid e·e^{\prime }\mid e+e^{\prime }\mid e^{\ast }\mid \langle e\rangle \mid \mu z.e\mathrm{where}a\in \Sigma ,z\in V$$

Each nested regular expression $e\in {\mathit{nRegExp}}_{\Sigma }$ is satisfied by a subset of hedges in $〚e〛\in {\mathcal{H}}_{\Sigma \cup V}$. We recall the definition of this set only informally. The expression $\epsilon$ is satisfied only by the empty word, expression $a\in \Sigma$ only by the hedge a only, and expression $z\in V$ only by the one letter hedge z. An expression $e·e^{\prime }$ with the concatenation operator and $e,e^{\prime }\in {\mathit{nRegExp}}_{\Sigma }$ is satisfied by the concatenations of hedges satisfying e and $e^{\prime }$, respectively. An expression $e^{\ast }$ with Kleene’s star operator and $e\in {\mathit{nRegExp}}_{\Sigma }$ is satisfied by repetitions of words satisfying e. The Kleene star provides for horizontal recursion on hedges. The $\mu$ expression $\mu z.e$, where $z\in V$ and $e\in {\mathit{nRegExp}}_{\Sigma }$, is satisfied by all hedges without recursion variable x in which z is repeatedly replaced by a hedge satisfying e until it disappears. We adopt the usual restriction [28] that in any nested regular expression $\mu z.e$, all occurrences of z in e are guarded by a tree constructor $\langle .\rangle$. In this way, the $\mu$-operator provides proper vertical recursion. Also note that, for defining sublanguages of ${\mathcal{H}}_{\Sigma }$, we only need such expressions in ${\mathit{nRegExp}}_{\Sigma }$ in which all occurrences of recursion variables z are bound in the scope of binder $\mu z$.

It is well known that nested regular expressions in ${\mathit{nRegExp}}_{\Sigma }$ can define all regular sublanguages of hedges in ${\mathcal{H}}_{\Sigma }$, i.e., they have the same expressiveness as hedge automata with alphabet $\Sigma$. Analogous results for regular tree expressions and tree automata are folklore [29].

Example 1 (Nested regular expressions of regular XPath filters).

For instance, we can define subsets of hedge encodings of Xml documents with two kinds of elements in $\Sigma =\{\mathtt{list},\mathtt{item}\}$ by expressions in ${\mathit{nRegExp}}_{\Sigma }$. For this article, we will use an encoding of the Xml document, which produces hedges that satisfy the following regular expression $\mathtt{doc}\in {\mathit{nRegExp}}_{\Sigma }$, where $z\in V$:

$$\mathtt{doc}{=}_{\mathit{def}}{\mathit{tree}}^{\ast }\mathrm{where}\mathit{tree}{=}_{\mathit{def}}\mu z.\langle (\mathtt{list}+\mathtt{item})·z^{\ast }\rangle$$

In our application, the encodings are a little richer in order to distinguish different types of Xml nodes such as elements, attributes, comments, and texts. An XPath filter such as [self::list/child::item] can then be expressed by the nested regular expression:

$$\mathit{filter}{=}_{\mathit{def}}\langle \mathtt{list}·\mathtt{doc}·\langle \mathtt{item}·\mathtt{doc}\rangle ·\mathtt{doc}\rangle ·\mathtt{doc}$$

General compilers from regular downward XPath filter queries to nested regular expressions were proposed in [8]. An implementation was developed in the context of the AStream tool [21].

We will use schemas $S\subseteq {\mathcal{H}}_{\Sigma }$ to restrict the inputs of our pattern-matching problem. As we are mostly interested in regular schemas, we do not assume regularity globally since safe-no-change projection does not rely on it. The pattern-matching problem for nested regular expressions with respect to a schema $\mathbf{S}\subseteq {\mathcal{H}}_{\Sigma }$ receives the following inputs:

• a nested regular expression $e\in {\mathit{nRegExp}}_{\Sigma }$ and
• a hedge $h\in \mathbf{S}$.

It then returns the truth value of the judgment $h\in 〚e〛$. The input hedge $h\in \mathbf{S}$ may either be given by a graph that resides in memory or by a stream containing the linearization $\mathit{nw}\left(h\right)$ of the input hedge, which can be read only once from left to right.

3.2. Irrelevant Subhedges

We define the concept of irrelevant occurrences of subhedges with respect to a given hedge pattern. What this means depends on the kind of algorithm that we will use for pattern matching. We will use algorithms that operate on the input hedge top-down, left-to-right, and bottom-up. This holds for streaming algorithms in particular.

Intuitively, when the pattern-matching algorithm reaches a node top-down or left-to-right whose subsequent subhedge is irrelevant, then it can jump over it without inspecting its structure. What jumping means should be clear if the hedge is given by a graph that is stored in memory. Notice that the full graph is to be constructed beforehand, even though some parts of it may turn out irrelevant. Still, one may win lots of time by jumping over irrelevant parts if either the graph was already constructed for other reasons or if many pattern-matching problems are to be solved on the same hedge.

In the case of streams, the irrelevant subhedge still needs to be parsed, but it will not be analyzed otherwise. Most typically, the possible analysis is carried out by automata, which may take two orders of magnitude more time than needed for parsing. Therefore, not having to do any analysis may considerably speed up the streaming algorithm.

Definition 1. 

Let $\mathit{S}\subseteq {\mathcal{H}}_{\Sigma }$ be a schema and $L\subseteq \mathit{S}$ a language satisfying this schema. We define ${\mathit{diff}}_{\mathit{S}}^L$ as the least symmetric relation on prefixes of nested words $u,u^{\prime }\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$:

$${\mathit{diff}}_{\mathit{S}}^L(u,u^{\prime })\iff \exists w\in \mathit{suffs}\left({\mathcal{N}}_{\Sigma }\right).u·w\in \mathit{nw}\left(L\right)\wedge u^{\prime }·w\in \mathit{nw}(\mathit{S}\backslash L).$$

A nested word prefix u is called subhedge relevant for L with schema $\mathit{S}$ if there exists nested words $v,v^{\prime }\in {\mathcal{N}}_{\Sigma }$ such that ${\mathit{diff}}_{\mathit{S}}^L(u·v,u·v^{\prime })$. Otherwise, the prefix u is called subhedge irrelevant for L with schema $\mathit{S}$.

So, ${\mathit{diff}}_{\mathbf{S}}^L(u,u^{\prime })$ states that u and $u^{\prime }$ have continuations that behave differently with respect to L and $\mathbf{S}$. Furthermore, a prefix u is subhedge irrelevant if language membership does not depend on hedge insertion at u under the condition that schema membership is guaranteed. The case without schema restrictions is subsumed by the above definition by choosing $\mathbf{S}={\mathcal{H}}_{\Sigma }$. In this case, the complement $\overline{{\mathit{diff}}_{{\mathcal{H}}_{\Sigma }}^L}$ is the Myhill–Nerode congruence of L, which is well-known in formal language theory from the minimization of DFAs (see any standard textbook on formal language theory or Wikipedia (Myhill–Nerode theorem: https://en.wikipedia.org/wiki/Myhill-Nerode_theorem, accessed on 23 July 2024)).

This congruence serves as the basis for Gold-style learning of regular languages from positive and negative examples (see [30] or Wikipedia (Gold-style learning in the limit https://en.wikipedia.org/wiki/Language_identification_in_the_limit, accessed on 23 July 2024)).

Schema restrictions are needed for our application to regular XPath patterns.

Example 2 (Regular XPath filters).

Consider the schema $\mathit{S}=〚\mathtt{doc}〛$ for hedges representing Xml documents with signature $\Sigma =\{\mathtt{list},\mathtt{item}\}$, and the regular hedge language $L=〚\mathit{filter}〛$, where

$$\mathit{filter}=\langle \mathtt{list}·{\mathit{tree}}^{\ast }·\langle \mathtt{item}·{\mathit{tree}}^{\ast }\rangle ·{\mathit{tree}}^{\ast }\rangle ·{\mathit{tree}}^{\ast }$$

is the nested regular expression from Example 1 for the XPath filter $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$. Recall that this nested regular expression can be applied to hedges representing Xml documents only. The nested word prefix $u=\langle \mathtt{list}·\langle \mathtt{item}$ is subhedge irrelevant for the language L with schema $\mathit{S}$. Note that its continuation $\langle \mathtt{list}·\langle \mathtt{item}\rangle \rangle$ with the suffix $w=\rangle \rangle$ belongs to L. Hence, for any $h\in {\mathcal{H}}_{\Sigma }$, if the continuation $\langle \mathtt{list}·\langle \mathtt{item}·h\rangle \rangle$ belongs to $\mathit{S}=〚\mathtt{doc}〛$, then it also belongs to L. Nevertheless, for the hedge $h_1=\langle \rangle$, the continuation $\langle \mathtt{list}·\langle \mathtt{item}·h_1\rangle \rangle$ does not belong to L since it does not satisfy the schema $\mathit{S}=〚\mathtt{doc}〛$.

The prefix $\langle \mathtt{item}$ is also irrelevant for the language L, even independently of the schema. This is because L does not contain any continuation of this prefix to some hedge. The prefix $\langle \mathtt{list}$ is not irrelevant for the language L wrt $\mathit{S}$. This can be seen with the suffix $w=\rangle$, since $\langle \mathtt{list}\rangle$ does not belong to L while the continuation $\langle \mathtt{list}·h\rangle$ with $h=\langle \mathtt{item}\rangle$ does belong to L and both continuations satisfy the schema $\mathit{S}=〚\mathtt{doc}〛$.

However, schema restrictions may also have surprising consequences that make the problem more tedious.

Example 3 (Surprising consequences of schema restrictions).

Consider the signature $\Sigma =\left\{a\right\}$, the pattern $L=\{\langle a\rangle \}$, and the schema $\mathit{S}=L\cup \{\langle \rangle ·a\}$. The prefix 〈 is indeed subhedge irrelevant to L and $\mathit{S}$. In order to see this, we consider all possible closing suffixes one by one. The smallest closing prefix is 〉. Note that a hedge $\langle h\rangle \in L$ if and only if $h=a$. So, membership to L seems to depend on the subhedge h. However, also note that $\langle h\rangle \in \mathbf{S}$ if and only if $h=a$. Therefore, when assuming that $\langle h\rangle$ belongs to the schema $\mathit{S}$, it must also belong to L. So, the pattern must match when assuming schema membership. The next closing suffix is $\rangle ·a$. Note that a hedge $\langle h\rangle ·a\in \mathit{S}$ if and only if $h=\epsilon$. However, $\langle h\rangle ·a\notin L$, so the pattern will not match for any subhedge h with this prefix when assuming the input hedge satisfies the schema. The situation is similar for larger suffixes, so in no case does language membership depend on the subhedge at prefix 〈 when assuming that the full hedge satisfies the schema $\mathit{S}$.

3.3. Basic Properties

We first show that subhedge irrelevant prefixes remain irrelevant when extended by the nested word of any hedge. This property should be expected for any reasonable notion of subhedge irrelevance.

Lemma 1. 

For any nested word prefix u, language $L, \mathit{S}\subseteq {\mathcal{H}}_{\Sigma }$, and nested word $v\in {\mathcal{N}}_{\Sigma }$, if the prefix u is subhedge irrelevant for L with schema $\mathit{S}$, then the prefix $u·v$ is so too.

Proof. 

Let u be subhedge irrelevant for L with schema $\mathbf{S}$ and $v\in {\mathcal{N}}_{\Sigma }$ a nested word. We fix arbitrary nested words $v^{\prime },v^{″}\in {\mathcal{N}}_{\Sigma }$ and a nested word suffix $w\in \mathit{suffs}\left({\mathcal{N}}_{\Sigma }\right)$ such that $u·v·v^{\prime }·w\in \mathit{nw}\left(\mathbf{S}\right)$ and $u·v·v^{″}·w\in \mathit{nw}\left(\mathbf{S}\right)$. Since $v·v^{\prime }$ and $v·v^{″}$ are also nested words, the subhedge irrelevance of u yields the following:  

$$u·v·v^{\prime }·w\in \mathit{nw}\left(L\right)\iff u·v·v^{″}·w\in \mathit{nw}\left(L\right).$$

Hence, the nested word prefix $u·v$ is subhedge irrelevant for L with schema $\mathbf{S}$.   □

Definition 2. 

We call a binary relation D on prefixes of nested words a difference relation on prefixes if for all prefixes of nested words $u,u^{\prime }\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$ and nested words $v\in {\mathcal{N}}_{\Sigma }$:

$$(u·v,u^{\prime }·v)\in D\Rightarrow (u,u^{\prime })\in D.$$

Lemma 2. 

${\mathit{diff}}_{\mathit{S}}^L$ is a difference relation.

Proof. 

We must show for all prefixes $u,u^{\prime }\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$ and nested words $v\in {\mathcal{N}}_{\Sigma }$ that $(u·v,u^{\prime }·v)\in {\mathit{diff}}_{\mathit{S}}^L$ implies $(u,u^{\prime })\in {\mathit{diff}}_{\mathit{S}}^L$. So, let $u,u^{\prime }$ be prefixes of nested words and v be a nested word such that $(u·v,u^{\prime }·v)\in {\mathit{diff}}_{\mathit{S}}^L$. Then, there exists a nested suffix w such that

$$u·v·w\in \mathit{nw}\left(L\right)\wedge u^{\prime }·v·w\in \mathit{nw}(\mathbf{S}\backslash L).$$

The suffix $\tilde{w}=v·w$ then satisfies $u·\tilde{w}\in \mathit{nw}\left(L\right)\wedge u^{\prime }·\tilde{w}\in \mathit{nw}(\mathbf{S}\backslash L)$. Hence, $(u,u^{\prime })\in {\mathit{diff}}_{\mathbf{S}}^L$ as required.   □

In the schema-free case of words, the complement $\overline{{\mathit{diff}}_{{\Sigma }^{\ast }}^L}$ is an equivalence relation that is known as the Myhill–Nerode congruence. In the case of schemas, however, the complement $\overline{{\mathit{diff}}_{\mathbf{S}}^L}$ may not be transitive; thus, it may not even be an equivalence relation. This might be surprising at first sight.

Example 4. 

Let $\Sigma =\{a,b\}$, $L=\{\epsilon ,b,aa\}$ and $\mathit{S}=L\cup \left\{aab\right\}$. Then, $(\epsilon ,a)\in \overline{{\mathit{diff}}_{\mathit{S}}^L}$ since there is no continuation that extends both ε and a into the schema. For the same reason, we have $(a,aa)\in \overline{{\mathit{diff}}_{\mathit{S}}^L}$. However, ${\mathit{diff}}_{\mathit{S}}^L(\epsilon ,aa)$ since $\epsilon ·b\in L$ and $aa·b\in \mathit{S}\backslash L$. Thus, $(\epsilon ,aa)\notin \overline{{\mathit{diff}}_{\mathit{S}}^L}$, so the complement $\overline{{\mathit{diff}}_{\mathit{S}}^L}$ is not transitive.

It should be noted for all languages L that $\overline{{\mathit{diff}}_{{\mathcal{H}}_{\Sigma }}^L}$ is reflexive and transitive, so it is an equivalence relation and thus a congruence. For general schemas $\mathbf{S}$, $\overline{{\mathit{diff}}_{\mathbf{S}}^L}$ may not be transitive, as shown by Example 4, and may thus not be congruent.

This indicates that schemas may make it more difficult to decide subhedge irrelevance. Indeed, the natural manner in which one might expect to eliminate schemas based on complements does not work, as confirmed by following lemma (in particular, the counter-example in proof $1.\nRightarrow 2$.):

Lemma 3. 

For any nested word prefix $u\in \mathit{suffs}\left({\mathcal{N}}_{\Sigma }\right)$ and $L, \mathit{S}\subseteq {\mathcal{H}}_{\Sigma }$, consider the following two properties:

1. 

u is irrelevant for L with schema $\mathit{S}$.

2. 

u is irrelevant for $L\cup \overline{\mathit{S}}$ with schema ${\mathcal{H}}_{\Sigma }$.

Then, property 2. implies property 1., but not always vice versa.

Proof. 

$1.\nRightarrow 2$.:

Here, a counter-example is presented. Let $L=\left\{\epsilon \right\}$ and $\mathbf{S}=\{\epsilon ,a\}$. Then, prefix a is irrelevant for L with schema $\mathbf{S}$. However, prefix a is relevant for $L\cup \overline{\mathbf{S}}={\mathcal{H}}_{\Sigma }\backslash \left\{a\right\}$ with schema ${\mathcal{H}}_{\Sigma }$, since for the nested words $v=\epsilon$ and $v^{\prime }=a$ we have $a·v\notin L\cup \overline{\mathbf{S}}$ and $a·v^{\prime }\in L\cup \overline{\mathbf{S}}$.

$1.\Leftarrow 2$.:

Next, we assume property 2. Consider arbitrary nested words $v,v^{\prime }\in {\mathcal{N}}_{\Sigma }$ and a nested word suffix $w\in \mathit{suffs}\left({\mathcal{N}}_{\Sigma }\right)$ such that $u·v·w,u·v^{\prime }·w\in \mathbf{S}$. It is sufficient to show that $u·v·w\in \mathbf{S}$ and $u·v^{\prime }·w\in \mathbf{S}$ implies $u·v·w\in L\iff u·v^{\prime }·w\in L$. This implication holds trivially if $u·v·w\in \overline{\mathbf{S}}$ or $u·v^{\prime }·w\in \overline{\mathbf{S}}$. Otherwise, we have $u·v·w\in \mathbf{S}$ and $u·v^{\prime }·w\in \mathbf{S}$. Property 2., as assumed, then implies $u·v·w\in L\iff u·v^{\prime }·w\in L$. So, property 1. holds.

   □

4. Hedge Automata

We recall the notion of stepwise hedge automata (Shas), introduce their downward extension (Sha^↓), discuss schema-completeness for both kinds of automata, define subhedge projection states for Sha^↓s, and show how to use them for evaluating Sha^↓s with subhedge projection. We limit ourselves to in-memory evaluation in this section but will discuss streaming evaluation in Section 9. The relation to several closely related notions of automata on hedges, forests, or nested words is discussed in some more detail in Section 12.

4.1. Stepwise Hedge Automata

Shas are a recent notion of automata for hedges [8] that mix up bottom-up tree and left-to-right word automata in a natural manner. They extend stepwise tree automata [31] in that they operate on hedges rather than unranked trees, i.e., on sequences of letters and trees containing hedges. They differ from nested word automata (Nwas) [11,15] in that they process hedges directly rather than their linearizations to nested words.

Definition 3. 

A stepwise hedge automaton (Sha) is a tuple $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ where Σ and $\mathcal{Q}$ are finite sets, $I,F\subseteq \mathcal{Q}$, and $\Delta =({\left(a^{\Delta }\right)}_{a\in \Sigma },{\langle \rangle }^{\Delta },$, where $a^{\Delta }\subseteq \mathcal{Q}\times \mathcal{Q}$, ${\langle \rangle }^{\Delta }\subseteq \mathcal{Q}$, and ${@}^{\Delta }\subseteq (\mathcal{Q}\times \mathcal{Q})\times \mathcal{Q}$. A Sha is deterministic or equivalently a dSha  if I and ${\langle \rangle }^{\Delta }$ contain at most one element, and all relations ${\left(a^{\Delta }\right)}_{a\in \Sigma }$ and ${@}^{\Delta }$ are partial functions.

The set of states $\mathcal{Q}$ subsumes a subset I of the initial state, a subset F of final states, and a subset ${\langle \rangle }^{\Delta }$ of tree initial states. The transition rules in $\Delta$ have three forms: if $(q,q^{\prime })\in a^{\Delta }$, then we have a letter rule that we write as $q\stackrel{a}{\to }{q}^{\prime }\mathrm{in}\Delta$. If $(q,p,q^{\prime })\in {@}^{\Delta }$, then we have an apply rule that we write as $q@p\to q^{\prime }\mathrm{in}\Delta$. Lastly, if $q\in {\langle \rangle }^{\Delta }\in \mathcal{Q}$, then we have a tree initial rule that we denote as $\stackrel{\langle \rangle }{\to }q\mathrm{in}\Delta$.

In Figure 4, we illustrate the graph of a dSha for the XPath filter $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$. The graph of Sha is obtained and drawn as usual for state automaton on words. The states are the nodes of the graph; in the example, $\mathcal{Q}=\{0,1,2,3,4,5\}$. The transition rules define the edges. The edge of a letter rule $q\stackrel{a}{\to }{q}^{\prime }$ in $\Delta$ is annotated by the letter $a\in \Sigma$. In the example, $\Sigma =\{\mathtt{list},\mathtt{item}\}$. Any apply rule $q@p\to q^{\prime }$ in $\Delta$ is drawn as an edge $q\stackrel{p}{\to }{q}^{\prime }$ annotated by state $p\in \mathcal{Q}$. The initial state $0\in I$ is indicated by an ingoing gray arrow, and the tree initial state $0\in {\langle \rangle }^{\Delta }$ by an ingoing gray arrow that is annotated by symbol $\langle \rangle$. Final states, such as $4\in F$, are indicated by a double circle.

4.1.1. Semantics

We next define the semantics of Shas, i.e., the transitions on hedges that it defines and the hedge language that it accepts. For any hedge $\mathit{h}\in {\mathcal{H}}_{\Sigma }$ we define the transition steps $q\stackrel{\mathit{h}}{\to }p\mathrm{wrt}\Delta$ such that for all $q,q^{\prime },p,p^{\prime }\in \mathcal{Q}$, $a\in \Sigma$, and $\mathit{h},{\mathit{h}}^{\prime }\in {\mathcal{H}}_{\Sigma }$

$$\frac{q\stackrel{a}{\to }{q}^{\prime }\mathrm{in}\Delta }{q\stackrel{a}{\to }{q}^{\prime }\mathrm{wrt}\Delta }\hspace{1em}\hspace{1em}\hspace{1em}\frac{q\stackrel{h}{\to }{q}^{\prime }\mathrm{wrt}\Delta q^{\prime }\stackrel{h^{\prime }}{\to }{q}^{″}\mathrm{wrt}\Delta }{q\stackrel{h·h^{\prime }}{\to }{q}^{″}\mathrm{wrt}\Delta }$$

$$\frac{q\in \mathcal{Q}}{q\stackrel{\epsilon }{\to }q\mathrm{wrt}\Delta }\hspace{1em}\hspace{1em}\hspace{1em}\frac{\stackrel{\langle \rangle }{\to }p\mathrm{in}\Delta p\stackrel{h}{\to }{p}^{\prime }q@p^{\prime }\to q^{\prime }\mathrm{in}\Delta }{q\stackrel{\langle h\rangle }{\to }{q}^{\prime }\mathrm{wrt}\Delta }$$

The transitions can be used to evaluate hedges nondeterministically bottom-up and left-to-right using Shas. The first three rules state how to evaluate sequences of trees and letters, such as a usual finite state automaton, while assuming that the trees were already evaluated to states. The last rule states how to evaluate a tree $\langle h\rangle$ from some state q to some state q′. This can be visualized as follows:

For any tree initial state $p\in {\langle \rangle }^{\Delta }$, one has to evaluate the subhedge h nondeterministically to some state $p^{\prime }$. For each $p^{\prime }$ obtained, one then returns to a state $q^{\prime }$ such that $q@p^{\prime }\to q^{\prime }\mathrm{in}\Delta$ nondeterministically.

A hedge is accepted by A if it permits a transition from some initial state to some final state. The language $\mathcal{L}\left(A\right)$ is the set of all accepted hedges:

$$\mathcal{L}\left(A\right)=\{\mathit{h}\in {\mathcal{H}}_{\Sigma }\mid q\stackrel{\mathit{h}}{\to }{q}^{\prime }\mathrm{wrt}\Delta ,q\in I,q^{\prime }\in F\}.$$

4.1.2. Runs

Runs can represent the whole history of a single choice of the nondeterministic evaluator on a hedge. For instance, the unique successful run of the dSha in Figure 4 on the hedge $h=\langle h^{\prime }\rangle$ with subhedge $h^{\prime }=list·\langle item\rangle$ is the hedge $0·\langle 0·list·1·\langle 0·item·2\rangle ·3\rangle ·4$ whose computation is illustrated in Figure 5. On the top level, the run justifies the transition $0\stackrel{h}{\to }4\mathrm{wrt}\Delta$ from the initial state 0 to a final state 4. When started in state 0, the subhedge $h^{\prime }$ needs to be evaluated first, so the run on the upper level needs to be suspended until this is done. The subrun on $h^{\prime }$ eventually justifies the transition $0\stackrel{h^{\prime }}{\to }3\mathrm{wrt}\Delta$. At this time point, the run of the upper level can be resumed and continue in state 4 since $0@3\to 4\mathrm{in}\Delta$.

When drawing runs, we illustrate any suspension/resumption mechanism with a box, for instance, the box on the edge 0  4. The box stands for a future value computed by the lower level, causing the upper level to suspend until its arrival. Eventually, the box is filled by state 3, as illustrated by 3 , so that the computation can be resumed on the upper level.

We next define runs of Shas on hedges formally. Whether a hedge with letters and states $R\in {\mathcal{H}}_{\Sigma \cup \mathcal{Q}}$ is a $\Delta$-run or simple a run on a hedge $h\in {\mathcal{H}}_{\Sigma }$, written $R\in {\mathit{run}}^{\Delta }\left(h\right)$, is defined by the following rules:

$$\frac{q\stackrel{a}{\to }{q}^{\prime }\mathrm{in}\Delta }{q·a·q^{\prime }\in {\mathit{run}}^{\Delta }\left(a\right)}\hspace{1em}\hspace{1em}\hspace{1em}\frac{q·R·q^{\prime }\in {\mathit{run}}^{\Delta }\left(h\right)q^{\prime }·R^{\prime }·q^{″}\in {\mathit{run}}^{\Delta }\left(h^{\prime }\right)}{q·R·q^{\prime }·R^{\prime }·q^{\prime \prime }\in {\mathit{run}}^{\Delta }(h·h^{\prime })}$$

$$\frac{true}{q\in {\mathit{run}}^{\Delta }\left(\epsilon \right)}\hspace{1em}\hspace{1em}\hspace{1em}\frac{\stackrel{\langle \rangle }{\to }p\mathrm{in}\Delta p·R·p^{\prime }\in {\mathit{run}}^{\Delta }\left(h\right)q@p^{\prime }\to q^{\prime }\mathrm{in}\Delta }{q·\langle R\rangle ·q^{\prime }\in {\mathit{run}}^{\Delta }\left(\langle h\rangle \right)}$$

Note that if $R\in {\mathit{run}}^{\Delta }\left(h\right)$, then h can be obtained by removing all states from R, i.e., ${\mathit{proj}}_{\Sigma }\left(R\right)=h$. Any run $R\in {\mathit{run}}^{\Delta }\left(h\right)$ can be identified with the mapping of prefixes of the nested word $\mathit{nw}\left(h\right)$ to states in $\mathcal{Q}$. Note that $\mathit{nw}\left(h\right)={\mathit{proj}}_{\Sigma }\left(\mathit{nw}\left(R\right)\right)$. For any prefix v of $\mathit{nw}\left(h\right)$, there exists a unique prefix r of the nested word $\mathit{nw}\left(R\right)$ that ends with some state $q\in \mathcal{Q}$ and satisfies ${\mathit{proj}}_{\Sigma }\left(r\right)=v$. We then call q the state of R at prefix v. The run $0·\langle 0·list·1·\langle 0·item·2\rangle ·3\rangle ·4$ on hedge $\langle list\langle item\rangle \rangle$ from Figure 5, for instance, assigns state 0 to the nested word prefixes $\epsilon$ and 〈 and state 1 to the nested word prefix $\langle ·\mathtt{list}$, etc.

A $\Delta$-run is called a run of $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ if it starts with some state in I. A run of A is successful if it ends with some state in F.

Lemma 4. 

$h\in \mathcal{L}\left(A\right)$ iff there exists a successful run $R\in {\mathit{run}}^{\Delta }\left(h\right)$ of A.

Proof. 

Straightforward with induction on h.   □

4.1.3. Determinization

Shas can always be determinized using the subset construction known from the determinization of finite state automata on words or of tree automata.

Given a Sha $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$, the state set of its determinization is ${\mathcal{Q}}^{\mathit{det}}=2^{\mathcal{Q}}$. The transition rules ${\Delta }^{\mathit{det}}$ of the determinization are presented in Figure 6.

4.1.4. In-Memory Membership Testing

Testing membership $h\in \mathcal{L}\left(A\right)$ for a hedge h, whose graph is stored in memory, can be carried out by computing the unique run of the determinization of A in a bottom-up and left-to-right manner, and testing whether it is successful. Alternatively, one can compute the unique set $P\subseteq \mathcal{Q}$ such that $I\stackrel{h}{\to }P$ with respect to the determinization of A and to test whether $P\cap F\ne \varnothing$. The computations are essentially the same, but the full history of the computation is lost when returning only the set P of states reached and not the run.

For determinizing Shas we can rely on a variant of the usual subset construction that is well-known from finite state automata on words or standard tree automata (see e.g., [8,16]). When it comes to membership testing, on-the-fly determinization is sufficient, which creates only the part of the deterministic automaton that is used by the run on h. This part is of linear size $O\left(\right|h\left|\right)$ while the full determinization of A may be of exponential size $O\left(2^{\left|A\right|}\right)$. In this way, membership $h\in \mathcal{L}\left(A\right)$ can be tested in time $O\left(\right|h\left|\right|A\left|\right)$.

4.1.5. Hedge Accessibility

For any set $P\subseteq \mathcal{Q}$ we define the set of states accessible from P via a hedge as follows:

$${\mathit{acc}}^{\Delta }\left(P\right)=\{q^{\prime }\in \mathcal{Q}\mid \exists q\in P.\exists h\in {\mathcal{H}}_{\Sigma }.q\stackrel{h}{\to }{q}^{\prime }\mathrm{wrt}\Delta \}.$$

We note that ${\mathit{acc}}^{\Delta }\left(P\right)$ can be computed from $\Delta$ and P in linear time in the size of $\Delta$. A state is hedge accessible from P if and only if it is accessible for P in the graph of $\Delta$. Or by computing the least fixed point of the following ground Datalog program depending on P and $\Delta$ that is of linear size in A:

$$\frac{q\in P}{acc\left(p\right).}\hspace{1em}\hspace{1em}\frac{q\stackrel{a}{\to }{q}^{\prime }\mathrm{in}\Delta }{acc\left(q^{\prime }\right):-acc\left(q\right).}\hspace{1em}\hspace{1em}\frac{q@p\to q^{\prime }\mathrm{in}\Delta }{acc\left(q^{\prime }\right):-acc\left(q\right),acc\left(p\right).}$$

Clearly, $\mathcal{L}\left(A\right)=\varnothing$ if and only if $F\cap {\mathit{acc}}^{\Delta }\left(I\right)=\varnothing$. Therefore, emptiness can be decided in linear time for Shas. This is in contrast to the more general notion of Sha^↓s that we introduce next.

4.2. Downward Stepwise Hedge Automata

The evaluation of Shas operates in a bottom-up and left-to-right manner, but cannot pass any information top-down. We next propose an extension of Shas to Sha^↓s, adding the ability to pass finite state information top-down.

Sha^↓s are basically equal to Neumann and Seidl’s pushdown forest automata [25] but lifted from (labeled) forests to (unlabeled) hedges. See Section 12 for a more detailed discussion on the relationship of Sha^↓s with other automata notions and Nwas in particular.

Definition 4. 

A downward stepwise hedge automaton (Sha^↓) is a tuple $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$, where Σ and $\mathcal{Q}$ are finite sets, $I,F\subseteq \mathcal{Q}$, and $\Delta =({\left(a^{\Delta }\right)}_{a\in \Sigma },{\langle \rangle }^{\Delta },$${@}^{\Delta })$. Furthermore, $a^{\Delta }\subseteq \mathcal{Q}\times \mathcal{Q}$, ${\langle \rangle }^{\Delta }\subseteq \mathcal{Q}\times \mathcal{Q}$, and ${@}^{\Delta }\subseteq (\mathcal{Q}\times \mathcal{Q})\times \mathcal{Q}$. A Sha^↓ is deterministic or equivalently a dSha^↓ if I contains at most one element, and all relations, ${\langle \rangle }^{\Delta }$, $a^{\Delta }$, and ${@}^{\Delta }$, are partial functions.

The only difference compared to Shas is the form of the tree-opening rules. If $(q,q^{\prime })\in {\langle \rangle }^{\Delta }\in \mathcal{Q}$, then we have a tree initial rule that we denote as follows: $q\stackrel{\langle \rangle }{\to }{q}^{\prime }\mathrm{in}\Delta$.

So, state $q^{\prime }$, where the evaluation of a subhedge starts, depends on the state q of the parent.

4.2.1. Semantics

The definition of transition steps for Sha^↓s differs from that of Shas only in the usage of opening rules in the following inference rule:

$$\frac{q\stackrel{\langle \rangle }{\to }p\mathrm{in}\Delta p\stackrel{h}{\to }{p}^{\prime }\mathrm{wrt}\Delta q@p^{\prime }\to q^{\prime }\mathrm{in}\Delta }{q\stackrel{\langle h\rangle }{\to }{q}^{\prime }\mathrm{wrt}\Delta }$$

This means that the evaluation of the subhedge h starts in a state p such that $q\stackrel{\langle \rangle }{\to }p\mathrm{in}\Delta$.

So, the restart state p that is chosen may now depend on the state q above. This is how finite state information is passed top-down by Sha^↓s. Shas, in contrast, operate purely bottom-up and left-to-right.

4.2.2. Runs

The notion of runs can be adapted straightforwardly from Shas to Sha^↓s. When in state q, it is sufficient to restart the computation in subhedges on the state p, such that $q\stackrel{\langle \rangle }{\to }p\in \Delta$. In this way, finite state information is passed down (while for Shas some tree initial is to be chosen that is independent of q). The only rule of runs to be changed is the following:

$$\frac{q\stackrel{\langle \rangle }{\to }p\mathrm{in}\Delta p·R·p^{\prime }\in {\mathit{run}}^{\Delta }\left(h\right)q@p^{\prime }\to q^{\prime }\mathrm{in}\Delta }{q·\langle R\rangle ·q^{\prime }\in {\mathit{run}}^{\Delta }\left(\langle h\rangle \right)}$$

An example of a run of the dSha^↓ in Figure 7 is shown in Figure 8. Here, the information on the level of nodes is passed down. It is represented by the number of primes. For instance, when opening the first subtree labeled with item we use the transition rule $1^{\prime }\stackrel{\langle \rangle }{\to }{0}^{″}$ stating that one moves from state 1 on the first level to state 0 on the second level.

One can see that all nodes of the subtrees $h_1$ and $h_2$ are evaluated to the projection state $\mathrm{\Pi }$. So, one can jump over these subtrees without reading them. This is justified by the fact that they are on level 2, the information that the evaluator of this Sha^↓ was passing down.

4.2.3. In-Memory Membership Testing

Testing membership $h\in \mathcal{L}\left(A\right)$ for a Sha^↓s in memory can again be carried out by computing the unique run of the determinization of A. As before, the membership tester can be based on on-the-fly determinization. However, the determinization procedure for Sha^↓s is more complicated then for Shas since Sha^↓s can pass information top-down and not only bottom-up and left-to-right. Determinization not only relies on the pure subset construction but also uses pairs of states in the subsets, basically in the same way as for nested word automata (Nwas) [14,15]. This is needed to deal with the stack of states that were seen above. Therefore, each determinization step may cost time $O\left(\right|A{|}^5)$ as stated for instance in [23]. The upper time bound for membership testing for Sha^↓s is thus in time $O\left(\right|h\left|\right|A{|}^5)$ and no longer combined linear as it is for Shas.

4.2.4. Relationship to Shas

Any Sha $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ can be mapped to a Sha^↓ $A^{\mathit{down}}=(\Sigma ,\mathcal{Q},{\Delta }^{\mathit{down}},$$I,$ $F)$ while preserving runs and determinism. The only change of ${\Delta }^{\mathit{down}}$ compared to $\Delta$ is described by the following rule:

$$\frac{\stackrel{\langle \rangle }{\to }p\mathrm{in}\Delta q\in \mathcal{Q}}{q\stackrel{\langle \rangle }{\to }p\mathrm{in}{\Delta }^{\mathit{down}}}$$

Independently of the current state $q\in \mathcal{Q}$, the Sha^↓ $A^{\mathit{down}}$ can start the evaluation of the subhedge of a subtree in any open tree state $p\in {\langle \rangle }^{\Delta }$. For instance, the dSha^↓ $A^{\mathit{down}}$ for dSha A from Figure 4 from the introduction is provided in Figure 9. Conversely, one can convert any dSha^↓ A into an equivalent Sha by introducing nondeterminism to eliminate top-down processing. See Section 12.2 for the construction.

4.2.5. Hedge Accessiblity

Hedge accessibility can be defined for Sha^↓s identically as for Shas. However, the set ${\mathit{acc}}^{\Delta }\left(P\right)$ can no longer be computed in linear time in the size of $\Delta$, in contrast to Shas. It can still be done in cubic time, similarly as for Nwas [22]. The problem is that hedge accessibility for Sha^↓s does no more coincide with the accessibility notion of the automaton’s graph.

For instance, in the Sha^↓ from Figure 7, we have ${\mathit{acc}}^{\Delta }\left(\left\{0\right\}\right)=\left\{4\right\}$. Note that $0^{\prime }$ does not belong to this set, even though there is a transition rule $0\stackrel{\langle \rangle }{\to }{0}^{\prime }$ in $\Delta$, making node $0^{\prime }$ accessible from node 0 in the graph of the automaton. This is because $0^{\prime }$ is not accessible over any hedge from 0; that is, there is no hedge h such that $0\stackrel{h}{\to }{0}^{\prime }$ wrt $\Delta$. Note that $0·\langle ·0^{\prime }$ is a factor of the nested word of some run of $\Delta$. This shows that $0^{\prime }$ is accessible from 0 over a nested word prefix nevertheless. For any Sha^↓ A, hedge accessibility can be computed by the following ground Datalog program of cubic size depending on the number of states of A:

$$\frac{p@q^{\prime }\to q\mathrm{in}\Delta p\stackrel{\langle \rangle }{\to }{p}^{\prime }\mathrm{in}\Delta }{acc(p,q):-acc(p^{\prime },q^{\prime }).}\hspace{1em}\frac{p\in \mathcal{Q}}{acc(p,p).}\hspace{1em}\frac{p\stackrel{a}{\to }q\mathrm{in}\Delta }{acc(p,q).}$$

$$\frac{p,q,r\in \mathcal{Q}}{acc(p,q):-acc(p,r),acc(r,q).}$$

Due to the cubic time of hedge accessibility for Sha^↓s, we will use hedge accessibility only for Shas where it is in linear time.

4.3. Schema Completeness

Schemas are inputs of the membership problem that we consider while completeness is a fundamental property of automata. To combine both aspects appropriately, we propose the notion of schema completeness, which we define uniformly for both Shas and Sha^↓s.

We call a set of $\Delta$ transition rules Sha^↓ complete if all its relations ${\left(a^{\Delta }\right)}_{a\in \Sigma }$, ${\langle \rangle }^{\Delta }$, and ${@}^{\Delta }$ are total. For Shas, we require that ${\langle \rangle }^{\Delta }$ is a nonempty set (instead of a total relation). A Sha or Sha^↓ A is called complete if $\Delta$ is complete and $I\ne \varnothing$. We can complete any Sha or Sha^↓ by adding a fresh sink state and also transition rules into the sink state for all left-hand sides, for which no other transition rule exists. The sink state is made initial if and only if there is no initial state before. It is not final though. We denote the completion of A by $compl\left(A\right)$.

Definition 5. 

A partial run of A on a hedge h is a prefix r of a run of $compl\left(A\right)$ on h, such that

• r ends with some state of $\mathcal{Q}$, and
• r does not contain the sink state of $compl\left(A\right)$.

A partial run r of A on h is called blocking if there does not exist any partial run $r^{\prime }$ of A on h, such that r is a strict prefix of $r^{\prime }$.

For instance, consider the hedge $h=\langle \mathtt{list}·\langle \mathtt{item}\rangle \rangle$. The dSha in Figure 4 has the unique run $R=0·\langle ·0·\mathtt{list}·1·\langle ·0·\mathtt{item}·2\rangle ·3·\rangle ·4$ on h that is illustrated in Figure 5. The partial runs of h are thus all prefixes of $\mathit{nw}\left(R\right)$ that end with some state, i.e., 0, $0·\langle ·0$, $0·\langle ·0·\mathtt{list}·1$, etc. None of these partial runs are blocking.

Definition 6. 

A schema for an automaton A is a hedge language $\mathit{S}\subseteq {\mathcal{H}}_{\Sigma }$, such that $\mathcal{L}\left(A\right)\subseteq \mathit{S}$. We call the automaton A schema-complete for schema $\mathit{S}$ if no hedge $h\in \mathit{S}$ has a blocking partial run of A.

Schemas $\mathbf{S}$ are usually used to restrict the type of hedges that a problem may accept as input. The dSha pattern-matching problem for a schema $\mathbf{S}$ takes two inputs: a hedge $h\in \mathbf{S}$ and an automaton A, rather than a nested regular expression e. The automaton A then selects those input hedges $h\in \mathbf{S}$ that match the pattern, i.e., such that $h\in \mathcal{L}\left(A\right)$. For this reason, we assume that $\mathbf{S}$ is a schema for A, i.e., $\mathcal{L}\left(A\right)\subseteq \mathbf{S}$. We will often assume that A is schema-complete for $\mathbf{S}$, so that no partial run of A on any input hedge $h\in \mathbf{S}$ may ever block. This can always be achieved next based on the next Lemma 5.

Example 5. 

The dSha in Figure 4 for the XPath filter $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$ has signature $\Sigma =\{\mathtt{item},\mathtt{list}\}$. It is schema-complete for schema $〚\mathtt{doc}〛$. To make this happen, we added state 5 to this automaton, which is not used by any successful run. Still, this  Sha  is not complete. For completing it, we would have to run 5 into sink states by adding many transitions into it, such as $0@4\to 5$ and $0@5\to 5$, and also add for all other states $q\in \{1,2,3,4,5\}$ the transition rules $q\stackrel{\mathtt{list}}{\to }5$, $q\stackrel{\mathtt{item}}{\to }5$, $q@4\to 5$, and $q@5\to 5$.

As we will see, automaton completion would make the safe-no-change algorithm incomplete for subhedge projection for Example 5. Without schema-completeness, however, safe-no-change projection may be unsound: it might overlook the rejection of hedges inside the schema. This is why it is much better to assume only schema-completeness rather than full completeness for the input automata of safe-no-change projection. For congruence projection, schema-completeness will be assumed implicitly.

We note that schema-completeness is well-defined even for nonregular schemas. For safe-no-change projection, we indeed do not have to restrict schemas to be regular, while for congruence projection, only regular schemas are considered.

Furthermore, if A is schema-complete for the universal schema $\mathbf{S}={\mathcal{H}}_{\Sigma }$ and does not have inaccessible states, then A is complete.

Schema-completeness of deterministic automata for regular schemas can always be made to hold as we will show next. For this we define that A is compatible with schema $\mathit{S}$ if for any hedge $h\in \mathbf{S}$ there exists a run of A in ${\mathit{run}}^{\Delta }\left(h\right)$. Schema-completeness implies compatibility, as shown by the next lemma. The converse is true only when assuming determinism.

Lemma 5. 

Let A be a deterministic  Sha  with schema $\mathit{S}$. Then, A is compatible with $\mathit{S}$ iff A is schema-complete for $\mathit{S}$.

Proof. 

Suppose that A is schema-complete for $\mathbf{S}$ Wlog., we can assume $\mathbf{S}\ne \varnothing$. Note that if $I=\varnothing$, then $\epsilon$ would be a blocking partial run for any hedge in $\mathbf{S}\backslash \left\{\epsilon \right\}$. Since $\mathbf{S}\ne \varnothing$, it follows that there exists $q_0\in I$. So, $q_0$ is a partial run for any hedge $h\in \mathbf{S}\backslash \left\{\epsilon \right\}$. Since there exist no blocking partial runs by schema-completeness, this run can be extended step by step to a run on any $h\in \mathbf{S}$. Therefore, for any hedge $h\in \mathbf{S}$, there exists some run by A, showing compatibility.

For the converse, let A be compatible with $\mathbf{S}$ and deterministic. We have to show that A is schema-complete for $\mathbf{S}$. Let v be a partial run of A on $h\in \mathbf{S}$. By compatibility, there exists a run $R\in {\mathit{run}}^{\Delta }\left(h\right)$ that starts in some initial state of A. By determinism, v must be a prefix of $\mathit{nw}\left(R\right)$. Thus, v is not blocking.   □

Proposition 1. 

Any  Sha  A with regular schema $\mathit{S}$ can be made schema-complete for $\mathit{S}$.

Proof. 

By Lemma 5, it is sufficient to make A compatible with $\mathbf{S}$. Let B be a dSha with $\mathbf{S}=\mathcal{L}\left(B\right)$. Automaton B is compatible with $\mathbf{S}$. Since B is deterministic, Lemma 5 implies that B is schema-complete for $\mathbf{S}$. Since $\mathbf{S}$ is a schema for A with $\mathcal{L}\left(A\right)\subseteq \mathcal{L}\left(B\right)$. We then compute the completion of $compl\left(A\right)$. The product $C=B\times compl\left(A\right)$ with final states $F^C=F^B\times F^A$ is schema-complete for schema $\mathbf{S}$. Furthermore, since $\mathcal{L}\left(A\right)\subseteq \mathbf{S}=\mathcal{L}\left(B\right)$, it follows that $\mathcal{L}\left(C\right)=\mathcal{L}\left(B\right)\cap \mathcal{L}\left(A\right)=\mathcal{L}\left(A\right)$.   □

Finally, note that the schema $\mathbf{S}$ can be recognized by the same product $C=B\times compl\left(A\right)$ except for replacing the set of final states $F^C$ by the set of schema final states $F^{\mathbf{S}}=F^B\times ({\mathcal{Q}}^A\cup \left\{\mathit{sink}\right\})$. We have $\mathbf{S}=\mathcal{L}\left(C[F^C/F^{\mathbf{S}}]\right)$, where $C[F^C/F^{\mathbf{S}}]$ is the dSha obtained from C by replacing the set of final states $F^C$ with $F^{\mathbf{S}}$.

5. Subhedge Projection

We next introduce the concept of subhedge projection states for Sha^↓s in order to distinguish prefixes of hedges that are subhedge irrelevant, and to evaluate Sha^↓s with subhedge projection.

5.1. Subhedge Projection States

Intuitively, a subhedge projection state can only loop in itself until a hedge closes (and then it is applied). When going down into a subtree, the subhedge projection state may still be changed into some other looping state. When leaving the subtree, however, one must come back to the original subhedge projection state. Clearly, any sink is a subhedge projection state. The even more interesting cases, however, are subhedge projection states that are not sinks.

We start with Sha^↓s without any restrictions but will impose schema-completeness and determinism later on.

Definition 7. 

We call a state $q\in \mathcal{Q}$ a subhedge projection state of $\Delta$ if there exists $q^{\prime }\in \mathcal{Q}$ called the witness of q, such that the set of transition rules of Δ containing $q^{\prime }$ or q on the leftmost position is included in the following set:

$$\begin{array}{c}\{q\stackrel{\langle \rangle }{\to }{q}^{\prime },q@q^{\prime }\to q,q^{\prime }\stackrel{\langle \rangle }{\to }{q}^{\prime },q^{\prime }@q^{\prime }\to q^{\prime }\}\hfill \\ \cup \{q^{\prime }\stackrel{a}{\to }{q}^{\prime },q\stackrel{a}{\to }q\mid a\in \Sigma \}\hfill \end{array}$$

The set of all subhedge projection states of Δ is denoted by ${\mathcal{Q}}_{\mathit{shp}}^{\Delta }$.

For any complete Sha^↓, the above set of transition rules must be equal to the set of all transition rules of $\Delta$ with q or $q^{\prime }$ on the leftmost position. But, if the Sha^↓ is only schema-complete for some schema $\mathbf{S}$, then not all these transitions must be present. Note also that a subhedge projection state q may be equal to its witness $q^{\prime }$. Therefore, any witness $q^{\prime }$ of some subhedge projection state is itself a subhedge projection state with witness $q^{\prime }$.

In the example dSha^↓ $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ in Figure 7, we have ${\mathcal{Q}}_{\mathit{shp}}^{\Delta }=\{\mathrm{\Pi },4,2^{\prime },3^{\prime },1^{″},2^{″}\}$. The witness of all these subhedge projection states is $\mathrm{\Pi }$. Note that not all possible transitions are present for the states in ${\mathcal{Q}}_{\mathit{shp}}^{\Delta }\backslash \left\{\mathrm{\Pi }\right\}$, given that this automaton is not complete (but still schema-complete for schema $〚\mathtt{doc}〛$).

Lemma 6. 

If $q\in {\mathcal{Q}}_{\mathit{shp}}^{\Delta }$ is a subhedge projection state and $q\stackrel{h}{\to }p$ wrt Δ, then $q=p$.

Proof. 

Suppose that $q\stackrel{h}{\to }p$ wrt $\Delta$ and that q is a subhedge projection state of $\Delta$ with witness $q^{\prime }$. So, the only possible transitions with q or $q^{\prime }$ on the left-hand side are the following:

Suppose that $h=t_1·\dots ·t_n$ is a sequence of trees or letters $t_i\in \langle {\mathcal{H}}_{\Sigma }\rangle \cup \Sigma$, where $1\le i\le n$. Then, there exists runs $R_i\in {\mathit{run}}^{\Delta }\left(t_i\right)$ and a run $R\in {\mathit{run}}^{\Delta }\left(h\right)$ that has the form $h=q_0·R_1·q_1·\dots ·R_n·q_n$, where $q_0=q$ and $q_n=p$. We prove for all $0\le i\le n$ that $q_i=q$ by induction on i. For $i=0$, this is trivial. For $i>0$, the induction hypothesis shows that $q_{i-1}=q$.

Case 

$t_i=a\in \Sigma$. Then, $q_{i-1}\stackrel{a}{\to }{q}_i\mathrm{in}\Delta$. Since $q_{i-1}=q$ is a subhedge projection state of $\Delta$, it follows that $q_i=q$, too.

Case 

$t_i=\langle h^{\prime }\rangle$ for some $h^{\prime }\in {\mathcal{H}}_{\Sigma }$. Since $q_{i-1}=q$ is a subhedge projection state of $\Delta$ with witness $q^{\prime }$, the subrun of R recognizing $t_i$ must be justified by the following diagram:

So, the subrun of R of $h^{\prime }$ may only contain the witness $q^{\prime }$ and, furthermore, $q_i=q$.

   □

5.2. Completeness for Subhedge Projection

Before defining when a Sha^↓ is complete for subhedge projection, we show that subhedge projection states in runs of deterministic Shas may only occur at irrelevant prefixes.

Proposition 2. 

Let $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ be a dSha^↓ with schema $\mathit{S}$, $R\in {\mathit{run}}^{\Delta }\left(h\right)$ is the run of $compl\left(A\right)$ on some hedge $\mathit{h}\in \mathit{S}$, and q is a subhedge projection state for Δ. Then, for any prefix $r·q$ of $\mathit{nw}\left(R\right)$, the prefix ${\mathit{proj}}_{\Sigma }\left(r\right)$ of $\mathit{nw}\left(h\right)$ is subhedge irrelevant for $\mathcal{L}\left(A\right)$ for schema $\mathit{S}$.

Proof. 

Let $R\in {\mathit{run}}^{\Delta }\left(h\right)$ be the unique run of $compl\left(A\right)$ on a hedge $h\in \mathbf{S}$. Suppose that $r·q·s=\mathit{nw}\left(R\right)$ for some subhedge projection states q of $\Delta$ and let $v={\mathit{proj}}_{\Sigma }\left(r\right)$ and $w={\mathit{proj}}_{\Sigma }\left(s\right)$. Since $v·w=\mathit{nw}\left(h\right)$ and $h\in \mathbf{S}$, it follows that $v·w\in \mathit{nw}\left(\mathbf{S}\right)$. We fix some hedge $h^{\prime }\in {\mathcal{H}}_{\Sigma }$, such that $v·\mathit{nw}\left(h^{\prime }\right)·w\in \mathit{nw}\left(\mathbf{S}\right)$ arbitrarily. Let $\tilde{h}\in {\mathcal{H}}_{\Sigma }$, such that $\mathit{nw}\left(\tilde{h}\right)=v·\mathit{nw}\left(h^{\prime }\right)·w$. For the subhedge irrelevance of v, we must show that

$$h\in \mathcal{L}\left(A\right)\iff \tilde{h}\in \mathcal{L}\left(A\right).$$

“⇒”

We suppose that $h\in \mathcal{L}\left(A\right)$ and have to show that $\tilde{h}\in \mathcal{L}\left(A\right)$. For the partial run $r·q$ of A on $\tilde{h}$, there must exist $R^{\prime }$ and p, such that $q·R^{\prime }·p\in {\mathit{run}}^{\Delta }\left(h^{\prime }\right)$. Since q is a subhedge projection state of $\Delta$, it follows by Lemma 6 that $q=p$. Hence, $r·q·\mathit{nw}\left(R^{\prime }\right)·q·s$ is a run of A on $\tilde{h}$. Since R was successful for A, s must end in F; thus, the above run on $\tilde{h}$ is successful for A too. Hence, $\tilde{h}\in \mathcal{L}\left(A\right)$, as we have shown.

“⇐”

We suppose that $\tilde{h}\in \mathcal{L}\left(A\right)$ and have to show that $h\in \mathcal{L}\left(A\right)$. Let $\tilde{R}$ be the unique run of $compl\left(A\right)$ on $\tilde{h}$. By determinism, $\tilde{R}$ must have the prefix $r·q$. So, there exist $R^{\prime },p^{\prime },s$, such that $p·R^{\prime }·q\in {\mathit{run}}^{\Delta }\left(h^{\prime }\right)$ and $r·q·R^{\prime }·p·s=\tilde{R}$. Lemma 6 shows that $p=q$. Hence, $r·q·s$ is the unique run of $compl\left(A\right)$ on h and this run is accepting. So, $h\in \mathcal{L}\left(A\right)$.

   □

We note that Proposition 2 would not hold without assuming determinism. To see this, we can add some sink to any dSha as a second initial state. One can then always go to this sink, which is a subhedge projection state. Nevertheless, no prefix going to the sink may be subhedge irrelevant.

Proposition 2 shows that subhedge projection states permit distinguishing prefixes that are subhedge irrelevant for deterministic Sha^↓s. An interesting question is whether all subhedge irrelevant prefixes can be found in this way. A closely related question is whether, for all regular patterns, there exist dShas that project all irrelevant subhedges.

Example 6. 

Reconsider Example 3 with $\Sigma =\left\{a\right\}$, a regular pattern $L=\{\langle a\rangle \}$, and a regular schema $\mathit{S}=L\cup \{\langle \rangle ·a\}$. The prefix $\langle \rangle$ is irrelevant for L and $\mathit{S}$, since all its continuations ending in the schema are rejected: there is only one such continuation, which is $\langle \rangle ·a$. However, a dSha  for L and S with maximal subhedge projection is given in Figure 17 but will not go into a subhedge projection state at this prefix. The reason is that it will go into the same state $\{3,4\}D0$ for any prefix in $\langle {\mathcal{N}}_{\Sigma }\rangle$, ignoring the nested word of the irrelevant subhedge. Therefore, this state must accept $\langle a\rangle$. But, when reading an a in this state, the dSha  goes into the rejection state $\left\{5\right\}D0$, showing that the hedge from the schema $\langle \rangle ·a\in S$ is rejected.

The example shows that we eventually reason with sets of prefixes. We first lift the notion of subhedge irrelevance to prefix sets.

Definition 8. 

Let $\mathit{S}\subseteq {\mathcal{H}}_{\Sigma }$ be a schema and $L\subseteq \mathit{S}$ a language satisfying this schema. A set of nested word prefixes $U\subseteq \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$ is called subhedge relevant for L with schema $\mathit{S}$ if there exist nested words $v^{\prime },v^{″}\in {\mathcal{N}}_{\Sigma }$ prefixes $u^{\prime },u^{″}\in U$, such that ${\mathit{diff}}_{\mathbf{S}}^L(u^{\prime }·v^{\prime },u^{″}·v^{″})$. Otherwise, the set U is called subhedge irrelevant for L wrt $\mathit{S}$.

In order to explain the problem of Example 6, we define for each nested word prefix $w\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$ a class $clas{s}_{\mathbf{S}}^L\left(w\right)$ of similar prefixes up replacing the nested word of any irrelevant subhedge by ${\mathcal{N}}_{\Sigma }$ from the left to the right. For all prefixes $w\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$, nested words $v\in {\mathcal{N}}_{\Sigma }$, letters $a\in \Sigma$, subsets of prefixes $U\subseteq \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$, and $l\in \widehat{\Sigma }$ letters or parenthesis, the following applies:

$$\begin{array}{ccc}\hfill clas{s}_{\mathbf{S}}^L\left(w\right)& =& class\_p{r}_{\left\{\epsilon \right\}}\left(w\right)\hfill \\ \hfill class\_p{r}_U\left(\epsilon \right)& =& \left\{\begin{array}{cc}U·{\mathcal{N}}_{\Sigma }\hfill & ifU\mathrm{irrelevant}\mathrm{for}L\mathrm{wrt} \mathbf{S}\hfill \\ U\hfill & \mathrm{else}\hfill \end{array}\right.\hfill \\ \hfill class\_p{r}_U(w·\langle ·v)& =& class\_in{s}_{U^{\prime }}(\langle ,v)\mathrm{where}{U}^{\prime }=class\_p{r}_U\left(w\right)\hfill \\ \hfill class\_p{r}_U(a·v)& =& class\_in{s}_{U^{\prime }}(a,v)\mathrm{where}{U}^{\prime }=U·a\hfill \\ \hfill class\_p{r}_U(\langle v\prime \rangle ·v)& =& class\_in{s}_{U^{\prime }}(\rangle ,v)\mathrm{where}{U}^{\prime }=class\_p{r}_U(\langle ·v\prime )\hfill \\ \hfill class\_in{s}_U(l,v)& =& \left\{\begin{array}{cc}U·{\mathcal{N}}_{\Sigma }\hfill & \mathrm{if}U\mathrm{irrelevant}\mathrm{for}L\mathrm{wrt} \mathbf{S}\hfill \\ class\_p{r}_{U·l}\left(v\right)\hfill & \mathrm{else}\hfill \end{array}\right.\hfill \end{array}$$

Definition 9. 

A prefix $u\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$ is strongly subhedge irrelevant for L wrt $\mathit{S}$ if the set $clas{s}_{\mathit{S}}^L\left(u\right)$ is subhedge irrelevant for L wrt $\mathit{S}$.

Example 7. 

In our running Example 6, where $L=\{\langle a\rangle \}$, $\mathit{S}=L\cup \{\langle \rangle ·a\}$, and $\Sigma =\left\{a\right\}$, we have $clas{s}_{\mathit{S}}^L(\langle )=\langle ·{\mathcal{N}}_{\Sigma }$, so the prefix 〈 is strongly subhedge irrelevant. The set $clas{s}_{\mathit{S}}^L\left(\langle a\rangle \right)=\langle {\mathcal{N}}_{\Sigma }\rangle$ is subhedge-relevant, so the prefix $\langle a\rangle$ is not strongly subhedge irrelevant. The set $clas{s}_{\mathit{S}}^L(\langle \rangle ·a)=\langle {\mathcal{N}}_{\Sigma }\rangle ·a·{\mathcal{N}}_{\Sigma }$ is subhedge irrelevant, so the prefix $\langle \rangle ·a$ is strongly subhedge irrelevant.

Definition 10. 

A dSha^↓ $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ is complete for subhedge projection for schema $\mathbf{S}$ if A is schema-complete for $\mathit{S}$ and for all hedges $h\in \mathbf{S}$ and all prefixes v of $\mathit{nw}\left(h\right)$ that are strongly subhedge irrelevant for Δ, the unique run of A on h goes to some subhedge projection state of Δ at v.

Example 8. 

The dSha  in Figure 17 is complete for subhedge projection for our running Example 6 where $L=\{\langle a\rangle \}$ and $\mathit{S}=L\cup \{\langle \rangle ·a\}$. It was obtained by congruence projection. On all prefixes of the subhedge irrelevant class $clas{s}_{\mathit{S}}^L(\langle )$, it goes to the subhedge projection state $\left\{0\right\}D1$. On all prefixes of the class $clas{s}_{\mathit{S}}^L\left(\langle a\rangle \right)$, which is not subhedge irrelevant, it goes to the state $\{3,4\}D0$, which is not a subhedge projection state. On all prefixes of the subhedge irrelevant class $clas{s}_{\mathit{S}}^L(\langle \rangle ·a)$, it goes to the subhedge projection state $\left\{5\right\}D0$.

Example 9. 

The dSha^↓ from Figure 7 is complete for subhedge projection with schema $〚\mathtt{doc}〛$. It can be obtained by safe-no-change projection. In general, however, dSha^↓s obtained by safe-no-change projection may not be complete for subhedge projection, as we will discuss in Section 6.3.

5.3. In-Memory Evaluator with Subhedge Projection

We next show how to refine the transitions for Sha^↓s by subhedge projection. We define the transition relation with subhedge projection ${\stackrel{h}{\to }}_{\mathit{shp}}\subseteq \mathcal{Q}\times \mathcal{Q}$ with respect to $\Delta$, such that, for all hedges $h,h^{\prime }\in {\mathcal{H}}_{\Sigma }$ and letters $a\in \Sigma$,

$$\frac{q\in {\mathcal{Q}}_{\mathit{shp}}^{\Delta }h\in {\mathcal{H}}_{\Sigma }}{q{\stackrel{h}{\to }}_{\mathit{shp}}q\mathrm{wrt}\Delta }\hspace{1em}\frac{q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }q\stackrel{a}{\to }{q}^{\prime }\mathrm{in}\Delta }{q{\stackrel{a}{\to }}_{\mathit{shp}}{q}^{\prime }\mathrm{wrt}\Delta }\hspace{1em}\frac{q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }}{q{\stackrel{\epsilon }{\to }}_{\mathit{shp}}q\mathrm{wrt}\Delta }$$

$$\frac{q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }q{\stackrel{h}{\to }}_{\mathit{shp}}{q}^{\prime }\mathrm{wrt}\Delta q^{\prime }{\stackrel{h^{\prime }}{\to }}_{\mathit{shp}}{q}^{\prime \prime }\mathrm{wrt}\Delta }{q{\stackrel{h·h^{\prime }}{\to }}_{\mathit{shp}}{q}^{″}\mathrm{wrt}\Delta }$$

$$\frac{q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }q\stackrel{\langle \rangle }{\to }p\mathrm{in}\Delta p{\stackrel{h}{\to }}_{\mathit{shp}}{p}^{\prime }q@p^{\prime }\to q^{\prime }\mathrm{in}\Delta }{q{\stackrel{\langle h\rangle }{\to }}_{\mathit{shp}}{q}^{\prime }\mathrm{wrt}\Delta }$$

The first rule says that subhedge projecting transitions stay in subhedge projection states until the end of the current subhedge is reached. This is correct by Lemma 6 under the condition that there are no blocking runs. The other rules state that the evaluator behaves without subhedge projection in other states.

Proposition 3. 

Let $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ be a Sha^↓ that is schema-complete for $\mathit{S}$. Then, for all hedges $h\in \mathit{S}$ and states $q,p\in \mathcal{Q}$, the following applies:

$$q\stackrel{\mathit{h}}{\to }p\mathit{wrt}\Delta \mathit{iff}q{\stackrel{\mathit{h}}{\to }}_{\mathit{shp}}p\mathit{wrt}\Delta$$

Proof. 

We distinguish two cases as follows:

Case 

$q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }$. Then for any $p\in \mathcal{Q}$, $q\stackrel{h}{\to }p\mathrm{wrt}\Delta$ iff $q{\stackrel{h}{\to }}_{\mathit{shp}}p\mathrm{wrt}\Delta$ by definition of the projecting transitions.

Case 

$q\in {\mathcal{Q}}_{\mathit{shp}}^{\Delta }$. Then, $q{\stackrel{h}{\to }}_{\mathit{shp}}q\mathrm{wrt}\Delta$. Since $h\in \mathbf{S}$ and A are schema-complete for $\mathbf{S}$, there exists some run of A on h; thus, the state is q, such that $q\stackrel{h}{\to }{q}^{\prime }\mathrm{wrt}\Delta$. Lemma 6 shows that $q^{\prime }=q$, thus $q\stackrel{\mathit{h}}{\to }q\mathrm{wrt}\Delta$.

   □

For evaluating a nondeterministic Sha^↓ $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ with subhedge projection on some hedge h whose graph is stored in memory, we can compute the unique subset $P\subseteq \mathcal{Q}$, such that $I{\stackrel{h}{\to }}_{\mathit{shp}}P$ with respect to the transition relation ${\Delta }^{det}$ of determinization of A and test whether $P\cap F\ne \varnothing$. When reaching any subhedge projection state $P^{\prime }$ while doing so, the evaluator can directly jump to the end of the current subhedge by applying the “stay” rule of the determinization of A:  

$$\frac{P^{\prime }\in {\mathcal{Q}}_{\mathit{shp}}^{{\Delta }^{det}}h\in {\mathcal{H}}_{\Sigma }}{P^{\prime }{\stackrel{h}{\to }}_{\mathit{shp}}{P}^{\prime }\mathrm{wrt}{\Delta }^{det}}$$

Thereby, it will avoid visiting any subhedge in a subhedge projection state of the determinization of A. The running time will thus depend linearly on the size of the non-projected part of h under the assumption that the determinization of A is available.

As before, only the needed part of the determinization of A for evaluating h is to be produced on-the-fly. Overall, this costs polynomial time in the size of the needed part of the determinization (but not necessarily linear time since the determinization of Sha^↓s is more costly than for Shas). Also note that whenever discovering a new state $P^{\prime }$ of the determinization of A, we have to test whether it is a subhedge projection state. For this, we must compute the state $P^{″}$, such that $P^{\prime }\stackrel{\langle \rangle }{\to }{P}^{″}$ wrt ${\Delta }^{det}$ and test whether only the transition rules allowed for subhedge projection states for $P^{\prime }$ with witness $P^{″}$ are available wrt ${\Delta }^{det}$.

6. Safe-No-Change Projection

We want to solve the regular pattern-matching problem for hedges with subhedge projection. We assume that the regular pattern is given by a dSha while the schema may be arbitrary, except that the dSha must be schema-complete (see Section 4.3). We then present the safe-no-change projection algorithm, which compiles the dSha into some dSha^↓ while introducing subhedge projection states.

The idea of the safe-no-change projection is to push information top-down by which to distinguish looping states that will safely no more be changed. Thereby, subhedge projection states are produced, as we will illustrate by examples. The soundness proof of safe-no-change projection is nontrivial and instructive for the soundness proof of congruence projection in Section 7.3. Completeness for subhedge projection cannot be expected, since the safe-no-change projection does only take simple loops into account. More general loops cannot be treated and it is not clear how that could be done. The congruence projection algorithm in Section 7.2 will eventually give an answer to this.

We keep the safe-no-change compiler independent of the schema and, therefore, do not have to assume its regularity. But, we have to assume the schema-completeness of the input dSha in order to show that the output dSha^↓ recognizes the same language within the schema. The regular pattern-matching problem can then be solved in memory and with subhedge projection by evaluating the dSha^↓ on the input hedge in memory and with subhedge projection, as discussed in Section 5.3.

6.1. Algorithm

We now describe the safe-no-change projection algorithm. Let $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ be a Sha with schema $\mathbf{S}$. The algorithm takes the dSha A as input and compiles it to some dSha^↓ $A^{\mathit{snc}}$. We will state why $A^{\mathit{snc}}$ is sound for subhedge projection under the condition that A is schema-complete for schema $\mathbf{S}$. We do not even have to assume that the schema $\mathbf{S}$ is regular. The proof is eight pages long and, therefore, delegated to the Appendix B.

For any set $P\subseteq \mathcal{Q}$, we define the set of states that safely lead to P:

$${\mathit{safe}}^{\Delta }\left(P\right)=\{q\in \mathcal{Q}\mid {\mathit{acc}}^{\Delta }\left(\left\{q\right\}\right)\subseteq P\}.$$

So, state q is safe for P if any hedge read from q on which the run with $\Delta$ does not block must reach some state in P. We note that ${\mathit{safe}}^{\Delta }\left(P\right)$ can be computed in linear time in the size of $\Delta$ using inverse hedge accessibility from P. We define the set of states that will no longer change:

$${\mathit{no-change}}^{\Delta }=\{q\mid q\in {\mathit{safe}}^{\Delta }\left(\left\{q\right\}\right)\}.$$

So, $q\in {\mathit{no-change}}^{\Delta }$ if and only if ${\mathit{acc}}^{\Delta }\left(\left\{q\right\}\right)\subseteq \left\{q\right\}$. This means that all transition rules starting in q must loop in q. In the example automaton from Figure 4, the self-looping states are those in ${\mathit{no-change}}^{\Delta }=\{2,3,4,5\}$.

Lemma 7. 

Let $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ be a  Sha  that is schema-complete for schema $\mathit{S}$. For any hedge $h\in \mathit{S}$ and state $q\in I\cap {\mathit{no-change}}^{\Delta }$, it then holds that $q\stackrel{h}{\to }q$ wrt Δ.

Proof. 

The schema-completeness of A for $\mathbf{S}$ applied to $h\in \mathbf{S}$ and $q\in I$ yields the existence of some state $q^{\prime }\in \mathcal{Q}$, such that $q\stackrel{h}{\to }{q}^{\prime }$ wrt $\Delta$. Let $q^{\prime }$ be any such state. Note that $q^{\prime }\in {\mathit{acc}}^{\Delta }\left(q\right)$. Since $q\in {\mathit{no-change}}^{\Delta }$, we have $q\in {\mathit{safe}}^{\Delta }\left(\left\{q\right\}\right)$, so that $q^{\prime }\in {\mathit{acc}}^{\Delta }\left(\left\{q\right\}\right)$ implies $q^{\prime }=q$. This proves $q\stackrel{h}{\to }q$ wrt $\Delta$.   □

For any state $q\in \mathcal{Q}$ and subset of states $\mathit{Q}\subseteq \mathcal{Q}$, we define the following:

$$\begin{array}{c}{\mathit{s-down}}^{\Delta }(q,\mathit{Q})={\mathit{safe}}^{\Delta }\left(\{p\in \mathcal{Q}\mid q{@}^{\Delta }p\subseteq \mathit{Q}\}\right)\hfill \\ {\mathit{s-no-change}}^{\Delta }\left(q\right)={\mathit{s-down}}^{\Delta }(q,\left\{q\right\})\hfill \end{array}$$

A state belongs to ${\mathit{s-down}}^{\Delta }(q,\mathit{Q})$ if all states $p\in {\mathit{acc}}^{\Delta }\left(q\right)$ satisfy $q{@}^{\Delta }p\subseteq \mathit{Q}$. So, p is a state in a down level that will safely go up to some state in $\mathit{Q}$. A state p belongs to ${\mathit{s-no-change}}^{\Delta }\left(q\right)$ if it safely does not change q, i.e., if $\left\{q\right\}{@}^{\Delta }{\mathit{acc}}^{\Delta }\left(\left\{p\right\}\right)\subseteq \left\{q\right\}$.

We next compile the Sha A to a Sha^↓ $A^{\mathit{snc}}=(\Sigma ,{\mathcal{Q}}^{\mathit{snc}},$${\Delta }^{\mathit{snc}},$$I^{\mathit{snc}},F^{\mathit{snc}})$. For this, let $\mathrm{\Pi }$ be a fresh symbol and consider the state set as follows:

$${\mathcal{Q}}^{\mathit{snc}}=\left\{\mathrm{\Pi }\right\}\uplus (\mathcal{Q}\times 2^{\mathcal{Q}}).$$

In practice, we restrict the state space to the those states that are accessible or clean the dShas keeping only those states that are used in some successful run. But, in the worst case, the construction may indeed be exponential. Example 15 can be adapted to show this.

A pair $(q,P)$ means the state q will safely no longer change in the current subhedge if $q\in P\cup {\mathit{no-change}}^{\Delta }$. In this case, the subhedge can be projected. The sets of initial and final states are defined as follows:

$$\begin{array}{c}{I}^{\mathit{snc}}=I\times \{\varnothing \}{F}^{\mathit{snc}}=F\times \{\varnothing \}\hfill \end{array}.$$

How to generate the transition rules of $A^{\mathit{snc}}$ from those of A is described in Figure 10. On states assigned on top-level $(q,P)$, the set P is empty so that only the states in ${\mathit{no-change}}^{\Delta }$ are safe for no change. This is why the definition of $I^{\mathit{snc}}$ and $F^{\mathit{snc}}$ use $P=\varnothing$. When going down from a state $(q,P)$ for which q is safe to not change, i.e., $q\in P\cup {\mathit{no-change}}^{\Delta }$, then the following rule is applied:

$$\frac{q\in P\cup {\mathit{no-change}}^{\Delta }}{(q,P)\stackrel{\langle \rangle }{\to }\mathrm{\Pi }\mathrm{in}{\Delta }^{\mathit{snc}}.}$$

The evaluation on the lower level goes to the extra state $\mathrm{\Pi }$, where it then loops until going back to q on the upper level. When going down from some state $(q,P)$, such that $q\notin P\cup {\mathit{no-change}}^{\Delta }$, then the following rule is applied:

$$\frac{\stackrel{\langle \rangle }{\to }{q}^{\prime }\mathrm{in}\Delta q\notin P\cup {\mathit{no-change}}^{\Delta }}{(q,P)\stackrel{\langle \rangle }{\to }(q^{\prime },{\mathit{s-no-change}}^{\Delta }\left(q\right))\mathrm{in}{\Delta }^{\mathit{snc}}}$$

The states in the set ${\mathit{s-no-change}}^{\Delta }\left(q\right)$ on the lower level safely will not make q change on the upper level for any subhedge to come (we could detect more irrelevant subhedges by allowing q to change but only in a unique manner. This can be obtained with the alternative definition: ${\mathit{s-no-change}}^{\prime }\Delta \left(q\right)={\cup }_{r\in \mathcal{Q}}{\mathit{s-down}}^{\Delta }(q,\left\{r\right\})$. Computing this set would require quadratic time $O\left(n\right|A\left|\right)$, while computing ${\mathit{s-no-change}}^{\Delta }\left(q\right)$ can be carried out in linear time $O\left(\right|A\left|\right)$).

When applied to the Sha in Figure 4 for $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$, the construction yields the Sha^↓ in Figure 11, which is indeed equal to the Sha^↓ from Figure 7 up to state renaming. When run on the hedge $\langle \mathtt{list}·\langle \mathtt{list}·h_1\rangle ·\langle \mathtt{item}·h_2\rangle \rangle$, as shown in Figure 8, it does not have to visit subhedges $h_1$ or $h_2$ since all of them will be reached starting from the projection state $\mathrm{\Pi }$.

6.2. Soundness

We next state and prove a soundness result for safe-no-change projection.

Theorem 1 (Soundness of safe-no-change projection.).

If a Sha A is schema-complete for some schema $\mathit{S}$, then safe-no-change projection for A preserves the language within this schema: $\mathcal{L}\left(A^{\mathit{snc}}\right)\cap \mathit{S}=\mathcal{L}\left(A\right)$.

The projecting in-memory evaluator of $A^{\mathit{snc}}$ will be more efficient than the non-projecting evaluator of A. Note that the size of $A^{\mathit{snc}}$ may be exponentially bigger than that of A. Therefore, for evaluating a dSha A with subhedge projection on a given hedge h, we may prefer only the needed part of $A^{\mathit{snc}}$ on-the-fly. This part has a size of $O\left(\right|h\left|\right)$ and can be computed in time $O\left(\right|A\left|\right|h\left|\right)$ so the exponential preprocessing time is avoided.

6.3. Incompleteness

Safe-no-change projection may be incomplete for subhedge projection so that not all prefixes that are strongly subhedge irrelevant are mapped to subhedge projection states. This is shown by the counter example dSha A for the XPath filter $[\mathtt{child}::\mathtt{item}]$ in Figure 12. Its safe-no-change projection $A^{\mathit{snc}}$ is given in Figure 13. Note that the prefix $u=\langle \mathtt{item}.\langle \mathtt{item}$ has the class $clas{s}_{\mathbf{S}}^{\mathcal{L}\left(A\right)}\left(u\right)=\langle \mathtt{item}.\langle \mathtt{item}.{\mathcal{N}}_{\Sigma }$, which is subhedge irrelevant so u is strongly subhedge irrelevant for the XPath filter $[\mathtt{child}::\mathtt{item}]$. Nevertheless, it leads to the state $(2,\{1,3,5,6\left\}\right)$, which is not a subhedge projection state since $2\notin \{1,3,5,6\}$. The problem is that this state can still be changed to $(4,\{1,3,5,6\left\}\right)$. This state is somehow equivalent with respect to the filter but not equal to $(2,\{1,3,5,6\left\}\right)$.

Another incompleteness problem should be mentioned: safe-no-change projection is sensitive to automata completion as noticed earlier in Example 5. This is because a state may belong to ${\mathit{no-change}}^{\Delta }$ before completion but no more after adding a sink. Nevertheless, such states never change on any tree satisfying the schema. This problem can be applied, for instance, to example dSha in Figure 4 for XPath query $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$. Therefore, it was important to assume only schema-completeness for safe-no-change projection and not to impose full completeness.

7. Congruence Projection

We present the congruence projection algorithm for detecting irrelevant subhedges for regular hedge patterns with regular schema restrictions. We prove that congruence projection is complete for subhedge projection, resolving the incompleteness of safe-no-change projection. For this, congruence projection may no more ignore the schema, so we have to assume that the input schema is regular too.

7.1. Ideas

The starting idea of congruence projection is to resolve the counter example for the completeness of safe-no-change projection in Figure 13. There, a state is changed when reading an irrelevant subhedge hedge. But, the state change moves to a somehow equivalent state, so it is not really relevant. Therefore, the idea is to detect when a state always remains equivalent rather than unchanged when reading an arbitrary subhedge. This is eventually done by the congruence projection in Figure 14 that we are going to construct.

The obvious question is as follows: which equivalence relation do we choose? Suppose that we want to test whether a hedge satisfying a regular schema $\mathbf{S}$ matches a regular pattern L. In the restricted case of words without schema restrictions, the idea could be to use Myhill–Nerode’s congruence $\overline{{\mathit{diff}}_{{\Sigma }^{\ast }}^L}$ but mapped to states. In the general case, however, the situation becomes more complex, given that in the general case $\overline{{\mathit{diff}}_{\mathbf{S}}^L}$ may fail to be an equivalence relation. So, it may not be a congruence, as already illustrated in Example 4. Furthermore, the treatment of the nesting of hedges will require to update the considered relation when moving down a hedge.

7.1.1. Difference Relations on States

The congruence projection algorithm will maintain a difference relation on states of the dSha which at the beginning will be induced by the difference relation on hedges ${\mathit{diff}}_{\mathbf{S}}^L$ and updated whenever moving down a hedge.

Definition 11. 

Let $(\Sigma ,\mathcal{Q},\Delta ,\_,\_)$ be a dSha. A difference relation for $\Delta$ is a symmetric relation on states $D\subseteq \mathcal{Q}\times \mathcal{Q}$, such that for all $h\in {\mathcal{H}}_{\Sigma }$,

$$(q\stackrel{h}{\to }{q}^{\prime }\mathit{wrt}\Delta \wedge p\stackrel{h}{\to }{p}^{\prime }\mathit{wrt}\Delta \wedge (q^{\prime },p^{\prime })\in D)\Rightarrow (q,p)\in D.$$

The set of all difference relations for Δ is denoted by ${\mathcal{D}}^{\Delta }$.

We call a subset $\mathit{Q}\subseteq \mathcal{Q}$ compatible with a difference relation $D\in {\mathcal{D}}^{\Delta }$ if ${\mathit{Q}}^2\cap D=\varnothing$. This means that no two states of $\mathit{Q}$ may be different with respect to D.

Definition 12. 

Let $(\Sigma ,\mathcal{Q},\Delta ,\_,\_)$ be a Sha and D a difference relation for Δ. A subset of states $\mathit{Q}\subseteq \mathcal{Q}$ is subhedge irrelevant for D wrt Δ if ${\mathit{acc}}^{\Delta }\left(\mathit{Q}\right)$ is compatible with D. The set of all subhedge irrelevant subsets of states for D wrt. Δ thus is as follows:

$${\mathit{dPrj}}^{\Delta }\left(D\right)=\{\mathit{Q}\subseteq \mathcal{Q}\mid {\mathit{acc}}^{\Delta }{\left(\mathit{Q}\right)}^2\cap D=\varnothing \}.$$

A state $q\in \mathcal{Q}$ is subhedge irrelevant for D if the singleton $\left\{q\right\}$ is subhedge irrelevant for D. The set of all subhedge irrelevant states of $\mathcal{Q}$ is denoted by ${\mathit{Prj}}^{\Delta }\left(D\right)$.

We consider subhedge-irrelevance for subsets of states since the congruence projection algorithm has to eliminate the nondeterminism that it introduces by a kind of Sha determinization. Determinization is necessary in order to recognize subhedge irrelevant prefixes properly: all single states in a subset may be subhedge irrelevant while the whole subset is not.

7.1.2. Least Difference Relations

For any binary relation $R\subseteq \mathcal{Q}\times \mathcal{Q}$, let ${\mathit{ldr}}^{\Delta }\left(R\right)$ be the least difference relation on states that contains R.

Lemma 8. 

$(p_1,p_2)\in {\mathit{ldr}}^{\Delta }\left(R\right)\iff \exists (q_1,q_2)\in R.\exists h\in {\mathcal{H}}_{\Sigma }.p_1\stackrel{h}{\to }{q}_1\mathit{wrt}\Delta \wedge p_2\stackrel{h}{\to }{q}_2\mathit{wrt}\Delta$.

Proof. 

The set $\{(p_1,p_2)\in {\mathcal{Q}}^2\mid \exists (q_1,q_2)\in R.\exists h\in {\mathcal{H}}_{\Sigma }.p_1\stackrel{h}{\to }{q}_1\mathrm{wrt}\Delta \wedge p_2\stackrel{h}{\to }{q}_2\mathrm{wrt}\Delta \}$ clearly is a difference relation that contains R, and thus, contains the least such difference relation ${\mathit{ldr}}^{\Delta }\left(R\right)$. Conversely, each pair in the above set must be contained in any difference relation containing R, and thus, in ${\mathit{ldr}}^{\Delta }\left(R\right)$.   □

Lemma 9. 

For any $R\subseteq \mathcal{Q}\times \mathcal{Q}$, the difference relation ${\mathit{ldr}}^{\Delta }\left(R\right)$ is the value of predicate D in the least fixed point of the ground datalog program generated by the following inference rules:

$$\frac{p,q\in \mathcal{Q}}{D(p,q):-R(p,q).}\hspace{1em}\frac{p_1\stackrel{a}{\to }{p}_2\mathrm{wrt}\Delta q_1\stackrel{a}{\to }{q}_2\mathrm{wrt}\Delta }{D(p_1,q_1):-D(p_2,q_2).}$$

$$\frac{p_1@p\to p_2\mathrm{wrt}\Delta q_1@q\to q_2\mathrm{wrt}\Delta }{D(p_1,q_1):-D(p_2,q_2).}\hspace{1em}\frac{p,q\in \mathcal{Q}}{D(q,p):-D(p,q).}$$

Proof. 

The first inference rule guarantees that $R\subseteq D$. The three later inference rules characterize difference relations $D\in {\mathcal{D}}^{\Delta }$. The second and third rules state that differences in D are propagated backwards over hedges h. This is done recursively by treating letter hedges $h=a$ by the second rule and tree hedges $h=\langle h^{\prime }\rangle$ by the third rule. The fourth rules expresses the symmetry of difference relations D. So, the least fixed point of the datalog program generated by the above rules contains R and is the least relation that satisfies all axioms of difference relations, so it is ${\mathit{ldr}}^{\Delta }\left(R\right)$.   □

Proposition 4. 

For any $R\subseteq \mathcal{Q}\times \mathcal{Q}$, the least difference relation ${\mathit{ldr}}^{\Delta }\left(R\right)$ can be computed in time $O\left(\right|A{|}^2)$.

Proof. 

The size of the ground datalog program computing ${\mathit{ldr}}^{\Delta }\left(R\right)$ from Lemma 9 is at most quadratic in the size of A, so its least fixed point can be computed in time $O\left(\right|A{|}^2)$.   □

7.2. Algorithm

We now define the congruence projection algorithm by a compiler from dShas and a set of schema final states to Sha^↓s, which, when run on a hedge, can detect all subhedge irrelevant prefixes.

7.2.1. Inputs and Outputs

As inputs, the congruence projection algorithm is given a dSha $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ for the regular pattern and a set $F_{\mathbf{S}}$ with $F\subseteq F_{\mathbf{S}}\subseteq \mathcal{Q}$. The dSha defines the regular language $L=\mathcal{L}\left(A\right)$ while the regular schema $\mathbf{S}$ is recognized by the dSha $A[F/F_{\mathbf{S}}]=(\Sigma ,\mathcal{Q},\Delta ,I,F_{\mathbf{S}})$. So, the same base automaton defines the regular language and regular schema by choosing different sets of final states.

Example 10. 

In the example dSha  in Figure 4 for the XPath filter $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$ with schema $〚\mathtt{doc}〛$, we have $F=\left\{4\right\}$ and $F_{\mathit{S}}=\{4,5\}$. In the automaton for the counter example $[\mathtt{child}::\mathtt{item}]$ for safe-no-change projection in Figure 12, we have $F=\left\{6\right\}$ and $F_{\mathit{S}}=\{5,6\}$.

We note that, if L and $\mathbf{S}$ were given by independent Shas, then we can obtain a common dSha A and a set $F_{\mathbf{S}}$ as above by computing the product of the dShas for L and $\mathbf{S}$ and completing it. As noticed in [16], it may be more efficient to determinize the Sha for $\mathbf{S}$ in a first step, build the product of the Sha for L with the dSha for $\mathbf{S}$ in a second, and determinize this product in the third step.

The congruence projection of A wrt. $F_{\mathbf{S}}$ will maintain in its current configuration a subset of states $\mathit{Q}\subseteq \mathcal{Q}$ and a difference relation on states in $D\in {\mathcal{D}}^{\Delta }$. Thereby, the congruence projection dSha^↓ can always detect whether the current prefix is subhedge irrelevant for L wrt. schema $\mathbf{S}$ by testing whether the current set of states $\mathit{Q}$ is subhedge irrelevant for the current difference relation D.

7.2.2. Initial Difference Relation

The initial difference relation on states $D_{init}^{A,F_{\mathbf{S}}}\in {\mathcal{D}}^{\Delta }$ is induced from the difference relation on prefixes ${\mathit{diff}}_{\mathbf{S}}^L$ as follows:

$$D_{init}^{A,F_{\mathbf{S}}}=\{(q^{\prime },q^{″})\mid \exists (v^{\prime },v^{″})\in {\mathit{diff}}_{\mathbf{S}}^L.\exists q_0\in I.q_0\stackrel{\mathit{hdg}\left(v^{\prime }\right)}{\to }{q}^{\prime }\mathrm{wrt}\Delta \wedge q_0\stackrel{\mathit{hdg}\left(v^{″}\right)}{\to }{q}^{″}\mathrm{wrt}\Delta \}.$$

The next lemma indicates how $D_{init}^{A,F_{\mathbf{S}}}$ can be defined from A and $F_{\mathbf{S}}$ without reference to the languages $L=\mathcal{L}\left(A\right)$ and $\mathbf{S}=\mathcal{L}\left(A[F/F_{\mathbf{S}}]\right)$.

Lemma 10. 

$D_{init}^{A,F_{\mathit{S}}}={\mathit{ldr}}^{\Delta }(F\times (F_{\mathit{S}}\backslash F))\cap {\mathit{acc}}^{\Delta }{\left(I\right)}^2$.

Proof. 

For one direction let $(p^{\prime },p^{″})\in D_{init}^{A,F_{\mathbf{S}}}$. Then, there exist nested words $(v^{\prime },v^{″})\in {\mathit{diff}}_{\mathbf{S}}^L$ and an initial state $q_0\in I$, such that $q_0\stackrel{\mathit{hdg}\left(v^{\prime }\right)}{\to }{p}^{\prime }\mathrm{wrt}\Delta$ and $q_0\stackrel{\mathit{hdg}\left(v^{″}\right)}{\to }{p}^{″}\mathrm{wrt}\Delta$. Since $v^{\prime },v^{″}$ are nested words, hedge accessibility $(p^{\prime },p^{″})\in {\mathit{acc}}^{\Delta }{\left(I\right)}^2$ follows. Furthermore, $(v^{\prime },v^{″})\in {\mathit{diff}}_{\mathbf{S}}^L$ requires the existence of a hedge $h\in {\mathcal{H}}_{\Sigma }$, such that $\mathit{hdg}\left(v^{\prime }\right)·h\in L$ and $\mathit{hdg}\left(v^{″}\right)·h\in \mathbf{S}\backslash L$. Hence, there are states $q^{\prime }\in F$ and $q^{″}\in F_{\mathbf{S}}\backslash F$, such that $p^{\prime }\stackrel{h}{\to }{q}^{\prime }$ and $p^{″}\stackrel{h}{\to }{q}^{″}$. Using Lemma 8, this implies that $(p^{\prime },p^{″})\in {\mathit{ldr}}^{\Delta }(F\times (F_{\mathbf{S}}\backslash F))$.

For the other direction, let $(p^{\prime },p^{″})\in {\mathit{ldr}}^{\Delta }(F\times (F_{\mathbf{S}}\backslash F))\cap {\mathit{acc}}^{\Delta }{\left(I\right)}^2$. Using Lemma 8, property $(p^{\prime },p^{″})\in {\mathit{ldr}}^{\Delta }(F\times (F_{\mathbf{S}}\backslash F))$ shows that there exist states $q^{\prime }\in F$, $q^{″}\in F_{\mathbf{S}}\backslash F$, and $h\in {\mathcal{H}}_{\Sigma }$, such that $p^{\prime }\stackrel{h}{\to }{q}^{\prime }\mathrm{wrt}\Delta$ and $p^{″}\stackrel{h}{\to }{q}^{″}\mathrm{wrt}\Delta$. From $(p^{\prime },p^{″})\in {\mathit{acc}}^{\Delta }{\left(I\right)}^2$, it follows that there are nested words $v^{\prime },v^{″}\in {\mathcal{N}}_{\Sigma }$ and an initial state $q_0\in I$, such that $q_0\stackrel{\mathit{hdg}\left(v^{\prime }\right)}{\to }{p}^{\prime }\mathrm{wrt}\Delta$ and $q_0^{\prime }\stackrel{\mathit{hdg}\left(v^{″}\right)}{\to }{p}^{″}\mathrm{wrt}\Delta$. Hence, $(v^{\prime },v^{″})\in {\mathit{diff}}_{\mathbf{S}}^L$, so that $(q^{\prime },q^{″})\in D_{init}^{A,F_{\mathbf{S}}}$.   □

As a consequence of Lemma 10 and Proposition 4, the initial difference relation $D_{init}^{A,F_{\mathbf{S}}}$ can be computed in time $O\left(\right|A{|}^2)$ from A and $F_{\mathbf{S}}$.

Proposition 5 (Soundness of the initial difference relation).

Let $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ be a complete dSha, $F\subseteq F_{\mathit{S}}\subseteq \mathcal{Q}$, $L=\mathcal{L}\left(A\right)$, and $\mathit{S}=\mathcal{L}\left(A[F/F_{\mathit{S}}]\right)$. For any hedge $h\in {\mathcal{H}}_{\Sigma }$ and state $q\in \mathcal{Q}$, such that $q_0\stackrel{h}{\to }q$ wrt Δ for some $q_0\in I$, the nested word $\mathit{nw}\left(h\right)$ is subhedge irrelevant for L and $\mathbf{S}$ if and only if q is subhedge irrelevant for $D_{init}^{A,F_{\mathbf{S}}}$ wrt. Δ.

Proof. 

We show that $\mathit{nw}\left(h\right)$ is subhedge relevant for L and $\mathbf{S}$ if and only if q is subhedge relevant for $D_{init}^{A,F_{\mathbf{S}}}$ wrt. $\Delta$.

“⇒”

Let $\mathit{nw}\left(h\right)$ be subhedge relevant for L and $\mathbf{S}$. Then, there exist hedges $h^{\prime },h^{″}\in {\mathcal{H}}_{\Sigma }$ and a nested word $w\in {\mathcal{N}}_{\Sigma }$, such that

$$\mathit{nw}\left(h\right)·\mathit{nw}\left(h^{\prime }\right)·w\in L\wedge \mathit{nw}\left(h\right)·\mathit{nw}\left(h^{″}\right)·w\in \mathbf{S}\backslash L.$$

Since A is deterministic and $q_0\stackrel{h}{\to }q$ wrt $\Delta$ for the unique $q_0\in I$, it follows that there exist states $p^{\prime },p^{″}\in \mathcal{Q}$, $q^{\prime }\in F$, and $q^{″}\in F_{\mathbf{S}}\backslash F$, such that

$$q\stackrel{h^{\prime }}{\to }{p}^{\prime }\stackrel{\mathit{hdg}\left(w\right)}{\to }{q}^{\prime }\mathrm{wrt}\Delta \wedge q\stackrel{h^{″}}{\to }{p}^{″}\stackrel{\mathit{hdg}\left(w\right)}{\to }{q}^{″}\mathrm{wrt}\Delta .$$

From $q^{\prime }\in F$ and $q^{″}\in F_{\mathbf{S}}\backslash F$, it follows that $(q^{\prime },q^{″})\in D_{init}^{A,F_{\mathbf{S}}}$. Since $D_{init}^{A,F_{\mathbf{S}}}$ is a difference relation, $p^{\prime }\stackrel{\mathit{hdg}\left(w\right)}{\to }{q}^{\prime }\mathrm{wrt}\Delta$ and $p^{″}\stackrel{\mathit{hdg}\left(w\right)}{\to }{q}^{″}\mathrm{wrt}\Delta$, this implies that $(p^{\prime },p^{″})\in D_{init}^{A,F_{\mathbf{S}}}$, too. Hence, $(p^{\prime },p^{″})\in {\mathit{acc}}^{\Delta }{\left(\left\{q\right\}\right)}^2\cap D_{init}^{A,F_{\mathbf{S}}}$, i.e., q, is relevant for $D_{init}^{A,F_{\mathbf{S}}}$ wrt. $\Delta$.

“⇐”

Let q be subhedge relevant for $D_{init}^{A,F_{\mathbf{S}}}$ wrt. $\Delta$. Then, there exist a pair $(p^{\prime },p^{″})\in {\mathit{acc}}^{\Delta }{\left(\left\{q\right\}\right)}^2\cap D_{init}^{A,F_{\mathbf{S}}}$. So, there are hedges $h^{\prime }$ and $h^{″}$, such that

$$q\stackrel{h^{\prime }}{\to }{p}^{\prime }\mathrm{wrt}\Delta \wedge q\stackrel{h^{″}}{\to }{p}^{″}\mathrm{wrt}\Delta .$$

Since $D_{init}^{A,F_{\mathbf{S}}}={\mathit{ldr}}^{\Delta }(F\times (F_{\mathbf{S}}\backslash F))$ by Lemma 10, there exist a nested word w and a pair $(q^{\prime },q^{″})\in {\mathcal{Q}}^2$ so that either $(q^{\prime },q^{″})$ or $(q^{″},q^{\prime })$ belongs to $F\times (F_{\mathbf{S}}\backslash F)$ and

$$p^{\prime }\stackrel{\mathit{hdg}\left(w\right)}{\to }{q}^{\prime }\mathrm{wrt}\Delta \wedge p^{″}\stackrel{\mathit{hdg}\left(w\right)}{\to }{q}^{″}\mathrm{wrt}\Delta .$$

Hence, $\mathit{nw}\left(h\right)·\mathit{nw}\left(h^{\prime }\right)·w$ in L and $\mathit{nw}\left(h\right)·\mathit{nw}\left(h^{″}\right)·w\in \mathbf{S}\backslash L$ or vice versa. This shows that $\mathit{nw}\left(h\right)$ is relevant for L and $\mathbf{S}$.

   □

7.2.3. Updating the Difference Relation

For any difference relation $D\subseteq {\mathcal{D}}^{\Delta }$ and subset of states $\mathit{Q}\subseteq \mathcal{Q}$, we define a binary relation ${\mathit{down}}_{\mathit{Q}}^{\Delta }\left(D\right)\subseteq \mathcal{Q}\times \mathcal{Q}$, such that for all states $p_1,p_2\in \mathcal{Q}$:

$$(p_1,p_2)\in {\mathit{down}}_{\mathit{Q}}^{\Delta }\left(D\right)\mathrm{iff}\exists q_1,q_2\in \mathit{Q}.(q_1{@}^{\Delta }{p}_1,q_2{@}^{\Delta }{p}_2)\in D.$$

For any subset of states $\mathit{Q}\subseteq \mathcal{Q}$ and difference relation $D\in {\mathcal{D}}^{\Delta }$, let $D_{\mathit{Q}}\in {\mathcal{D}}^{\Delta }$ be the least difference relation that contains ${\mathit{down}}_{\mathit{Q}}^{\Delta }\left(D\right)$:

$$D_{\mathit{Q}}={\mathit{ldr}}^{\Delta }({\mathit{down}}_{\mathit{Q}}^{\Delta }(D)).$$

Lemma 11. 

The difference relation $D_{\mathit{Q}}$ can be computed in time $O\left(\right|A{|}^2)$ from $\mathit{Q}$, Δ, and D.

Proof. 

The binary relation ${\mathit{down}}_{\mathit{Q}}^{\Delta }\left(D\right)$ can be computed in time $O\left(\right|\Delta {|}^2)$ from $\mathit{Q}$, D, and $\Delta$. The least difference relation ${\mathit{ldr}}^{\Delta }({\mathit{down}}_{\mathit{Q}}^{\Delta }(D))$ can be computed by ground datalog in time $O\left(\right|A{|}^2)$ from ${\mathit{down}}_{\mathit{Q}}^{\Delta }\left(D\right)$ using Proposition 4.   □

Example 11. 

Reconsider the dSha  for the XPath filter $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$ in Figure 4 with the set of schema-final states $F_{\mathit{S}}=\{4,5\}$. Since $F=\left\{4\right\}$, we have $F_{\mathit{S}}\backslash F=\left\{5\right\}$. The initial difference relation is $D0=D_{init}^{A,F_{\mathit{S}}}$, which is the symmetric closure of $\left(\right\{0\}\times \{1,4,5\left\}\right)\cup \left\{\right(1,4),(4,5\left)\right\}$. The subhedge irrelevant states are ${\mathit{Prj}}^{\Delta }\left(D0\right)=\{1,2,3,4,5\}$ since only state 0 can access two states in the difference relation $D0$, the final state in $F=\left\{4\right\}$, and a nonfinal state that is schema-final in $F_{\mathit{S}}\backslash F=\left\{5\right\}$ of some hedges. The difference relation ${\mathit{down}}_{\left\{0\right\}}^{\Delta }\left(D_{init}^{A,F_{\mathit{S}}}\right)$ is the symmetric closure of $\{1,2\}\times \left\{3\right\}$, which is equal to the difference relation $D1={D0}_{\left\{0\right\}}={\mathit{ldr}}^{\Delta }({\mathit{down}}_{\left\{0\right\}}^{\Delta }(D0))$. Hence, the projection states are ${\mathit{Prj}}^{\Delta }\left(D1\right)=\{2,3,4,5\}$ since from states 0 and 1 one can still reach both 1 and 3 while $(1,3)\in D1$. The difference relation $D2={\mathit{down}}_{\left\{1\right\}}^{\Delta }\left(D1\right)$ is the symmetric closure of $\left\{\right(1,2),(2,3\left)\right\}$. Hence, the projection states are ${\mathit{Prj}}^{\Delta }\left(D2\right)=\{1,2,3,4,5\}$. From state 1, one can only reach states 1 and 3, which are not different for $D2$.

Projection states for the initial difference relation contain all states that are safe for selection or safe for rejection with respect to the schema:

Lemma 12. 

All states in ${\mathit{safe}}^{\Delta }\left(F\right)$ and ${\mathit{safe}}^{\Delta }(F_{\mathit{S}}\backslash F)$ are subhedge irrelevant for $D_{init}^{A,F_{\mathit{S}}}$.

• We omit the proof since the result is not used later on.

Given a dSha $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ and a set $F\subseteq F_{\mathbf{S}}\subseteq \mathcal{Q}$, we now construct the congruence projection $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ as the following dSha^↓:

$$A^{\mathit{cgr}\left(\mathbf{S}\right)}=(\Sigma ,{\mathcal{Q}}^{\mathit{cgr}},{\Delta }^{\mathit{cgr}},I^{\mathit{cgr}\left(\mathbf{S}\right)},F^{\mathit{cgr}\left(\mathbf{S}\right)}),$$

where the set of states, initial states, and final states are as follows:

$$\begin{array}{ccc}\hfill {\mathcal{Q}}^{\mathit{cgr}}& \subseteq & 2^{\mathcal{Q}}\times {\mathcal{D}}^{\Delta }\hfill \\ \hfill I^{\mathit{cgr}\left(\mathbf{S}\right)}& =& \left\{(I,D_{init}^{A,F_{\mathbf{S}}})\right\}\hfill \\ \hfill F^{\mathit{cgr}\left(\mathbf{S}\right)}& =& \left\{(\mathit{Q},D_{init}^{A,F_{\mathbf{S}}})\mid \left(\begin{array}{c}Q\notin {\mathit{dPrj}}^{\Delta }\left(D_{init}^{A,F_{\mathbf{S}}}\right)\Rightarrow \mathit{Q}\cap F\ne \varnothing \wedge \hfill \\ Q\in {\mathit{dPrj}}^{\Delta }\left(D_{init}^{A,F_{\mathbf{S}}}\right)\Rightarrow {\mathit{acc}}^{\Delta }\left(\mathit{Q}\right)\cap F\ne \varnothing \hfill \end{array}\right)\right\}\hfill \end{array}$$

The transition rules in ${\Delta }^{\mathit{cgr}}$ are provided by the inference rules in Figure 15. To illustrate construction and why determinization is needed, we reconsider Example 3, where schema restrictions have consequences that at first sight may be counter-intuitive.

Example 12 (Determinization during congruence projection is needed.).

Reconsider Example 3 with signature $\Sigma =\left\{a\right\}$, pattern $L=\{\langle a\rangle \}$, and schema $\mathit{S}=L\cup \{\langle \rangle ·a\}$. This language can be defined by the dSha A in Figure 16. The schema $\mathit{S}$ can be defined by the same automaton but with the schema-final states $F_{\mathit{S}}=\{3,5\}$ and $F=\left\{3\right\}$. The dSha $A^{\mathit{cgr}\left(\mathit{S}\right)}$ produced by congruence projection is shown in Figure 17. The unique hedge $\langle a\rangle$ of the language $L=\mathcal{L}\left(A\right)$ is accepted in state $\left(\right\{3,4\},D\mathit{0})$, where D0$=\left\{\right(3,5),(5,3\left)\right\}$. The unique hedge $\langle \rangle ·a$ in $\mathit{S}\backslash L$ is rejected: after reading the tree $\langle \rangle$, the automaton goes to state $\left(\right\{3,4\},D\mathit{0})$ where it goes to a projecting sink $\left(\right\{5\},D\mathit{0})$ when reading the subsequent letter a. We note that any hedge in $\langle {\Sigma }^{\ast }\rangle$ is accepted in state $\left(\right\{3,4\},D\mathit{0})$ by $A^{\mathit{cgr}\left(\mathit{S}\right)}$, even though only the single hedge $\langle a\rangle$ belongs to L. This is sound given that the hedges in $\langle (\epsilon +a·a·a^{\ast })\rangle$ do not belong to schema $\mathit{S}$ anyway. We notice that $\left(\right\{3,4\},D\mathit{0})$ cannot be a projection state since the run on hedge $\langle \rangle ·a$ must continue to the sink $\left(\right\{5\},D\mathit{0})$; therefore, state $\left(\right\{3,4\},D\mathit{0})$ is subhedge relevant.

Note that both individual states 3 and 4 are subhedge irrelevant for D0 while the subset with both states $\{3,4\}$ is subhedge relevant for D0 since $(3,5)\in {\mathit{acc}}^{\Delta }{\left(\{3,4\}\right)}^2\cap$ D0. This shows that the deterministic construction is indeed needed to decide subhedge irrelevance for cases such as in this example. Also, notice that state 0 is subhedge irrelevant for D1 = ∅. This is fine since the acceptance of hedges of the form $\langle h\rangle ·h^{\prime }$ by $A^{\mathit{cgr}\left(\mathit{S}\right)}$ does indeed not depend on h. In contrast, it depends on $h^{\prime }$, so that the subset of states $\{3,4\}$ cannot be subhedge irrelevant for D0.

Finally, let us discuss how the transition rules of dSha^↓ $A^{\mathit{cgr}\left(\mathit{S}\right)}$ are inferred. The most interesting transition rule is $\left(\right\{0\},D\mathit{0})@\left(\right\{0\},D\mathit{1})\to \left(\right\{3,4\},D\mathit{0})$ in ${\Delta }^{\mathit{cgr}}$. It is produced by the following inference rule, where $\mathit{Q}=P=\left\{0\right\}$, ${\mathit{acc}}^{\Delta }\left(P\right)=\{0,1,3,4,5\}$, ${\mathit{Q}}^{\prime }=\{3,4\}$, $D=D\mathit{0}$, and $D_{\mathit{Q}}=D\mathit{1}$:

$$\frac{\mathit{Q}@{\mathit{acc}}^{\Delta }\left(P\right)\to {\mathit{Q}}^{\prime }\mathrm{in}{\Delta }^{\mathit{det}}\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)P\in {\mathit{dPrj}}^{\Delta }\left(D_{\mathit{Q}}\right)}{(\mathit{Q},D)@(P,D_{\mathit{Q}})\to ({\mathit{Q}}^{\prime },D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

This rule states that if P is subhedge irrelevant for $D_{\mathit{Q}}$, then all states accessible from P must be tried out since all of them could be reached by some different subhedge that got projected away. As a result, one cannot know due to subhedge projection into which state of ${\mathit{Q}}^{\prime }$ one should go. In order to stay deterministic, we thus go into all possible states, i.e., into the subset ${\mathit{Q}}^{\prime }$. In the example, these are all the states that can be reached when reading some hedge in the pattern $\{\langle h\rangle \mid h\in {\mathcal{H}}_{\Sigma }\}$, given that the subhedges $\mathit{h}\in \Sigma$ are not inspected with subhedge projection.

Example 13 (XPath filter $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$ with schema $〚\mathtt{doc}〛$.).

The congruence projection of the example dSha in Figure 4 is given in Figure 18. This dSha is similar to the safe-no-change projection in Figure 7, except that the useless state $2^{\prime }$ is removed and the four projection states are now looping in themselves rather than going to a shared looping state Π. It should also be noticed that only singleton state sets are used there, so no determinization is needed. As we will see, this is typical for experiments on regular XPath queries where larger subsets rarely occur.

Example 14 (Counter-example for completeness of the safe-no-change projection.).

Reconsider the counter example for the completeness of the safe-no-change projection, i.e., the dSha in Figure 12 for the XPath query $[\mathtt{child}::\mathtt{item}]$ with schema final states $F_{\mathit{S}}=\{0,5,6\}$. Its congruence projection is shown in Figure 14. We note that the prefix $\langle item.\langle item$ leads to the state {2}D3, which is a subhedge projection state since 2 is subhedge irrelevant for D3. In particular, note that ${\mathit{acc}}^{\Delta }\left(\left\{2\right\}\right)=\{2,4\}$ so no two states accessible from 2 are different in D3. This means that state 2 may still be changed to 4 but then it does not become different with respect to D3. This resolves the incompleteness issue with the safe-no-change projection on this example.

The first property for states $(\mathit{Q},D)$ assigned by the congruence projection is that $\mathit{Q}$ is compatible with D. This means that no two states in $\mathit{Q}$ are different with respect to D. Intuitively, each state of $\mathit{Q}$ is as good as each other except for leading out of the schema. So, if $(\mathit{Q},D)$ is assigned to a prefix, and a suffix leads from some state in $\mathit{Q}$ to F, then it cannot lead to $F_{\mathbf{S}}\backslash F$ from some other state in $\mathit{Q}$.

Lemma 13. 

If the partial run of $A^{\mathit{cgr}\left(\mathit{S}\right)}$ assigns a state $(\mathit{Q},D)$ to a prefix, then $\mathit{Q}$ is compatible with D.

We omit the proof since this instructive result will not be used directly later on. Still, compatibility will play an important role in the soundness proof.

7.3. Soundness

We next adapt the soundness result and proof from the safe-no-change projection to congruence projection.

Theorem 2 (Soundness of congruence projection.).

For any dSha $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ and set $F\subseteq F_{\mathit{S}}\subseteq \mathcal{Q}$, the congruence projection of A with respect to $F_{\mathit{S}}$ preserves the language of A within schema $\mathit{S}=\mathcal{L}\left(A[F/F_{\mathit{S}}]\right)$, i.e.,

$$\mathcal{L}\left(A^{\mathit{cgr}\left(\mathit{S}\right)}\right)\cap \mathit{S}=\mathcal{L}\left(A\right).$$

We note that $\mathbf{S}$ is a schema for A since $F\subseteq F_{\mathbf{S}}$. Furthermore, A is schema-complete for $\mathbf{S}$ since A is deterministic and $\mathbf{S}=\mathcal{L}\left(A[F/F_{\mathbf{S}}]\right)$.

Proof. 

The proof of the inclusion $\mathcal{L}\left(A\right)\subseteq \mathcal{L}\left(A^{\mathit{cgr}\left(\mathbf{S}\right)}\right)\cap \mathbf{S}$ will be based on the following two claims:

Claim 2.1a. 

For all $h\in {\mathcal{H}}_{\sum },D\in {\mathcal{D}}^{\Delta }$, and $Q\in dPr{j}^{\Delta }\left(D\right),(Q,D)\stackrel{h}{\to }(Q,D)\mathrm{wrt}{\Delta }^{cgr}.$

Proof. 

We prove Claim 2.1a by induction on the structure of h. Suppose that $\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$.

Case 

$h=\langle h^{\prime }\rangle$. The induction hypothesis applied to $h^{\prime }$ yields $(\mathit{Q},D)\stackrel{h^{\prime }}{\to }(\mathit{Q},D)\mathrm{wrt}{\Delta }^{\mathit{cgr}}$. We can thus apply the following two inference rules:

$$\frac{\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)\stackrel{\langle \rangle }{\to }(\mathit{Q},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}\hspace{1em}\frac{\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D))@(\mathit{Q},D)\to (\mathit{Q},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

in order to close the following diagram with respect to ${\Delta }^{\mathit{cgr}}$:

This proves $(\mathit{Q},D)\stackrel{h}{\to }(\mathit{Q},D)$ wrt ${\Delta }^{\mathit{cgr}}$, as required by the claim.

Case 

$h=a$. Since $q\in {\mathit{dPrj}}^{\Delta }\left(D\right)$, we can apply the inference rule:

$$\frac{a\in \Sigma \mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)\stackrel{a}{\to }(\mathit{Q},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

This proves this case of the claim.

Case 

$h=\epsilon$. We trivially have $(\mathit{Q},D)\stackrel{\epsilon }{\to }(\mathit{Q},D)$ wrt ${\Delta }^{\mathit{cgr}}$.

Case 

$h=h^{\prime }·h^{″}$. By the induction hypothesis applied to $h^{\prime }$ and $h^{″}$, we obtain the following: $(\mathit{Q},D)\stackrel{h^{\prime }}{\to }(\mathit{Q},D)$ and $(\mathit{Q},D)\stackrel{h^{″}}{\to }(\mathit{Q},D)$ wrt ${\Delta }^{\mathit{cgr}}$. Hence, $(\mathit{Q},D)\stackrel{h^{\prime }·h^{″}}{\to }(\mathit{Q},D)$ wrt ${\Delta }^{\mathit{cgr}}$.

This ends the proof of Claim 2.1a. The next claim is the key of the soundness proof.   □

For any difference relation D, we define a binary relation ${\mathit{dep}}^{\mathit{a}}\left(D\right)\subseteq 2^{\mathcal{Q}}\times 2^{\mathcal{Q}}$, such that, for any two subsets of states ${\mathit{Q}}^{\prime },{\mathit{Q}}^{″}\subseteq \mathcal{Q}$,

$$({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{a}}\left(D\right)\iff \left\{\begin{array}{cc}{\mathit{Q}}^{\prime }\notin {\mathit{dPrj}}^{\Delta }\left(D\right)\Rightarrow {\mathit{Q}}^{\prime }\subseteq {\mathit{Q}}^{″}\wedge \hfill & \hfill \left(1^a\right)\\ {\mathit{Q}}^{\prime }\in {\mathit{dPrj}}^{\Delta }\left(D\right)\Rightarrow ({\mathit{Q}}^{\prime }\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}^{″}\right)\wedge {\mathit{Q}}^{″}\in {\mathit{dPrj}}^{\Delta }(D\left)\right)\hfill & \hfill \left(2^a\right)\end{array}\right.$$

Furthermore, note that ${\mathit{Q}}^{″}\in {\mathit{dPrj}}^{\Delta }\left(D\right)\wedge {\mathit{Q}}^{\prime }\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}^{″}\right)$ implies ${\mathit{Q}}^{\prime }\in {\mathit{dPrj}}^{\Delta }\left(D\right)$, and thus, $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$.

Claim 2.2a. 

Let $h\in {\mathcal{H}}_{\sum }$ a hedge, $\mathit{Q},{\mathit{Q}}^{\prime }\subseteq \mathcal{Q}$ subsets of states, and $D\in {\mathcal{D}}^{\Delta }$ a difference relation. If $\mathit{Q}\stackrel{h}{\to }{\mathit{Q}}^{\prime } \mathrm{wrt} {\Delta }^{det}$, then there exists ${\mathit{Q}}^{″}\subseteq \mathcal{Q}$, such that $(\mathit{Q},\mathit{D})\stackrel{h}{\to }({\mathit{Q}}^{\prime \prime },\mathit{D}) \mathrm{wrt} {\Delta }^{cgr}$ and $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {dep}^{\mathit{a}}\left(\mathit{D}\right)$.

Proof. 

If $\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$, then $(\mathit{Q},D)\stackrel{h}{\to }(\mathit{Q},D)\mathrm{wrt}{\Delta }^{\mathit{cgr}}$ by Claim 2.1a. Let ${\mathit{Q}}^{″}=\mathit{Q}$. We then have ${\mathit{Q}}^{\prime }\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}^{″}\right)$ and ${\mathit{Q}}^{″}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$, and thus, ${\mathit{Q}}^{\prime }\in {\mathit{dPrj}}^{\Delta }\left(D\right)$ so that ($1^a$) and ($2^a$) of $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$. So, it is sufficient to prove the claim under the assumption that $\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$. The proof is by induction on the structure of h.

Case 

$h=\langle h^{\prime }\rangle$. The assumption $\mathit{Q}\stackrel{h}{\to }{\mathit{Q}}^{\prime }\mathrm{wrt}{\Delta }^{\mathit{det}}$ shows that there exists a subset of states $P\subseteq \mathcal{Q}$ closing the following diagram:

In particular, we have $\mathit{Q}@P\to {\mathit{Q}}^{\prime }$ in ${\Delta }^{\mathit{det}}$. Let $D^{\prime }=D_{\mathit{Q}}$. Since $\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$, we can infer the following:

$$\frac{\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)\stackrel{\langle \rangle }{\to }({\langle \rangle }^{\Delta },D^{\prime })\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

We have to distinguish two cases depending on whether ${\langle \rangle }^{\Delta }$ belongs to ${\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$ or not.

Subcase 

${\langle \rangle }^{\Delta }\notin {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$. The induction hypothesis applied to $h^{\prime }$ yields the existence of $P^{\prime }\subseteq \mathcal{Q}$, such that $(P,P^{\prime })\in {\mathit{dep}}^{\mathit{a}}\left(D^{\prime }\right)$ and

$$({\langle \rangle }^{\Delta },D^{\prime })\stackrel{h^{\prime }}{\to }(P^{\prime },D^{\prime })\mathrm{wrt}{\Delta }^{\mathit{cgr}}.$$

Subsubcase 

$P\notin {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$. Since $(P,P^{\prime })\in {\mathit{dep}}^{\mathit{a}}\left(D^{\prime }\right)$, it follows that $P\subseteq P^{\prime }$. Hence, $P^{\prime }\notin {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$. Let ${\mathit{Q}}^{″}\subseteq \mathcal{Q}$ be such that $\mathit{Q}@P^{\prime }\to {\mathit{Q}}^{″}\mathrm{in}{\Delta }^{\mathit{det}}$. We can then apply the following inference rule:

$$\frac{\mathit{Q}@P^{\prime }\to {\mathit{Q}}^{″}\mathrm{in}{\Delta }^{\mathit{det}}\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)P^{\prime }\notin {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)}{(\mathit{Q},D)@(P^{\prime },D^{\prime })\to ({\mathit{Q}}^{″},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

Hence, we can close the diagram as follows:

This shows that $(\mathit{Q},D)\stackrel{h}{\to }({\mathit{Q}}^{″},D)\mathrm{wrt}{\Delta }^{\mathit{cgr}}$. Since $P\subseteq P^{\prime }$, $\mathit{Q}@P\to {\mathit{Q}}^{\prime }$, and $\mathit{Q}@P^{\prime }\to {\mathit{Q}}^{″}\mathrm{in}{\Delta }^{\mathit{det}}$, it follows that ${\mathit{Q}}^{\prime }\subseteq {\mathit{Q}}^{″}$, and thus, $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$. This shows the claim in this case.

Subsubcase 

$P\in {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$. Since $(P,P^{\prime })\in {\mathit{dep}}^{\mathit{a}}\left(D^{\prime }\right)$, it follows that $P\subseteq {\mathit{acc}}^{\Delta }\left(P^{\prime }\right)$ and $P^{\prime }\in {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$. Hence, we can apply the following inference rule for some ${\mathit{Q}}^{″}\subseteq {\mathcal{H}}_{\Sigma }$:

$$\frac{\mathit{Q}@{\mathit{acc}}^{\Delta }\left(P^{\prime }\right)\to {\mathit{Q}}^{″}\mathrm{in}{\Delta }^{\mathit{det}}\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)P^{\prime }\in {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)}{(\mathit{Q},D)@(P^{\prime },D^{\prime })\to ({\mathit{Q}}^{″},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

Hence, we can close the diagram as follows:

This shows that $(\mathit{Q},D)\stackrel{h}{\to }({\mathit{Q}}^{″},D)\mathrm{wrt}{\Delta }^{\mathit{cgr}}$. Since $P\subseteq {\mathit{acc}}^{\Delta }\left(P^{\prime }\right)$, it follows from $\mathit{Q}@P\to {\mathit{Q}}^{\prime }$ and $\mathit{Q}@{\mathit{acc}}^{\Delta }\left(P^{\prime }\right)\to {\mathit{Q}}^{″}$ in ${\Delta }^{\mathit{det}}$ that ${\mathit{Q}}^{\prime }\subseteq {\mathit{Q}}^{″}$. Thus, $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$, so the claim holds in this case too.

Subcase 

${\langle \rangle }^{\Delta }\in {\mathit{Prj}}^{\Delta }\left(D^{\prime }\right)$. Claim 2.1a shows that $({\langle \rangle }^{\Delta },D^{\prime })\stackrel{h^{\prime }}{\to }({\langle \rangle }^{\Delta },D^{\prime })$ wrt ${\Delta }^{\mathit{cgr}}$. Let ${\mathit{Q}}^{″}$ be such that $\mathit{Q}@{\mathit{acc}}^{\Delta }\left({\langle \rangle }^{\Delta }\right)\to Q^{″}\mathrm{in}{\Delta }^{\mathit{det}}$. We can apply the following inference rule:

$$\frac{\mathit{Q}@{\mathit{acc}}^{\Delta }\left({\langle \rangle }^{\Delta }\right)\to {\mathit{Q}}^{″}\mathrm{in}{\Delta }^{\mathit{det}}\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right){\langle \rangle }^{\Delta }\in {\mathit{Prj}}^{\Delta }\left(D^{\prime }\right)}{(\mathit{Q},D)@({\langle \rangle }^{\Delta },D^{\prime })\to ({\mathit{Q}}^{″},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

We can thus close the diagram below as follows with respect to ${\Delta }^{\mathit{cgr}}$:

This shows that $(\mathit{Q},D)\stackrel{h}{\to }({\mathit{Q}}^{″},D)\mathrm{wrt}{\Delta }^{\mathit{cgr}}$. Since $P\subseteq {\mathit{acc}}^{\Delta }\left({\langle \rangle }^{\Delta }\right)$, it follows from $\mathit{Q}@P\to {\mathit{Q}}^{\prime }$ and $\mathit{Q}@{\mathit{acc}}^{\Delta }\left({\langle \rangle }^{\Delta }\right)\to {\mathit{Q}}^{″}$ in ${\Delta }^{\mathit{det}}$ that ${\mathit{Q}}^{\prime }\subseteq {\mathit{Q}}^{″}$, and thus, $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$.

Case 

$h=a$. Since $\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$, we can apply the inference rule:

$$\frac{\mathit{Q}\stackrel{a}{\to }{\mathit{Q}}^{\prime }\mathrm{in}{\Delta }^{\mathit{det}}\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)\stackrel{a}{\to }({\mathit{Q}}^{\prime },D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

Hence, $(\mathit{Q},D)\stackrel{h}{\to }({\mathit{Q}}^{\prime },D)\mathrm{wrt}{\Delta }^{\mathit{cgr}}$. With ${\mathit{Q}}^{″}={\mathit{Q}}^{\prime }$, the claim follows since $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$.

Case 

$h=\epsilon$. In this case, we have $\mathit{Q}={\mathit{Q}}^{″}$ and $(\mathit{Q},D)\stackrel{\epsilon }{\to }(\mathit{Q},D)\mathrm{wrt}{\Delta }^{\mathit{cgr}}$, so the claim holds.

Case 

$h=h_1·h_2$. Since $\mathit{Q}\stackrel{h}{\to }{\mathit{Q}}^{\prime }\mathrm{wrt}{\Delta }^{\mathit{det}}$, there exists ${\mathit{Q}}_1\subseteq \mathcal{Q}$, such that $\mathit{Q}\stackrel{h_1}{\to }{\mathit{Q}}_1$ and ${\mathit{Q}}_1\stackrel{h_2}{\to }{\mathit{Q}}^{\prime }\mathrm{wrt}{\Delta }^{\mathit{det}}$. By the induction hypothesis applied to $h_1$ there exists ${\mathit{Q}}_1^{\prime }\subseteq \mathcal{Q}$, such that $(\mathit{Q},D)\stackrel{h_1}{\to }({\mathit{Q}}_1^{\prime },D)\mathrm{wrt}{\Delta }^{\mathit{cgr}}$ and $({\mathit{Q}}_1,{\mathit{Q}}_1^{\prime })\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$.

Subcase 

${\mathit{Q}}_1\in {\mathit{dPrj}}^{\Delta }\left(D\right)$. Since $({\mathit{Q}}_1,{\mathit{Q}}_1^{\prime })\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$, it follows that ${\mathit{Q}}_1\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}_1^{\prime }\right)$ and ${\mathit{Q}}_1^{\prime }\in {\mathit{dPrj}}^{\Delta }\left(D\right)$. Furthermore, Claim 2.1a shows that $({\mathit{Q}}_1^{\prime },D)\stackrel{h_2}{\to }({\mathit{Q}}_1^{\prime },D)$, and hence, $(\mathit{Q},D)\stackrel{h}{\to }({\mathit{Q}}_1^{\prime },D)$ wrt ${\Delta }^{\mathit{cgr}}$. Since ${\mathit{Q}}^{\prime }\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}_1\right)$ and ${\mathit{Q}}_1\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}_1^{\prime }\right)$, it follows that ${\mathit{Q}}^{\prime }\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}_1^{\prime }\right)$. Hence, $({\mathit{Q}}_1^{\prime },{\mathit{Q}}^{\prime })\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$ and $(\mathit{Q},D)\stackrel{h}{\to }({\mathit{Q}}_1^{\prime },D)$.

Subcase 

${\mathit{Q}}_1\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$. In this case, we can apply the induction hypothesis to ${\mathit{Q}}_1\stackrel{h_2}{\to }{\mathit{Q}}^{\prime }\mathrm{wrt}{\Delta }^{\mathit{det}}$, showing that there exists ${\mathit{Q}}^{″}\subseteq \mathcal{Q}$, such that $({\mathit{Q}}_1^{\prime },D)\stackrel{h_2}{\to }({\mathit{Q}}^{″},D)\mathrm{wrt}{\Delta }^{\mathit{cgr}}$ and $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$. Hence, $(\mathit{Q},D)\stackrel{h}{\to }({\mathit{Q}}^{″},D)\mathrm{wrt}{\Delta }^{\mathit{cgr}}$ and $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$.

This ends the proof of Claim 2.2a.   □

Proof of Inclusion $\mathcal{L}\left(A\right)\subseteq \mathcal{L}\left(A^{\mathit{cgr}\left(\mathbf{S}\right)}\right)\cap \mathbf{S}$. 

Since $\mathbf{S}$ is a schema for A, we have $\mathcal{L}\left(A\right)\subseteq \mathbf{S}$, so that it is sufficient to show $\mathcal{L}\left(A\right)\subseteq \mathcal{L}\left(A^{\mathit{cgr}\left(\mathbf{S}\right)}\right)$. Let $h\in \mathcal{L}\left(A\right)$. Then, there exist $q_0\in I$ and $q\in F$, such that $q_0\stackrel{h}{\to }q$ wrt $\Delta$. Let $\mathit{Q}=\left\{q\right\}$. Since A is deterministic, it follows that $I\stackrel{h}{\to }\mathit{Q}$ wrt ${\Delta }^{\mathit{det}}$. Using Claim 2.2a, this implies the existence of a subset of states ${\mathit{Q}}^{\prime }\in \mathcal{Q}$, such that $(\mathit{Q},{\mathit{Q}}^{\prime })\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$ and $(I,D_{init}^{A,F_{\mathbf{S}}})\stackrel{h}{\to }({\mathit{Q}}^{\prime },D_{init}^{A,F_{\mathbf{S}}})$ wrt ${\Delta }^{\mathit{cgr}}$. Furthermore, $(I,D_{init}^{A,F_{\mathbf{S}}})\in I^{\mathit{cgr}\left(\mathbf{S}\right)}$. In order to prove $h\in \mathcal{L}\left(A^{\mathit{cgr}\left(\mathbf{S}\right)}\right)$, it is thus sufficient to show that $({\mathit{Q}}^{\prime },D_{init}^{A,F_{\mathbf{S}}})\in F^{\mathit{cgr}\left(\mathbf{S}\right)}$. We distinguish two cases:

Case 

${\mathit{Q}}^{\prime }\notin {\mathit{dPrj}}^{\Delta }\left(D_{init}^{A,F_{\mathbf{S}}}\right)$. Condition ($1^a$) of $(\mathit{Q},{\mathit{Q}}^{\prime })\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$, shows that $\mathit{Q}\subseteq {\mathit{Q}}^{\prime }$ so that $q\in {\mathit{Q}}^{\prime }$. Since $q\in F$, it follows that ${\mathit{Q}}^{\prime }\cap F\ne \varnothing$. Thus, $({\mathit{Q}}^{\prime },D_{init}^{A,F_{\mathbf{S}}})\in F^{\mathit{cgr}\left(\mathbf{S}\right)}$, so that $h\in \mathcal{L}\left(A^{\mathit{cgr}\left(\mathbf{S}\right)}\right)$.

Case 

${\mathit{Q}}^{\prime }\in {\mathit{dPrj}}^{\Delta }\left(D_{init}^{A,F_{\mathbf{S}}}\right)$. Condition ($2^a$) of $(\mathit{Q},{\mathit{Q}}^{\prime })\in {\mathit{dep}}^{\mathit{a}}\left(D\right)$ yields $\mathit{Q}\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}^{\prime }\right)$. Hence, $q\in {\mathit{acc}}^{\Delta }\left({\mathit{Q}}^{\prime }\right)\cap F$ so that $({\mathit{Q}}^{\prime },D_{init}^{A,F_{\mathbf{S}}})\in F^{\mathit{cgr}\left(\mathbf{S}\right)}$, and thus, $h\in \mathcal{L}\left(A^{\mathit{cgr}\left(\mathbf{S}\right)}\right)$.

This ends the proof of the first inclusion.   □

We next want to show the inverse inclusion $\mathcal{L}\left(A^{\mathit{cgr}\left(\mathbf{S}\right)}\right)\cap \mathbf{S}\subseteq \mathcal{L}\left(A\right)$. It will eventually follow from the next two claims.

Claim 2.1b. 

For any hedge $h\in {\mathcal{H}}_{\Sigma }$, difference relation $D\subseteq {\mathcal{D}}^{\Delta }$, projection state $\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$ and state $\mu \in {\mathcal{Q}}^{\mathit{cgr}}$: if $(\mathit{Q},D)\stackrel{h}{\to }\mu \mathrm{wrt}{\Delta }^{\mathit{cgr}}$, then $\mu =(\mathit{Q},D)$.

Proof. 

By induction on the structure of $h\in {\mathcal{H}}_{\Sigma }$, suppose that $(\mathit{Q},D)\stackrel{h}{\to }\mu \mathrm{wrt}{\Delta }^{\mathit{cgr}}$:

Case 

$h=\langle h^{\prime }\rangle$. There must exist states ${\mu }_1,{\mu }_1^{\prime }\in {\mathcal{Q}}^{\mathit{cgr}}$ closing the following diagram:

Since $\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$, the following rule must have been applied to infer $(\mathit{Q},D)\stackrel{\langle \rangle }{\to }{\mu }_1\mathrm{wrt}{\Delta }^{\mathit{cgr}}$:

$$\frac{\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)\stackrel{\langle \rangle }{\to }(\mathit{Q},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

Therefore, ${\mu }_1=(\mathit{Q},D)$. The induction hypothesis applied to $(\mathit{Q},D)\stackrel{h^{\prime }}{\to }{\mu }_1^{\prime }\mathrm{wrt}{\Delta }^{\mathit{cgr}}$ shows that ${\mu }_1^{\prime }=(\mathit{Q},D)$, too. So, $\mu$ must have been inferred by applying the rule:

$$\frac{\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)@(\mathit{Q},D)\to (\mathit{Q},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

Hence, $\mu =(\mathit{Q},D)$, as required.

Case 

$h=a$. The following rule must be applied:

$$\frac{a\in \Sigma \mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)\stackrel{a}{\to }(\mathit{Q},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

Hence, $\mu =(\mathit{Q},D)$.

Case 

$h=\epsilon$. Obvious.

Case 

$h=h_1·h_2$. There must exist some ${\mu }_1$, such that $(\mathit{Q},D)\stackrel{h_1}{\to }{\mu }_1\stackrel{h_2}{\to }\mu$ wrt ${\Delta }^{\mathit{cgr}}$. Using the induction hypothesis applied to $h_1$, we have ${\mu }_1=(\mathit{Q},D)$. We can thus apply the induction hypothesis to $h_2$ to obtain ${\mu }_2=(\mathit{Q},D)$.

This ends the proof of Claim 2.1b.   □

We next need an inverse of Claim 2.2a. For relating runs $A^{\mathit{cgr}}$ back to runs of A, we define for any difference relation D another binary relation ${\mathit{dep}}^{\mathit{b}}\left(D\right)\subseteq 2^{\mathcal{Q}}\times 2^{\mathcal{Q}}$, such that for any two subsets of states ${\mathit{Q}}^{\prime },{\mathit{Q}}^{″}\subseteq \mathcal{Q}$,

$$({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D\right)\iff \left\{\begin{array}{cc}({\mathit{Q}}^{\prime }\times {\mathit{Q}}^{″})\cap D=\varnothing \wedge \hfill & \hfill \left(0^b\right)\\ {\mathit{Q}}^{\prime }\notin {\mathit{dPrj}}^{\Delta }\left(D\right)\Rightarrow {\mathit{Q}}^{″}\subseteq {\mathit{Q}}^{\prime }\wedge \hfill & \hfill \left(1^b\right)\\ {\mathit{Q}}^{\prime }\in {\mathit{dPrj}}^{\Delta }\left(D\right)\Rightarrow {\mathit{Q}}^{″}\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}^{\prime }\right)\hfill & \hfill \left(2^b\right)\end{array}\right.$$

Claim 2.2b. 

Let $\mathit{Q}\in \mathcal{Q}$ be a subset of states that is compatible with a difference relation $D\in {\mathcal{D}}^{\Delta }$. For any hedge $h\in {\mathcal{H}}_{\Sigma }$ and state $\mu \in {\mathcal{Q}}^{\mathit{cgr}}$ with $(\mathit{Q},D)\stackrel{h}{\to }\mu \mathrm{wrt}{\Delta }^{\mathit{cgr}},$ there exist a pair of subsets of states $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$, such that $\mu =({\mathit{Q}}^{\prime },D)$ and $\mathit{Q}\stackrel{h}{\to }{\mathit{Q}}^{″}$ wrt. ${\Delta }^{\mathit{det}}$.

Proof. 

If $\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$, then Claim 2.1b shows that $\mu =(\mathit{Q},D)$. Let ${\mathit{Q}}^{\prime }=\mathit{Q}$ and ${\mathit{Q}}^{″}$ be the unique subset of states, such that $\mathit{Q}\stackrel{h}{\to }{\mathit{Q}}^{″}$ wrt. ${\Delta }^{\mathit{det}}$. We have to show that $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$. Since $\mathit{Q}\stackrel{h}{\to }{\mathit{Q}}^{″}$, we have ${\mathit{Q}}^{″}\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}^{\prime }\right)$, so condition ($2^b$) holds. Condition ($1^b$) holds trivially since $\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$. For condition ($0^b$), note that $\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$ implies ${\mathit{acc}}^{\Delta }{\left(\mathit{Q}\right)}^2\cap D=\varnothing$. Furthermore, ${\mathit{Q}}^{\prime }\times {\mathit{Q}}^{″}\subseteq {\mathit{acc}}^{\Delta }{\left(\mathit{Q}\right)}^2$, so that $({\mathit{Q}}^{\prime }\times {\mathit{Q}}^{″})\cap D=\varnothing$ as required.

We can thus assume that $\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$. The proof is by induction on the structure of $h\in {\mathcal{H}}_{\Sigma }$. We distinguish all the possible forms of hedges $h\in {\mathcal{H}}_{\Sigma }$:

Case 

$h=\langle h^{\prime }\rangle$. By definition of $(\mathit{Q},P)\stackrel{h}{\to }\mu \mathrm{wrt}{\Delta }^{\mathit{cgr}}$, there must exist ${\mu }_1,{\mu }_1^{\prime }\in {\mathcal{Q}}^{\mathit{cgr}}$, such that the following diagram can be closed:

Since $\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$, the following inference rule was applied to infer $(\mathit{Q},D)\stackrel{\langle \rangle }{\to }{\mu }_1$ wrt ${\Delta }^{\mathit{cgr}}$, where $D^{\prime }=D_{\mathit{Q}}$:

$$\frac{\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)\stackrel{\langle \rangle }{\to }({\langle \rangle }^{\Delta },D^{\prime })\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

Hence, ${\mu }_1=({\langle \rangle }^{\Delta },D^{\prime })$. If ${\langle \rangle }^{\Delta }\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$, then the induction hypothesis applied to $({\langle \rangle }^{\Delta },D^{\prime })\stackrel{h^{\prime }}{\to }{\mu }_1^{\prime }$ wrt ${\Delta }^{\mathit{cgr}}$ shows that there exists $(P^{\prime },P^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D^{\prime }\right)$, such that ${\mu }_1^{\prime }=(P^{\prime },D^{\prime })$ and ${\langle \rangle }^{\Delta }\stackrel{h^{\prime }}{\to }{P}^{″}\mathrm{wrt}{\Delta }^{\mathit{det}}$. Otherwise, the same can be concluded as we argued at the beginning.

Subcase 

$P^{\prime }\in {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$. The transition rule $(\mathit{Q},D)@{\mu }_1\to \mu$ must be inferred as follows for ${\mathit{Q}}^{\prime }\subseteq \mathcal{Q}$:

$$\frac{\mathit{Q}@{\mathit{acc}}^{\Delta }\left(P^{\prime }\right)\to {\mathit{Q}}^{\prime }\mathrm{in}{\Delta }^{\mathit{det}}\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)P^{\prime }\in {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)@(P^{\prime },D^{\prime })\to ({\mathit{Q}}^{\prime },D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

This shows that $\mu =({\mathit{Q}}^{\prime },D)$. So, we have the following diagram:

Let ${\mathit{Q}}^{″}$ be the unique subset of states, such that $\mathit{Q}@P^{″}\to {\mathit{Q}}^{″}$ wrt ${\Delta }^{\mathit{det}}$. We can then close the following diagram:

From $(P^{\prime },P^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D^{\prime }\right)$, it follows $(P^{\prime }\times P^{″})\cap D^{\prime }=\varnothing$, and thus, $({\mathit{Q}}^{\prime }\times {\mathit{Q}}^{″})\cap D=\varnothing$, i.e., condition ($0^b$) of $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$. Since $P^{\prime }\in {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$, condition ($2^b$) of $(P^{\prime },P^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D^{\prime }\right)$ yields $P^{″}\subseteq {\mathit{acc}}^{\Delta }\left(P^{\prime }\right)$. Furthermore, $\mathit{Q}@P^{″}\to {\mathit{Q}}^{″}$ and $\mathit{Q}@{\mathit{acc}}^{\Delta }\left(P^{\prime }\right)\to {\mathit{Q}}^{\prime }$ in ${\Delta }^{\mathit{det}}$, which yields ${\mathit{Q}}^{″}\subseteq {\mathit{Q}}^{\prime }$ so that conditions ($1^b$) and ($2^b$) of $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$ follow. Hence, $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$. In summary, we show that $\mu =({\mathit{Q}}^{\prime },D)$, $\mathit{Q}\stackrel{\langle h^{\prime }\rangle }{\to }{\mathit{Q}}^{″}\mathrm{wrt}{\Delta }^{\mathit{det}}$, and $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$, as required by the claim.

Subcase 

$P^{\prime }\notin {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$. The transition rule $(\mathit{Q},D)@{\mu }_1\to \mu$ must thus be inferred as follows for ${\mathit{Q}}^{\prime }\subseteq \mathcal{Q}$:

$$\frac{\mathit{Q}@P^{\prime }\to {\mathit{Q}}^{\prime }\mathrm{in}{\Delta }^{\mathit{det}}\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)P^{\prime }\notin {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)@(P^{\prime },D^{\prime })\to ({\mathit{Q}}^{\prime },D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

This shows that $\mu =({\mathit{Q}}^{\prime },D)$, and we can close the following diagram:

Condition ($0^b$) of $(P^{\prime },P^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D^{\prime }\right)$ yields $(P^{\prime }\times P^{″})\cap D^{\prime }=\varnothing$. Let ${\mathit{Q}}^{″}$ be the unique subset of states, such that $\mathit{Q}@P^{″}\to {\mathit{Q}}^{″}$ wrt. ${\Delta }^{\mathit{det}}$. Since $D^{\prime }\subseteq {\mathit{down}}_{\mathit{Q}}^{\Delta }\left(D\right)$ and $(P^{\prime }\times P^{″})\cap D^{\prime }=\varnothing$, it follows from $\mathit{Q}@P^{\prime }\to {\mathit{Q}}^{\prime }$ and $\mathit{Q}@P^{″}\to {\mathit{Q}}^{″}$ wrt. ${\Delta }^{\mathit{det}}$ that $({\mathit{Q}}^{\prime }\cap {\mathit{Q}}^{″})\cap D=\varnothing$. That is, condition ($0^b$) of $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$. Since $P^{\prime }\notin {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$, condition ($1^b$) of $(P^{\prime },P^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D^{\prime }\right)$ is $P^{″}\subseteq P^{\prime }$. From $\mathit{Q}@P^{″}\to {\mathit{Q}}^{″}$ and $\mathit{Q}@P^{\prime }\to {\mathit{Q}}^{\prime }$, it thus follows that ${\mathit{Q}}^{″}\subseteq {\mathit{Q}}^{\prime }$. Hence, conditions ($1^b$) and ($2^b$) of $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$ are valid, so that $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$ holds. Furthermore,

This shows that $\mathit{Q}\stackrel{\langle h^{\prime }\rangle }{\to }{\mathit{Q}}^{″}$ wrt $\Delta$. The other two requirements of the claim, $\mu =({\mathit{Q}}^{\prime },D)$ and $({\mathit{Q}}^{\prime },{\mathit{Q}}^{″})\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$, were shown earlier.

Case 

$h=a$. Since $\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$, the following inference rule must be used:

$$\frac{\mathit{Q}\stackrel{a}{\to }{\mathit{Q}}^{\prime }\mathrm{in}\Delta \mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)\stackrel{a}{\to }({\mathit{Q}}^{\prime },D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

So $\mu =({\mathit{Q}}^{\prime },D)$, $\mathit{Q}\stackrel{a}{\to }{\mathit{Q}}^{\prime }\mathrm{wrt}{\Delta }^{\mathit{det}}$. Furthermore, since $\mathit{Q}$ is compatible with D, and D is a difference relation, ${\mathit{Q}}^{\prime }$ is compatible with D too. Hence, $({\mathit{Q}}^{\prime }\times {\mathit{Q}}^{\prime })\cap D=\varnothing$, showing condition ($0^b$) of $({\mathit{Q}}^{\prime },{\mathit{Q}}^{\prime })\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$. Trivially, ${\mathit{Q}}^{\prime }\subseteq {\mathit{Q}}^{\prime }\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}^{\prime }\right)$ so conditions ($1^b$) and ($2^b$) of $({\mathit{Q}}^{\prime },{\mathit{Q}}^{\prime })\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$ hold, too. Hence, $({\mathit{Q}}^{\prime },{\mathit{Q}}^{\prime })\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$.

Case 

$h=\epsilon$. Obvious.

Case 

$h=h_1·h_2$. Since $(\mathit{Q},D)\stackrel{h}{\to }\mu$ wrt. ${\Delta }^{\mathit{cgr}}$, there exists some ${\mu }_1\in {\mathcal{Q}}^{\mathit{cgr}}$, such that $(\mathit{Q},D)\stackrel{h_1}{\to }{\mu }_1$ and ${\mu }_1\stackrel{h_2}{\to }\mu$ wrt ${\Delta }^{\mathit{cgr}}$. We can apply the induction hypothesis to $h_1$. Hence, there exist subsets of states ${\mathit{Q}}_1,{\mathit{Q}}_1^{\prime }\subseteq \mathcal{Q}$, such that ${\mu }_1=({\mathit{Q}}_1,D)$, $\mathit{Q}\stackrel{h_1}{\to }{\mathit{Q}}_1^{\prime }$ wrt $\Delta$, and $({\mathit{Q}}_1,{\mathit{Q}}_1^{\prime })\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$. We distinguish two cases:

Subcase 

${\mathit{Q}}_1\in {\mathit{dPrj}}^{\Delta }\left(D\right)$. Since ${\mu }_1=({\mathit{Q}}_1,D)\stackrel{h_2}{\to }\mu$ wrt ${\Delta }^{\mathit{cgr}}$, Claim 2.1b shows that $\mu =({\mathit{Q}}_1,D)$ so that $({\mathit{Q}}_1,D)\stackrel{h_2}{\to }({\mathit{Q}}_1,D)$ wrt. ${\Delta }^{\mathit{cgr}}$. Condition ($2^b$) of $({\mathit{Q}}_1,{\mathit{Q}}_1^{\prime })\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$ and ${\mathit{Q}}_1\in {\mathit{dPrj}}^{\Delta }\left(D\right)$ imply that ${\mathit{Q}}_1^{\prime }\in {\mathit{acc}}^{\Delta }\left({\mathit{Q}}_1\right)$. Let ${\mathit{Q}}_2$ be the unique subset of states, such that ${\mathit{Q}}_1^{\prime }\stackrel{h_2}{\to }{Q}_2$ wrt. ${\Delta }^{\mathit{det}}$. Then, ${\mathit{Q}}_2\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}_1^{\prime }\right)$ so that ${\mathit{Q}}_2\subseteq {\mathit{acc}}^{\Delta }\left({\mathit{Q}}_1\right)$, and thus, condition ($2^b$) of $({\mathit{Q}}_1,{\mathit{Q}}_2)\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$ holds. Condition ($1^b$) of $({\mathit{Q}}_1,{\mathit{Q}}_2)\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$ is trivial since ${\mathit{Q}}_1\in {\mathit{dPrj}}^{\Delta }\left(D\right)$. Since ${\mathit{Q}}_1\in {\mathit{dPrj}}^{\Delta }\left(D\right)$ and ${\mathit{Q}}_1\times {\mathit{Q}}_2\in {\mathit{acc}}^{\Delta }{\left(\mathit{Q}\right)}^2$, it follows that ${\mathit{Q}}_1\times {\mathit{Q}}_2\cap D=\varnothing$ so condition ($0^b$) of $({\mathit{Q}}_1,{\mathit{Q}}_2)\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$ holds, too. Hence, $\mathit{Q}\stackrel{h}{\to }{\mathit{Q}}_2$ wrt ${\Delta }^{\mathit{det}}$ and $({\mathit{Q}}_1,{\mathit{Q}}_2)\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$.

Subcase 

${\mathit{Q}}_1\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$. Since ${\mathit{Q}}_1\stackrel{h_1}{\to }{\mathit{Q}}_1^{\prime }$ and ${\mathit{Q}}_1$ is compatible with difference relation D, it follows that ${\mathit{Q}}_1^{\prime }$ is compatible with D, too. We can thus apply the induction hypothesis to $({\mathit{Q}}_1,D)\stackrel{h_2}{\to }\mu$, showing the existence of subsets of states $({\mathit{Q}}_2,{\mathit{Q}}_2^{\prime })\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$, such that $\mu =({\mathit{Q}}_2,D)$ and ${\mathit{Q}}_1\stackrel{h_2}{\to }{\mathit{Q}}_2^{\prime }$ wrt $\Delta$. So, we have $\mathit{Q}\stackrel{h}{\to }{\mathit{Q}}_2^{\prime }$ wrt. $\Delta$ and $({\mathit{Q}}_2,{\mathit{Q}}_2^{\prime })\in {\mathit{dep}}^{\mathit{b}}\left(D\right)$, as required.

This ends the proof of Claim 2.2b.   □

Proof of Inclusion $\mathcal{L}\left(A^{\mathit{cgr}\left(\mathbf{S}\right)}\right)\cap \mathbf{S}\subseteq \mathcal{L}\left(A\right)$. 

Let $h\in \mathcal{L}\left(A^{\mathit{cgr}\left(\mathbf{S}\right)}\right)\cap \mathbf{S}$. Then, there exists a final subset of states $\mathit{Q}\in F^{\mathit{cgr}\left(\mathbf{S}\right)}$, such that $(I,D_{init}^{A,F_{\mathbf{S}}})\stackrel{h}{\to }(\mathit{Q},D_{init}^{A,F_{\mathbf{S}}})$ wrt ${\Delta }^{\mathit{cgr}}$. Since I is a singleton or empty, it is compatible with $D_{init}^{A,F_{\mathbf{S}}}$. Claim 2.2b shows that there exists a subset of states ${\mathit{Q}}^{\prime }\subseteq \mathcal{Q}$, such that $I\stackrel{h}{\to }{\mathit{Q}}^{\prime }$ wrt. ${\Delta }^{\mathit{det}}$ and $(\mathit{Q},{\mathit{Q}}^{\prime })\in {\mathit{dep}}^{\mathit{b}}\left(D_{init}^{A,F_{\mathbf{S}}}\right)$. Condition ($0^b$) of $(\mathit{Q},{\mathit{Q}}^{\prime })\in {\mathit{dep}}^{\mathit{b}}\left(D_{init}^{A,F_{\mathbf{S}}}\right)$ shows that $(\mathit{Q},{\mathit{Q}}^{\prime })\cap D_{init}^{A,F_{\mathbf{S}}}=\varnothing$. Since $h\in S$, it follows that ${\mathit{Q}}^{\prime }\cap F_{\mathbf{S}}\ne \varnothing$. The determinism of A shows that ${\mathit{Q}}^{\prime }$ is a singleton. So, there exists a state $q^{\prime }\in F_{\mathbf{S}}$, such that ${\mathit{Q}}^{\prime }=\left\{q^{\prime }\right\}$.

Case 

$\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D_{init}^{A,F_{\mathbf{S}}}\right)$. Condition ($1^b$) shows that ${\mathit{Q}}^{\prime }\subseteq \mathit{Q}$ so that $q^{\prime }\in \mathit{Q}$. Since $\mathit{Q}\in F^{\mathit{cgr}}$, we have $\mathit{Q}\cap F\ne \varnothing$. Since $\mathit{Q}$ is compatible with $D_{init}^{A,F_{\mathbf{S}}}$, $q^{\prime }\in \mathit{Q}$, and $\mathit{Q}\cap F\ne \varnothing$, it follows that $q^{\prime }\notin F_{\mathbf{S}}\backslash F$. Since $q^{\prime }\in F_{\mathbf{S}}$, this implies $q^{\prime }\in F$, and thus, $h\in \mathcal{L}\left(A\right)$.

Case 

$\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D_{init}^{A,F_{\mathbf{S}}}\right)$. Condition ($2^b$) shows that $Q^{\prime }\subseteq {\mathit{acc}}^{\Delta }\left(\mathit{Q}\right)$ so that $q^{\prime }\in {\mathit{acc}}^{\Delta }\left(\mathit{Q}\right)$. Since $\mathit{Q}\in F^{\mathit{cgr}}$, we have ${\mathit{acc}}^{\Delta }\left(\mathit{Q}\right)\cap F\ne \varnothing$. Let $q\in {\mathit{acc}}^{\Delta }\left(Q\right)\cap F$ be arbitrary. Since $\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D_{init}^{A,F_{\mathbf{S}}}\right)$ and $(q^{\prime },q)\in {\mathit{acc}}^{\Delta }{\left(Q\right)}^2$, it follows that $(q,q^{\prime })\notin D_{init}^{A,F_{\mathbf{S}}}$. Furthermore, $q\in F$ so that $q^{\prime }\notin F_{\mathbf{S}}\backslash F$. In combination with $q^{\prime }\in F_{\mathbf{S}}$, this implies $q^{\prime }\in F_{\mathbf{S}}$ so $h\in \mathcal{L}\left(A\right)$.

This ends the proof of the inverse inclusion, and thus, of $\mathcal{L}\left(A\right)=\mathcal{L}\left(A^{\mathit{cgr}\left(\mathbf{S}\right)}\right)\cap \mathbf{S}$.   □

7.4. Completeness

We next show the completeness of congruence projection for subhedge projection according to Definition 10. Let $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ be a complete dSha and $F\subseteq F_{\mathbf{S}}\subseteq \mathcal{Q}$. Automaton A defines the regular pattern $L=\mathcal{L}\left(A\right)$ with the schema-final states in $F_{\mathbf{S}}$ the regular schema $\mathbf{S}=\mathcal{L}\left(A[F/F_{\mathbf{S}}]\right)$. We first show that all subhedge irrelevant states of difference relations are subhedge projection states $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ according to Definition 7.

Lemma 14. 

If $\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$, then $(\mathit{Q},D)$ is a subhedge projection state of ${\Delta }^{\mathit{cgr}}$ with witness $(\mathit{Q},D)$.

Proof. 

We assume that $q\in {\mathit{Prj}}^{\Delta }\left(D\right)$ and have to show that $(\mathit{Q},D)$ is a subhedge projection state of ${\Delta }^{\mathit{cgr}}$. We have to show that all transition rules starting with $(\mathit{Q},D)$ are permitted for a subhedge projection state $(\mathit{Q},D)$ with $(\mathit{Q},D)$. They are generated by the following rules, all of which are looping:

$$\frac{a\in \Sigma \mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)\stackrel{a}{\to }(\mathit{Q},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}\frac{\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)@(\mathit{Q},D)\to (\mathit{Q},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}\frac{\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)}{(\mathit{Q},D)\stackrel{\langle \rangle }{\to }(\mathit{Q},D)\mathrm{in}{\Delta }^{\mathit{cgr}}}$$

   □

So, if a partial run of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ on a prefix u assigns some state $(P,D)$ with $P\in {\mathit{dPrj}}^{\Delta }\left(D\right)$, then $(P,D)$ is a subhedge projection state by Lemma 14, and thus, subhedge irrelevant by Proposition 2.

Lemma 15. 

Let $A^{\prime }=A[I/{\langle \rangle }^{\Delta }]$, $L^{\prime }=\mathcal{L}\left(A^{\prime }\right)$, and ${\mathit{S}}^{\prime }=\mathcal{L}\left(A^{\prime }[F/F_{\mathit{S}}]\right)$. If $({\langle \rangle }^{\Delta },D)\stackrel{\mathit{hdg}\left(u\right)}{\to }(\mathit{Q},D)$ wrt $A^{\mathit{cgr}\left(\mathit{S}\right)}$ for some nested word $u\in {\mathcal{N}}_{\Sigma }$ and either

• $\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$ and $q\in \mathit{Q}$, or
• $\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$ and $q\in {\mathit{acc}}^{\Delta }\left(\mathit{Q}\right)$,

then there exists a nested word $u^{\prime }\in clas{s}_{{\mathit{S}}^{\prime }}^{L^{\prime }}\left(u\right)$ and $q_0\in {\langle \rangle }^{\Delta }$, such that $q_0\stackrel{\mathit{hdg}\left(u^{\prime }\right)}{\to }q$ wrt Δ.

Proof. 

By induction on the length of u, suppose that $({\langle \rangle }^{\Delta },D)\stackrel{\mathit{hdg}\left(u\right)}{\to }(\mathit{Q},D)$ wrt $A^{\mathit{cgr}\left(\mathbf{S}\right)}$. In the base case, we have $u=\epsilon$. Hence, $\mathit{Q}={\langle \rangle }^{\Delta }$.

Case 

$\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$ and $q\in \mathit{Q}$. Then, ${\langle \rangle }^{\Delta }=\left\{q\right\}$. The unique run of $A^{\prime }$ on $u=\epsilon$ starts and ends in the tree initial state $q\in {\langle \rangle }^{\Delta }$.

Case 

$\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$ and $q\in {\mathit{acc}}^{\Delta }\left(\mathit{Q}\right)$. Since $\mathit{Q}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$, Lemma 14 shows that $(\mathit{Q},D)$ is a subhedge projection state, so that $\epsilon$ is subhedge irrelevant for $L^{\prime }$ and ${\mathbf{S}}^{\prime }$ by Proposition 2. Hence, $clas{s}_{{\mathbf{S}}^{\prime }}^{L^{\prime }}\left(\epsilon \right)={\mathcal{N}}_{\Sigma }$. Furthermore, since $q\in {\mathit{acc}}^{\Delta }\left(\mathit{Q}\right)$, there exists a hedge h and $q_0\in I$, such that $q_0\stackrel{h}{\to }q$ wrt $\Delta$. Let $u^{\prime }=\mathit{nw}\left(h\right)$. The run of A on $u^{\prime }$ ends in q and $u^{\prime }\in clas{s}_{{\mathbf{S}}^{\prime }}^{L^{\prime }}\left(u\right)$.

For the induction step, we distinguish the possible forms of the nested word u. So, there exist $\tilde{u},v\in {\mathcal{N}}_{\Sigma }$ and $a\in \Sigma$, such that either of the following cases holds:

Case 

$u=\tilde{u}·\langle ·v·\rangle$. Let $(\tilde{\mathit{Q}},D)$ be the state in which the run of ${\left(A^{\prime }\right)}^{\mathit{cgr}\left(\mathbf{S}\right)}$ on $\tilde{u}$ ends.

Subcase 

$\tilde{\mathit{Q}}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$. Let $D^{\prime }=D_{\tilde{\mathit{Q}}}$. Then, there exist $P\subseteq \mathcal{Q}$, such that $({\langle \rangle }^{\Delta },D^{\prime })\stackrel{v}{\to }(P,D^{\prime })$ wrt $A^{\mathit{cgr}\left(\mathbf{S}\right)}$. So, the run of ${\left(A^{\prime }\right)}^{\mathit{cgr}\left(\mathbf{S}\right)}$ on v goes to state $(P,D^{\prime })$.

Subsubcase 

$P\in {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$. By construction of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$, we then have $\mathit{Q}=\tilde{\mathit{Q}}{@}^{\Delta }{\mathit{acc}}^{\Delta }\left(P\right)$. Since $q\in \mathit{Q}$, there exist $\tilde{q}\in \tilde{\mathit{Q}}$ and $p\in {\mathit{acc}}^{\Delta }\left(P\right)$, such that $\tilde{q}@p\to q$ in $\Delta$. By induction hypothesis, there exists $v^{\prime }\in clas{s}_{{\mathbf{S}}^{\prime }}^{L^{\prime }}\left(v\right)$, such that the run of $A^{\prime }$ on $v^{\prime }$ ends in p. Hence, the run of A on $u^{\prime }=\tilde{u}·\langle ·v^{\prime }·\rangle$ ends in q, and furthermore, $u^{\prime }\in clas{s}_{\mathbf{S}}^L\left(u\right)$.

Subsubcase 

$P\notin {\mathit{dPrj}}^{\Delta }\left(D^{\prime }\right)$. By construction of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$, we then have $\mathit{Q}=\tilde{\mathit{Q}}{@}^{\Delta }P$. Since $q\in \mathit{Q}$, there exist $\tilde{q}\in \tilde{\mathit{Q}}$ and $p\in P$, such that $\tilde{q}@p\to q$ in $\Delta$. By induction hypothesis, there exists $v^{\prime }\in clas{s}_{{\mathbf{S}}^{\prime }}^{L^{\prime }}\left(v\right)$, such that the run of $A^{\prime }$ on $v^{\prime }$ ends in p. Hence, the run of A on $u^{\prime }=\tilde{u}·\langle ·v^{\prime }·\rangle$ ends in q, and furthermore, $u^{\prime }\in clas{s}_{\mathbf{S}}^L\left(u\right)$.

Subcase 

$\tilde{\mathit{Q}}\in {\mathit{dPrj}}^{\Delta }\left(D\right)$. Then, $\mathit{Q}=\tilde{\mathit{Q}}$ and $q\in {\mathit{acc}}^{\Delta }\left(\tilde{\mathit{Q}}\right)$. Using the induction hypothesis applied to $\tilde{u}$, there exists ${\tilde{u}}^{\prime }\in clas{s}_{\mathbf{S}}^L\left(\tilde{u}\right)$, such that ${\langle \rangle }^{\Delta }\stackrel{\mathit{hdg}\left({\tilde{u}}^{\prime }\right)}{\to }q$. Since $\tilde{u}$ is irrelevant for L and ${\mathbf{S}}^{\prime }$, we have ${\tilde{u}}^{\prime }\in clas{s}_{\mathbf{S}}^L\left(u\right)$.

Case 

$u=\tilde{u}·a$. This is easier to achieve than the previous cases and is thus omitted.

   □

Lemma 16. 

A partial run of $A^{\mathit{cgr}\left(\mathit{S}\right)}$ assigns a state $(\mathit{Q},D)$ to a prefix $u\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$. If $\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$ and $q\in \mathit{Q}$, then there exists a prefix $u^{\prime }\in clas{s}_{\mathit{S}}^L\left(u\right)$, such that a partial run of A assigns state q to $u^{\prime }$.

Proof. 

By induction on the length of u. In the base case, we have $u=\epsilon$. The partial run of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ on u assigns state $(\mathit{Q},D)$, where $\mathit{Q}=I$ and $D=D_{init}^{A,F_{\mathbf{S}}}$. Suppose that $\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$ and $q\in \mathit{Q}$. Then, $I=\left\{q\right\}$. The unique partial run of A on $u=\epsilon$ ends in the initial state $q\in I$. For the induction step, let $A^{\prime }=A[I/{\langle \rangle }^{\Delta }]$, $L^{\prime }=\mathcal{L}\left(A^{\prime }\right)$, and ${\mathbf{S}}^{\prime }=\mathcal{L}(A^{\prime }[F/F_{\mathbf{S}}]$. We distinguish the possible forms of u. So, $\tilde{u}\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$, $v\in {\mathcal{N}}_{\Sigma }$, and $a\in \Sigma$ exist, such that either of the following hold:

Case 

$u=\tilde{u}·\langle ·v$. Let $(\tilde{\mathit{Q}},\tilde{D})$ be the state in which the partial run of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ on $\tilde{u}$ ends. From $\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$, it follows that $\tilde{\mathit{Q}}\notin {\mathit{dPrj}}^{\Delta }\left(\tilde{D}\right)$. Furthermore, $({\langle \rangle }^{\Delta },D)\stackrel{v}{\to }(\mathit{Q},D)$ wrt $A^{\mathit{cgr}\left(\mathbf{S}\right)}$. So, the partial run of ${\left(A^{\prime }\right)}^{\mathit{cgr}\left(\mathbf{S}\right)}$ on v goes to state $(\mathit{Q},D)$. Using the induction hypothesis, there exists $v^{\prime }\in clas{s}_{{\mathbf{S}}^{\prime }}^{L^{\prime }}\left(v\right)$, such that the partial run of $A^{\prime }$ on $v^{\prime }$ ends in q. Hence, the partial run of A on $u^{\prime }=\tilde{u}·\langle ·v^{\prime }$ ends in q, and furthermore, $u^{\prime }\in clas{s}_{\mathbf{S}}^L\left(u\right)$.

Case 

$u=\tilde{u}·\langle ·v·\rangle$. Let $(\tilde{\mathit{Q}},\tilde{D})$ be the state in which the partial run of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ on $\tilde{u}$ ends. From $\mathit{Q}\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$, it follows that $\tilde{\mathit{Q}}\notin {\mathit{dPrj}}^{\Delta }\left(\tilde{D}\right)$. Furthermore, $({\langle \rangle }^{\Delta },D)\stackrel{v}{\to }(P,D)$ wrt $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ for some $P\subseteq \mathcal{Q}$.

Subcase 

$P\in {\mathit{dPrj}}^{\Delta }\left(D\right)$. Then, $Q=\tilde{\mathit{Q}}@\Delta {\mathit{acc}}^{\Delta }\left(P\right)$. So, there exists $\tilde{q}\in \mathit{Q}$ and $p\in {\mathit{acc}}^{\Delta }\left(P\right)$, such that $\tilde{q}@p\to q$ in $\Delta$. Using Lemma 15, there exists $v^{\prime }\in clas{s}_{{\mathbf{S}}^{\prime }}^{L^{\prime }}\left(v\right)$, such that ${\langle \rangle }^{\Delta }\stackrel{\mathit{hdg}\left(v^{\prime }\right)}{\to }p$ wrt $\Delta$. Using the induction hypothesis applied to $\tilde{u}$, there exists an initial state $q_0\in I$ and a prefix ${\tilde{u}}^{\prime }\in clas{s}_{\mathbf{S}}^L\left(\tilde{u}\right)$, such that the partial run of A on ${\tilde{u}}^{\prime }$ goes on to state $\tilde{q}$. Hence, the partial run of A on $u^{\prime }={\tilde{u}}^{\prime }·\langle ·v^{\prime }·\rangle$ goes on to q and $u^{\prime }\in clas{s}_{\mathbf{S}}^L\left(u\right)$.

Subcase 

$P\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$. Then, $Q=\tilde{\mathit{Q}}@\Delta P$. So, there exists $\tilde{q}\in \mathit{Q}$ and $p\in P$, such that $\tilde{q}@p\to q$ in $\Delta$. Using Lemma 15, there exists $v^{\prime }\in clas{s}_{{\mathbf{S}}^{\prime }}^{L^{\prime }}\left(v\right)$, such that ${\langle \rangle }^{\Delta }\stackrel{\mathit{hdg}\left(v^{\prime }\right)}{\to }p$ wrt $\Delta$. Using the induction hypothesis applied to $\tilde{u}$, there exists an initial state $q_0\in I$ and a prefix ${\tilde{u}}^{\prime }\in clas{s}_{\mathbf{S}}^L\left(\tilde{u}\right)$, such that the partial run of A on ${\tilde{u}}^{\prime }$ goes on to state $\tilde{q}$. Hence, the partial run of A on $u^{\prime }={\tilde{u}}^{\prime }·\langle ·v^{\prime }·\rangle$ goes on to q and $u^{\prime }\in clas{s}_{\mathbf{S}}^L\left(u\right)$.

Case 

$u=\tilde{u}·a$. This is easier to achieve than the previous case and is thus omitted.

   □

The next lemma states the key invariant of congruence projection, which eventually proves its completeness for subhedge projection. For this, we define for any $F\subseteq F_{\mathbf{S}}\subseteq F$ the binary relation ${\not\approx }_{F_{\mathbf{S}}}^F$ as the symmetric closure of $F\times (F_{\mathbf{S}}\backslash F)$, i.e., for all $(q^{\prime },q^{″})\in {\mathcal{Q}}^2$:

$$q^{\prime }{\not\approx }_{F_{\mathbf{S}}}^F{q}^{″}\iff \left\{\begin{array}{cccc}(q^{\prime }\in F\hfill & \wedge & q^{″}\in F_{\mathbf{S}}\backslash F)\hfill & \vee \hfill \\ (q^{\prime }\in F_{\mathbf{S}}\backslash F\hfill & \wedge & q^{″}\in F)\hfill \end{array}\right.$$

Lemma 17 (Key Invariant).

Let $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ a dSha, $F\subseteq F_{\mathit{S}}\subseteq \mathcal{Q}$, $L=\mathcal{L}\left(A\right)$, and $\mathit{S}=\mathcal{L}\left(A[F/F_{\mathit{S}}]\right)$. If $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ has a partial run on prefix $u\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$ to state $(P,D)\in 2^{\mathcal{Q}}\times {\mathcal{D}}^{\Delta }$, then for all $(p^{\prime },p^{″})\in P^2$, $(r^{\prime },r^{″})\in D$, and $(v^{\prime },v^{″})\in {\mathcal{N}}_{\Sigma }^2$, such that

$$p^{\prime }\stackrel{\mathit{hdg}\left(v^{\prime }\right)}{\to }{r}^{\prime }\mathrm{wrt}\Delta \wedge p^{″}\stackrel{\mathit{hdg}\left(v^{″}\right)}{\to }{r}^{″}\mathrm{wrt}\Delta$$

there exist $(u^{\prime },u^{″})\in clas{s}_{\mathbf{S}}^L\left(u\right)$, $w\in \mathit{suffs}\left({\mathcal{N}}_{\Sigma }\right)$, $q^{\prime }{\not\approx }_{F_{\mathbf{S}}}^F{q}^{″}$, and $q_0\in I$ such that

$$q_0\stackrel{\mathit{hdg}(u^{\prime }·v^{\prime }·w)}{\to }{q}^{\prime }\mathrm{wrt}\Delta \wedge q_0\stackrel{\mathit{hdg}(u^{″}·v^{″}·w)}{\to }{q}^{″}\mathrm{wrt}\Delta$$

Proof. 

Induction on the number of dangling opening parenthesis of u.

In the base case, u does not have any dangling opening parenthesis so $u\in {\mathcal{N}}_{\Sigma }$ is a nested word. In this case, $D=D_{init}^{A,F_{\mathbf{S}}}$. So, $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ has a partial run on nested word $u\in {\mathcal{N}}_{\Sigma }$ to $(P,D_{init}^{A,F_{\mathbf{S}}})$, where $P\subseteq \mathcal{Q}$. Let $(p^{\prime },p^{″})\in P^2$, $(r^{\prime },r^{″})\in D_{init}^{A,F_{\mathbf{S}}}$, and $(v^{\prime },v^{″})\in {\mathcal{N}}_{\Sigma }^2$, such that

$$p^{\prime }\stackrel{\mathit{hdg}\left(v^{\prime }\right)}{\to }{r}^{\prime }\mathrm{wrt}\Delta \wedge p^{″}\stackrel{\mathit{hdg}\left(v^{″}\right)}{\to }{r}^{″}\mathrm{wrt}\Delta$$

It then follows that $P\notin {\mathit{dPrj}}^{\Delta }\left(D_{init}^{A,F_{\mathbf{S}}}\right)$. Therefore, we can apply Lemma 16. It shows that there exist nested words $(u^{\prime },u^{″})\in {\left(clas{s}_{\mathbf{S}}^L\right(u\left)\right)}^2$ and $q_0\in I$, such that

$$q_0\stackrel{\mathit{hdg}\left(u^{\prime }\right)}{\to }{p}^{\prime }\mathrm{wrt}\Delta \wedge q_0\stackrel{\mathit{hdg}\left(u^{″}\right)}{\to }{p}^{″}\mathrm{wrt}\Delta$$

Since $D_{init}^{A,F_{\mathbf{S}}}\subseteq {\mathit{ldr}}^{\Delta }(F\times (F_{\mathbf{S}}\backslash F))$ by Lemma 10, there exist a nested word $w\in {\mathcal{N}}_{\Sigma }$ and states $q^{\prime }{\not\approx }_{F_{\mathbf{S}}}^F{q}^{″}$, such that

$$r^{\prime }\stackrel{\mathit{hdg}\left(w\right)}{\to }{q}^{\prime }\mathrm{wrt}\Delta \wedge r^{″}\stackrel{\mathit{hdg}\left(w\right)}{\to }{q}^{″}\mathrm{wrt}\Delta$$

Then, we have the following:

$$q_0\stackrel{\mathit{hdg}(u^{\prime }·v^{\prime }·w)}{\to }{q}^{\prime }\mathrm{wrt}\Delta \wedge q_0\stackrel{\mathit{hdg}(u^{″}·v^{″}·w)}{\to }{q}^{″}\mathrm{wrt}\Delta$$

For the induction step, let $u=u_1·\langle ·u_2$ for some prefix $u_1\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$ and nested word $u_2\in {\mathcal{N}}_{\Sigma }$ so that $u_1$ has one open dangling bracket less than u. Let $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ have a partial run on nested word $u\in {\mathcal{N}}_{\Sigma }$ to $(P,D)$. Let $(P_1,D_1)$ be the state that this run assigns to prefix $u_1$. Then, $D={\left(D_1\right)}_{P_1}$. Let $(p^{\prime },p^{″})\in P^2$, $(r^{\prime },r^{″})\in D$, and $(v^{\prime },v^{″})\in {\mathcal{N}}_{\Sigma }^2$, such that

$$p^{\prime }\stackrel{\mathit{hdg}\left(v^{\prime }\right)}{\to }{r}^{\prime }\mathrm{wrt}\Delta \wedge p^{″}\stackrel{\mathit{hdg}\left(v^{″}\right)}{\to }{r}^{″}\mathrm{wrt}\Delta$$

Since $D={\left(D_1\right)}_{P_1}$, there exist $(p_1^{\prime },p_1^{″})\in {\left(P_1\right)}^2$ and states $(r_1^{\prime },r_1^{″})\in D_1$, such that $p_1^{\prime }@q^{\prime }\to r_1^{\prime }$ and $p_1^{″}@q^{″}\to r_1^{″}$. In particular, $P_1\notin {\mathit{dPrj}}^{\Delta }\left(D_1\right)$ so that we can apply Lemma 16. It shows that there exist $(u_2^{\prime },u_2^{″})\in clas{s}_{{\mathbf{S}}^{\prime }}^{L^{\prime }}\left(u_2\right)$, such that ${\langle \rangle }^{\Delta }\stackrel{u_2^{\prime }}{\to }{p}^{\prime }$ and ${\langle \rangle }^{\Delta }\stackrel{u_2^{″}}{\to }{p}^{″}$. Let $v_1^{\prime }=\langle ·u_2^{\prime }·v^{\prime }·\rangle$ and $v_1^{″}=\langle ·u_2^{\prime }·v^{″}·\rangle$. Then, $p_1^{\prime }\stackrel{\mathit{hdg}\left(v_1^{\prime }\right)}{\to }{r}_1^{\prime }$, $p_1^{\prime }\stackrel{\mathit{hdg}\left(v_1^{″}\right)}{\to }{r}_1^{″}$ wrt $\Delta$. Using the induction hypothesis applied to $u_1$, on which $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ has a partial run to state $(P_1,D_1)$, such that $(p_1^{\prime },p_1^{″})\in {\left(P_1\right)}^2$, $p_1^{\prime }\stackrel{\mathit{hdg}\left(v_1^{\prime }\right)}{\to }{r}_1^{\prime }$, $p_1^{\prime }\stackrel{\mathit{hdg}\left(v_1^{″}\right)}{\to }{r}_1^{″}$, and $(r_1^{\prime },r_1^{″})\in D$, there exist $(u_1^{\prime },u_1^{″})\in clas{s}_{\mathbf{S}}^L\left(u_1\right)$, $w_1\in \mathit{suffs}\left({\mathcal{N}}_{\Sigma }\right)$, $q^{\prime }{\not\approx }_{F_{\mathbf{S}}}^F{q}^{″}$, and $q_0\in I$, such that

$$q_0\stackrel{\mathit{hdg}(u_1^{\prime }·v_1^{\prime }·w_1)}{\to }{q}^{\prime }\mathrm{wrt}\Delta \wedge q_0\stackrel{\mathit{hdg}(u_1^{″}·v_1^{″}·w_1)}{\to }{q}^{″}\mathrm{wrt}\Delta$$

Let $u^{\prime }=u_1^{\prime }·\langle ·u_2^{\prime }$, $u^{″}=u_1^{″}·\langle ·u_2^{″}$, and $w=\rangle ·w_1$. The above then yields the following:

$$q_0\stackrel{\mathit{hdg}(u^{\prime }·v^{\prime }·w)}{\to }{q}^{\prime }\mathrm{wrt}\Delta \wedge q_0\stackrel{\mathit{hdg}(u^{″}·v^{″}·w)}{\to }{q}^{″}\mathrm{wrt}\Delta$$

Furthermore, $(u^{\prime },u^{″})\in {\left(clas{s}_{\mathbf{S}}^L\left(u\right)\right)}^2$, so this was to be shown.   □

Proposition 6. 

Let $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ a dSha, $F\subseteq F_{\mathit{S}}\subseteq \mathcal{Q}$, $L=\mathcal{L}\left(A\right)$, and $\mathit{S}=\mathcal{L}\left(A[F/F_{\mathit{S}}]\right)$. If $A^{\mathit{cgr}\left(\mathit{S}\right)}$ has a partial run on prefix $u\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$ to some state $(P,D)\in 2^{\mathcal{Q}}\times {\mathcal{D}}^{\Delta }$ so that $P\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$, then $clas{s}_{\mathit{S}}^L\left(u\right)$ is subhedge relevant for L wrt $\mathit{S}$.

Proof. 

Suppose that $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ has a partial run on prefix $u\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$ to some state $(P,D)\in 2^{\mathcal{Q}}\times {\mathcal{D}}^{\Delta }$ so that $P\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$. Since $P\notin {\mathit{dPrj}}^{\Delta }\left(D\right)$, there exist ${\mathit{acc}}^{\Delta }{\left(P\right)}^2\cap D\ne emptyset$. So, there exist $(p^{\prime },p^{″})\in P^2$, $(r^{\prime },r^{″})\in D$, and $(v^{\prime },v^{″})\in {\mathcal{N}}_{\Sigma }^2$, such that  

$$p^{\prime }\stackrel{\mathit{hdg}\left(v^{\prime }\right)}{\to }{r}^{\prime }\mathrm{wrt}\Delta \wedge p^{″}\stackrel{\mathit{hdg}\left(v^{″}\right)}{\to }{r}^{″}\mathrm{wrt}\Delta$$

Using Lemma 17, there exist $(u^{\prime },u^{″})\in clas{s}_{\mathbf{S}}^L\left(u\right)$, $w\in \mathit{suffs}\left({\mathcal{N}}_{\Sigma }\right)$, $q^{\prime }{\not\approx }_{F_{\mathbf{S}}}^F{q}^{″}$, and $q_0\in I$, such that

$$q_0\stackrel{\mathit{hdg}(u^{\prime }·v^{\prime }·w)}{\to }{q}^{\prime }\mathrm{wrt}\Delta \wedge q_0\stackrel{\mathit{hdg}(u^{″}·v^{″}·w)}{\to }{q}^{″}\mathrm{wrt}\Delta$$

Hence, $clas{s}_{\mathbf{S}}^L\left(u\right)$ is relevant for L wrt $\mathbf{S}$.   □

Theorem 3 (Completeness of congruence projection.).

For any complete dSha $(\Sigma ,\mathcal{Q},\Delta ,I,F)$ and set $F\subseteq F_{\mathit{S}}\subseteq \mathcal{Q}$, the congruence projection $A^{\mathit{cgr}\left(\mathit{S}\right)}$ is sound and complete for subhedge projection for the regular pattern $\mathcal{L}\left(A\right)$ wrt the regular schema $\mathit{S}=\mathcal{L}\left(A[F_{\mathit{S}}/F]\right)$.

Proof. 

The soundness of congruence projection is shown in Theorem 2. For proving completeness, let $u\in \mathit{prefs}\left({\mathcal{N}}_{\Sigma }\right)$ be a nested word prefix that is strongly subhedge irrelevant for L wrt $\mathbf{S}$. By definition of strong irrelevance, the class $clas{s}_{\mathbf{S}}^L\left(u\right)$ is irrelevant for L and $\mathbf{S}$. Let $(P,D)$ be the unique state assigned by the partial run of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ to prefix u. Since $clas{s}_{\mathbf{S}}^L\left(u\right)$ is irrelevant for L and $\mathbf{S}$, Proposition 6 shows that $P\in {\mathit{dPrj}}^{\Delta }\left(D\right)$. Using Lemma 14, $(P,D)$ is then a subhedge projection state of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$.   □

7.5. In-Memory Complexity

We next discuss the complexity of membership testing with complete subhedge projection based on the in-memory evaluation of the input hedge by the congruence projection of the input automaton.

Lemma 18. 

The number of states $|{\mathcal{Q}}^{\mathit{cgr}}|$ is in $O(2^{n^2+n})$, where n is the number of states of A.

Proof. 

With the deterministic construction, the states of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ are pairs in $2^{\mathcal{Q}}\times {\mathcal{D}}^{\Delta }$. So, the maximal number of states of the congruence projection is $|Q^{\mathit{cgr}\left(\mathbf{S}\right)}|=2^{\left|Q\right|}\left|{\mathcal{D}}^{\Delta }\right|\le 2^n{2}^{n^2}=2^{n^2+n}$.   □

Let $\mathit{diff-rel}\left({\mathcal{Q}}^{\mathit{cgr}\left(\mathbf{S}\right)}\right)$ be the set of difference relations used in the states of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$.

Corollary 1. 

Let the signature Σ be fixed. For any complete dSha  A with schema $\mathit{S}\subseteq {\mathcal{H}}_{\Sigma }$, the membership of any hedge $h\in \mathit{S}$ can be tested in memory in time $O\left(\right|1\left|\right)$ per nonprojected node of h after a preprocessing time of $O\left(\right|A^{\mathit{cgr}\left(\mathit{S}\right)}|+n^{3}d)$, where d is the cardinality of $\mathit{diff-rel}\left({\mathcal{Q}}^{\mathit{cgr}\left(\mathit{S}\right)}\right)$ and n is the number of states of A.

Since $|{\mathcal{Q}}^{\mathit{cgr}\left(\mathbf{S}\right)}|$ is in $O\left(2^{n^2+n}\right)$ by Lemma 18, the preprocessing time $O\left(\right|A^{\mathit{cgr}\left(\mathbf{S}\right)}|+n^{3}d|)$ is in $O\left(2^{2n^2+2n}\right)$, too.

Proof. 

Since A is complete, Thereom 3 shows that $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ is complete for subhedge projection for schema $\mathbf{S}$. The in-memory evaluator of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ can be run on any input hedge $h\in \mathbf{S}$ in time $O\left(\right|1\left|\right)$ per nonprojected node of h after a preprocessing time of $O\left(\right|A^{\mathit{cgr}\left(\mathbf{S}\right)}|+n^{3}d)$, where d is the cardinality of $\mathit{diff-rel}\left(A^{\mathit{cgr}\left(\mathbf{S}\right)}\right)$. Since the signature $\Sigma$ is fixed and $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ deterministic, $|A^{\mathit{cgr}\left(\mathbf{S}\right)}|$ is in $O\left(\right|{\mathcal{Q}}^{\mathit{cgr}\left(\mathbf{S}\right)}{|}^2)$. The dominating part $|A^{\mathit{cgr}\left(\mathbf{S}\right)}|$ is in $O\left(\right|{\mathcal{Q}}^{\mathit{cgr}}{|}^2)$.   □

When applied to regular XPath query answering as in our experiments in Section 11, we have $n\le 80$ and $d\le 25$ so the cubic part $O\left(n^{3}d\right)$ is not limiting. The sizes of the Sha^↓s are below 2600 counting both states and rules. The upper bound $O\left(\right|A^{\mathit{cgr}\left(\mathbf{S}\right)}|+n^{3}d)$ thus suggests that the preprocessing time may be feasible in practice, even though exponential in the worst case.

We must be careful here since the sizes of the Sha^↓s reported in Section 11 do not refer to the pure congruence projection $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ but to the earliest congruence projection $A_{e\left(\mathbf{S}\right)}^{\mathit{cgr}\left(\mathbf{S}\right)}$ that we will introduce only in Section 8. The noncreation of transition rules to states that are safe for rejection may lead to a radical size reduction. Alternatively, an important size reduction in $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ could be obtained by removing all useless states and transition rules. But, this would not reduce the precomputation time, just the opposite: it woud increase the preprocessing time since cleaning automata a posteriori also needs time, which for Sha^↓s may be nonlinear.

Instead of precomputing $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ from the inputs A and $F_{\mathbf{S}}$ statically, we can compute only the part needed of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ for evaluating the input hedge h on-the-fly. We note that the number of transition rules of the needed part is bounded by $O\left(\right|h\left|\right)$ but may be way smaller due to projection. The preprocessing time then goes down to $O\left(\right|A\left|\right)$. Since the full automaton $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ may be exponential in n in the worst case, this may reduce the overall processing time.

On the positive side, on-the-fly evaluation reduces the overall computation time to $O\left(\right|h|+n^{3}d\left(h\right))$, where $d\left(h\right)$ is the number of difference relations in the part of $A^{\mathit{cgr}\left(\mathbf{S}\right)}$ needed for the evaluation of h, so $d\left(h\right)\le min\left(\right|h|,d)$. On the down side, $d\left(h\right)$ steps will no more be in constant but cubic time in $O\left(n^3\right)$. It should also be noted that the overall time of the safe-no-change projection on-the-fly evaluation is in $O\left(\right|A|+|h|+ns(h\left)\right)$, where $s\left(h\right)$ is the number of subsets of safe states of A in states of ${\mathcal{Q}}^{\mathit{cgr}}$, i.e., $s\left(h\right)\le \left|h\right|$. This is since each subset of safe states can be computed in linear time in $O\left(n\right)$. As argued above, the overall time of the congruence projection obtained by on-the-fly evaluation is in $O\left(\right|A|+|h|+n^{3}d\left(h\right))$. This is because the computation of a difference relation is in $O\left(n^3\right)$ by using ground Datalog programs of cubic size. The additional cubic factor in $n^{3}d\left(h\right)$ instead of a linear factor in $ns\left(h\right)$ is the price to be paid for achieving complete subhedge projection with on-the-fly evaluation. Note that this price is largely acceptable in cases where $n\le 80$, $d\left(h\right)\le 15$, and $s\left(h\right)\le 15$.

Example 15 (Exponential blow-up.).

In order to see how the exponential worst case may happen, we consider a family of regular languages, for which the minimal left-to-right Dfa is exponentially bigger than the minimal right-to-left Dfa. The classical example languages with this property are $L_n={\Sigma }^{\ast }.a.{\Sigma }^n$, where $n\in \mathbb{N}$ and $\Sigma =\{a,b\}$. Intuitively, a word in ${\Sigma }^{\ast }$ belongs to $L_n$ if and only its $n+1$-th letter from the end is equal to $‘a’$. The minimal left-to-right Dfa for $L_n$ has $2^{n+1}$ many states since it needs to memorize a window of $n+1$ letters. In contrast, its minimal right-to-left Dfa has only $n+1$ states; in this direction, it is sufficient to memorize the distance from the end modulo $n+1$.

We next consider the schema $\mathit{S}$ defined by $\mu z.{\langle (a+b)·z\rangle }^{\ast }$. It contains all hedges having in all its subtrees a label from $\Sigma =\{a,b\}$ as the first letter, and no further occurrences of letters of Σ elsewhere. We then consider the family of hedge languages $H_n\in {\mathcal{H}}_{\Sigma }$ with schema $\mathit{S}$, such that the sequence of labels of some root-to-leaf path of $h_n$ belongs to $L_n$. Note that $H_n$ can be recognized in a bottom-up manner by the dSha $A_n$ with $O(n+1)$ states, which simulates the minimal deterministic  Dfa  of $L_n$ on all paths of the input hedge. For an evaluator with subhedge projection, the situation is different. When moving top-down, it needs to memorize the sequence of labels of the $n+1$-last ancestors, possibly filled with $b^{\prime }s$, and there are $2^{n+1}$ of such sequences. If, for some leaf, its sequence starts with an “a” then the following subhedges with the following leaves can be projected away. As a consequence, there cannot be any dSha^↓ recognizing $H_n$ that projects away all irrelevant subhedges with less than $2^{n+1}$ states. In particular, the size of $A_n^{\mathit{cgr}\left(\mathit{S}\right)}$ must be exponential in the size of $A_n$.

8. Earliest Membership with Subhedge Projection

We next enhance our compilers for introducing subhedge projection states, such that they can also detect certain membership or nonmembership in an earliest manner. For this, we show how to combine the previous earliest membership tester for dShas from [21] with any subtree projection algorithm so that it applies to both, to safe-no-change projection and to congruence projection. By combining both aspects orthogonally, we obtain an earliest dSha with subhedge projection, which can be evaluated either in in memory or in streaming mode.

8.1. Earliest Membership

We start with a semantic characterization of earliest membership, i.e., of prefixes of nested words of hedges whose suffixes are irrelevant for membership, when assuming top-down and left-to-right processing. Note that the same characterization up to various presentation choices was given earlier in the context of stream processing [21,22].

Definition 13. 

Let $\mathit{S}\subseteq {\mathcal{H}}_{\Sigma }$ be a hedge schema and $L\subseteq \mathit{S}$ a hedge language satisfying this schema. A nested word prefix v is certain for membership in L with schema $\mathit{S}$ for all suffixes w of nested words such that $v·w\in \mathit{nw}\left(\mathit{S}\right)$ it holds that $v·w\in \mathit{nw}\left(L\right)$. It is called certain for nonmembership in L with schema $\mathit{S}$ for all suffixes w of nested words such that $v·w\in \mathit{nw}\left(\mathit{S}\right)$ it holds that $v·w\notin \mathit{nw}\left(L\right)$.

We are now interested in Sha^↓ that are able to detect certain membership and nonmembership syntactically at the earliest possible prefix or node of the run.

Definition 14. 

A dSha^↓ $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ with schema $\mathit{S}\subseteq {\mathcal{H}}_{\Sigma }$ is called earliest for schema $\mathit{S}$ if there exists a state $\mathit{sel}\in \mathcal{Q}$, such that for all hedges $h\in \mathit{S}$, the unique run R of $compl\left(A\right)$ on h satisfies the following:

• it goes to the state $\mathit{sel}$ for exactly all those prefixes of $\mathit{nw}\left(h\right)$ that are certain for membership in $\mathcal{L}\left(A\right)$ wrt $\mathit{S}$;
• it goes to the sink for all prefixes of $\mathit{nw}\left(h\right)$ that are certain for nonmembership in $\mathcal{L}\left(A\right)$ with $\mathit{S}$.

For dSha A, schema-complete for schema $\mathbf{S}=\mathcal{L}\left(A[F/F_{\mathbf{S}}]\right)$, we can construct a dSha^↓ $A_{e\left(\mathbf{S}\right)}$ that is the earliest for $\mathbf{S}$, as shown in Appendix A. The idea is to adapt the earliest membership tester for deterministic Shas from [21] so that it takes schema restrictions into account. Furthermore, we need to compile the input dShas to dSha^↓s rather than to dNwas, as used there. This change is straightforward once the relationship between both automata models becomes clear.

Given that the construction of $A_{e\left(\mathbf{S}\right)}$ is not part of the core of the present paper, we only illustrate its outcome at our running example: for the dSha A with schema $\mathbf{S}=〚\mathtt{doc}〛$, in Figure 4, we obtain the Sha^↓ $A_{e\left(\mathbf{S}\right)}$ in Figure 19. Note that, on the prefix $\langle \mathtt{list}\langle \mathtt{item}$, it goes to state to $\mathit{sel}$, showing that it is certain for membership. On prefix $\langle \mathtt{item}$, it blocks showing that it is certain for nonmembership. However, on the prefix $\langle \mathtt{list}\langle \mathtt{list}$, it does not go into a subhedge projection state even though this prefix is subhedge irrelevant.

8.2. Adding Subhedge Projection

We now enhance any earliest membership for dShas with subhedge projection. The idea is to combine subhedge projection with earliest membership. So, suppose that we are given a schema $\mathbf{S}$ and two automata, $A^{\pi }$ and $A_e$, with schema $\mathbf{S}$ that recognize the same language up to the schema, so $\mathcal{L}\left(A^{\pi }\right)\cap \mathcal{L}\left(\mathbf{S}\right)=\mathcal{L}\left(A_e\right)\cap \mathcal{L}\left(\mathbf{S}\right)$. Furthermore, assume that $A_e$ has a selection state $\mathit{sel}$, indicating certain membership at the earliest possible prefix.

In order to obtain the earliest membership with subhedge projection, we combine the two Sha^↓s into a single Sha^↓, $A_e^{\pi }=(\Sigma ,{\mathcal{Q}}_e^{\pi },$ ${\Delta }_e^{\pi },$$I_e^{\pi },F_e^{\pi })$, basically running both automata in parallel but under shared control. The state set of $A_e^{\pi }$ are as follows:

$$\begin{array}{c}{\mathcal{Q}}_e^{\pi }=({\mathcal{Q}}^{\pi }\times ({\mathcal{Q}}_e\backslash \left\{\mathit{sel}\right\}))\cup \left\{\mathit{sel}\right\}\hfill \\ I_e^{\pi }=(I^{\pi }\times (I_e\backslash \left\{\mathit{sel}\right\}))\cup \{\mathit{sel}\mid \mathit{sel}\in I_e\}\hfill \\ F_e^{\pi }=(F^{\pi }\times (F_e\backslash \left\{\mathit{sel}\right\}))\cup \left\{\mathit{sel}\right\}\hfill \end{array}$$

The transition rules in ${\Delta }_e^{\pi }$ are given in Figure 20. Any run of $A_e^{\pi }$ synchronizes parallel runs of $A^{\pi }$ and $A_e$ as follows: whenever $A_e$ goes to $\mathit{sel}$, then $A_e^{\pi }$ does so too, and whenever $A^{\pi }$ goes into a subhedge projection state, then it makes $A_e$ jump over the subsequent subhedge.

For illustration, we reconsider the dSha from Figure 4. The earliest dSha^↓ with subhedge projection $A_{e\left(\mathbf{S}\right)}^{\mathit{snc}}$ is given in Figure 21. Here, we combine the safe-no-change projection $A^{\mathit{snc}}$ from Figure 11 with the earliest membership tester $A_{e\left(\mathbf{S}\right)}$ from Figure 19. Their combination $A_{e\left(\mathbf{S}\right)}^{\mathit{snc}}$ represents the pattern $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$ with schema $〚\mathtt{doc}〛$ in a concise manner. After prefix $\langle \mathtt{list}\langle \mathtt{item}$, it detects certain membership: for the prefix $\langle \mathtt{list}\langle \mathtt{list}$, it detects that the subhedge is irrelevant, and after prefix $\langle \mathtt{item}$, it detects certain nonmembership.

8.3. Soundness and Completeness

We next show that the soundness and completeness of a projection algorithm are preserved when combined with some earliest membership tester when assuming determinism.

Proposition 7. 

Let $A^{\pi }$ and $A_e$ be dSha^↓s with the same schema $\mathit{S}$ and the same language up to the schema, i.e., $\mathcal{L}\left(A^{\pi }\right)\cap \mathcal{L}\left(\mathit{S}\right)=\mathcal{L}\left(A_e\right)\cap \mathcal{L}\left(\mathit{S}\right)$. The combination dSha^↓$A_e^{\pi }$ then has the same language up to schema $\mathit{S}$, too. Furthermore, if $A_e$ is earliest for S, then $A_e^{\pi }$ is earliest for S and goes into some subhedge projection state whenever $A^{\pi }$ does.

Proof. 

If q is a subhedge projection state of $A^{\pi }$ and r a state of $A_e$, then $(q,r)$ is a subhedge projection state of $A_e^{\pi }$. The evaluator for automaton $A_e^{\pi }$ runs $A^{\pi }$ and $A_e$ in parallel on the input hedge, while skipping subhedges starting in subhedge projection states of $A^{\pi }$. These include the subhedges starting in a projection state when evaluating $A_e^{\pi }$ on h. Applying Proposition 2 to $A^{\pi }$, such subhedges are irrelevant for $\mathcal{L}\left(A^{\pi }\right)$ with respect to $\mathbf{S}$, since $A^{\pi }$ is assumed to be deterministic. Since $\mathcal{L}\left(A^{\pi }\right)\cap \mathbf{S}=\mathcal{L}\left(A_e\right)\cap \mathbf{S}$, skipping such a subhedge does not affect acceptance by $A_e$ for hedges inside schema $\mathbf{S}$. Therefore, the evaluator of $A_e^{\pi }$ is earliest with respect to $\mathbf{S}$, too.   □

The above proposition shows that if $A^{\pi }$ is complete for subhedge projection, then $A_e^{\pi }$ is also complete for subhedge projection while also being earliest.

Theorem 4. 

For any complete dShas A with schema $\mathit{S}$, the dSha  $A^{\mathit{cgr}}{\left(\mathit{S}\right)}_{e\left(\mathit{S}\right)}$ is earliest and complete for subhedge projection for schema $\mathbf{S}$.

Proof. 

The congruence projection $A^{\mathit{cgr}}\left(\mathbf{S}\right)$ is sound by Proposition 2 and complete for subhedge projection wrt $\mathbf{S}$ by Theorem 3. The automaton $A_{e\left(\mathbf{S}\right)}$ discussed in Section 8.1 is earliest for S. So, their combination $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ yields the earliest automaton with complete subhedge projection wrt $\mathbf{S}$ using Proposition 7.   □

8.4. In-Memory Complexity

We next discuss the complexity of earliest membership testing with complete subhedge projection by an in-memory evaluator for $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$.

Lemma 19. 

The number of states in ${\mathcal{Q}}^{\mathit{cgr}}{\left(\mathit{S}\right)}_{e\left(\mathit{S}\right)}$ is in $O(2^{n^2+2n+log\left(n\right)})$, where $n=\left|\mathcal{Q}\right|$.

Proof. 

Using Lemma 18, the number of states of $A^{\mathit{cgr}}\left(\mathbf{S}\right)$ is in $O(2^{n^2+n})$, where $n=\left|\mathcal{Q}\right|$. The number of states of $A_{e\left(\mathbf{S}\right)}$ is in $O\left(n{2}^n\right)$, and thus, in $O(2^{n+log\left(n\right)})$. The number of states of $Q^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ is thus in $O(2^{n^2+n}{2}^{n+log\left(n\right)})$, which is in $O(2^{n^2+2n+log\left(n\right)})$   □

Corollary 2. 

Let the signature Σ be fixed. For any complete dSha  A with schema $\mathit{S}\subseteq {\mathcal{H}}_{\Sigma }$, earliest membership on a input hedge $h\in \mathit{S}$ can be tested in memory in time $O\left(\right|1\left|\right)$ per nonprojected node of h after a preprocessing time of $O\left(\right|A^{\mathit{cgr}}{\left(\mathit{S}\right)}_{e\left(\mathit{S}\right)}|+n^{3}d+ns)$, where d is the number of difference relations and s is the number of subset of safe states in ${\mathcal{Q}}^{\mathit{cgr}}{\left(\mathit{S}\right)}_{e\left(\mathit{S}\right)}$. The preprocessing time is also in $O(2^{2n^2+4n+2log\left(n\right)})$, where n is the number of states of A.

Proof. 

Since A is complete, Thereom 4 shows that $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ is the earliest, sound, and complete for subhedge projection for schema $\mathbf{S}$. The in-memory evaluator of $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ can be run on any input hedge $h\in \mathbf{S}$ in time $O\left(\right|1\left|\right)$ per nonprojected node of h after a preprocessing time of $O\left(\right|A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}|+n^{3}d+ns$. $\Sigma$ is fixed to the size $|A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}|$ bounded by $O\left(\right|Q^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}{|}^2)$. Lemma 19 shows that $|{\mathcal{Q}}^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}|$ is in $O(2^{n^2+2n+log\left(n\right)})$, so $|A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}|$ is in $O(2^{2n^2+4n+2log\left(n\right)})$.   □

As before, instead of precomputing $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ from the input dSha A and $F_{\mathbf{S}}$, we can compute only the part needed for evaluating the input hedge h on-the-fly. If the needed part of $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ for evaluating h is small, the overall time and space go down considerably.

9. Streaming Algorithms

We show that dSha^↓s can be evaluated on the nested word of a hedge in a streaming manner, so that they produce the same results and projection behavior as by evaluating them on hedges in-memory in a top-down and left-to-right manner.

All aspects related to subhedge projection remain unchanged. Earliest query answering in streaming mode means that nested word suffixes are ignored if irrelevant, and not only nested words as with subhedge projection. What changes between both evaluation modes is memory consumption. For in-memory evaluation, the whole graph of the input hedge must be stored, while in streaming mode, only a stack and a state are maintained at any event. For testing regular language membership, the state space is finite, while the stack space become infinite, since the depth of the stack depends on the depth of the input hedge.

The property of having equivalent streaming and in-memory evaluators was already noticed in [26] for Neumann and Seidl’s pushdown forest automata [25] and for nested word automata [11,15]. We here show how to transpose this result to dSha^↓s. This permits us to show the correctness of our evaluation algorithms based on properties of the constructed automata. It also permits obtaining correctness properties of streaming algorithms by reasoning about the in-memory evaluator of Sha^↓s.

9.1. Streaming Evaluators for Downward Hedge Automata

We define the streaming evaluator of a Sha^↓ $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ by a visibly pushdown machine. The set of configurations of this machine is $\mathcal{K}=\mathcal{Q}\times {\mathcal{Q}}^{\ast }$, containing pairs of states and stacks of states. The visibly pushdown machine provides for any word $v\in {\widehat{\Sigma }}^{\ast }$ a streaming transition relation ${\stackrel{v}{\to }}^{str}\subseteq \mathcal{K}\times \mathcal{K}$, such that for all states $q,q^{\prime }\in \mathcal{Q}$ and stacks $\sigma \in {\mathcal{Q}}^{\ast }$ and words $v,v^{\prime }\in {\widehat{\Sigma }}^{\ast }$:

$$\frac{\mathit{true}}{(q,\sigma ){\stackrel{\epsilon }{\to }}^{\mathit{str}}(q,\sigma ) \mathrm{wrt}\Delta }\hspace{1em}\frac{(q,\sigma ){\stackrel{v}{\to }}^{\mathit{str}}(q\prime ,\sigma )\hspace{1em}(q\prime ,\sigma ){\stackrel{v\prime }{\to }}^{\mathit{str}}(q″,\sigma ) \mathrm{wrt}\Delta }{(q\prime ,\sigma ){\stackrel{v·v\prime }{\to }}^{\mathit{str}}(q″,\sigma ) \mathrm{wrt}\Delta }$$

$$\frac{q\stackrel{a}{\to }q\prime \mathrm{in}\Delta }{q{\stackrel{a}{\to }}^{\mathit{str}}q\prime \mathrm{wrt}\Delta }\hspace{1em}\frac{q\stackrel{\langle \rangle }{\to }q\prime \mathrm{in}\Delta }{(q,\sigma ){\stackrel{\langle }{\to }}^{\mathit{str}}(q\prime ,\sigma ·q) \mathrm{wrt}\Delta }\hspace{1em}\frac{q@p\to q\prime \mathrm{in}\Delta }{(p,\sigma ·q){\stackrel{\rangle }{\to }}^{\mathit{str}}(q\prime ,\sigma ) \mathrm{wrt}\Delta }$$

We note that the same visibly pushdown machine can be obtained by compiling the Sha^↓ to an Nwa, whose streaming evaluator is defined by a visibly pushdown machine too (see Section 12.3 on related work).

The notion of partial runs of the in-memory evaluator is closely related to streaming runs. To see this, we first note that each partial run v wrt $\Delta$ naturally defines a stack as follows. Since v is a nested word prefix, there exist unique nested words $v_0,\dots ,v_n\in \mathit{nw}\left({\mathcal{H}}_{\Sigma \cup \mathcal{Q}}\right)$, such that

$$v=v_0·\langle ·v_1·\langle ·\dots ·\langle ·v_n.$$

These nested words must be partial runs of $\Delta$ too, so there exist states $p_0,\dots p_n,q_0,\dots ,q_n\in \mathcal{Q}$, such that $v_i$ goes from $p_i$ to $q_i$ for all $0\le i\le n$. We define the stack $\sigma$ of v by $\sigma =q_0·\dots ·q_{n-1}$ and say that v goes from $p_0$ to $q_n$ and has stack $\sigma$. Note that the stack is empty if v is a nested word, since in this case $n=0$ so that $v=v_0$ and $\sigma =\epsilon$. For instance, the partial run $v_0=0·\langle ·0·list·1·\langle ·0·item·2$ has the stack ${\sigma }_0=0·1$ while the partial run $v_1=v_0·\rangle ·3·\rangle ·4$ has stack ${\sigma }_1=\epsilon$.

Lemma 20. 

Consider a Sha  $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$, states $q,q^{\prime }\in \mathcal{Q}$, and a stack $\sigma \in {\mathcal{Q}}^{\ast }$. Let v be a prefix of some nested word in $\mathit{nw}\left({\mathcal{H}}_{\Sigma }\right)$. Then, there exists a partial run r wrt Δ from q to $q^{\prime }$ with stack σ such that ${\mathit{proj}}_{\Sigma }\left(r\right)=v$ iff $(q,\epsilon ){\stackrel{\epsilon }{\to }}^{\mathit{str}}(q^{\prime },\sigma )$ wrt Δ.

Proof. 

This is carried out by induction on the length of prefix v.   □

This lemma implies that any in-memory transition of A on a hedge h from q to $q^{\prime }$ can be identified with some streaming transition of A on $\mathit{nw}\left(h\right)$ from $(q,\epsilon )$ to $(q^{\prime },\epsilon )$.

Proposition 8. 

$q\stackrel{\mathit{h}}{\to }{q}^{\prime }\mathit{wrt}\Delta \iff (q,\epsilon ){\stackrel{\mathit{nw}\left(h\right)}{\to }}^{\mathit{str}}(q^{\prime },\epsilon )\mathit{wrt}\Delta$.

Proof. 

If $q\stackrel{\mathit{h}}{\to }{q}^{\prime }\mathrm{wrt}\Delta$, there exists a run R of $\mathit{h}$ with $\Delta$ that starts with q and ends with $q^{\prime }$. Consider the partial run $v=\mathit{nw}\left(R\right)$ on h. Since v is a nested word, its stack is empty. By Lemma 20, we have $(q,\epsilon ){\stackrel{{\mathit{proj}}_{\Sigma }\left(v\right)}{\to }}^{\mathit{str}}(q^{\prime },\epsilon )$ wrt $\Delta$ and ${\mathit{proj}}_{\Sigma }\left(v\right)=\mathit{nw}\left(h\right)$. For the inverse implication, assume that $(q,\epsilon ){\stackrel{\mathit{nw}\left(h\right)}{\to }}^{\mathit{str}}(q^{\prime },\epsilon )\mathrm{wrt}\Delta$. Let $u=\mathit{nw}\left(h\right)$. By Lemma 20, there exists a partial run v of $\Delta$ from q to $q^{\prime }$ with stack $\epsilon$. Then, there exists a run R of $\Delta$ on h such that $v=\mathit{nw}\left(R\right)$. Hence, $q\stackrel{h}{\to }{q}^{\prime }\mathrm{wrt}\Delta$.   □

For any deterministic dSha^↓ A, hedge h, and state q, the streaming transition on $\mathit{nw}\left(h\right)$ with respect to $\Delta$ starting with $(q,\epsilon )$ can be computed in time $O\left(1\right)$ per letter after a precomputation in time $O\left(\right|A\left|\right)$. So, the overall computation time is in $O\left(\right|A|+|h\left|\right)$. The streaming memory needed to store a configuration is of size $O\left(depth\right(h)+|A\left|\right)$ since the size of the visible stack is bounded by the depth of the input hedge.

9.2. Adding Subhedge Projection

We next extend the streaming transition relation with subhedge projection, in analogy to what we did for the in-memory transition relation. We define a transition relation with subhedge projection ${\stackrel{\mathit{h}}{\to }}_{\mathit{shp}}^{\mathit{str}}\subseteq \mathcal{K}\times \mathcal{K}$ with respect to $\Delta$, such that for all words $v,v^{\prime }\in {\widehat{\Sigma }}^{\ast }$, letters $a\in \Sigma$, states $p,q,q^{\prime },q^{″}\in \mathcal{Q}$, and stacks $\sigma ,{\sigma }^{\prime },{\sigma }^{″}\in {\mathcal{Q}}^{\ast }$:

$$\frac{q\in {\mathcal{Q}}_{\mathit{shp}}^{\Delta }\hspace{1em}v\in \mathit{nw}\left({\mathcal{H}}_{\Sigma }\right)}{(q,\sigma ){\stackrel{v}{\to }}_{\mathit{shp}}^{\mathit{str}}(q,\sigma ) \mathrm{wrt}\Delta }$$

$$\frac{q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }\hspace{1em}q\stackrel{a}{\to }{q}^{\prime }\mathrm{in}\Delta }{(q,\sigma ){\stackrel{a}{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },\sigma ) \mathrm{wrt}\Delta }\hspace{1em}\frac{q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }}{(q,\sigma ){\stackrel{\epsilon }{\to }}_{\mathit{shp}}^{\mathit{str}}(q,\sigma ) \mathrm{wrt}\Delta }$$

$$\frac{q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }\hspace{1em}(q,\sigma ){\stackrel{v}{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },{\sigma }^{\prime }) \mathrm{wrt}\Delta \hspace{1em}(q^{\prime },{\sigma }^{\prime }){\stackrel{v\prime }{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{″},{\sigma }^{″}) \mathrm{wrt}\Delta }{(q,\sigma ){\stackrel{v·v\prime }{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{″},{\sigma }^{″}) \mathrm{wrt}\Delta }$$

$$\frac{q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }\hspace{1em}q\stackrel{\langle \rangle }{\to }{q}^{\prime }\mathrm{in}\Delta }{(q,\sigma ){\stackrel{\langle }{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },\sigma ·q) \mathrm{wrt}\Delta }\hspace{1em}\frac{p\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }\hspace{1em}q@p\to q^{\prime }\mathrm{in}\Delta }{(p,\sigma ·q){\stackrel{\rangle }{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },\sigma ) \mathrm{wrt}\Delta }$$

The projecting transition relation stays in a configuration with a subhedge projection state until the end of the current subhedge is reached. This is correct since a subhedge projection state cannot be changed anyway.

Proposition 9. 

For any Sha^↓ $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$, states $q,q^{\prime }\in \mathcal{Q}$, nested word v on which Δ has no blocking partial run starting from q:

$$(q,\epsilon ){\stackrel{v}{\to }}^{\mathit{str}}(q^{\prime },\epsilon ) \mathit{wrt}\Delta \iff (q,\epsilon ){\stackrel{v}{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },\epsilon ) \mathit{wrt}\Delta .$$

Proof. 

Let $h\in {\mathcal{H}}_{\Sigma }$ be the hedge such that $v=\mathit{nw}\left(h\right)$. The proof is by induction on the size of v. For the implication from the left to the right we assume that $(q,\epsilon ){\stackrel{v}{\to }}^{\mathit{str}}(q^{\prime },\epsilon ) \mathrm{wrt}\Delta$. Since $v=\mathit{nw}\left(h\right)$, Proposition 8 proves $q{\stackrel{h}{\to }}^{\mathit{str}}{q}^{\prime }$ wrt $\Delta$.

Case 

$q\in {\mathcal{Q}}_{\mathit{shp}}^{\Delta }$. By Lemma 6, it follows from $q{\stackrel{h}{\to }}^{\mathit{str}}{q}^{\prime }$ wrt $\Delta$ that $q=q^{\prime }$. We can then apply the following rule since v is a nested word:

$$\frac{q\in {\mathcal{Q}}_{\mathit{shp}}^{\Delta }\hspace{1em}v=\mathit{nw}\left({\mathcal{H}}_{\Sigma }\right)}{(q,\epsilon ){\stackrel{v}{\to }}_{\mathit{shp}}^{\mathit{str}}(q,\epsilon ) \mathrm{wrt}\Delta }$$

Hence, $(q,\epsilon ){\stackrel{v}{\to }}_{\mathit{shp}}^{\mathit{str}}(q,\epsilon ) \mathrm{wrt}\Delta$.

Case 

$q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }$. We distinguish all possible forms of hedge h.

Subcase 

$h=h^{\prime }·h^{″}$. Let $v^{\prime }=\mathit{nw}\left(h^{\prime }\right)$ and $v^{″}=\mathit{nw}\left(h^{″}\right)$ so $v=v^{\prime }·v^{″}=\mathit{nw}\left(h\right)$. The following rule must have been applied:

$$\frac{(q,\epsilon ){\stackrel{v\prime }{\to }}^{\mathit{str}}(p,\sigma ) \mathrm{wrt}\Delta \hspace{1em}(p,\sigma ){\stackrel{v″}{\to }}^{\mathit{str}}(q^{\prime },\epsilon ) \mathrm{wrt}\Delta }{(q,\epsilon ){\stackrel{v\prime ·v″}{\to }}^{\mathit{str}}(q^{\prime },\epsilon ) \mathrm{wrt}\Delta }$$

Since $v^{\prime }$ is a nested word, it follows that $\sigma =\epsilon$. By the induction hypothesis applied to the nested words $v^{\prime }$ and $v^{″}$, we get $(q,\epsilon ){\stackrel{v\prime }{\to }}_{\mathit{shp}}^{\mathit{str}}(p,\epsilon ) \mathrm{wrt}\Delta$ and $(p,\epsilon ){\stackrel{v″}{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },\epsilon ) \mathrm{wrt}\Delta$. Hence, we can apply the following rule:

$$\frac{q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }\hspace{1em}(q,\epsilon ){\stackrel{v\prime }{\to }}_{\mathit{shp}}^{\mathit{str}}(p,\epsilon ) \mathrm{wrt}\Delta \hspace{1em}(p,\epsilon ){\stackrel{v″}{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },\epsilon ) \mathrm{wrt}\Delta }{(q,\epsilon ){\stackrel{v\prime ·v″}{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },\epsilon ) \mathrm{wrt}\Delta }$$

This shows $(q,\epsilon ){\stackrel{v}{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },\epsilon )$ as required.

Subcases

$h=\langle h^{\prime }\rangle$ or $h=a$ or $h=\epsilon$. Straightforward.

For the implication from the right to the left we assume that $(q,\epsilon ){\stackrel{v}{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },\epsilon ) \mathrm{wrt}\Delta$.

Case 

$q\in {\mathcal{Q}}_{\mathit{shp}}^{\Delta }$. Then, the following rule must have been applied with $q=q^{\prime }$:

$$\frac{q\in {\mathcal{Q}}_{\mathit{shp}}^{\Delta }\hspace{1em}v=\mathit{nw}\left({\mathcal{H}}_{\Sigma }\right)}{(q,\epsilon ){\stackrel{v}{\to }}_{\mathit{shp}}^{\mathit{str}}(q,\epsilon ) \mathrm{wrt}\Delta }$$

Since we assume that v does not have any blocking partial runs with $\Delta$, there exists a partial run on v with $\Delta$ that starts with q. Since v is a nested word, any partial run on v is the nested word of some run, so there exists a run on the hedge h that starts with q. This shows the existence of some state p such that $q\stackrel{h}{\to }p$ wrt $\Delta$. Since $q\in {\mathcal{Q}}_{\mathit{shp}}^{\Delta }$, Lemma 6 shows $q=p$, and thus, $q\stackrel{h}{\to }q$ wrt $\Delta$. Proposition 8 then proves that $(q,\epsilon ){\stackrel{\mathit{nw}\left(h\right)}{\to }}^{\mathit{str}}(q,\epsilon )$ wrt $\Delta$, which is equal to $(q,\epsilon ){\stackrel{v}{\to }}^{\mathit{str}}(q^{\prime },\epsilon )$ wrt $\Delta$, as required.

Case 

$q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }$. We distinguish all possible forms of the hedge h.

Subcase 

$h=h^{\prime }·h^{″}$. Let $v^{\prime }=\mathit{nw}\left(h^{\prime }\right)$ and $v^{″}=\mathit{nw}\left(h^{″}\right)$ so $v=v^{\prime }·v^{″}=\mathit{nw}\left(h\right)$. The following rule must have been applied:

$$\frac{q\notin {\mathcal{Q}}_{\mathit{shp}}^{\Delta }\hspace{1em}(q,\epsilon ){\stackrel{v\prime }{\to }}_{\mathit{shp}}^{\mathit{str}}(p,\sigma ) \mathrm{wrt}\Delta \hspace{1em}(p,\sigma ){\stackrel{v″}{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },\epsilon ) \mathrm{wrt}\Delta }{(q,\epsilon ){\stackrel{v\prime ·v″}{\to }}_{\mathit{shp}}^{\mathit{str}}(q^{\prime },\epsilon ) \mathrm{wrt}\Delta }$$

Since $v^{\prime }$ is a nested word, it follows that $\sigma =\epsilon$ too. By the induction hypothesis applied to the nested words $v^{\prime }$ and $v^{″}$, we get $(q,\epsilon ){\stackrel{v\prime }{\to }}^{\mathit{str}}(p,\epsilon ) \mathrm{wrt}\Delta$ and $(p,\epsilon ){\stackrel{v″}{\to }}^{\mathit{str}}(q^{\prime },\epsilon ) \mathrm{wrt}\Delta$. Hence, we can apply the following rule:

$$\frac{(q,\epsilon ){\stackrel{v\prime }{\to }}^{\mathit{str}}(p,\epsilon ) \mathrm{wrt}\Delta \hspace{1em}(p,\epsilon ){\stackrel{v″}{\to }}^{\mathit{str}}(q^{\prime },\epsilon )\mathrm{wrt}\Delta }{(q,\epsilon ){\stackrel{v\prime ·v″}{\to }}^{\mathit{str}}(q^{\prime },\epsilon ) \mathrm{wrt}\Delta }$$

This shows $(q,\epsilon ){\stackrel{v}{\to }}^{\mathit{str}}(q^{\prime },\epsilon )$, as required.

Subcases

$h=\langle h^{\prime }\rangle$ or $h=a$ or $h=\epsilon$. Straightforward.

This ends the proofs of both directions.   □

A streaming evaluator with subhedge projection for deterministic Sha^↓s A on hedges h can thus be obtained by computing the streaming transition relation with subhedge projection for A of $\mathit{nw}\left(h\right)$ starting with the initial configuration. This costs time at most $O\left(1\right)$ per letter of $\mathit{nw}\left(h\right)$, i.e., constant time per event of the stream.

9.3. Streaming Complexity of Earliest Membership with Subhedge Projection

Using streaming evaluators for Sha^↓s, we can test earliest membership with subhedge projection in streaming mode by running the Sha^↓ $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$.

Corollary 3. 

For any complete dShas $(\Sigma ,\mathcal{Q},\Delta ,I,F)$, $F\subseteq F_{\mathit{S}}\subseteq \mathcal{Q}$, language $L=\mathcal{L}\left(A\right)$, and schema $\mathit{S}=\mathcal{L}\left(A[F/F_{\mathit{S}}]\right)$, earliest membership with complete subhedge projection for L wrt $\mathit{S}$ can be tested in streaming mode for any hedge $h\in \mathit{S}$ in time $O\left(\right|1\left|\right)$ per nonprojected letter of $\mathit{nw}\left(h\right)$ after a preprocessing time of $O\left(\right|A^{\mathit{cgr}}{\left(\mathit{S}\right)}_{e\left(\mathit{S}\right)}|+n^{3}d+ns)$, where d is the number of difference relations and s the number of safe subsets in states of ${\mathcal{Q}}_{e\left(\mathit{S}\right)}^{\mathit{cgr}}\left(\mathit{S}\right)$. The space required is in $O(depth\left(h\right)+|A^{\mathit{cgr}}{\left(\mathit{S}\right)}_{e\left(\mathit{S}\right)}\left|\right)$.

Proof. 

By Theorem 4, the dSha^↓ $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ is earliest and sound and complete for subhedge projection. By Propositions 8 and 9, we can thus obtain a streaming membership tester with subhedge projection for a hedge $h\in \mathbf{S}$ by running the streaming evaluator with subhedge projection of $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ on the nested word $\mathit{nw}\left(h\right)$. Since this Sha^↓ is deterministic, the streaming evaluation can be done in time $O\left(\right|1\left|\right)$ per letter of $\mathit{nw}\left(h\right)$ after a preprocessing time of $O\left(\right|A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}|+n^{3}d+ns)$.   □

As for safe-no-change projection, the exponential preprocessing time can again be avoided by creating the part of the Sha^↓ $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ needed for the evaluation of the hedge on the fly. The size of the needed part is in $O\left(\right|h\left|\right)$. Hence, the space requirements can also be reduced to $O\left(\right|h\left|\right)$, which may be too much for large streams $\mathit{nw}\left(h\right)$. The needed part of $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$, however, may be much smaller than $\left|h\right|$. Furthermore, a stack of size $depth\left(h\right)$ has to be maintained but this dependence on h may be acceptable even for large streams $\mathit{nw}\left(h\right)$.

10. Monadic Regular Queries and XPath

We next consider the problem of how to answer monadic regular queries on hedges in an earliest manner and with subhedge projection. Each monadic query defines a set of positions for each input hedge. The XPath queries self::list[child::item], for instance, can be applied to the hedge $\langle \mathtt{list}.\langle \mathtt{list}\rangle .\langle \mathtt{item}\rangle \rangle .\langle \mathtt{list}.\langle \mathtt{item}\rangle \rangle$, where it selects the position of the first top-most tree labeled by $\mathtt{list}$ since it contains a child tree labeled by $\mathtt{item}$. We identify positions of hedges by integers. For instance, the selected $\mathtt{list}$ element in the hedge above is identified by the integer 2. Therefore, the answer set of the above query on the above hedge is $\left\{2\right\}$. Note that monadic queries do not return any other information about the selected positions, in contrast to the subtree semantics of XPath in the official W3C standard.

Answering this query is more complicated than verifying whether the XPath filter [self::list/child::item] matches at the root of the first subtree of the hedge, since the answer set needs to be constructed, too. Concerning memory consumption, alive answer candidates need to be buffered for monadic query answering. Therefore, the memory consumption of streaming evaluators of monadic queries may grow linearly in the size of the stream to the number of the alive candidates does.

10.1. Monadic Regular Queries

Monadic queries on hedges in ${\mathcal{H}}_{\Sigma }$ can be identified with languages of annotated hedges in ${\mathcal{H}}_{\Sigma \cup \left\{x\right\}}$, where $x\notin \Sigma$ in an arbitrarily fixed selection variable. A monadic query on hedges in ${\mathcal{H}}_{\Sigma }$ can then be identified with the language of all annotated hedges in ${\mathcal{H}}_{\Sigma \cup \left\{x\right\}}$, in which a single position got x-annotated and this position is selected by query.

Therefore, regular monadic queries on hedges in ${\mathcal{H}}_{\Sigma }$ can be defined by nested regular expressions in ${\mathit{nRegExp}}_{\Sigma \cup \left\{x\right\}}$ or by a dSha with the same signature. The regular XPath query self::list[child::item], for instance, can be defined by following nested regular expression:

$$\langle \mathtt{list}·x·\top ·\langle ·\mathtt{item}·\top ·\rangle ·\top \rangle ·\top ,$$

where ⊤ is like a wildcard accepting any hedge in ${\mathcal{H}}_{\Sigma }$ without x, i.e., for some recursion variable $z\in V$:

$$\top =\mu z.{({+}_{a\in \Sigma }a+\langle z\rangle )}^{\ast }.$$

Note that the nested regular expression for the XPath filter [self::list/child::item] in ${\mathit{nRegExp}}_{\Sigma }$ can be obtained from that of the XPath query self::list[child::item] by removing the x. In addition, for any monadic query on hedges, we have to restrict the annotated hedges to be recognized by the language of the query such that exactly one position in each hedge is annotated by x. This can be done by intersection with the following nested regular expression $on{e}_x$:

$$\mu z.(T.x.T+T.\langle z\rangle .T)$$

The previous schema $〚\mathtt{doc}〛$ for pattern matching has now to be changed from to $\mathbf{S}=〚{\mathtt{doc}}_x〛\cap 〚on{e}_x〛$, where ${\mathtt{doc}}_x\in {\mathit{nRegExp}}_{\Sigma \cup \left\{x\right\}}$ is like $\mathtt{doc}$ but over the signature $\Sigma \cup \left\{x\right\}$. For the regular XPath query self::list[child::item] we obtain the dSha in Figure 22. The transition rules with the special letter x indicate, where positions may be bound to the selection variable: x can be bound only in state 2, that is in subtrees starting with a $\mathtt{list}$ element. The same automaton can be used to define the schema $\mathbf{S}=〚{\mathtt{doc}}_x〛\cap 〚on{e}_x〛$ by using the schema-final states $F_{\mathbf{S}}=\{14,20\}$ instead of the final states $F=\left\{20\right\}$. Indeed, we constructed the automaton by intersecting the automaton obtained from the XPath expression with the automata computed from the regular expressions ${\mathtt{doc}}_x$ and $on{e}_x$ for the schema. Note that 20 is a final state for the query (and the schema), while 14 is a sink state for the query (but not for the schema).

10.2. Earliest Query Answering with Subhedge Projection

A naive query evaluation algorithm for a hedge automaton with signature $\Sigma \cup \left\{x\right\}$ receives an input hedge $h\in {\mathcal{H}}_{\Sigma }$, guesses all possible positions of where to insert x into h, and tests for all of them whether the annotated hedge obtained is accepted by A. In a less naive manner, one inserts x only at those position where the current state has an outgoing x transition. All this can be improved if the hedge automaton under consideration supports selection, rejection, or projection states. In all cases, the algorithm has to buffer all binding candidates, that are not yet in a selection or rejection state. In the worst case it has to buffer the bindings to all positions. Therefore, the state space of the evaluator of monadic queries is no more finite.

Selection and rejection states can be identified by adapting the earliest query answering algorithm from [21]. This can be made such that one obtains an Sha^↓ $A_{e\left(\mathbf{S}\right)}$ with signature $\Sigma \cup \left\{x\right\}$ depending on the schema. It should be noticed that automata used in [21] are dNwas instead of Sha^↓s, and that schemas are ignored there. The needed adaptations are therefore discussed in Appendix A. How to obtain an efficient query answering algorithm in streaming mode from $A_{e\left(\mathbf{S}\right)}$ is the main topic of [21]. This is the base algorithm that we will extend with subhedge projection for our experiments.

In order to add congruence projection to the earliest query answering algorithm, we have to run the automaton $A_{e\left(\mathbf{S}\right)}^{\mathit{cgr}}\left(\mathbf{S}\right)$. The earliest congruence projection for the XPath query self::list[child::item] is shown in Figure 23. It binds x in state 2D1 after having read a top-level $\mathtt{list}$-element and then checks for the existence of a child element $\mathtt{item}$, going into selection state 1D3 once it is found. All other children of the top-level $\mathtt{list}$ element are projected in state 2D3. For adding earliest query answering to self-no-change projection, we have to run the automaton $A_{e\left(\mathbf{S}\right)}^{\pi }$ instead of $A_{e\left(\mathbf{S}\right)}^{\mathit{cgr}}$. We do not give the details, but claim for the present query that the projection power of both automaton is the same.

Fortunately, the soundness and completeness theorems for earliest membership testing with subhedge projection do only affect the constructions of these Sha^↓s, but not how they are to be evaluated. However, there is one additional issue: Nested word prefixes cannot be considered as irrelevant for the subsequent subhedge, if the selection variable x can be bound there, even if the acceptance doesn’t depend on where it will be bound. This is since the position of the binding is to be returned by any query answering algorithm.

Definition 15. 

We call a prefix v binding irrelevant for L and $\mathit{S}$ it there does not exist any hedge h containing x and suffix w such that $v·w\in \mathit{nw}\left(\mathit{S}\right)$ and $v·\mathit{nw}\left(h\right)·w\in \mathit{nw}\left(L\right)$.

Our subhedge projecting evaluator can project only at subhedge irrelevant prefixes that are also binding irrelevant. Given an dSha A and a set of schema final-states $F_{\mathbf{S}}$, one can decide whether a prefix is subhedge irrelevant if the current state q does accept hedges containing x but can access some other state that is not safe for rejection and permits x-bindings. In ground Datalog, we can distinguish binding irrelevant states q by imposing the following rules for all $q,q^{\prime }\in \mathcal{Q}$:

• binding_irrelev(q) :- state(q), not binding_relev(q).
• binding_relev(q) :- not bind_x(q),  acc(q,q’), bind_x(q’), not rej(q’).

Note in our example of the XPath query self::list[child::item] that state 2D3 of the dSha^↓ $A^{\mathit{cgr}}{(〚\mathtt{doc}〛\cap 〚{one}_x〛)}_{e(〚\mathtt{doc}〛\cap 〚{one}_x〛)}$ in Figure 23 is binding irrelevant, since x must be bound on the top-level $\mathtt{list}$ element in state 2D1, so it cannot be bound on any $\mathtt{list}$ element below.

11. Experiments with Streaming XPath Evaluation

We compare experimentally three streaming evaluators for dSha, for earliest monadic query answering, without subhedge projection, with safe-no-change projection, and with congruence projection.

11.1. Benchmark dSha for XPath Queries

We start from dSha defining monadic queries for the regular XPath queries A1-A8 from the XPathMark benchmark [7] that are given in

Table 1. Regular XPath queries A1-A8 from the XPathMark collection.

Query ID	XPath Expression
A1	/site/closed_auctions/closed_auction/annotation/description/ text/keyword
A2	//closed_auction//keyword
A3	/site/closed_auctions/closed_auction//keyword
A4	/site/closed_auctions/closed_auction[annotation/description/ text/keyword]/date
A5	/site/closed_auctions/closed_auction [descendant::keyword]/date
A6	/site/people/person[profile/gender and profile/age]/name
A7	/site/people/person[phone or homepage]/name
A8	/site/people/person[address and (phone or homepage) and (creditcard or profile)]/name

. These XPath queries show most of the features of the regular fragment of XPath. In

Table 2. Additional regular XPath queries for the XPathMark documents.

A0	/site
A1_0a	/site/*
A1_0b	/site/@*
A1_0c	/site//@*
A1_1a	//bidder/personref[starts-with(@person,’person0’)]
A1_1d	//bidder/personref[@person=’person0’]
A1_2	//person
A1_3	/site/regions/africa//@*
A1_4	/site/regions/africa/*
A1_5	/site/regions/*
A1_6	//closed_auction/annotation//keyword
A2_1	//closed_auction[descendant::keyword]
A4_0	/site/closed_auctions/closed_auction[annotation]/date
A4_1	/site[open_auctions]/closed_auctions

, we added 14 further XPath path queries that we found useful for testing too.

Deterministic Shas for A1–A8 were provided earlier in [21]. For the other XPath queries we compiled them to dShas via nested regular expression.

In order to produce the input dShas for our evaluators, we intersect these automata with a dSha for the schema $〚{\mathtt{doc}}_x^{\prime }〛\cap 〚on{e}_x〛$ where ${\mathtt{doc}}_x^{\prime }$ is the schema we use for hedge encodings of real-world Xml documents with x annotations. Thereby, we could identify the schema final states $F_{\mathbf{S}}$. We minimized and completed the result.

Table 3. Overall size and number n of states of the input dSha A for the query with schema-final states $F_{\mathbf{S}}$, the overall size and number of states of the Sha^↓$A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ obtained by earliest congruence projection, and the number d its difference relations.

Query	dSha A	dSha↓	#Diff-	Query	dSha A	dSha↓	#Diff-
ID	Size(n)	Size(#State)	Rel. d	ID	Size(n)	Size(#State)	Rel. d
A1	482(68)	1296(324)	16	A1_0c	230(43)	238(62)	6
A2	224(42)	316(82)	7	A1_1a	305(54)	384(101)	8
A3	320(53)	662(156)	10	A1_1d	305(54)	382(101)	8
A4	629(74)	1651(404)	18	A1_2	194(39)	152(42)	5
A5	438(63)	1226(269)	13	A1_3	318(53)	672(159)	10
A6	675(76)	2090(500)	22	A1_4	312(52)	479(132)	10
A7	394(59)	728(184)	12	A1_5	266(47)	293(84)	8
A8	648(79)	2091(504)	24	A1_6	265(47)	588(142)	9
A0	203(40)	158(44)	5	A2_1	232(42)	295(78)	7
A1_0a	224(42)	145(44)	6	A4_0	392(59)	719(184)	12
A1_0b	203(39)	68(23)	5	A4_1	292(49)	287(78)	8

reports the size of the input dShas for our evaluators obtained by the above procedure. For each dSha A, the size is given by two numbers size(n), the first for the overall size and the second for the number of states n.

For the input dSha A of each query, we statically computed the whole dSha^↓s $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ while using the necessary parts of the determinization algorithm from [16]. The size of the dSha^↓s obtained and the number d of difference relations are reported in Table 3 too. The biggest size is 2091(504) for the input dSha for A8. The largest number of difference relations d = 24 is also obtained for this query. So, indeed the size of these automata is much smaller than one might expect from a construction that is highly exponential in the worst case. However, the only query that we couldn’t obtain its correspondent deterministic earliest congruence projection $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ is A5.

The time for computing the earliest congruence projection took between 2.3 and 26 s.

We note that we have not yet computed the complete dSha^↓s statically for safe-no-change projection $A^{\mathit{snc}}$, pure congruence projection $A^{\mathit{cgr}}\left(\mathbf{S}\right)$ and earliest query answering $A_{e\left(\mathbf{S}\right)}$. We believe that these automata may become bigger than $A^{\mathit{cgr}}{\left(\mathbf{S}\right)}_{e\left(\mathbf{S}\right)}$ that we constructed in a single shot.

11.2. Streaming Evaluation Tool: AStream

We are using and developing the AStream tool for answering monadic dSha queries on Xml streams with schema restrictions. Version 1.01 of AStream was presented in [21] supports earliest query answering without projection. Given a dSha A defining a monadic query, a set of schema final states $F_{\mathbf{S}}$ and an Xml stream w, it constructs on-the-fly the needed part of the Sha^↓ $A_{e\left(\mathbf{S}\right)}$ while parsing w and evaluating A on the hedge encoding w.

For the FCT conference version of the present paper [24], we enhanced AStream with safe-no-change projection. This leads to version AStream 2.01. It constructs the earliest safe-no-change projection dSha^↓ $A_{e\left(\mathbf{S}\right)}^{\mathit{snc}}$ on the fly while evaluating the monadic query defined by A on the hedge encoding the Xml stream w. Subhedge projection can be switched on or off in order to compare both versions without further implementation differences.

For this present journal version, we added earliest congruence projection and integrated it into AStream leading to AStream 3.0.

AStream 3.0 is different from the two previous versions in that the dSha^↓ $A_{e\left(\mathbf{S}\right)}^{\mathit{cgr}}$ is constructed statically and entirely, independently of the input hedge. We then use a generic earliest streaming evaluator for Sha^↓s that rejects in rejection states, selects in selection states, and projects in all subhedge projection states. This evaluator could also be run with $A_{e\left(\mathbf{S}\right)}^{\mathit{snc}}$ or $A_{e\left(\mathbf{S}\right)}$, as long as these do not grow too big.

It should be noticed that $A_{e\left(\mathbf{S}\right)}^{\mathit{cgr}}$ turned out to be nicely small for our whole benchmark, with the other dSha^↓s $A_{e\left(\mathbf{S}\right)}^{\mathit{snc}}$ and $A_{e\left(\mathbf{S}\right)}$ risk becoming bigger. As stated earlier, we did not construct these dSha^↓s so far for our benchmark queries. This is why we continued to run the earliest streaming with safe-no-change projection with AStream 2.01, while we used AStream 3.0 for earliest streaming with congruence projection.

All AStream versions rely on Java’s abc-Datalog for computing the least fixed points of Datalog programs in a bottom-up manner. Datalog programs are needed for all logical reasoning: for computing subsets of states that are safe for rejection, selection, or subhedge projection on all levels. Also, the difference relations are computed based on Datalog. Note that earliest on-the-fly query evaluation requires running Datalog during query evaluation, while with a static approach for constructing dSha^↓s entirely, Datalog is only needed at preprocessing time during the automaton construction.

11.3. Evaluation Measures

We want to measure the time for running the three streaming query evaluators for all dShas obtained from the XPath expressions in the collection in Table 1 and Table 2.

One advantage of the XPathMark benchmark is that it comes with a generator of Xml documents to which the queries can be applied and that can be scaled in size. We created an Xml document of size 1.1 GB, which is sufficiently large to make streaming mandatory. Our experiments show that the efficiency of query evaluation grows linearly with the size of the non-projected part of the Xml document. This holds for each of our three evaluators. Therefore, measuring the time of the evaluator on Xml documents of other sizes would not show any new insights, except that memory consumption remains moderate too.

The time gain of projection for a query Q is the percentage of time saved by projection during query evaluation when ignoring the pure document parsing time ${\mathtt{t}}_{\mathtt{parse}}$ seconds:

$${\mathit{time-gain}}_Q=100\ast (1-({\mathtt{t}}_{{\mathtt{noproject}}_{\mathtt{Q}}}-{\mathtt{t}}_{\mathtt{parse}})/({\mathtt{t}}_{{\mathtt{project}}_{\mathtt{Q}}}-{\mathtt{t}}_{\mathtt{parse}})).$$

We can measure the time gain for earliest safe-no-change projection by AStream 2.01 and for earliest congruence projection by AStream 3.0, since projection can be switched off in both of them. The disadvantage of the time gain is that it depends on the details of the implementation.

The event gain is a better measure for the projection power. The parser sends a stream of needed events to the Sha^↓, while ignoring parts of the input document that can be projected away. For this, it must always be informed by the automaton about what kind of events can be projected in the current state. The event gain of projection ${\mathit{event-gain}}_Q$ is then the percentage of events that are recognized to be irrelevant and thus no more created by a projecting evaluator for query Q. One might expect that the time gain is always smaller than the event gain.

$${\mathit{time-gain}}_Q\le {\mathit{event-gain}}_Q$$

Indeed, this will be confirmed by our experiments. We believe that these discrepancies between these two measure indicate how much room remains for optimizing the implementation of an evaluator. In this way, we can be observe that some room for further optimizations AStream 3.0 still remains.

11.4. Experiments

We used Java’s $XMLStreamReader$ Interface v1.0 from the $javax.xml.stream$ package to parse and stream Xml files in Scala. Parsing a 1.1 GB Xml document required ${\mathtt{t}}_{\mathtt{parse}}=\mathtt{15}$ s, while querying it without projection took us in average ${\mathtt{t}}_{{\mathtt{noproject}}_{\mathtt{avg}}}=\mathtt{752}$ s, while varying from 600 to 900 s. The average processing time for 1 MB was thus 0.72 s. Once we knew this, we predicted the expected evaluation time from the size of the non-projected part of the Xml document. It is as follows:

$${\mathit{event-gain}}_Q\ast 1100\ast 0.72\mathrm{s}+{\mathtt{t}}_{\mathtt{parse}}\mathrm{s}.$$

Without projection, the parser generates $1,012,018,728$ events for the 1.1 GB Xml document. This is the baseline for computing ${\mathit{event-gain}}_Q$ for all queries Q.

We ran AStream with Scala v2.13.3 on a machine with the operating system Ubuntu 20.04.06 LTS (64-bit). The machine is a Dell Precision-7750 machine (Round Rock, TX, USA), equipped with an Intel^® Core™ i7-10875H CPU @ 2.30GHz × 16 and 32 GB of RAM (Santa Clara, CA, USA).

11.4.1. Earliest Congruence Projection

We present the measures of our evaluator with the earliest congruence projection in

Table 4. Time gain and event gain by earliest congruence projection with AStream3.01 on a 1.1 GB XPathMark stream.

Query	#Ans-	Time	Time	Event	Query	#Ans-	Time	Time	Event
ID	Wer	in s	Gain	Gain	ID	Wer	in s	Gain	Gain
A1	38,267	29.3	98.1%	98.9%	A1_0c	3,600,816	226.6	72.9%	75.7%
A2	117,389	171.8	77.9%	81.1%	A1_1a	2	187.8	78.1%	80.3%
A3	117,389	33.4	97.4%	97.8%	A1_1d	2	217.3	74.8%	80.3%
A4	24,959	33.2	97.8%	98.9%	A1_2	1,062,965	238.2	68.2%	76.0%
A5	50,186				A1_3	25,074	16.2	99.1%	99.8%
A6	30,307	45.9	96.9%	98.2%	A1_4	5170	17.5	99.7%	100.0%
A7	179,889	30.5	98.0%	98.7%	A1_5	6	14.3	100.0%	100.0%
A8	67,688	37.8	97.3%	98.7%	A1_6	117,389	188.7	75.4%	81.1%
A0	1	21.3	99.1%	100.0%	A2_1	50,186	199.7	76.5%	81.1%
A1_0a	6	15.2	99.9%	100.0%	A4_0	91,650	21.8	99.0%	99.3%
A1_0b	0	0.0	100.0%	100.0%	A4_1	1	12.0	100.0%	100.0%

. The event gain is high, varying between 75.7% for A1_0c and 100% for A0, A1_0a, A1_0b, A1_4, A1_5, and A4_1. The event gain for queries without descendant axis is above 98.2%. For these queries, all subtrees that are not on the main path of the query can be projected away. In particular, only subtrees until a fixed depth have to be inspected. These are the queries A1, A4, A6-A8 and A4_0 and the queries with event gain 100% listed above. The time gain for all these queries is a little smaller than the event gain but still above 98.2%.

We next consider the second type of queries having the descendant axis, specifically A2, A1_0c, A1_1a, A1_1d, A1_2, A1_6, and A2_1. Intuitively, a lower event gain is expected in these cases because subhedge projection with the descendant axis cannot directly exclude an entire subtree under a given element; all descendant elements must be inspected. For instance, the query A1_0c equal to $/\mathtt{site}//@\ast$ must select all attributes of all elements under the root element $site$, requiring the inspection of every Xml element in the document, for attribute presence, except for the root element $site$.

The lower event gain reported for the aforementioned queries align with the above expectation, showing percentages ranging from 75.7% to 81.1%. Also, these queries exhibit a larger gap between time gain and event gain, which ranges from 2.2% to 5.7% and will be subject to future optimization. It is worth noting that some of these queries combine multiple features at once, like queries A1_1a and A1_1d, which have a descendant axis, filter with attributes, and string comparison.

The only exception to this trend for the queries having descendant axis are A3 and A1_3, with respective event gains of 97.8% and 99.8%. The reason behind this high gain is that these queries starts from the root, with a child axis path of depth 3 before querying the descendant axis. This initial path allows for the exclusion and projection of most elements under $site$ that do not fit the specified path with their entire subtrees.

11.4.2. Earliest Safe-No-Change Projection

We compare earliest safe-no-change projection with the earliest congruence projection in

Table 5. Time gain for earliest safe-no-change and eaerliest congruence projection, and the event gain of earliest congrunce projection.

Query	Time (s)	Time (s)	Time Gain	Time Gain	Event Gain
ID	Snc	Congr	Snc	Congr	Congr
A1	72.8	29.3	92.3%	98.1%	98.9%
A2	664.6	171.8	8.5%	77.9%	81.1%
A3	666.1	33.4	10.9%	97.4%	97.8%
A4	78.5	33.2	92.4%	97.8%	98.9%
A5	77.7		92.3%
A6	65.2	45.9	95.1%	96.9%	98.2%
A7	24.6	30.5	98.8%	98.0%	98.7%
A8	24.8	37.5	98.9%	97.3%	98.7%

. For this, we restrict ourselves to the queries $A1$–$A8$ from the XPathMark in Table 1.

The XPath queries A1, A4, A6, A7, and A8 do not have descendant axes. The time gain for safe-no-change projection on these queries is between $92.3$–$98.9\%$. For earliest congruence projection, the time gain was better for A1, A4, and A6 with at least 96.6%. For A7 and A7, the time gain of safe-no-change projection is close to the event gain of congruence projection but the time gain of congruence projection is 1.5% lower. So, we hope that an optimized version of congruence projection could outperform safe-no-change projection in all cases.

The XPath queries with descendant axis are A2, A3, and A5. For A2 and A3 in particular, the time gain of safe-no-change projection is very low with even not 10.9% while congruence projection yields at least 77.9%. We believe that this is bug in our current implementation of safe-no-change projection, which may be prohibiting the projection of attributes and text nodes for these queries (but not for the others). Congruence projection on A2 gains 77.9% of time and 81.1% of events. This is much better, but still far from the projection power for queries without descendant axi s. For A3, congruence projection gains 97.4% of time. This is much better than for A2 since the descendant axis in A3 is at the end, while for A2 it is at the beginning of the path. But, even for A2, congruence projection is very helpful.

11.4.3. Comparison to External Tools

QuiXPath was shown to be the most efficient large coverage external tool for regular XPath evaluation on Xml streams [4]. Therefore, we compare the efficiency of congruence projection to QuiXPath in

Table 6. Comparison of of parse-free timings in seconds for XPathMark queries with QuiXPath and earliest congruence projection.

Query	Parse-Free	Parse-Free	Fract.	Query	Parse-Free	Parse-Free	Fract.
ID	Time (s)	Time (s)		ID	Time (s)	Time (s)
	QuiXPath	Congr. Prj.			QuiXPath	Congr. Prj.
A1	11.0	14.3	1.3	A2	11.4	156.8	13.8
A4	11.6	18.2	1.6	A3	11.5	18.4	1.6
A6	10.7	30.9	2.9	A5	12.0
A7	8.6	15.5	1.8
A8	8.8	22.5	2.6

, and thereby, by transitivity to the many alternative tools. We note that QuiXPath performs early query answering by compilation to possibly nondeterministic Nwas with selection states, without always being earliest. Apart from this, it bases its efficiency on both subtree projection and descendant projection.

The experiments with QuiXPath in [4] use the parse-free time. Therefore, we do the same for congruence projection by reducing the measured overall time by 15 s of parsing time.

Compared to the parsing-free times for the XPath queries without descendant axis, the earliest congruence projection demonstrates significant improvements. On average, our projection is only slower by a factor of 1.3–2.9 than QuiXPath. This shows that our implementation is already close to be competitive with the existing Xml streaming tools, while being the only one guaranteeing earliest query answering.

For query A2 with a descendant axis, congruence projection is by a factor of 13.8 slower than QuiXPath. The reason is that QuiXPath supports descendant projection, while congruence projection concerns only subhedge. For query A3 with a descendant axis at the end, congruence projection is only by a factor of 1.6 behind, so descendant projection seems less relevant here.

12. Related Automata Models

A large number of alternative automata notions for trees, forests, and hedges were proposed in the literature. We compare them to Shas and Sha^↓s in the following subsection and discuss the implications.

12.1. Shas versus Tree, Forest, and Hedge Automata

• Standard Hedge Automata. Standard hedge automata [9,10,32,33] operate on labeled hedges in a bottom-up manner, similarly to Shas. Also they have the same expressiveness, but horizontal languages are specified differently, leading to a problematic notion of determinism [34]. For this reason, unique minimization fails for deterministic standard hedge automata. Still, they have the same expressiveness as Shas when restricted to labeled hedges.

• Standard Tree Automata and SHA Minimization. Syntactically, any Sha A is a standard tree automaton over the following ranked signature:
  • a unary symbol for any letter $a\in \Sigma$;
  • a binary symbol @;
  • a constant $\langle \rangle$.
  Semantically, there is no perfect general correspondence. Only for subclasses a perfect correspondence can be made up via binary encoding. This was shown in [8] for the subclass of dShas with $I={\langle \rangle }^{\Delta }$. In [35], it could also be shown for the subclass of multi-module dShas. This leads to unique minimization results for both subclasses of deterministic Shas.

• Bojanczyk’s Forest Automata. Standard hedge automata also have the same expressiveness as Bojanczyk’s forest automata (see Section 3.3 of [36]). These are based on the idea of transition monoids, rather than on the idea of transition rules such as Shas or finite state automata or Neuman’s and Seidl’s forest automata. As a consequence, there is an exponential difference in succinctness between Shas and Bojanczyk’s forest automata [35].

12.2. SHA^↓s versus SHAs

We start explaining why Sha^↓’s have the same expressiveness as Shas. Basically, it is the same reason for why Nwas have the same expressiveness as Shas (see e.g., [8]). For the easier direction, we argued already that any Sha $A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$ can converted to a Sha^↓ $down\left(A\right)$.

• Downward Elimination. Conversely, we can convert any dSha^↓A into an equivalent Sha ${\mathit{elim}}^{\downarrow }\left(A\right)$ while possibly introducing nondeterminism as follows:

  $$\begin{array}{c}{I}^{{\mathit{elim}}^{\downarrow }}=I\times \mathcal{Q}\hfill \\ F^{{\mathit{elim}}^{\downarrow }}=F\times \mathcal{Q}\hfill \end{array}\hspace{1em}\frac{q\stackrel{\langle \rangle }{\to }{q}^{\prime }\mathrm{in}\Delta }{\stackrel{\langle \rangle }{\to }(q,q^{\prime }) \mathrm{in}{\Delta }^{{\mathit{elim}}^{\downarrow }}}$$

  $$\frac{q\stackrel{a}{\to }{q}^{\prime }\mathrm{in}\Delta r\in \mathcal{Q}}{(r,q)\stackrel{a}{\to }(r,q^{\prime }) \mathrm{in}{\Delta }^{{\mathit{elim}}^{\downarrow }}}\hspace{1em}\frac{q@p\to q^{\prime }\mathrm{in}\Delta r\in \mathcal{Q}}{(r,q)@(q,p)\to (r,q^{\prime }) \mathrm{in}{\Delta }^{{\mathit{elim}}^{\downarrow }}}$$

Proposition 10. 

$\mathcal{L}\left(A\right)=\mathcal{L}({\mathit{elim}}^{\downarrow }\left(A\right))$.

The construction is analogous to the conversion of Nwas to Shas [8] or to hedge automata [26]. The correctness proofs for these compilers are standard.

• Unique Minimization. Unique minimization fails for dSha^↓s as usual for deterministic multiway automata. This even happens for deterministic two-way finite state automata on words. In contrast, the subclass of dShas with the restriction that the initial state and the tree initial state are the same enjoys unique minimization [8].

• Neuman and Seidl’s Pushdown Forest Automata. Standard hedge automata are also known to have the same expressiveness than Neumann and Seidl’s pushdown forest automata [25]. The latter can be identified with Sha^↓s for labeled hedges. What is needed to map pushdown forest automata to standard hedge automata is downward elimination, as for mapping Sha^↓s to Shas.

12.3. SHA^↓s versus NWAs

The streaming evaluator for Sha^↓s via a visibly pushdown machine can also be obtained by compiling a Sha^↓ to a nested word automaton (Nwa) [11] whose streaming evaluator is given by a visibly pushdown machine [12], previously known as input driven automata [13,14,15].

Definition 16. 

A nested word automata (Nwa) is a tuple $(\Sigma ,\mathcal{Q},\mathrm{\Gamma },\Delta ,I,F)$, where Σ, Γ, and $\mathcal{Q}$ are sets, $I,F\subseteq \mathcal{Q}$, and $\Delta =({\left(a^{\Delta }\right)}_{a\in \Sigma },{{\langle }^{\Delta },\rangle }^{\Delta })$ contains the following relations: $a^{\Delta }\subseteq \mathcal{Q}\times \mathcal{Q}$, ${\langle }^{\Delta }\subseteq \mathcal{Q}\times (\mathrm{\Gamma }\times \mathcal{Q})$, and ${\rangle }^{\Delta }\subseteq \mathcal{Q}\times \mathrm{\Gamma }\times \mathcal{Q}$. A  Nwa  is deterministic or equivalently a dNwa  if I contains at most one element and all above relations are partial functions.

The elements of $\mathrm{\Gamma }$ are called stack symbols. The transition rules in $\Delta$ again have three forms: internal rules $q\stackrel{a}{\to }{q}^{\prime }$ in $\Delta$ as for Shas, opening rules $q\stackrel{\langle \downarrow \gamma }{\to }{q}^{\prime }$ in $\Delta$ if ${\langle }^{\Delta }\left(q\right)=(q^{\prime },\gamma )$, and closing rules $q\stackrel{\rangle \uparrow \gamma }{\to }{q}^{\prime }$ in $\Delta$ if ${\rangle }^{\Delta }(q,\gamma )=q^{\prime }$.

• Streaming evaluators for Nwas. The streaming evaluator for Nwas can be seen as pushdown machines for evaluating the nested words of hedges in streaming manner. A configuration of the pushdown machine is a pair in $\mathcal{K}=\mathcal{Q}\times {\mathrm{\Gamma }}^{\ast }$ containing a state and a stack. For any word $v\in {\widehat{\Sigma }}^{\ast }$, we define a streaming transition relation ${\stackrel{v}{\to }}^{\mathit{str}}\subseteq \mathcal{K}\times \mathcal{K}$, such that, for all $q,q^{\prime }\in \mathcal{Q}$ and $\sigma \in {\mathrm{\Gamma }}^{\ast }$, the following is applicable:

  $$\frac{true}{(q,\sigma ){\stackrel{\epsilon }{\to }}^{\mathit{str}}(q,\sigma ) \mathrm{wrt}\Delta }\hspace{1em}\frac{(q,\sigma ){\stackrel{v}{\to }}^{\mathit{str}}(q^{\prime },\sigma )\hspace{1em}(q^{\prime },\sigma ){v^{\prime }}^{\mathit{str}}(q^{″},\sigma ) \mathrm{wrt}\Delta }{(q,\sigma ){\stackrel{v·v^{\prime }}{\to }}^{\mathit{str}}(q^{″},\sigma ) \mathrm{wrt}\Delta }$$

  $$\frac{q\stackrel{a}{\to }{q}^{\prime } \mathrm{in}\Delta }{q{\stackrel{a}{\to }}^{\mathit{str}}{q}^{\prime } \mathrm{wrt}\Delta }\hspace{1em}\frac{q\stackrel{\langle \downarrow \gamma }{\to }{q}^{\prime } \mathrm{in}\Delta }{(q,\sigma ){\stackrel{\langle }{\to }}^{\mathit{str}}(q^{\prime },\sigma ) \mathrm{wrt}\Delta }\hspace{1em}\frac{q\stackrel{\rangle \uparrow \gamma }{\to }{q}^{\prime } \mathrm{in}\Delta }{(q,\sigma ·\gamma ){\stackrel{\rangle }{\to }}^{\mathit{str}}(q^{\prime },\sigma ) \mathrm{wrt}\Delta }$$

• From SHA^↓s to NWAs. For any Sha^↓$A=(\Sigma ,\mathcal{Q},\Delta ,I,F)$, we can define the Nwa $A^{\mathit{nwa}}=(\Sigma ,\mathcal{Q},$$\mathrm{\Gamma },{\Delta }^{\mathit{nwa}},I^{\mathit{nwa}},F^{\mathit{nwa}})$ while preserving determinism, such that $\Gamma =\mathcal{Q}$, while ${\Delta }^{\mathit{nwa}}$ contains for all $a\in \Sigma$ and $q,p\in \mathcal{Q}$ transition rules:

  $$\frac{q\stackrel{a}{\to }{q}^{\prime }\mathrm{in}\Delta }{q\stackrel{a}{\to }{q}^{\prime }\mathrm{in}{\Delta }^{\mathit{nwa}}}\hspace{1em}\frac{q\stackrel{\langle \rangle }{\to }{q}^{\prime }\mathrm{in}\Delta }{q\stackrel{\langle \downarrow q}{\to }{q}^{\prime }\mathrm{in}{\Delta }^{\mathit{nwa}}}\hspace{1em}\frac{q@p\to q^{\prime }\mathrm{in}\Delta }{p\stackrel{\rangle \uparrow q}{\to }{q}^{\prime }\mathrm{in}{\Delta }^{\mathit{nwa}}}$$

For instance, the dNwa $A^{\mathit{nwa}}$ of the dSha^↓ A in Figure 9 obtained from the dSha for the $[\mathtt{self}::\mathtt{list}/\mathtt{child}::\mathtt{item}]$ from the introduction is given in Figure 24. The dNwa $A^{\mathit{nwa}}$ of the dSha^↓ A in Figure 21 is given in Figure 25.

Lemma 21. 

$\mathcal{L}\left(A^{\mathit{nwa}}\right)=\mathcal{L}\left(A\right)$.

The runs of Sha^↓s A and Nwas $A^{\mathit{nwa}}$ can be identified.

• Projection for NWAs. Projecting evaluators for Nwas were proposed in the context of projecting Nwas [4]. They are based on the following notion of states of irrelevant subtrees for Nwas.

Definition 17. 

(Variant of Definition 3 of [4]). We call a state q of an Nwa a state of irrelevant subtrees if there exist two different stack symbols $\gamma ,{\gamma }^{\prime }$ and a state $q^{\prime }$ such that the transitions shown on the right exist but no further opening transitions with γ, no further transitions with ${\gamma }^{\prime }$, and no further closing transition in q popping γ. In this case, we write $q\in \mathit{i-tree}{e}_{\Sigma \backslash \varnothing }$.

Lemma 22. 

Any subhedge projection state of a complete Sha^↓ A is a state of irrelevant subtrees of $A^{\mathit{nwa}}$.

It was then shown in [4] how to map an Nwa with a subset ${\mathcal{Q}}_{\mathit{shp}}^{\Delta }$ of states of irrelevant subtrees to a projecting Nwa, whose streaming semantics yield a streaming evaluator with subhedge projection. The presentation of subhedge projection for Sha^↓s without passing via Nwas yields the same result in a more direct manner.

It should also be noticed that projecting Nwas supports descendant projection besides subhedge projection. For Shas, this is left to future research.

13. Conclusions and Future Work

We introduced the notion of irrelevant subhedges for membership to hedge language under schema restrictions. We developed algorithms with subhedge projection that jump over irrelevant subhedges for testing membership to the regular hedge languages defined by dShas. All our algorithms can be run in memory mode and in streaming mode. The difficulty was how to push the needed finite state information for subtree projection top-down given that Shas operate bottom-up. We solved it based on compilers from Shas to downward Shas.

This first compiler propagates safety information regarding non-changing states. The second compiler, which we prove to be complete for subhedge projection, propagates difference relations. We then combined subhedge projection with earliest membership testing and combined it earliest monadic query answering. We integrated both safe-no-change and complete congruence in our AStream tool and confirmed their usefulness experimentally: we showed that it can indeed speed up the the AStream tool for answering regular XPath queries on Xml streams. Moreover, congruence projection showed much higher event projection gain rates than safe-no-change, ranging between 75.7% and 100% for the class of queries that does not have descendant axis, becoming competitive in time with the best existing streaming tools for regular XPath queries without a recursive axis.

A remaining open theoretical and practical question is how to obtain descendant projection for Shas, as needed to speed up the evaluation of XPath queries with a descendant axis. On the implementation and experimental side, it would be nice to investigate the in-memory version of our subhedge algorithms too. Finally, it would be nice to optimize streaming earliest-query answering tools further until they outperform the best streaming tools in all cases, while being better for queries where the other tools are not earliest.