*3.2. Experimental Design*

In this HIT, a subject was asked to start from the beginning fragment of a sentence, and then guess character after character of the remainder of the sentence. Figure 1 shows the interface used in the experiment. As shown, a subject received three types of information:


In this framework, once a subject decides on their guess, they input it and press enter to submit it. If the guess is correct, the context is updated to length *n*, and the task continues with the prediction of the *n* + 1-th character. If the answer is incorrect, the subject must guess what the *n*-th character is until obtaining the correct answer. Subjects were informed in advance of the number of characters in the remaining phrase to avoid anyone abandoning the task.

**Figure 1.** Our user interface for our cognitive experiment on Amazon Mechanical Turk. It provides: (**i**) the number of characters still available for use, (**ii**) the preceding *n* − 1 characters, and (**iii**) the set of incorrect characters already used.

If a phrase is too long, subjects become easily distracted. Therefore, it was necessary to adjust the length of time provided for an experimental session. Too short a time raises the cognitive load, whereas too long a time decreases a subject's interest. After multiple trials across multiple options, such as putting a constant cap on the time allowed for each guess, we chose to allow a maximum number of guesses for every phrase. After some preliminary tests, this number was fixed to the character length of the phrase. Therefore, a subject was able to complete the task only if they always guessed all of the characters correctly. Most of the time, then, a subject was unable to finish a phrase.

The phrases were taken from the *Wall Street Journal*. In particular, 225 sentences were randomly extracted for this experiment and used as the experimental phrases. Their average length was 150.97. All characters were capitalized, and non-alphabetical symbols other than spaces were removed, duplicating the settings in previous works [1,9–11]. Hence, the characters were limited to the 26 letters of the alphabet, all in capital letters, and the space symbol. Table 2 lists the top ten most frequently used words and two successive words used in the experiment. As shown, they are relatively simple words that do not require specialized knowledge to predict correctly.


**Table 2.** The top ten most frequently used words along with two subsequent words appearing in the phrases used in our experiment.

We considered multiple variations of Shannon's experiment. The experiment could have consisted of guessing a character of a different phrase every time; thus, increasing the cognitive load for the subject by having them read through a different phrase every time. Another possibility was to proceed even if the character guess was incorrect. Since multiple subjects participated, it would then still be possible to acquire the probability of a correct guess. Such a method would decrease the task load substantially. However, this idea was not adopted since some subjects could choose random characters for all predictions. Finally, we reached the conclusion that Shannon's framework was well designed and utilized it in this work.
