Predicting Question Popularity for Community Question Answering

Abstract

In this paper, we study the problem of predicting popularities of questions in Community Question Answering (CQA). To address this problem, we propose a Posterior Attention Recurrent Point Process Model (PARPP) to take both the interaction of users and the Matthew effect into account for question popularity prediction. Our PARPP uses long short-term memory (LSTM) to encode the observed history and another LSTM network to record each step of decoding information. At each decoding step, it uses prior attention to capture answers that have a greater impact on the problem. When a new answer is observed, it uses Bayes’ rule to modify prior attention and obtain posterior attention. Then, the posterior attention is used to update the decoding status. We further introduce a convergence strategy to capture the Matthew effect in CQA. We conduct experiments on a Zhihu dataset crawled from a famous Chinese CQA forum. The experimental results show that our model outperforms several state-of-the-art methods. We further analyze the attention mechanism in our model. Our analysis shows that the proposed attention mechanism can better capture the impact of each answer on the future popularity of the question, which makes our model more interpretable. Our study would shed light on other similar studies such as answer ranking in response to the question and finding experts who have expertise on the topics of the questions.

1. Introduction

In recent years, Community Question Answering (CQA) forums, such as Zhihu, http://zhihu.com/, Quora, https://www.quora.com/, and yahoo! answer, https://answers.yahoo.com/, have been booming and become important online platforms for users to exchange ideas and find information. CQA provides a convenient platform where people can ask their own questions and share their answers. Moreover, like most social networks, users in CQA can interact with each other; e.g., users can vote and write reviews for their favorite answers. In CQA, given a question, its accumulated interactions will have an impact on the ordering of its answers. In recent years, research on CQA has been focusing on question retrieval and answer ranking to help users find relevant questions and high-quality answers better and faster. Predicting question popularity is of great importance to allow communities to better organize their questions both to promote community development and to improve user experience.

Previous work [1] on CQA addresses a wide spectrum of applications such as application question retrieval [2], answer selection [3], heterogeneous question answering community detection [4], and the automatic moderation of user-generated content [5]. Not many studies have focused on predicting question popularity. Yet, predicting question popularity, such as when the next answer appears or how many answers will appear in the future, is crucial to the development of the community. Accurate prediction would allow the community to rank content better, discover the trends of questions better, further improve its content-delivery networks and more effectively push ads. Existing predicting question popularity methods simply perform a qualitative analysis of a problem’s popularity [6,7]; i.e., they only determine whether a given problem will be popular but lack a more fine-grained quantitative analysis.

Given a question and the trace of its past answers, when will the next answer appear? How many answers will be provided by users to the question? A number of temporal stochastic process models [8] have been proposed and proven to be able to effectively capture the arrival of information in online media. Although these models have made a great progress in predicting the popularity of online cascade, they suffer from the following three drawbacks:

• All these temporal point process models do not take into account the impact of user interaction on an event cascade in the online community. An important feature of the online community is that people can interact with others anytime, anywhere. For instance, in CQA, users can vote for their favorite answers or leave their own opinions in the comments section. These interactions will affect the ranking of the answers. Figure 1 shows answers of a question related to interpersonal socialization in Zhihu, which is a famous Chinese Community Question Answering platform. As we can see, top-ranked answers, though they were posed a long time ago (four years before), received more than 10k thumbs-up votes, so they were ranked ahead. In contrast, new answers received a small number of votes, and thus they are ranked at the bottom and unobserved at the top. It is generally believed that the impact of recent events in the future is greater than that of past events [8,9,10], but due to the special interaction mechanism in CQA, a long-term but high-quality answer will gain users’ attention because of the higher ranking, which will have a lasting and important impact on the future of a question.
• There is an obvious Matthew effect [11] in CQA, which has not been considered in previous models. The Matthew effect is a concept originating in the biblical Gospel of Matthew. It is sometimes summarized by the adage “the rich get richer, while the poor get poorer”. Robert Merton, a sociologist, first observed the Matthew effect in the academic field in that eminent scientists will often receive more credit than a comparatively unknown researcher even if their works are similar. The Matthew effect has been found in many aspects of our social life [12,13,14,15,16,17,18].
  In CQA, this effect is very common and is amplified by answer visibilities. The rapid development of online social media has brought about the outbreak of information. However, people are usually only attracted by a few top-rank high-quality answers, while the rest go unnoticed [19]. Moreover, these excellent answers will accumulate advantages, while a higher ranking makes them easier to attract users’ attention and obtain a better ranking. Due to the existence of the Matthew effect, an early dominant answer will continue to widen the gap with other answers. As shown in Figure 1, the number of votes for the first answer is nearly five times more than that of the second answer. The Matthew effect affects the ranking of answers, and ranking affects the users’ first impression of a question, which in turn affects the popularity of the question. However, existing approaches do not take into account this common phenomenon in CQA.
• For the convenience of analyzing information diffusion, with very few exceptions [8,20,21], most existing point process models make specific assumptions about the functional forms of the underlying generative processes. For example, the Hawkes process [22,23,24], a famous non-homogenous Poisson process, supposes that the arrival of an event causes the probability of future events to increase, assuming that historical events influence future events through a positive, additive and time decay kernel function. However, the true mechanism behind event cascade is hard to specify or verify in practice, and thus, the fixed simple parametric representations may constrain the expressive power of these models.

Accordingly, we propose a Posterior Attention Recurrent Point Process model, abbreviated as PARPP, that takes into account both user interactions and the Matthew effect for predicting question popularity in CQA. Specifically, inspired by the common encoding–decoding framework in language translation [25,26], our PARPP uses long short-term memory (LSTM) to encode the observed history and another LSTM network to record each step of decoding information. At each decoding step, we use prior attention to capture answers that have a greater impact on the problem at this time. When a new answer is observed, we use Bayes’ rule to modify prior attention and obtain posterior attention at this time. Then, posterior attention is used to update the decoding status. Furthermore, we introduce a convergence strategy in the proposed attention mechanism to capture the Matthew effect. That is, we focus our attention on the direction of high values to simulate the phenomenon that high-ranking answers will continue to accumulate advantages. We have four motivations for applying attention mechanisms in predicting the popularity of community Q&A questions: (1) not all answers in response to the questions contribute equally to the prediction of the popularity; (2) the popularity of a question can depend on the different Matthew effect associated with it; (3) the factors influencing Q&A question popularity can change over time in the answers; and (4) without an attention mechanism to focus on relevant answers, the popularity prediction model may dilute its predictive power by considering too many irrelevant answers. To evaluate the performance of our proposed PARPP model, we conduct experiments on a Zhihu dataset and aim at answering these research questions: Given a question and the trace of its observed answers, when will the next answer appear? How many answers will be posted for a particular period? Can the proposed attention mechanism capture the most influential answers?

Our contributions can be summarized as the following:

• We propose a Posterior Attention Recurrent Point Process Model, PARPP, to predict the popularity of questions in CQA. Our PARPP is a new point process model that takes into account the impact of user interaction in the modeling of the point process.
• In CQA, users’ interactions will affect the ranking of the answers and thus the popularity of the question. Our proposed PARPP fully utilizes users’ interactions by a posterior attention mechanism.
• We explore the Matthew effect existing in CQA, such that a higher praised answer will accumulate advantages and gain more attention from users. To our knowledge, we are the first to study the Matthew effect in CQA.
• To model the Matthew effect, we further introduce a convergence strategy in the proposed attention-based model that yields better prediction performance.
• Our proposed model, PARPP, yields powerful results compared to existing state-of-the-art methods in two cascade prediction tasks. The analysis of the experimental results shows that the learned alignments in our model can accurately capture the changes of users’ attention to different answers over time.

The remainder of this paper is organized as follows. Section 2 discusses related work. Section 3.1 defines the research problem we study. Section 3 details the proposed Posterior Attention Recurrent Point Process model (PARPP). Section 4 describes our experimental setup. Section 5 discusses our experimental results. Finally, Section 6 concludes the paper.

2. Related Work

In this section, we discuss five lines of the most related work: Community Question Answering, cascade prediction, point process, attention mechanism, and Matthew effect.

2.1. Community Question Answering

Community Question Answering (CQA) has seen a spectacular increase in popularity recently. With the advent and popularity of sites like Zhihu, Quora, and Stack Overflow, an increasing number of people now use these web forums to seek answers to their questions and share their views. Like other online social networking sites, users in CQA can interact with each other. Users can not only post relevant (or irrelevant) answers based on their opinions/expertise but also upvote or downvote answers from other users based on the validity, significance and content of the responses.

With the explosive growth in the number of questions and answers, how to find the most useful information for users has become a challenge. A number of related algorithms have been proposed, which can be divided into two categories. (i) The first is question retrieval, which aims to find the existing questions that are semantically equivalent or relevant to the queried questions from users. Specifically, Zhou et al. [27] proposed a way to use the semantic relations extracted from the world knowledge of Wikipedia in order to enhance the question similarity in the concept space. Zhou et al. [28] proposed a framework using Fisher kernel to learn continuous word embeddings with metadata of category information within CQA pages for question retrieval. In [29], a convolutional neural tensor network architecture was proposed to encode the sentences in semantic space and model their interactions with a tensor layer. (ii) The second category is answer selection, which aims to pick out good answers from a set of candidate answers. In CQA, a question is often represented as two parts: a subject that summarizes the main points of the question and a body that elaborates on the subject in detail. To make use of the subject–body relationship of community questions, Wu et al. [30] proposed the Question Condensing Networks. Zhou et al. [31] proposed a recurrent convolutional neural network to capture both the semantic matching between question and answer and the semantic correlations embedded in the sequence of answers. There are some other studies that focus on predicting question quality [32,33] and finding expert users in Community Question Answering [34,35]. More recently, Liska et al. [36] and Zhuang et al. [37] proposed a benchmark for adaptation to new knowledge over time in question-answering models. Via heterogeneous semantic fusion, Wu et al. [7] proposed a community answer recommendation, which combines text semantic and multi-aspect features to enhance answer recommendations.

Although research on CQA has been ongoing for some time, to our knowledge, there is very little existing studies in the literature on question popularity prediction. Existing methods just simply perform a qualitative analysis of the problem’s popularity [2,3,6,38]. Specifically, Quan et al. [38] used three popularity-related features of questions (potential hits, popular terms and tedious unpopular terms) in the proposed framework to identify popular questions. Liu et al. [6] proposed a supervised machine learning approach to predict question popularity by modeling content-related, user behavior and user profile features. However, these methods only work for simple qualitative analysis, i.e., to judge whether a question is a popular question. In practical application, we often need more fine-grained prediction: for example, given a question and the existing answers, when will the next new answer appear? How many new answers will appear? This is the focus of this paper.

2.2. Cascade Prediction

Predicting the popularity of questions in CQA can be seen as a cascade prediction problem, and research on this kind of problem is a rich and active field [39]. Recent models for predicting the size of an information cascades are generally characterized by two types of approaches: feature-based methods and point process-based methods. We briefly introduce the related work of feature-based methods here. The related work of point process methods are discussed in the next subsection.

Feature-based methods first construct an exhaustive list of potentially relevant features that contribute to the growth and spreading of cascades, such as content features, original poster/resharer features, structural features, and temporal features [38,40]. Then, different learning or statistical methods are applied to classify the cascades and predict their future size, including simple regression models [40], support vector machine (SVM) [41], probabilistic collaborative filtering [42], naive Bayes [43], and multi-scale decomposition [44]. Recent studies on cascade prediction include the use of hierarchical attention neural networks [45], feature-based models [39], and post hoc deferral mechanisms [46].

There are some drawbacks with such approaches: the features used for training need to be extracted manually, and it is difficult to confirm the quality of the selected features, which makes the effect of the models very sensitive to the features [47]. Instead, a point process models the formation of an information cascade in a network, avoiding expensive and cumbersome features extraction. Thus, in this work, we choose to use the point process to predict question popularity.

2.3. Point Process

Point processes [48] are collections of random points falling in some space whose values are “point patterns” such as time and location. Point processes provide statistical language to describe the time and properties of events and have been applied in many fields. In geophysics, the occurrences of earthquakes and aftershocks can be considered to be point processes, and suitable modeling of the conditional intensity function of a point process is useful for the investigation of various statistical features of seismic activity [49,50,51]. In biology, a spatio-temporal point process model is used to understand and predict a species’ distribution [52]. In finance, financial time-series data, such as the transactions of the stock, can be effectively modeled by point processes [53,54]. In the analysis of online social media, Zhao et al. [55] build on the theory of self-exciting point processes to develop a statistical model that allows us to make an accurate prediction of tweet popularity. Kobayashi and Lambiotte [56] take into account the circadian nature of the users and the aging of information and propose a time-dependent Hawkes process for predicting retweet dynamics.

For the convenience of analyzing discrete events in continuous time (“event streams”), typical point process models make specific assumptions about the functional forms of the underlying generative processes. For instance, a standard Possion process [57], the simplest point process, assumes that events occur independently of each other. The Hawkes process [23], a famous non-homogenous point process, supposes that the arrival of an event causes the probability of future events to increase, assuming that historical events influence future events through a positive, additive and time decay kernel function. On the contrary, a self-correcting point process [58] assumes that the occurrence of past points inhibits the occurrence of future points. However, real-world patterns often seem to violate these assumptions. For instance, in CQA, a controversial answer will inspire more answers to refute it, while for a good answer, people may prefer to upvote it rather than to write a new one. In this case, it is difficult to express the influence of an answer on the subsequent events in a fixed function. Although traditional point process methods have achieved certain results in many fields, these methods rely too much on the hypothesis of the underlying diffusion model, which greatly limits their expressive power.

Recent research studies have attempted to extend the expressive power of temporal point processes with deep learning techniques. Dheur et al. [59] propose RMTPP, which is a temporal point process that used a recurrent neural network (RNN) to embed an event history to a vector. Wang et al. [8] explore the cross-dependence problem when modeling a cascade dynamic with RNN and introduce an attention mechanism to solve the cross-dependence in a cascade. In [21], a competing recurrent point process is proposed for modeling visibility dynamics in information diffusion. Zuo et al. [22] propose a differentially private estimation for the Hawkes process. Borrajo et al. [60] define a goodness-of-fit test to check the adequacy of a model in point processes via nonparametric techniques. Kındap and Godsill [61] present novel point process simulation methods based on subordination with a generalized inverse Gaussian process and using a generalized shot-noise representation that involves the random thinning of infinite series of decreasing jump sizes.

However, all of these methods do not take into account the impact of user interaction on event streams. Especially in CQA, users’ behavior will form an obvious Matthew effect. To the best of our knowledge, we are the first to introduce the Matthew effect in the point process.

2.4. Attention Mechanism

The attention mechanism first emerged as an improvement over the encoder–decoder-based neural machine translation system in natural language processing (NLP). Typically, neural machine translation is based on encoder–decoder recurrent neural networks (RNNs) or long short-term memories (LSTMs) [62]. Usually, there are two RNNs/LSTMs, one we call the encoder, which is used to process the entire input sentence and encode it into a context vector, and the other we call the decoder, which uses context vectors as input and produces the words in a sentence one after another. However, RNNs cannot remember longer sentences and sequences due to the vanishing/exploding gradient problems. The performance of the encoder–decoder network degrades rapidly as the length of the input sentence increases [63]. Another problem is that there is no way to give more importance to some of the input words compared to others while translating the sentence. In order to solve these shortcomings, Bahdanau et al. [64] propose the first attention model, suggesting that not only can all the input words be taken into account in the context vector but relative importance should also be given to each one of them. This idea is called “Attention”. A more common understanding is that attention points out which input should be related to the output.

Although the attention mechanism comes from natural language translation, it can be applied in most sequence-to-sequence modeling, especially in the field of dialogue systems, abstract generated, point processes, etc. There are many variations of attention mechanisms. Soft attention is proposed for translation in [64] and refined further in [65,66]. It computes attention for each output as a polynomial distribution over the input states and uses the polynomial probabilities to weight the sum of input vectors. The weighted value serves as relevant context for the output and participates in subsequent calculations. Because soft attention is end-to-end differentiable and easy to implement, it has become the most widely used attention mechanism in sequence-to-sequence learning. Hard attention is another attention mechanism proposed in [67]. Unlike the soft attention mechanism, hard attention makes the output determined from a single input rather than an average of all inputs. An important drawback of hard attention is that it cannot be guided. Another important drawback of hard attention is that it cannot be differentiable. Some work [68,69] has been put forward to try to solve this problem. In addition, there are some other attention mechanisms. Luong et al. [65] attempt to bridge the gap between soft and hard attention and propose local attention that averages a window of input. Yang et al. [70] consider the relationship between the attentions at different time steps and propose a recurrent history mechanism. Some works model attention as latent alignments [71,72]. Because traditional attention mechanisms can only be serially calculated, in order to increase concurrency and improve computational efficiency, Vaswani et al. [73] propose a self-attention mechanism. Fazil et al. [74] propose an attention mechanism based a neural network model for socialbot detection. Mazzia et al. [75] propose an action transformer, which is a self-attention model, for short-time pose-based human action recognition. Shen et al. [66] propose an attention model to convert a continuous sign language video clip into a spoken language. Shaik et al. [76] propose an adaptive unlearning algorithm using an attention mechanism to adapt to changing data distributions and participant characteristics in single-modality and multimodality scenarios.

Prevalent attention architectures do not know what the real output is when calculating attention, because of which they are assumed to be unable to adequately model the dependence among the attention and output Shankar and Sarawagi [77]. In order to take the output into account for a more accurate alignment, Shankar and Sarawagi [77] propose posterior attention models to yield better predictions and alignment accuracy. It is worth mentioning that our work is inspired by them.

2.5. Matthew Effect

The term the Matthew effect was coined by a sociologist, Robert K. Merton [11], who took the name from the parable of the talents or minas in the biblical Gospel of Matthew. Robert K. Merton found that eminent scientists will often receive more credit than a comparatively unknown researcher even if their works are similar; it also means that credit will usually be given to researchers who are already famous. Merton furthermore argued that in the scientific community, the Matthew effect reaches beyond simple reputation to influence the wider communication system, playing a part in social selection processes and resulting in a concentration of resources and talent. Because of this, the Matthew effect has a more intuitive expression: “the rich get richer while the poor get poorer”.

The Matthew effect can be observed in many aspects of life and fields of activities. For instance, the wealth share owned by the top 1.0% of the richest families in the US has continued increasing over the years [14]. A study of science funding in the Netherlands discovered that winners just above the funding threshold were found to accumulate more than twice as much funding during the subsequent eight years as non-winners with near-identical review scores that fell just below the threshold [15]. In almost all European countries, the Matthew effect makes disadvantaged children less likely to use childcare than more advantaged children [16]. In social commerce, a disproportionately higher percentage of votes went to early-posted lengthy reviews due to the Matthew effect [17]. In education, a Matthew effect phenomenon was observed in which early success in acquiring reading skills usually leads to later successes in reading as the learner grows, while failing to learn to read before the third or fourth year of schooling may be indicative of lifelong problems in learning new skills [78]. Gómez-Bengoechea and Jung [79] explore the relationship between information and communication technologies diffusion and labor productivity at the firm level, and they found a Matthew effect on firms’ digitalization distributional effects.

However, as far as we know, little research has paid attention to analyze the Mathew effect in CQA. In fact, we also observed the existence of the Matthew effect in CQA. An early and quality answer will accumulate its advantage after it occupies the ranking advantage, which leads to a better but later answer having trouble attracting users’ attention. We will provide a more detailed analysis in later sections.

3. Model

In this section, we first provide the definition of our task; then, we detail our proposed Posterior Attention Recurrent Point Process model, PARPP, for predicting CQA popularity. We begin by the definition of the task, subsequently review the temporal point process model, and then describe our attention-based recurrent point process model.

3.1. Problem Formulation

The problem we aim to address is to predict the popularity of the problem based on its observed history. The input data are a collection of M question-answering cascades $\mathcal{C}={\left\{{\mathcal{S}}^i\right\}}_{i=1}^M$, with each cascade ${\mathcal{S}}^i=\left\{(t_k^i,m_k^i)\right|t_k^i\in [0,+\infty ),k=1,\dots ,N^i\}$ being a sequence of answering behaviors of a question, where $t_k^i$ is the time at which an answer is posted, $m_k^i$ is the marker of the answer, and $N^i$ is the total number of answers of a question that have appeared so far. In CQA, the marker includes the textual content associated with an answer and the number of the poster’s followers. Let the history ${\mathcal{H}}_k$ be the list of observed events up to the k-th answer and $N_{t_k}$ be the number of answers up to time $t_k$. The objective of our model is to predict the time $t_{k+1}$ of the next answer and how many answers $N_{t_k+\Delta t}$ there will be in the future $t_k+\Delta t$ given a sequence of past answers ${\mathcal{H}}_k$, which can be formally define as the following:

• Predicting Questions’ Popularities in CQA. Given a question-answering history ${\mathcal{H}}_k$, the goal is to seek a function f that aims at predicting the time of the next answer and predicting the number of answers at the future time $t_k+\Delta t$:

  $${\mathcal{H}}_k=\{(t_1,m_1),\dots ,(t_k,m_k)\}\stackrel{f}{⟶}{t}_{k+1},N_{t_k+\Delta t},$$

  where $\Delta t$ is the window size we want to predict.

  Table 1. Main notations used across the whole paper.

  Symbol	Description
  $\mathcal{C}$	a collection of cascades
  $\mathcal{S}$	a question-answer cascade
  M	total number of question-answer cascades
  i	sequence number of a question-answer cascade
  k	represents the k-th answer
  $t_k$	the time when the k-th answer being posted
  ${\tilde{t}}_k$	model’s estimate of the time of the k-th answer
  $m_k$	the marker of the k-th answer
  ${\mathcal{H}}_k$	observed history up to the k-th answers
  $N^i$	total number of answers in a question-answer cascade i
  $N_{t_k}$	the number of answers up to time $t_k$
  ${\tilde{N}}_{t_k}$	model’s estimate of the number of answers at time $t_k$
  $\lambda$	conditional intensity function
  $f\left(t\right|{\mathcal{H}}_k)$	conditional probability density function
  $F\left(t\right|{\mathcal{H}}_k)$	conditional cumulative distribution function
  $\mathcal{L}$	the negative logarithmic likelihood function

  summarizes our main notations used across the paper.

3.2. Temporal Point Process

The temporal point process is a powerful stochastic model to capture event streams. One way to characterize a temporal point process is to specify the distribution function of the next arrival time conditional on the past. Given the history up to the last event ${\mathcal{H}}_k$, the conditional cumulative distribution function $F\left(t\right|{\mathcal{H}}_k)$ is

$$F\left(t\right|{\mathcal{H}}_k)={\int }_{t_k}^{t}P(t_{k+1}\in [s,s+ds)|{\mathcal{H}}_k)ds={\int }_{t_k}^{t}f\left(s\right|{\mathcal{H}}_k)ds,$$

(1)

where $f\left(s\right|{\mathcal{H}}_k)$ is the conditional probability density function for the event’s next arrival time $t_{k+1}$. The joint probability of an event cascade $t_1,t_2,\dots ,t_k$ is then obtained by the chain rule as

$$f(t_1,t_2,\dots ,t_k)=\prod _{i=1}^{k}f\left(t_i\right|{\mathcal{H}}_{i-1}).$$

(2)

However, it is difficult to work with the conditional arrival distribution $f\left(t\right|{\mathcal{H}}_k)$. Instead, we use the conditional intensity function to characterize a temporal point process. Let $N_t$ count the number of events up to time t; then, the conditional intensity function can be specified as

$$\lambda \left(t\right|{\mathcal{H}}_k)=\underset{h\to 0}{lim}{\displaystyle \frac{P\{N_{t_k+h}-N_{t_k}=1|{\mathcal{H}}_k\}}{h}}.$$

(3)

It turns out that the conditional intensity function $\lambda \left(t\right|{\mathcal{H}}_k)$ in the above equation can be expressed in terms of the conditional density f and its corresponding conditional cumulative distribution function F:

$$\lambda \left(t\right|{\mathcal{H}}_k)={\displaystyle \frac{f\left(t\right|{\mathcal{H}}_k)}{1-F\left(t\right|{\mathcal{H}}_k)}}.$$

(4)

As a consequence, the conditional density function can be alternatively specified by

$$f\left(t\right|{\mathcal{H}}_k)=\lambda \left(t\right|{\mathcal{H}}_k)exp(-{\int }_{t_k}^t\lambda \left(\tau \right|{\mathcal{H}}_k)d\tau ).$$

(5)

Then, we can estimate the time of the next arriving event ${\tilde{t}}_{k+1}$ using the following expectation:

$${\tilde{t}}_{k+1}={\int }_{t_k}^{+\infty }t·f\left(t\right|{\mathcal{H}}_k)dt.$$

(6)

In general, the integration in Equation (6) does not have analytic solutions. In order to obtain the estimate of $t_{k+1}$, we apply commonly used numerical integration techniques [80] to calculate Equation (6) instead. Conditional intensity function $\lambda \left(t\right|{\mathcal{H}}_k)$ can be seen as the expected rate of arrivals conditioned on ${\mathcal{H}}_k$, so we can predict the future event count ${\tilde{N}}_{t_k+\Delta t}$ as

$${\tilde{N}}_{t_k+\Delta t}-N_{t_k}={\int }_{t_k}^{t_k+\Delta t}\lambda \left(s\right|{\mathcal{H}}_k)ds.$$

(7)

3.3. Recurrent Point Process

Our model extends the temporal point process by integrating recurrent neural networks (RNNs) in a unique way. An RNN is a feedforward neural network which has been proven to successfully model sequences. For each answer with the posting time $t_k$ and the marker $m_k$, we vectorize the pair $(t_k,m_k)$ into ${\mathit{x}}_k$ as the input to a recurrent neural network in order. After RNN’s nonlinear transformation, we can obtain the hidden state ${\mathit{h}}_k=RNN({\mathit{x}}_k,{\mathit{h}}_{k-1})$. The embedding ${\mathit{h}}_k$ can be seen as the representative of the k-th answers, and the output is trained to maximize the likelihood of a CQA cascade,

$$P\left(\mathcal{S}\right)=\prod _{k=1}^{N-1}P\left(t_{k+1}\right|{\mathcal{H}}_k)=\prod _{k=1}^{N-1}f\left(t_{k+1}\right|{\mathit{h}}_k),$$

(8)

where $P\left(\mathcal{S}\right)$ denotes a question–answer cascade.

3.4. Attention Mechanism

In CQA, a question may have dozens or even hundreds of answers. Users with limited time would only be attracted by a small number of high-quality answers. Although these answers are often not the latest posts, they occupy the optimal sort position. Intuitively, we believe that these answers will have a greater impact on the popularity of the question. However, due to the chain structure of an RNN, it fails to effectively capture the impact of long history on the present. The attention mechanism is an effective way to solve this problem. The attention mechanism was originally used in the encoder–decoder-based neural machine translation system [64]. It is proposed as an improvement to solve the vanishing/exploding gradient problems and obtain a better alignment between output and input. We refer to the practice in natural language translation and use the attention mechanism in our model to find answers that have the greatest impact on the popularity of the question.

In the existing encoder–decoder model, a general way of applying attention mechanism is to calculate a context vector ${\mathit{c}}_k$:

$${\mathit{c}}_k=\sum _{i=1}^k{a}_{k,i}{\mathit{h}}_i,s.t\sum _{i=1}^k{a}_{k,i}=1,$$

(9)

where the weight of each attention variable $a_{k,i}$ is computed as a function of the decoder state and encoder state as $a_{k,i}\propto e^{A_{\theta }({\mathit{h}}_i,{\mathit{s}}_k)}$. Here, $A_{\theta }(.,.)$ is an end-to-end trained function of an input embedding ${\mathit{h}}_i$ and a decoder state ${\mathit{s}}_k$.

However, the treatment of the attention variables has been rather ad hoc. For instance, in natural language translation, attention is interpreted as the alignment of the source words and the target words. Only when we observe target words can we accurately know which source words it is aligned with. Therefore, in order to better capture the impact of historical events at the present, we propose a Posterior Attention Recurrent Point Process Model, PARPP, for CQA popularity prediction.

The framework of the proposed PARPP is shown in Figure 2. According to the figure, the input is a sequence of pairs of answers and the corresponding answers, i.e, $(t_1,m_1),\dots ,(t_k,m_k)$, and the output is the predicted timestamp that the next answer will appear at. The proposed PARPP encodes the input sequential data into sequential embeddings $h_1,h_2,\dots ,h_k$ via an encoder. Each embedding here is the representation of each answer as a whole rather than a word of an answer. To predict the timestamp of the next answer that will appear, our proposed model utilizes two attention layers, i.e., the “posterior attention” and the “prior attention” layers, which are the two main components in the decoder. In the decoder, the model takes together the inferred embeddings $h_1,h_2,\dots ,h_k$ into the posterior attention layer and outputs the inferred embedding vector $s_k$. In the prior attention layer, the input consists of a sequence of predicted timestamps $t_{k+1}^1,t_{k+1}^2,\dots ,t_{k+1}^k$ that services the prediction of the timestamp when the next answer appears. Finally, the decoder of our model predicts the next timestamp $t_{k+1}$ of the next answer via weighting all the timestamps $t_{k+1}^1,t_{k+1}^2,\dots ,t_{k+1}^k$ via the prior attention layer. We first factorize the conditional probability of the next answering behavior via chain rule as

$$\begin{array}{cc}\hfill P\left(t_{k+1}\right|{\mathcal{H}}_k)& =\sum _{a_k=1}^{k}P(t_{k+1},a_k|{\mathcal{H}}_k)=\sum _{a_k=1}^{k}P\left(t_{k+1}\right|a_k,{\mathcal{H}}_k)P\left(a_k\right|{\mathcal{H}}_k)\hfill \\ \hfill & =\sum _{a_k=1}^{k}P\left(t_{k+1}\right|a_k,{\mathcal{H}}_k)Prio{r}_k\left(a_k\right),\hfill \end{array}$$

(10)

where we denote $P\left(a_k\right|{\mathcal{H}}_k)$ as $Prio{r}_k\left(a_k\right)$ and regard $P\left(a_k\right|{\mathcal{H}}_k)$ as the prior attention $Prio{r}_k\left(a_k\right)$ because it is the attention distribution before observing the next answer. According to the framework, we represent the conditional probability as

$$P\left(t_{k+1}\right|a_k,{\mathcal{H}}_k)=f(t;{\mathit{s}}_k,{\mathit{h}}_{a_k})=\lambda \left(t\right)exp\left(-{\int }_{t_k}^t\lambda \left(\tau \right)d\tau \right).$$

(11)

With ${\mathit{h}}_{a_k}$ and the decoder state ${\mathit{s}}_k$, we can formulate the conditional intensity function by

$$\begin{array}{cc}\hfill \lambda \left(t\right)& ={\lambda }_0{e}^{-\alpha (t-t_k)}\hfill \\ \hfill s.t.{\lambda }_0& =exp({\mathit{w}}_{\lambda }^T{\mathit{s}}_k+{\mathit{u}}_{\lambda }^T{\mathit{h}}_{a_k}+b_{\lambda })\hfill \\ \hfill \alpha & =exp({\mathit{w}}_{\alpha }^T{\mathit{s}}_k+{\mathit{u}}_{\alpha }{\mathit{h}}_{a_k}+b_{\alpha }),\hfill \end{array}$$

(12)

where the column vector ${\mathit{w}}_{\lambda },{\mathit{u}}_{\lambda },{\mathit{w}}_{\alpha },{\mathit{u}}_{\alpha }$ and scalar $b_{\lambda },b_{\alpha }$ are parameters in the model, ${\lambda }_0$ is the initial value of $\lambda \left(t\right)$ and $\alpha$ is the decay rate. In order to avoid the explosion of $\lambda$, we limit $\alpha$ to be greater than zero.

When we observe the next answer, we can calculate the posterior distribution of attention. Similarly, we call $P\left(a_k\right|{\mathcal{H}}_{k+1})=P\left(a_k\right|{\mathcal{H}}_k,(t_{k+1},m_{k+1}))$ as the posterior attention $Post{r}_k\left(a_k\right)$, since this is the attention distribution after observing the next answer at the corresponding step, unlike that in the prior attention.

We can obtain $Post{r}_k\left(a_k\right)$ by applying the Bayes rule as follows:

$$Post{r}_k\left(a_k\right)=P\left(a_k\right|{\mathcal{H}}_{k+1})={\displaystyle \frac{P\left(t_{k+1}\right|a_k,{\mathcal{H}}_k)P\left(a_k\right|{\mathcal{H}}_k)}{P\left(t_{k+1}\right|{\mathcal{H}}_k)}}={\displaystyle \frac{P\left(t_{k+1}\right|a_k,{\mathcal{H}}_k)Prio{r}_k\left(a_k\right)}{{\sum }_{a_k=1}^{k}P\left(t_{k+1}\right|a_k,{\mathcal{H}}_k)Prio{r}_k\left(a_k\right)}}.$$

(13)

We expect this attention to be more accurate as it is computed with the knowledge of the posting time of the next answer. We then use the posterior attention $Post{r}_k\left(a_k\right)$ to update the decoding state ${\mathit{s}}_{k+1}$ by

$${\mathit{s}}_{k+1}=RNN({\mathit{s}}_k,\sum _{a_k=1}^{k}Post{r}_k\left(a_k\right){\mathit{h}}_{a_k}).$$

(14)

Next, we calculate the prior attention of step $k+1$. It is remarkable that at step $k+1$, we observe an event more than before, which changes the range of attention variables, i.e., $a_k\in [1,\dots ,k],a_{k+1}\in [1,\dots ,k,k+1]$. Intuitively, we assume that the larger the k, the more similar the distribution of attention in steps k and $k+1$. The greater the number of answers, the more the interest of people is allocated to a few high-quality answers. When a new answer is overwhelmed by other answers, it is difficult to attract people’s attention. Based on this observation, we compute $Prio{r}_{k+1}\left(a_{k+1}\right)$ by

$$Prio{r}_{k+1}\left(a_{k+1}\right)=\left\{\begin{array}{c}Post{r}_k\left(a_k\right)(1-{\displaystyle \frac{1}{k+1}}),a=1,\dots ,k\hfill \\ {\displaystyle \frac{1}{k+1}},a=k+1\hfill \end{array}\right..$$

(15)

As k increases, the attention gained in the latest answer will become smaller and smaller, and the attention distribution of the old answer will be more and more similar to the previous one. The example in Figure 3a illustrates our idea. Since the new answer is sorted backwards, it is difficult to attract the attention of the user, and thus the attention distribution at this time should be similar to the previous moment $t_k$.

3.5. Convergence Strategy

In CQA, because users are usually attracted by a small number of high-quality answers, and because of the Matthew effect, users’ attention will become more and more concentrated. To capture the Matthew effect existing in CQA, we propose a convergence strategy in our PARPP to adjust the posterior attention mechanism. The idea is depicted in Figure 3b. Due to the existence of the Matthew effect, the popular answer will always accumulate popularity and form a gap with other answers. As time progresses, the votes for the top-ranked answers will far exceed those for the other answers. In extreme cases, the user’s attention will be focused entirely on the top-ranked answers. The upper right of Figure 3b shows the distribution of attention in this case. Formally, let $P_{t_k}\left(a\right)$ indicate the distribution of attention when there are k answers, and we assume that the attention assigned to the first answer is the greatest. To facilitate the narrative, we assume that only the answer ranked first will continue to accumulate advantages, and the gap with the remaining answers will grow. Then, we expect

$$\underset{\Delta t\to +\infty }{lim}{P}_{t_k+\Delta t}\left(a\right)=P_{+\infty }\left(a\right)=\left\{\begin{array}{c}1,a=1\hfill \\ 0,a=2,\dots ,k\hfill \end{array}\right..$$

(16)

At time $t_k+\Delta t$, we expect that the attention distribution will converge toward the direction of $P_{+\infty }$. Inspired by this, we calculate the attention distribution of time $t_{k+\Delta t}$ by

$$P_{t_{k+\Delta t}}=\underset{P_{t_{k+\Delta t}}}{min}\left((1-\delta )D_{KL}(P_{t_k}\left|\right|P_{t_{k+\Delta t}})+\delta D_{KL}(P_{+\infty }\left|\right|P_{t_{k+\Delta t}})\right),$$

(17)

where $D_{KL}$ is the Kullback–Leibler divergence, $\delta$ is a function of t and $\delta \propto t,\delta \in (0,1)$. $\delta$ can be regarded as the degree of convergence of $P_{t_k}$ to $P_{+\infty }$. The larger $\Delta t$ is, the larger $\delta$ is, and the closer $P_{t_{k+\Delta t}}$ is to $P_{+\infty }$. The optimization problem of Formulation (17) can be solved using the method of Lagrange multipliers. Then, we have

$$P_{t_k+\Delta t}\left(a\right)=\left\{\begin{array}{ccc}(1-\delta )P_{t_k}\left(a\right)+\delta ,\hfill & \hfill a& =1\hfill \\ (1-\delta )P_{t_k}\left(a\right),\hfill & \hfill a& =2,\dots ,k\hfill \end{array}\right..$$

(18)

We use this convergence strategy in the attention mechanism of our proposed model. For convenience of description, as before, we assume that the attention assigned to the first answer in the posterior of the previous step is the greatest. Then, we change the calculation of prior attention in step $k+1$ by

$$Prio{r}_{k+1}\left(a\right)=\left\{\begin{array}{ccc}((1-\delta )Post{r}_k\left(a\right)+\delta )(1-{\displaystyle \frac{1}{k+1}}),\hfill & \hfill a& =1\hfill \\ (1-\delta )Post{r}_k\left(a\right)(1-{\displaystyle \frac{1}{k+1}}),\hfill & \hfill a& =2,\dots ,k\hfill \\ {\displaystyle \frac{1}{k+1}},\hfill & \hfill a& =k+1\hfill \end{array}\right.,$$

(19)

where we let $\delta =1-e^{-\theta dt}$, with $dt=t_{k+1}-t_k$, and $\theta$ being a positive parameter learning in the model.

A convergence strategy can also be extended to converge to the top-k attention variables. For example, we assume that the two highest attentions at time $t_k$ are $P_k(a=1)=\mu$ and $P_k(a=2)=\nu$; to converge toward the highest two attentions, we can make

$$P_{+\infty }\left(a\right)=\left\{\begin{array}{ccc}{\displaystyle \frac{\mu }{\mu +\nu }},\hfill & \hfill a& =1\hfill \\ {\displaystyle \frac{\nu }{\mu +\nu }},\hfill & \hfill a& =2\hfill \\ 0,\hfill & \hfill a& =3,\dots ,k\hfill \end{array}\right..$$

(20)

The rest of the calculations for $P_{k+1}\left(a\right)$ are similar to Equations (17)–(19). We denote PARPP with a convergence strategy as PARPP-CON.

3.6. Parameter Learning

In this section, we introduce the learning process of our proposed models. Given a collection of QA sequences $\mathcal{C}={\left\{{\mathcal{S}}^i\right\}}_{i=1}^M$, where ${\mathcal{S}}^i=\left({(t_j^i,m_j^i)}_{j=1}^{N^i}\right)$, we assume that each question–answer cascade is independent of each other. As a result, the logarithmic likelihood of the QA set $\mathcal{C}$ is the sum of the logarithmic likelihood of the individual cascade. Then, we can train the model by minimizing the joint negative logarithmic likelihood $\mathcal{L}\left(\mathcal{C}\right)$:

$$\begin{array}{cc}\hfill \mathcal{L}\left(\mathcal{C}\right)=\mathcal{L}\left(\left\{{\mathcal{S}}^i\right\}\right)& =-\sum _{i=1}^M\sum _{k=1}^{N^i-1}logP\left(t_{k+1}^i\right|{\mathcal{H}}_k^i)\hfill \\ \hfill & =-\sum _{i=1}^M\sum _{k=1}^{N^i-1}\sum _{a=1}^k\left(logP\left(t_{k+1}^i\right|a,{\mathcal{H}}_k^i)+logPrio{r}_k\left(a\right)\right).\hfill \end{array}$$

(21)

We exploit the Back Propagation Through Time (BPTT) for training PARPP. In each training iteration, we vectorize the question as the first input and then its answering behaviors. Each input includes not only temporal features, such as the number of followers of users for each answer, but also the embedding of text information. The temporal features are assembled by the logarithm time interval $log(t_k-t_{k-1})$ and the discretization of numerical attributes according to the month, day, hour, minute and second. In order to obtain the encoder of the text information, we use bert-as-service [81], which is a toolkit based on the famous natural language processing (NLP) model developed by Google for pre-training language representations, and BERT [82], to map sentences into fixed-length representation. We then put the sentences representation to an RNN to obtain the text representation of an answer. For better results, we adopt LSTM [62], a modern variant of RNN, in our model. Then, we apply stochastic gradient descent (SGD) with mini-batch algorithms and update the parameters using the Adam optimizer [83]. We also employ an early stopping method [84] to prevent overfitting.

4. Experimental Setup

In this section, we describe the experimental dataset crawled from Zhihu, which is a famous Chinese CQA. We then compare our PARPP model to the state-of-the-art modeling methods of cascade prediction. Finally, we analyze how the proposed attention mechanism performs.

4.1. Research Questions

The research questions guiding the remainder of the paper are outlined below:

• (RQ1) Given a question and the trace of observed answers, when will the next answer appear?
• (RQ2) Given a question and the trace of observed answers, how many answers will be posted in the future?
• (RQ3) Can the proposed attention mechanism capture the most influential answers?

4.2. Dataset

Our experiment is conducted on a real-world dataset: Zhihu, which is a famous Chinese CQA. In order to make the data more representative, we use a Zhihu API. (the API is publicly available at: https://syaning.github.io/zhihuapi-py/, accessed on 1 June 2024) to crawl the data from 25 distinguished topics in Zhihu, and there are up to 700 questions under each topic. In the end, we crawled 8272 questions published before 7 September 2019 (there are no restrictions on their posting timestamps), and the corresponding answers, which number more than 1.3 million. These questions were posted anywhere from two to three days ago to a few years ago. For each question and answer, the dataset includes the question/answer ID, their timestamps, the number of votes for the question, the number of followers of the poster, the text content and the comments.

Table 2. Statistics of the Zhihu dataset used in our experiments.

Topic	#Questions	#Answers	#Comments	#Votes	Average Duration (Days)
Lifestyle	529	133,782	582,868	7,372,546	1165
Economics	751	71,871	278,613	2,809,376	1319
Sports	539	58,885	387,740	3,856,046	1196
Internet	209	37,985	132,640	2,950,844	698
Art	544	60,455	270,134	3,559,153	1249
Read	432	111,631	292,375	4,336,772	1177
Food	194	62,467	342,557	3,834,906	1006
Animation	475	48,005	152,896	1,870,583	410
Movie	254	47,006	136,307	2,296,818	472
Game	132	51,471	206,775	2,604,300	910
…	…	…	…	…	…
Total	8272	1,315,316	5,910,928	74,061,774	979

shows the statistics of some topics in the dataset. To illustrate the details of the structure of the data in our dataset, Figure 4 shows an example data with an example question and its corresponding answers with timestamps.

In this paper, we focus on questions that have been submitted more than half a year before and have more than 50 answers within half a year. We remove answers that appear six months after the question was posted because if the time is too far away, it is no longer meaningful to predict the popularity of the question. Eventually, we filtered out 1888 questions with more than 840,000 answers. Details of the dataset are shown in Figure 5. Figure 5a shows the empirical distribution for the inter-event times, and the mean value is 25.15 h. For visualization convenience, we have taken the logarithm of the time interval. Figure 5b shows the distribution histogram of the answers’ posting time. From this figure, we can see that most of the answers appeared within 10 days after the question being asked, and the trend of entire answers conformed to the exponential decay. Therefore, in Equation (12), our conditional intensity function uses an exponential form. Finally, Figure 5c displays the evolution of the number of answers in 90 days. We take the number of answers within half a year after the question is raised as the final number and mark it with dashed lines in the figure. On average, the final number of answers to a question in half a year is ∼318, and a question receives nearly 80% of its retweets in the first 90 days.

We divide the dataset into a training set and a test set in a ratio of 9:1. We input the answers in each question answer cascade into the model in turn and predict when the next answer will appear and the number of answers in a specific point in time.

4.3. Matthew Effect in CQA

To examine the existence and effectiveness of the Matthew effect in CQA, we ranked all answers to each question based on the number of votes it received. Then, we calculated the percentage of votes received by the 10% and 20% top-ranked answers, respectively, against the total votes received by all answers. We also calculated the percentage of votes received by the 10% and 20% earliest posted answers (based on their posting dates), respectively, against total votes received by all answers. We randomly selected 10 topics from the Zhihu dataset and recorded the statistics of their number of questions and the average percentage of total votes that each of the ten topics received in

Table 3. Concentration of total votes among top and early answers (%).

Topic	Nums	Top 10%	Top 20%	Earliest 10%	Earliest 20%
Lifestyle	529	86.0	92.8	26.7	40.8
Economics	751	79.8	88.3	33.3	45.2
Sports	539	82.5	90.3	26.8	37.9
Internet	209	82.5	90.9	30.2	44.2
Art	544	77.1	86.5	26.5	40.2
Read	432	83.1	90.6	29.5	41.4
Food	194	90.5	95.6	28.0	42.9
Animation	475	77.9	86.7	30.6	45.0
Movie	254	82.8	90.8	33.9	48.5
Game	132	79.2	88.9	30.6	47.0
…	…	…	…	…	…
Mean	—	82.1	90.1	29.6	43.3

.

We found that the top 10% of answers received an average of 82.1% of the total votes, while the top 20% of answers received an average of 90.1% of the total votes. In contrast, the earliest 10% of answers received an average of 29.6% of the total votes, and the earliest 20% of reviews received an average of 43.3% of the total votes. From the data, we found that a disproportionately higher percentage of votes went to early-posted answers. This shows that an early-posted quality answer will accumulate advantages over the time, which is evidence of existence of the Matthew effect in CQA.

Figure 6 provides another perspective on the Matthew effect in CQA. A direct result of the Matthew effect is the concentration of resources. Figure 6 shows the distribution histogram of the number of votes obtained by answers. Figure 6a shows the average proportion of votes received by the ten most popular answers in each question. Surprisingly, although the most popular answer is just one answer, it receives more than 40% of the votes for all the answers in a question. The distribution of the entire voting number is similar to the power law distribution, which is a typical distribution formed by the concentration of resources caused by the Matthew effect. Figure 6b shows the statistics result of votes of top-k% answers. We can see that on average, 5% of the answers received more than 80% of the votes, and 10% of the responses almost ruled the whole question. All these data indicated the existence and significant influence of the Matthew effect in CQA.

4.4. Baselines

To evaluate the predictive performance of our proposed model, we compare our proposed model PARPP with the following five prediction baselines. Methods (iv) and (v) can model both the next event type and the next activation time. Here, we only use them to predict the time of the next event.

• Naive Prediction. This is a naive baseline that uses the time difference between the last two answeres to predict when the next answer will appear.
• Poisson Process [85]. This is a point process that assumes the conditional intensity function is independent of the history ${\mathcal{H}}_k$ and remains constant over time:

  $$\lambda \left(t\right)={\lambda }_0,$$

  (22)

  where ${\lambda }_0$ is a fixed constant. In fact, the model produces an estimate of the average inter-event gaps.
• Hawkes Process [86]. This is a self-exciting process in which the arrival of an event causes the conditional intensity function to increase. The intensity function is parameterized by

  $$\lambda \left(t\right)={\lambda }_0+\alpha \sum _{t_k<t}\varphi (t-t_k),$$

  (23)

  where ${\lambda }_0⩾0$ is the base rate and $\varphi (·)$ is the triggering kernel capturing temporal dependencies. In our experiments, we choose $\varphi \left(t\right)=e^{-\beta t}$. The parameters ${\lambda }_0$, $\alpha$, and $\beta$ are optimized by maximizing the likelihood function.
• RMTPP [59]. The recurrent marked temporal point process (RMTPP) is a temporal point processes method based on the RNN. The original model not only predicts the time of the next event but also predicts the type of the next event. Here, we only use it to predict the next time. Specifically, RMTPP applies an RNN to learn a general representation ${\mathit{h}}_k$ of a nonlinear dependency over both the time and the marker information from past events:

  $${\mathit{h}}_k=max\{{\mathit{W}}^m{\mathit{m}}_k+{\mathit{W}}^t{\mathit{t}}_k+{\mathit{W}}^h{\mathit{h}}_{k-1},0\},$$

  (24)

  where ${\mathit{W}}^m$, ${\mathit{W}}^t$, and ${\mathit{W}}^h$ are weight matrices, ${\mathit{m}}_k$ represents the marker features, and ${\mathit{t}}_k$ represents the temporal features. Then, the conditional intensity function can be formulated by

  $$\lambda \left(t\right)=exp({{\mathit{v}}^t}^T·{\mathit{h}}_k+w^t(t-t_k)+b^t),$$

  (25)

  where ${\mathit{v}}^t$ is a column vector, and $w^t$ and $b^t$ are scalars.
• CYAN-RNN [8]. This is an attention-based RNN method to model cascade dynamics. Similarly, the conditional intensity function is computed by

  $$\lambda \left(t\right)=exp({\mathit{v}}^T·{\mathit{h}}_k+{\mathit{u}}^T·{\mathit{s}}_k+{\mathit{z}}^T·{\mathit{x}}_k+w(t-t_k)),$$

  (26)

  where $\mathit{v}$, $\mathit{u}$, $\mathit{z}$, and w are learning parameters, ${\mathit{x}}_k$ is the vector of the k-th event ($t_k,m_k$), and ${\mathit{s}}_k$ is the decoding state which can be calculated through an attention mechanism by

  $${\mathit{s}}_k=\sigma ({\mathit{x}}^k,{\mathit{s}}_{k-1},\sum _a{P}_k\left(a\right){\mathit{h}}_a),$$

  (27)

  where $\sigma$ is a nonlinear activation function. The attention weight $P_k\left(a\right)$ is formalized as

  $$P_k\left(a\right)={\displaystyle \frac{e^{A_{\theta }({\mathit{h}}_a,{\mathit{s}}_k)}}{{\sum }_{i=1}^k{e}^{A_{\theta }({\mathit{h}}_i,{\mathit{s}}_k)}}},$$

  (28)

  where $A_{\theta }$ is a non-linear function.
  The author [8] further proposed a coverage strategy to adjust the misallocation of attention. We denote CYAN-RNN with coverage as CYAN-RNN(cov).

4.5. Evaluation Metrics

The evaluation metrics used to evaluate the performance of answers’ popularities are the ones widely used in point processes [8,20,55,56]: root mean square error (RMSE), mean absolute percentage error (MAPE) and mean absolute error per day, of which RMSE and $MAP{E}_t$ measure the performance in next-answer prediction, while $MAP{E}_N$ and absolute error per day measure the performance in the answers’ count prediction. At step k, models output the corresponding conditional density function $\lambda \left(t\right|{\mathcal{H}}_k)$ each time a prediction is made. For predicting the time of the next answer, we use Equation (6) to obtain the next time prediction ${\tilde{t}}_{k+1}$. For predicting the number of answers, we use Equation (7) to obtain the prediction ${\tilde{N}}_{t_k+\Delta t}$. Then, these metrics can be computed as follows:

• RMSE. Every time we make a prediction, we have an estimate ${\tilde{t}}_{k+1}$. We can obtain RMSE for time prediction by

  $$RMSE=\sqrt{{\displaystyle \frac{{({\sum }_n{\tilde{t}}_{k+1}-t_{k+1})}^2}{n}}},$$

  (29)

  where $t_{k+1}$ is the ground-truth prediction and n is the number of predictions.
• MAPE. $MAP{E}_t$ reports the mean deviation of the estimated timestamps of the next future events from their actual times:

  $$MAP{E}_t={\displaystyle \frac{100\%}{n}}\sum _n{\displaystyle \frac{|{\tilde{t}}_{k+1}-t_{k+1}|}{t_{k+1}}}.$$

  (30)

Additionally, $MAP{E}_N$ measures the count prediction errors defined as below:

$$MAP{E}_N={\displaystyle \frac{100\%}{n}}\sum _n{\displaystyle \frac{|{\tilde{N}}_{t_k+\Delta t}-N_{t_k+\Delta t}|}{N_{t_k+\Delta t}}}.$$

(31)

• Mean Absolute Error per Day: This indicator can measure the impact of forecast duration on forecast results and is defined as follows:

  $$ϵ={\displaystyle \frac{1}{n\Delta t}}\sum _n|{\tilde{N}}_{t_k+\Delta t}-N_{t_k+\Delta t}|,$$

  (32)

  where $N_{t_k+\Delta t}$ is the ground-truth result.

The evaluation metrics used to evaluate the alignment of the attention mechanism is P@k (Precision at k) [87] and NDCG@k (Normalized Discounted Cumulative Gain at k [88], as we rank and select the top-k answers with the highest attention for precise prediction purposes in the model. We sort the attention of each prediction and select the top-k answers with the highest attention (denoted as the predicted rank list ${\tilde{R}}_k$) and compare them with the actual situation, which is sorted by the number of comments (denoted as the ground truth list $R_k$). Because each comment has a timestamp, we use the number of comments instead of votes to track the dynamic changes in user attention. The details of the calculation are as follows:

• Precision. Precision is a widely used metric to judge whether a model can correctly predict the positive samples. We calculate how many answers are in both the set ${\tilde{R}}_k$ and the ground-truth set $R_k$ and treat them as true positive. We calculate P@k as shown below:

  $$P@k={\displaystyle \frac{|{\tilde{R}}_k\cap R_k|}{k}}.$$

  (33)
• NDCG. Because P@k does not consider the ranking position, we also use NDCG@k. We denote $an{s}_i$ as the i-th answer in ${\tilde{R}}_k$ and define the relevance ($re{l}_i$) of $an{s}_i$ as follows:

  $$re{l}_i=\left\{\begin{array}{c}\hfill 1,an{s}_i\in R_k\\ \hfill 0,an{s}_i\notin R_k\end{array}\right..$$

  (34)

  Then, we can obtain DCG (Discounted Cumulative Gain) as below:

  $$DCG@k=\sum _{i=1}^k{\displaystyle \frac{re{l}_i}{{log}_2(i+1)}}.$$

  (35)

  And we obtain IDCG (Ideal Discounted Cumulative Gain) as below:

  $$IDCG@k=\sum _{i=1}^k{\displaystyle \frac{1}{{log}_2(i+1)}}.$$

  (36)

  Finally, we calculate NDCG@k as follows:

  $$NDCG@k={\displaystyle \frac{DCG@k}{IDCG@k}}.$$

  (37)

Higher P@k and NDCG@k scores, and lower RMSE and MAPE scores, indicate better performance.

5. Results and Analysis

In this section, we answer the research questions and analyze our experimental results.

5.1. Predict the Time of the Next Answer

RQ1: We first compare the performance of our proposed model (PARPP) and the baselines in predicting the time at which the next answer appears.

Table 4. Performance of predicting the time of the next answer.

	Zhihu
Model	RMSE (Day)	${\mathbf{MAPE}}_{\mathit{t}}$ (%)
Naive	4.90	39.82
Possion	5.02	118.21
Hawkes	4.83	398.11
RMTPP	4.66	23.44
CYAN-RNN	4.60	23.49
CYAN-RNN(cov)	4.58	23.34
PARPP	4.21	20.87
PARPP-CON(Top1%)	4.17	16.92
PARPP-CON(Top5%)	4.17	16.20
PARPP-CON(Top10%)	4.14	16.01
PARPP-CON(Top15%)	4.11	15.93
PARPP-CON(Top20%)	4.16	15.99

shows the results of our experiment. PARPP-CON is the same as PARPP except that it uses a convergence strategy. The hyperparameters of PARPP and PARPP-CON are set as the following: learning rate {0.1, 0.01, 0.001}; hidden layer size of encoder and decoder {8, 16, 32}; and batch size {8, 16, 32}. We choose the best results from each configuration. In order to verify the optimal effect of our proposed convergence strategy, we set up five comparison groups {Top1%, Top5%, Top10%, Top15%, Top20%}, which correspond to the convergence of the 1%, 5%, 10%, 15% and 20%, respectively.

It can be seen from Table 4 that in the prediction task, the RMSE and ${\mathrm{MAPE}}_t$ evaluation scores obtained by PARPP and PARPP-CON are lower than the other baselines, indicating the validity of our proposed model. Moreover, we can observe that PARPP-CON consistently performs better than PARPP, especially in the case of the Top15%. Compared to the second best model CYAN-RNN(cov), PARPP-CON reduces 10.60% relative errors on RMSE and reduces relative errors on ${\mathrm{MAPE}}_t$ by 31.7%, implying that the convergence strategy can better capture the impact of each answer on the problem.

An interesting finding in the table is that the performances of the Possion process and Hawkes process in RMSE are not too prominent, but they perform quite poorly in terms of the evaluation metric MAPE. We plot the change in MAPE in Figure 7. The x-axis is the observation time, and we make a time prediction every six hours. For brevity, we have only drawn the model PARPP-CON with the convergence of Top15% because it performs the best. At early stages, traditional point process methods, the Possion process and the Hawkes process, find it difficult to obtain accurate predictions. It may be that their assumptions about the underlying diffusion model do not fit the situation in CQA. On the contrary, the model based on the recurrent neural network works well, and our model is optimal.

5.2. Predict the Number of Answers

RQ2: We then test the performance of our model in predicting the number of answers in the future. We set 10 observation time points T, starting from the moment when the problem occurs, with the time interval varying from 1 day to 10 days. At each observation point, we enter the observed answers into the model and then use the resulting conditional intensity to predict the number of answers for the future. The window sizes ${\Delta }_{pred}$ we aim to predict are 1 to 10 days. Figure 8 shows the prediction results based on the mean absolute error per day for each model and Figure 9 based on ${\mathrm{MAPE}}_N$. As the plots for the three models, i.e., Hawkes, PARPP, and PARPP-CONT(Top15%), overlap each other in Figure 8a, which makes them hard to be distinguished, we zoom in the plots of these three models and show them in Figure 8b. The same plots applied to Figure 8c and Figure 9a,c, resulting in the zoomed-in figures, i.e., Figure 8d and Figure 9b,d, respectively. Because the naive method does not take into account the fact that the popularity of the problem will decline, its prediction performance is poor. For convenience of demonstration, we did not draw the results. Similarly, for the sake of brevity, we only draw the results of the PARPP-CON(Top15%) model, which performs the best in the previous prediction task.

From Figure 8 and Figure 9, we can see that the neural network methods RMTPP, CYAN-RNN and CYAN-RNN(cov), which did well in next-answer prediction, have very poor results in this prediction task. One important reason is that their models are trained to maximize the probability of the next occurrence rather than the number of answers in the future. Point processes learned by these models are more likely to predict the next point in time and lose the ability to predict the number of future events. Surprisingly, although our model is also trained with the maximum likelihood of the next event, in this prediction task, our model exhibits competitive performance compared to the traditional methods, which reflects the strong adjustment ability of our posterior mechanism. It can be seen that in most cases, the prediction effect of our model is the best, and the longer the observation time, the greater the advantage of our model. This is also what we expect. After all, the longer the observation time is, the more accurate the Bayesian criterion will be. Compared with the suboptimal model Hawkes, our best performing model, PARPP-CON(Top15%) can obtain up to 15.8% (18.9%) improvement on mean error and up to 20.8% (14.6%) improvement on ${\mathrm{MAPE}}_N$ in the case of ${\Delta }_{pred}=1$ day (10 days). These results demonstrate the power of our proposed model to achieve optimal results in predicting answer times while maintaining competitive results in predicting answer quantities.

5.3. Evaluation on Attention Mechanism

RQ3: We expect to check whether our proposed posterior attention mechanism and convergence strategy can capture high-quality answers. Ideally, the answer’s quality can be judged based on the number of votes it receives. Intuitively, the higher the vote number of the answer, the higher its ranking, and the greater the impact on the problem. However, since the Zhihu CQA does not provide a timestamp for each vote, we cannot obtain the dynamic change of the number of votes for the answers, so we use the number of comments on the answers instead to track their popularity. Whenever a new answer appears, we sort all the observed answers by their cumulative number of comments at this time point and then select the top-k answers. We consider these top-k answers as the answers that have the biggest impact on the issue at this moment and treat them as ground true. Then, we compare the answers of the attention distribution top-k in the model with ground truth and calculate the number of overlaps. We use P@k (precision at k) and NDCG@k (normalized discounted cumulative gain at k) as evaluation metrics.

Table 5. Performance of attention mechanism. The table shows the performance of the attention mechanism in aligning the top-k’s most popular answers. Scores in bold indicate the best performance per column.

	P@		NDCG@
Model	10	20	10	20
CYAN-RNN	0.122	0.231	0.107	0.199
CYAN-RNN(cov)	0.115	0.215	0.109	0.191
PARPPprior	0.135	0.241	0.114	0.206
PARPP-CON(Top1%)prior	0.157	0.262	0.109	0.196
PARPP-CON(Top5%)prior	0.159	0.241	0.112	0.203
PARPP-CON(Top10%)prior	0.159	0.252	0.113	0.206
PARPP-CON(Top15%)prior	0.170	0.247	0.120	0.216
PARPP-CON(Top20%)prior	0.170	0.239	0.114	0.208
PARPPpostr	0.155	0.251	0.115	0.209
PARPP-CON(Top1%)postr	0.156	0.262	0.110	0.199
PARPP-CON(Top5%)postr	0.157	0.246	0.115	0.212
PARPP-CON(Top10%)postr	0.166	0.268	0.115	0.210
PARPP-CON(Top15%)postr	0.171	0.267	0.119	0.220
PARPP-CON(Top20%)postr	0.170	0.248	0.113	0.206

shows the results. From the table, we can see that our model can be more accurately aligned with the heat of the answer than the CYAN-RNN model, which also uses the attention mechanism. We also compared the performance of prior attention and posterior attention on alignment. It can be seen that posterior attention is generally better than prior attention in alignment, which is also in line with our expectations. From the table, we can also see that the models that perform well in the two prediction tasks, PARPP-CON(Top10%) and PARPP-CON(Top15%), also work well in alignment. This confirms our hypothesis that popular answers have a greater impact on the question and also confirms the effectiveness of the convergence strategy.

6. Conclusions

In this paper, we studied the problem of predicting popularities of questions in Community Question Answering (CQA): given a question and its historical answers, predict the time of the next arriving answer and the number of answers at a particular point in time. To tackle this problem, we propose a Posterior Attention Recurrent Point Process (PARPP) model that takes advantages of RNNs’ abilities to model cascade conveniently and effectively, and we add a posterior attention mechanism to better capture the impact of each answer on the problem. In addition, in CQA, due to the interaction of users, the ranking of answers has an obvious Matthew effect, that is, early and good-quality answers more easily obtain a better ranking, and the advantages of ranking can make them more easily noticed by users and accumulate these advantages. In order to capture the impact of the Matthew effect in CQA on the popularity of the problem, we further proposed a convergence strategy. We evaluate the effectiveness of our proposed models on a real dataset crawled from the Zhihu platform. The experimental results demonstrate that our proposed models can consistently outperform state-of-the-art methods when predicting the time of the next answer and yield competitive performance when predicting the number of answers in the future. Additionally, our revised model, PARPP-CON, performs consistently better than PARPP, implying that the convergence strategy can effectively capture the impact of the Matthew effect on the question.

Limitations of this study include the following: (1) We only take into account the textual information of the answers for population prediction of the answers. Multimodal information that is naturally embedded in the answers such as images, number of thumbs-up, etc., should be taken into account as well. (2) We format the data of questions and their answers as sequential data and assume that questions to each other are independent. Formatting the questions and their answers as a social network sounds more reliable. (3) Most recent advances in deep learning such as large language models are not taken into account for the improvement of the question population prediction, and how to integrate and utilize large language models into our prediction task is unexplored in this paper.

As future work, besides the limitations that we take as future directions, there are still a number of unexplored avenues that are of value to investigated. Is our proposed model still effective at predicting popularities in platforms other than Zhihu? We will conduct experiments on more CQA datasets to investigate and test if the proposed model still outperforms the state-of-the-art models. In addition, the proposed model simply encodes text information with BERT [82]. How to effectively extract rich text information in CQA and take them as input to the proposed model requires more exploration. How to take social network information into consideration is also another direction worth exploring. We are planning to take GPT-based models as baselines in our experiments for comparisons. We are also planning to leverage external resources such as the news articles published in news online platforms such as BBC News and Toutiao to boost questions’ popularity prediction performance. Finally, we intend to simulate a dataset and conduct experiments on this simulation dataset to further explore the performance of our proposed model.