Event-Based Probabilistic Embedding for POI Recommendation

Abstract

Location-based social networks (LBSNs) have collected massive geo-tagged information, enabling the derivation of user preference for point of interests (POIs) in support of personalized recommendation. The existing embedding techniques deal with multiple factors by embedding a separate model for each factor. As a result, the interaction amongst various factors cannot be captured properly. In addition, we notice that the effectiveness of personalized recommendation is closely related to the current time and location. It is obvious that users would check into a POI which fits their interests, even if the current location is far away from the POI or the time is inappropriate. Therefore, it is necessary to recommend the right POI according to the time and geographic location of the user. In other words, it is necessary to predict the most likely visiting event, including users, POI, event time, and event location. In this paper, we propose a probabilistic embedding model called Topic And Region Embedding (TARE), which embeds events by simulating the users’ decision-making process. The results of TARE not only take various factors and their interaction into consideration but also consider the time and geographic location of events. Extensive experiments on three location-based social network datasets show that TARE achieves better performance in recommendation accuracy than existing state-of-the-art methods.

1. Introduction

The development of location-based social networks (LBSNs) enables the easy acquisition of massive geo-tagged information which can be used in the derivation of user preference for point-of-interests (POIs) to provide personalized recommendations. POI recommendation [1] aims to model user behavior in support of future prediction regarding which POIs to visit. The behavior can be regarded as a synthesized activity influenced by various factors including geographical effect, temporal effect, user preference, POI categories, and so on. For example, for a person who usually goes to a punk-style bar near his home at night, his behavior may be subject to time (the user likes night activities), user preferences (the user likes to drink), geographic location (the user likes to visit places near home), and POI category (the user likes punk-style POIs).

To tackle the POI recommendation problem, traditional Matrix Factorization (MF)-based recommendation algorithms [2,3,4,5,6,7] decompose the check-in matrix into the user–topic matrix and the topic–POI matrix. Then, the user–topic matrix multiplies the topic–POI matrix to fill the missing values in the check-in matrix. Accordingly, a user’s behavior is predicted by using the filled-in value in the check-in matrix. However, the users’ behavior in the real world is complicated. The linear models based on MF take various influential factors into consideration, but the nature of a linear model limits its performance in predicting the user’s behavior pattern where the various factors can interact with each other.

To address this issue, a growing number of researchers have integrated deep learning techniques into recommendation systems, e.g., PACE [8], POI2Vec [9], Geo-teaser [10] and so on [11,12,13,14]. The major ideas behind these systems include two steps. The first step is to obtain user and POI embeddings. The second step takes the POI and user embedding into a non-linear model [8] or a linear model [15] for further joint training. There are two main types of embedding algorithms. The first one is a variant of the word2vec algorithm [9,10]. It generates the POI sequence according to the check-in time, and then, it uses the POI sequence to embed the POI. The other one is a variant of the node2vec algorithm [8,16,17]. It constructs a graph of users and POIs, such as the user’s social network graph, the geographical graph of the POI, and so on, in order to use the graphs to perform POI embedding.

Nevertheless, these existing embedding methods have certain deficiencies. Firstly, the existing embedding methods consider multiple factors by embedding a separate model for each factor without exploring the intrinsic relationship between different factors. Notice that the intrinsic relationship between different factors has a great impact on the user’s behavior. For example, users may like to visit a POI near their home at night while preferring to visit a POI near their work location during the day (time affects the user’s geographic preferences). Second, we notice that the effectiveness of personalized recommendation is closely related to the current time and location. The certain time and user history check-in records can be used to infer the user preferences at event time. Event locations can filter out POIs that are too far away and thus improve the accuracy of user behavior predictions. So, the embedding methods should take into account the time and geographic location for better recommendation performance.

To address the above issues in existing methods, we propose a novel embedding method called Topic And Region Embedding (TARE), which takes into account a variety of factors including geographical effect, temporal effect, user preference and POI categories. Unlike the existing embedding method, the TARE embedded object is not a user or a POI but an event that includes user, POI, event time and event location. The results obtained by TARE not only take into account user preferences and POI information but also event time and event locations.

To explore the intrinsic link between multiple factors, we use the probability generation model to embed events. The basic idea is to “mimic” user behaviors in a process of decision making. So, we use a hidden variable topic to explore the joint impact of POI category and time factors on the user behavior and use a hidden variable region to explore the joint impact of geographic location and time factors on user behavior. As region and topic are interdependent, TARE combines the time factor, geographic factor, POI category, and user preference into a unified model.

The main contributions of this work are summarized as follows:

• We propose a novel embedding method called TARE which takes into account a variety of factors including geographical effect, temporal effect, user preference and POI categories. In addition, TARE is also able to explore the intrinsic link between time and geographic factors and the intrinsic link between time factors and POI categories to improve the accuracy of recommendations.
• The objects embedded in TARE are events. Compared with the existing embedding methods, TARE not only considers user preference and POI information but also time and location of the event. To the best of our knowledge, this is the first attempt to explore event-based embedding in POI recommendation.
• We conduct extensive experiments on three real-world datasets to evaluate the performance enhancement in recommendation accuracy of TARE, and our approach achieves up to 12.39% performance gains in recommendation accuracy compared to existing state-of-the-art methods.

2. Topic And Region Embedding

In this section, we first formulate the problem and then present our proposed embedding method TARE (Topic And Region Embedding).

2.1. Preliminary

Definition 1

(POI). A POI is a uniquely identified site (e.g., a bar, gym or restaurant). In our model, a POI has three attributes: identifier, location and category. We use v to represent a POI identifier and $l_v$ to denote its corresponding geographical attribute in terms of longitude and latitude coordinates. We use the notation $W_v$ to denote the set of its category.

Definition 2

(Check-in Activity). A user check-in activity is represented by a five-tuple $(u,v,l_v,W_v,t)$ denoting user u visits POI v at time t.

Definition 3

(Event). An event is represented by a four-tuple $(u,v,t_q,l_q)$ denoting user u to visit POI v at time $t_q$ at location $l_q$.

Definition 4

(User Profile). For each user u, we create a user profile $D_u$, which consists of a set of check-in activities associated with u. The dataset D used in our model consists of user profiles, i.e., $D=\{D_u:u\in U\}$.

Problem 1

(Event-based Embedding). Given a check-in activity dataset D and an event $e=(u,v,t_q,l_q)$, our goal is to embed e according to the u’s profile, $l_q$, $t_q$, the category $W_v$ and location $l_v$ of POI v.

2.2. Model Description

The POI recommendation is mainly used to predict user behavior. In order to handle POI recommendations better, we analyze which factors influence user behavior and how these factors affect user behavior. Figure 1 shows the decision-making process of users. An intuitive factor that affects user behavior is time. Furthermore, the time affects the activity type of a user which progressively affects the user’s geographic preferences and POI category preferences. For example, users are inclined to decide which place to eat between 12:00 p.m. and 1:00 p.m. Meanwhile, geographic information and POI category are also affecting the decision. Users may choose an appropriate POI geographic location and POI category according to personal preference and current location. Finally, users select a POI to visit based on activity type, POI location and POI category.

According to the above process, this paper proposes a new embedding method called Topic And Region Embedding (TARE), which embeds events according to user preferences, event time, event location and POI by simulating the user decision process. TARE divides the factors that influence user behavior into four categories: geographic information, time factor, POI category, and personal preference. User behavior has a similar periodic pattern, and this is manifested in the fact that users always visit similar POIs at specific times [18]. In other words, time factors affect user behavior all the time. To this end, TARE uses two hidden variables (r, z) to discover the relationship between time, POI category and geographic information.

Table 1. Notations of parameters.

Symbol	Description
$N,V,R,Z$	The number of users, POI, region, topic
$D_u$	The profile of user u
$v_{u,i}$	The POI of i-th record in $D_u$
$l_{u,i}$	The location of POI $v_{u,i}$
${\theta }_{z|u},{\theta }_{r|u}$	The interest of user u, distribution over topic, region
${\theta }_{w|z}$	The topic z’s distribution over POI categories
${\mu }_r,{\Sigma }_r$	The mean location, location convariance of region r
${\theta }_{v|r,z},{\theta }_{t|r,z}$	The region r and topic z distribution over POI and time

lists the notations of our model.

Table 2. The abbreviation and full name of concepts.

Abbreviation	Full Name
LBSNs	Location-Based Social Networks
POIs	Point Of Interests
MF	Matrix Factorization
SGD	Stochastic Gradient Descent
PSSG	Projected Scaled Sub-Gradient
TARE	Topic And Region Embedding

lists the abbreviation and full name of different concepts in this paper. Figure 2 shows the graphical representation of TARE. TARE consists of two parts: embedding of events and prediction of event occurrence, which will be described below.

2.3. Embedding of Events

Users’ time preferences and category preferences often interact with each other. For example, people who like midnight activities often like to go to the bar. To this end, we explore the interaction between time preferences and category preferences through the hidden variable topic z. Technically, each topic z in our model is not only associated with a multinomial distribution ${\theta }_{w|z}$ over categories of POIs but also related with a multinomial distribution over a POI’s identifier ${\theta }_{v|r,z}$ and a check-in time ${\theta }_{t|r,z}$. Users liking a topic z indicates that the time preference and category preference corresponding to topic z are in line with the users’ preference. So, we use a user’s distribution ${\theta }_{z|u}$ of interest on a set of topics to simulate a user’s interest in topic z.

In addition, we use the hidden variable region r to explore the interaction between time preference and geographical preference. Unlike Wang et al. [19], we use two hidden variables (topic z and region r) instead of just one hidden variable, because using two hidden variables has the following advantages. Firstly, the use of two hidden variables can better explain the meaning of the model. The two hidden variables, respectively, represent the geographical regions and semantic topics. Second, using two hidden variables can make the model perform better with the same number of free variables. Similar to topic z, each region r is associated with a multinomial distribution over the POI’s identifier ${\theta }_{v|r,z}$, a check-in time ${\theta }_{t|r,z}$ and a Gaussian distribution $N({\mu }_r,{\Sigma }_r)$. Then, we use N distributions ${\theta }_{r|u}$ to represent each user’s preference for r. Finally, we model the user decision process as shown in Algorithm 1, and the time complexity of Algorithm 1 is $O\left(\right|D_u|\ast |W_v\left|\right)$.

Algorithm 1 Modeling user decision process.

1: for each $D_u$ in D do 2:  for the each check-in record (u, v, $l_v$, $W_v$, t) in $D_u$ do 3:   Draw a topic index z ∼ Multi (${\theta }_{z|u}$) 4:   Draw a region index r ∼ Multi (${\theta }_{r|u}$) 5:   Draw a POI v ∼ Multi (${\theta }_{v|r,z}$) 6:   Draw a time t ∼ Multi (${\theta }_{t|r,z}$) 7:   for each word $w\in W_v$ do 8:    Draw w ∼ Multi (${\theta }_{w|z}$) 9:   end for 10:   Draw a location $l_v$ ∼ N (${\mu }_r$, ${\Sigma }_r$) 11:  end for 12: end for

To incorporate the check-in time information to the topic discovery process, we employ the widely adopted discretization method by [20] to split a day into hourly-based slots. The joint probability is characterized by $P(u,v,l_v,W_v,t,z,r)$ as follows.

$$\begin{array}{c}\hfill P(u,v,l_v,W_v,t_q,z,r)={\theta }_{z|u}^{\left(z\right)}{\theta }_{r|u}^{\left(r\right)}{\theta }_{t|r,z}^{\left(t_q\right)}{\theta }_{v|r,z}^{\left(v\right)}p\left(l_v\right|{\mu }_r,{\Sigma }_r)\prod _{w_i\in W_v}{\theta }_{w|z}^{w_i}\end{array}$$

(1)

where ${\theta }_{z|u}^i$ is the probability that user u generates topic i, ${\theta }_{r|u}^i$ is the probability that user u generates region i, ${\theta }_{t|r,z}^i$ is the probability that region r and topic z jointly generate time i, ${\theta }_{v|r,z}^i$ is the probability that region r and topic z jointly generate POI i and ${\theta }_{w|z}^{w_i}$ is the probability that topic z generates word $w_i$.

The probability that POI v appears in region r is characterized by $P\left(l_v\right|{\mu }_r,\Sigma )$ as follows.

$$\begin{array}{c}\hfill p\left(\overrightarrow{l_v}\right|\overrightarrow{{\mu }_r},{\Sigma }_r)=\frac{1}{2\pi \sqrt{{\Sigma }_r}}exp\left(\frac{-(\overrightarrow{l_v}-{\overrightarrow{{\mu }_r)}}^T{\Sigma }_r^{-1}(\overrightarrow{l_v}-\overrightarrow{{\mu }_r})}{2}\right)\end{array}$$

(2)

where ${\mu }_r$ and ${\Sigma }_r$ denote the mean vector and covariance matrix, respectively.

The event-embedding result is characterized by $E_{u,v,t_q,l_q}$ as follows.

$$\begin{array}{cc}\hfill E_{u,v,t_q,l_q}& =[E_r^{\left(1\right)},E_r^{\left(2\right)},\dots ,E_r^{\left(R\right)},E_z^{\left(1\right)},E_z^{\left(2\right)},\dots ,E_z^{\left(Z\right)}]\hfill \\ \hfill & =ReLU\left([P_r^{\left(1\right)},P_r^{\left(2\right)},\dots ,P_r^{\left(R\right)},P_z^{\left(1\right)},P_z^{\left(2\right)},\dots ,P_z^{\left(Z\right)}]\right)\hfill \end{array}$$

(3)

where $ReLU\left(x\right)=max(0,x)$.

Given event $e=[u,v,l_q,t_q]$, $P_r^{\left(i\right)}$ and $P_z^{\left(i\right)}$ are defined as:

$$\begin{array}{cc}\hfill & P_r^{\left(i\right)}={\theta }_{t_q|r}^{\left(i\right)}·{\theta }_{v|r}^{\left(i\right)}\ast {\theta }_{r|u}^{\left(i\right)}\ast p\left(\overrightarrow{l_v}\right|\overrightarrow{{\mu }_r},{\Sigma }_r)\ast p\left(\overrightarrow{l_q}\right|\overrightarrow{{\mu }_r},{\Sigma }_r)\hfill \\ \hfill & P_z^{\left(i\right)}={\theta }_{t_q|z}^{\left(i\right)}·{\theta }_{v|z}^{\left(i\right)}\ast {\theta }_{z|u}^{\left(i\right)}\ast \prod _{w_i\in W_v}{\theta }_{w|z}^{w_i}\hfill \end{array}$$

(4)

where · represents the dot product of two vectors, ${\theta }_{z|u}^i$ is the probability that user u generates topic i, ${\theta }_{r|u}^i$ is the probability that user u generates region i and ${\theta }_{w|z}^{w_i}$ is the probability that topic z generates word $w_i$.

In addition, ${\theta }_{t_q|r}^{\left(i\right)}$, ${\theta }_{v|r}^{\left(i\right)}$, ${\theta }_{t_q|z}^{\left(i\right)}$ and ${\theta }_{v|z}^{\left(i\right)}$ are defined as follows.

$$\begin{array}{cc}\hfill & {\theta }_{t|r}^{\left(i\right)}=[{\theta }_{t|i,z=1}^{\left(t\right)},{\theta }_{t|i,z=2}^{\left(t\right)},\dots ,{\theta }_{t|i,z=Z}^{\left(t\right)}]\hfill \\ \hfill & {\theta }_{v|r}^{\left(i\right)}=[{\theta }_{v|i,z=1}^{\left(v\right)},{\theta }_{v|i,z=2}^{\left(v\right)},\dots ,{\theta }_{v|i,z=Z}^{\left(v\right)}]\hfill \\ \hfill & {\theta }_{t|z}^{\left(i\right)}=[{\theta }_{t|r=1,i}^{\left(t\right)},{\theta }_{t|r=2,i}^{\left(t\right)},\dots ,{\theta }_{t|r=R,i}^{\left(t\right)}]\hfill \\ \hfill & {\theta }_{v|z}^{\left(i\right)}=[{\theta }_{v|r=1,i}^{\left(v\right)},{\theta }_{v|r=2,i}^{\left(v\right)},\dots ,{\theta }_{v|r=R,i}^{\left(v\right)}]\hfill \end{array}$$

(5)

where ${\theta }_{t|r,z}^i$ is the probability that region r and topic z jointly generate time i and ${\theta }_{v|r,z}^i$ is the probability that region r and topic z jointly generate POI i.

2.4. Prediction of Event Occurrence

We feed the embedded results to the deep neural network. At the time $t_q$, the prediction of user u’s preference for POI v at location $l_q$ is as follows.

$$\begin{array}{c}\hfill {\widehat{y}}_{u,v,t_q,l_q}=\mathcal{H}\left(E_{u,v,t_q,l_q}\right|{\theta }_{neural})\end{array}$$

(6)

where $E_{u,v,t_q,l_q}$ is the embedding result of the event, ${\theta }_{neural}$ denotes the parameters in the hidden layers and ${\widehat{y}}_{u,v,t_q,l_q}$ is user u’s preference for POI v at location $l_q$ and time $t_q$.

Given the input feature vector x, the q-th hidden layer is denoted as $h^q$, which is a non-linear function of the previous hidden layer $h^{q-1}$ defined as:

$$h^q\left(x\right)=ReLu(W^q{h}^{q-1}\left(x\right)+b^q)$$

(7)

where $W^q$ and $b^q$ are parameters of the q-th layer. We adopt the rectified linear unit $ReLU\left(x\right)=max(0,x)$ as the non-linear function.

By combining Equation (7), the hidden layers can be defined as follows.

$$\begin{array}{cc}\hfill x_{pre}& =H\left(E_{u,v,t_q,l_q}\right)\hfill \\ & =h^Q(\dots h^1\left(h^0\left(E_{u,v,t_q,l_q}\right)\right)\dots )\hfill \end{array}$$

(8)

Due to the one-class nature of implicit feedback in POI recommendation, we first connect a binary softmax layer on the top of H, which is basically a logistic regression with sigmoid function, so we have:

$$\begin{array}{cc}\hfill {\widehat{y}}_{u,v,t_q,l_q}& =\mathcal{H}\left(E_{u,v,t_q,l_q}\right|{\theta }_{neural})\hfill \\ \hfill & =\sigma \left(H{\left(E_{u,v,t_q,l_q}\right)}^T{w}_y\right)\hfill \end{array}$$

(9)

where the sigmoid function is defined as $\sigma \left(x\right)=1/(1+e^{-x})$.

3. Model Inference and Learning

Our goal with model inference is to learn the parameters that maximize the likelihood of observed random variables and minimize the loss function of neural network. TARE’s optimization goal can be expressed as follows.

$$\begin{array}{c}\hfill \mathcal{J}={\mathcal{J}}_{PGM}+{\mathcal{J}}_{neural}\end{array}$$

(10)

where ${\mathcal{J}}_{PGM}$ is the optimization target of event embedding and ${\mathcal{J}}_{neural}$ is the loss function of the neural network.

The ${\mathcal{J}}_{PGM}$ is defined as follows.

$$\begin{array}{cc}\hfill & {\mathcal{J}}_{PGM}\hfill \\ & =-P(\overrightarrow{z},\overrightarrow{r},\overrightarrow{w},\overrightarrow{t},\overrightarrow{v}|\overrightarrow{u},{\theta }_{embedding})\hfill \\ \hfill & =-P\left(\overrightarrow{z}\right|\overrightarrow{u},{\theta }_z)P\left(\overrightarrow{r}\right|\overrightarrow{u},{\theta }_r)P\left(\overrightarrow{t}\right|\overrightarrow{r},\overrightarrow{z},{\theta }_t)P\left(\overrightarrow{v}\right|\overrightarrow{r},\overrightarrow{z},{\theta }_v)P\left(\overrightarrow{w}\right|\overrightarrow{z},{\theta }_w)P\left(\overrightarrow{l}\right|\overrightarrow{\mu },\overrightarrow{\Sigma })\hfill \\ \hfill & =-\prod _{u=1}^N\prod _{i=1}^{|D_u|}\prod _{w\in W_{v_{u,i}}}{\theta }_{w|z_{u,i}}^{\left(w\right)}{\theta }_{z|u}^{\left(z_{u,i}\right)}{\theta }_{r|u}^{\left(r_{u,i}\right)}{\theta }_{t|r_{u,i},z_{u,i}}^{\left(t_{u,i}\right)}{\theta }_{v|r_{u,i},z_{u,i}}^{\left(v_{u,i}\right)}p\left(l_{v_{u,i}}\right|{\mu }_{r_{u,i}},{\Sigma }_{r_{u,i}})\hfill \end{array}$$

(11)

The loss function ${\mathcal{J}}_{neural}$ of the neural network is defined as follows.

$$\begin{array}{c}\hfill {\mathcal{J}}_{neural}=\sum _{u=1}^N\sum _{i=1}^{|D_u|}∥y_{u,v_{u,i},t_{u,i},l_q}-\mathcal{H}\left(E_{u,v_{u,i},t_{u,i},l_q}\right)∥+\lambda {∥W∥}_F^2\end{array}$$

(12)

where $D_u$ represents the set of $u$’s check-in activities, and $|D_u|$ represents the number of $u$’s check-in activities. We use L2 regularization to prevent overfitting.

However, it is difficult to minimize $\mathcal{J}$. Therefore, we apply the Gibbs EM algorithm [21], which is a mixture between EM and a Monte Carlo sampler, to minimize $\mathcal{J}$. In the E-step, we sample regions and topics by using Gibbs sampling. In the M-step, we optimize the model parameters ${\theta }_{neural}$ and ${\theta }_{PGM}$ by fixing all topic and region assignments. The two steps are conducted repeatly until convergence.

More specifically, we draw region r and topic z for each check-in activity in the E-step. We use Gibbs sampling to sample region r and topic z according to Equations (13) and (14). When we sample region r, we assume all other variables are fixed. $r_{u,i}$ represents the region assignments for all user activities except the $i$th activity of user u, and the sampling topic z is similar to the sampling region r.

$$\begin{array}{cc}\hfill & P\left(z_{u,i}\right|z_{\neg u,i},\overrightarrow{u},\overrightarrow{v},\overrightarrow{w},\overrightarrow{l},\overrightarrow{t})\hfill \\ \hfill & \propto \sum _{r=1}^R{\theta }_{z|u}^{\left(z_{u,i}\right)}\ast {\theta }_{r|u}^{\left(r\right)}\ast {\theta }_{t|r,z_{u,i}}^{\left(t_{u,i}\right)}\ast {\theta }_{v|r,z_{u,i}}^{\left(v_{u,i}\right)}\ast p\left(l_{v_{u,i}}\right|{\mu }_r,{\Sigma }_r)\ast \prod _{w_i\in W_v}{\theta }_{w|z_{u,i}}^{w_i}\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill & P\left(r_{u,i}\right|r_{\neg u,i},\overrightarrow{u},\overrightarrow{v},\overrightarrow{w},\overrightarrow{l},\overrightarrow{t})\hfill \\ \hfill & \propto \sum _{z=1}^Z{\theta }_{z|u}^{\left(z\right)}\ast {\theta }_{r|u}^{\left(r_{u,i}\right)}\ast {\theta }_{t|r_{u,i},z}^{\left(t_{u,i}\right)}\ast {\theta }_{v|r_{u,i},z}^{\left(v_{u,i}\right)}\ast p\left(l_{v_{u,i}}\right|{\mu }_{r_{u,i}},{\Sigma }_{r_{u,i}})\ast \prod _{w_i\in W_v}{\theta }_{w|z}^{w_i}\hfill \end{array}$$

(14)

In the M-step, we optimize the parameters ${\theta }_{neural}$ and ${\theta }_{PGM}$ to minimize $\mathcal{J}$ with all topic and region assignments fixed. We use Stochastic Gradient Descent (SGD) to update the parameters ${\theta }_{neural}$ and ${\theta }_{PGM}$ for minimizing ${\mathcal{J}}_{neural}$.

There are no negative samples in the training data because this is one-class problem. So, we generate negative samples by negative sampling. For each positive sample, we generate five negative samples by randomly changing one of the elements (u, v, $t_q$ or $l_q$) in positive samples. Due to space limitations, the gradient of the weight of the neural network is not listed, and the gradients of model parameters ${\theta }_{t|r,z}^t$, ${\theta }_{v|r,z}^v$, ${\theta }_{r|u}^r$, ${\theta }_{z|u}^z$ and ${\theta }_{w|z}^w$ are provided as follows.

$$\begin{array}{cc}\hfill & \frac{\partial {\theta }_{neural}}{\partial {\theta }_{t|r,z}^t}=\frac{\partial {\theta }_{neural}}{\partial P_{rz}}\frac{\partial P_{rz}}{\partial {\theta }_{t|r,z}^t}\hfill \\ \hfill & ={\delta }_{E,r}^{\left(r\right)}{\theta }_{v|r,z}^{\left(v\right)}P\left(l_v\right|{\mu }_r,{\Sigma }_r){\theta }_{r|u}^{\left(r\right)}+{\delta }_{E,z}^{\left(z\right)}{\theta }_{v|r,z}^{\left(v\right)}\prod _{w_i\in W_v}{\theta }_{w|z}^{\left(w_i\right)}{\theta }_{z|u}^{\left(z\right)}\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill & \frac{\partial {\theta }_{neural}}{\partial {\theta }_{v|r,z}^t}=\frac{\partial {\theta }_{neural}}{\partial P_{rz}}\frac{\partial P_{rz}}{\partial {\theta }_{v|r,z}^t}\hfill \\ \hfill & ={\delta }_{E,r}^{\left(r\right)}{\theta }_{t|r,z}^{\left(t\right)}P\left(l_v\right|{\mu }_r,{\Sigma }_r){\theta }_{r|u}^{\left(r\right)}+{\delta }_{E,z}^{\left(z\right)}{\theta }_{t|t,z}^{\left(t\right)}\prod _{w_i\in W_v}{\theta }_{w|z}^{\left(w_i\right)}{\theta }_{z|u}^{\left(z\right)}\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill & \frac{\partial {\theta }_{neural}}{\partial {\theta }_{r|u}^r}=\frac{\partial {\theta }_{neural}}{\partial P_{rz}}\frac{\partial P_{rz}}{\partial {\theta }_{r|u}^r}={\delta }_{E,r}^{\left(r\right)}({\theta }_{t|r}^r·{\theta }_{v|r}^{\left(r\right)})P\left(l_v\right|{\mu }_r,{\Sigma }_r)\hfill \end{array}$$

(17)

$$\frac{\partial {\theta }_{neural}}{\partial {\theta }_{z|u}^z}=\frac{\partial {\theta }_{neural}}{\partial P_{rz}}\frac{\partial P_{rz}}{\partial {\theta }_{z|u}^z}={\delta }_{E,z}^{\left(z\right)}({\theta }_{t|z}^{\left(z\right)}·{\theta }_{\left(v\right|z)}^{\left(z\right)})\prod _{w_i\in W_v}{\theta }_{w|w}^{\left(w_i\right)}$$

(18)

$$\begin{array}{cc}\hfill & \frac{\partial {\theta }_{neural}}{\partial {\theta }_{w|z}^w}=\frac{\partial {\theta }_{neural}}{\partial P_{rz}}\frac{\partial P_{rz}}{\partial {\theta }_{w|z}^w}\hfill \\ \hfill & ={\delta }_{E,z}^{\left(z\right)}({\theta }_{t|z}^{\left(z\right)}·{\theta }_{\left(v\right|z)}^{\left(z\right)}){\theta }_{z|u}^z\prod _{w_i\in W_v\&w_i\ne w}{\theta }_{w|z}^{\left(w\right)}\hfill \end{array}$$

(19)

where $P_{rz}=[P_r^{\left(1\right)},P_r^{\left(2\right)},\dots ,P_r^{\left(R\right)},P_z^{\left(1\right)},P_z^{\left(2\right)},\dots ,P_z^{\left(Z\right)}]$, ${\delta }_E$ is the residual of the embedded layer E, ${\delta }_{E,r}^i$, ${\delta }_{E,z}^i$ represents that the residual of $E_r^i$, $E_z^i$. ${\delta }_E$ can be calculated by neural network back-propagation. Due to space limitations, the derivation process of ${\delta }_E$ is not listed.

To update the parameters ${\theta }_{PGM}$ for maximizing ${\mathcal{J}}_{PGM}$, we use the gradient descent learning algorithm Projected Scaled Sub-Gradient (PSSG) [22], which is designed to solve optimization problems with L1 regularization on parameters. The gradients of model parameters ${\theta }_{t|r,z}^t$, ${\theta }_{v|r,z}^v$, ${\theta }_{r|u}^r$, ${\theta }_{z|u}^z$ and ${\theta }_{w|z}^w$ are provided as follows.

$$\begin{array}{ccc}\hfill \frac{\partial {\theta }_{PGM}}{\partial {\theta }_{t|r,z}^t}& =& d(t,z,r)-\sum _{j=1}^{|D_t|}{\theta }_{z|u_j}^{\left(z\right)}{\theta }_{r|u_j}^{\left(r\right)}{\theta }_{t|r,z}^t\hfill \end{array}$$

(20)

$$\begin{array}{ccc}\hfill \frac{\partial {\theta }_{PGM}}{\partial {\theta }_{v|r,z}^v}& =& d(v,z,r)-\sum _{j=1}^{|D_v|}{\theta }_{z|u_j}^{\left(z\right)}{\theta }_{r|u_j}^{\left(r\right)}{\theta }_{v|r,z}^v\hfill \end{array}$$

(21)

$$\begin{array}{ccc}\hfill \frac{\partial {\theta }_{PGM}}{\partial {\theta }_{r|u}^r}& =& d(u,r)-{\theta }_{r|u}\ast N_u\hfill \end{array}$$

(22)

$$\begin{array}{ccc}\hfill \frac{\partial {\theta }_{PGM}}{\partial {\theta }_{z|u}^z}& =& d(u,z)-{\theta }_{z|u}\ast N_u\hfill \end{array}$$

(23)

$$\begin{array}{ccc}\hfill \frac{\partial {\theta }_{PGM}}{\partial {\theta }_{w|z}^w}& =& d(z,w)-d\left(z\right){\theta }_{w|z}^{\left(w\right)}\hfill \end{array}$$

(24)

where $d(t,z,r)$ is the number of activities assigned to topic z and region r on time t; $d(v,z,r)$ is the number of activities where the POI v is assigned to topic z and region r; $d(u,r)$, $d(u,z)$ is the number of u’s activities which are assigned to topic r and region z; $d(z,w)$ is the number of activities where the word w is assigned to topic z; $d\left(z\right)$ is the number of activities which is assigned to topic z; $D_t$ denotes the set of activities occurring on time t; $D_v$ denotes the set of activities occurring at POI v; and $N_u$ is the number of u’s activities.

For geographical modeling, the parameters $\overrightarrow{\mu }$ and $\overrightarrow{\Sigma }$ can be obtained in closed form as Equations (25) and (26).

$$\begin{array}{c}\hfill {\mu }_r=\frac{1}{d\left(z\right)}\sum _{j=1}^{|D_z|}{\overrightarrow{l}}_j\end{array}$$

(25)

$$\begin{array}{c}\hfill {\Sigma }_r=\frac{1}{d\left(z\right)-1}\sum _{j=1}^{|D_z|}{(\overrightarrow{l_j}-\overrightarrow{{\mu }_r})}^T(\overrightarrow{l_j}-\overrightarrow{{\mu }_r})\end{array}$$

(26)

where $D_z$ represents the set of check-in activities assigned to region r and $\overrightarrow{l_j}$ is the location where the $j$th activity in $D_z$ occurs.

4. Experiments

In this section, we evaluate our model for POI recommendation with extensive experiments on real-world check-in datasets.

4.1. Dataset

Gowalla. The Gowalla check-in dataset (http://snap.stanford.edu/data/loc-gowalla.html, accessed on 12 January 2023) is generated from February 2009 to October 2010. We use the records generated within the United States and filter out those users with fewer than 20 check-ins and the POIs with fewer than 15 visitors. After filtering, the sparsity of the user–POI check-in matrix is 99.72%.

Foursquare. The Foursquare dataset (https://sites.google.com/site/yangdingqi/home/foursquare-dataset, accessed on 12 January 2023) includes check-in data from April 2012 to September 2013. We use the records generated within New York, Philadelphia and Trenton, and we filter those users with fewer than 10 check-ins as well as those POIs with fewer than 15 visitors. After filtering, the sparsity of the user–POI check-in matrix is 99.56%.

Yelp. The Yelp’s Challenge Dataset (Yelp dataset challenge round 7 (https://www.yelp.com/datasetchallenge, accessed on 20 February 2016) comes from the Yelp Dataset Challenge round 7 in 2016. The data contain a large number of geotagged businesses and reviews within several cities. We use the records generated within Phoenix and eliminate those users with fewer than 20 check-ins as well as those POIs with fewer than 15 visitors. The sparsity of the user–POI check-in matrix is 99.78%.

The statistics of the used data sets are shown in

Table 3. The statistics of the used data sets.

Dataset	#User	#POI	#Check-In	Sparsity
Gowalla	19,017	11,681	643,620	99.72%
Foursquare	7961	10,629	459,221	99.56%
Yelp	17,624	15,040	602,445	99.78%

. We partition each dataset into a training set and test set. For each user, we use the earliest 80% check-ins as the training data and the most recent 20% check-ins as the test data. Different learning rate schedule have a significant impact on performance [23]. so in order to ensure the fairness of the experiment, we apply constant learning rate for all models in the experiment. We implement TARE with Tensorflow (http://www.tensorflow.org/, accessed on 12 January 2023) for the training of our model.

4.2. Baselines for Comparison

We compare TARE with the following six state-of-the-art methods with well-tuned parameters.

• LRT: LRT [20] is a time-enhanced MF model based on observed temporal properties that characterize each user by different latent vectors for different time slots.
• GeoMF: GeoMF [4] is a geographical Weighted Matrix Factorization (WMF) model which integrates geographical influence by modeling users’ activity regions and the influence propagation on geographical space.
• JIM: JIM [24] is a joint probabilistic generative model that mimic users’ check-in behaviors by integrating the factors of temporal effect, content effect, geographical influence and word-of-mouth effect.
• TRM: TRM [25] is a unified probabilistic generative model that simultaneously discovers the semantic, temporal, and spatial patterns of users’ check-in activities, and it models their joint effect on users’ decision making for the selection of POIs to visit.
• PACE: PACE [8] is a general and principled combination of CF and SSL based on neural networks that model user preference for POIs. PACE considers both social effect and geographic effect. On the other hand, PACE is an embedding algorithm based on the node2vec algorithm.
• Geo-Teaser: Geo-Teaser [10] is a temporal POI-embedding model that captures the check-ins’ sequential contexts and the various temporal characteristics on different days. On the other hand, Geo-Teaser is an embedding algorithm based on the word2vec algorithm.

4.3. Evaluation Methods and Metrics

As for the performance comparison with baselines, we apply two widely used metrics, i.e., $Accuracy@k$ and $MAP@k$, where accuracy measures the number of correct recommendation in the result, while MAP considers the rank of the recommendation.

Particularly, for each test case $(u,v,l_v,W_v,t_q)$, we use Gaussian function to generate a coordinate $l_q$ within the circle of radius d centered at $l_v$ to represent the current standing point of user u. Thus, a query $q=(u,l_q,t_q)$ is formed for the test case. The set of all queries is represented as $S_{test}$.

For each query $(u,l_q,t_q)$ in $S_{test}$, we calculate the user preferences for POIs and generate a ranked list based on their preference scores. Let p denote the rank of the test item v within this list. Finally, we obtain a top-k recommendation list. If $p<k$, we have a hit (i.e., the test item v is recommended to the user).

The computation of $Accuracy@k$ proceeds as follows. For a single test, we define $hit@k$ as 1 when v appears in the top-k recommendation list; otherwise, $hit@k$ is 0. The overall $Accuracy@k$ is defined by averaging all test cases:

$$Accuracy@k=\frac{\#hit@k}{|S_{test}|}$$

(27)

where $\#hit@k$ denotes the number of hits in the test set, and $|S_{test}|$ is the number of all test cases.

$MAP@k$ is highly relative to the position of v in the list (i.e., p). When we have a hit, we focus on the value of p. $MAP@k$ is defined as follows:

$$MAP@k=\frac{{\sum }_{i=1}^{|S_{test}|}\frac{hit@k_i}{p_i}}{|S_{test}|}$$

(28)

4.4. Performance Analysis

In this part, we present the effectiveness of the recommendation methods with well-tuned parameters. Figure 3 show the performance under $Accuracy@k$ and $MAP@k$ on the Gowalla, Yelp and Foursquare datasets, respectively.

Although the relative performance among the baselines varies across different datasets and metrics, TARE outperforms other competitor models significantly. Compared with the strongest baseline, TARE yields approximately a 12.39% relative improvement in terms of $Accuracy@10$ and 10.14% relative improvement in terms of $MAP@10$. Such consistent improvements clearly demonstrate the strength of TARE over the baseline methods. Taking a closer look at the results, several observations are made: (1) Both LRT and GeoMF are based on the MF model, but the performances vary greatly. The result shows that temporal influence is more critical than geographical property for POI recommendation. (2) The experimental results show that the methods (TARE, TRM, JIM) which consider query time and query location are much better than other methods. Therefore, query time and query location are crucial to the accuracy of recommendation. (3) The embedded method PACE which considers geographic and social factors has a better performance than Geo-Teaser, which only considers geographical factors. The result shows that embedded methods which effectively consider a large number of factors are superior to those only considering a small number of factors. (4) Although TARE and TRM consider the same factors, TARE takes into account the intrinsic link between time and other factors (POI category and POI location). So, TARE can achieve better performance compared to TRM.

The main reasons why TARE is superior to other methods can be concluded as follows. First, the objects embedded in TARE are events. TARE takes full advantage of all the information in an event. Compared to TARE, other embedding methods only take advantage of a small portion of the information in an event. Second, TARE considers the intrinsic link between multiple factors. TARE explores the intrinsic link between time factors and POI category and the intrinsic relationship between time factors and geographic factors.

4.5. Impact of Different Parameters

We fix the number of hidden layers as 4 and the sizes of layers as $128\to 64\to 32\to 16$ for neural networks. However, the effectiveness of TARE is subject to the number of topics and regions (i.e., Z and R ). To this end, we study the effectiveness of event-based embedding by varying Z and R. The results are showen in

Table 4. Accuracy@10 with different #topic and #region.

Capacity	R = 20	R = 40	R = 60	R = 80	Z = 100
Z = 20	0.221	0.292	0.355	0.378	0.431
Z = 40	0.254	0.337	0.403	0.430	0.461
Z = 60	0.314	0.375	0.417	0.472	0.481
Z = 80	0.317	0.364	0.448	0.465	0.527
Z = 100	0.349	0.380	0.465	0.523	0.540

. From the results, we observe that the recommendation accuracy of TARE is first enhanced with an increasing number of topics, and then, it does not change drastically when the number of topics exceeds 80. Similar observations can be made for increasing the number of regions (i.e., R): the recommendation accuracy of TARE is imporved with an increasing number of regions, and then, it remains stable when the number of regions is larger than 80. The reason is that Z and R represent the model complexity. Thus, when Z and R are too small, the model has limited ability to describe the data. It should be noted that the performance reported in Figure 3 is achieved with $Z=100$ and $R=100$.

5. Releated Work

5.1. Latent Factor Model

The latent factor model (LFM) is also popular in recommender systems for both implicit and explicit feedback which assumes the existence of latent factor space and leverages matrix factorization to obtain low-dimensional matrices. Probabilistic matrix factorization (PMF) is developed to optimize regularized MF by introducing Gaussian prior. Many researchers try to improve and optimize models based MF and PMF. Cheng et al. [3] proposed PMFSR to integrate social influence into PMF with social regularization. In [20], Gao et al. proposed a Location Recommendation framework with Temporal effects (LRT), which incorporates temporal influence into an MF. In addition, Lian et al. proposed the GeoMF model [4] to incorporate geographical influence into a weighted regularized matrix factorization model (WRMF) [26,27]. However, MF-based methods’ computational performance will be significantly reduced facing large-scale data. Furthermore, MF-based methods rely on global calculation resulting in difficulty in incremental updates.

In addition, the probabilistic generative models [28] are also applied to POI recommendation due to their strong interpretability. Zhao et al. [29] proposed a unified sentiment–aspect–region model (SAR) to learn user preference based on the reviews, categories, and locations of POIs. However, it just obtains topical regions and topical aspects by applying the topic model and sets parameters manually to fuse different influence factors. JIM [24] is further developed later, which integrates the factors of temporal effect, content effect, geographical influence, and word-of-mouth effect, especially for out-of-town users. Different from the fused model, JIM is the joint model [30] which learns several influential factors together. LSARS [31] takes crowd sentiment into account to improve the effectiveness of JIM. However, the nature of the linear model limits the performance of the above model.

5.2. Deep Learning Model

Neural networks [32,33,34] are widely used in various fields, and have been proven to have superior performance through research and practice. In recent years, a growing number of researchers have integrated deep learning techniques into recommendation systems. These recommendation systems can basically be divided into two modules. The first module is an embedded module that embeds users or POIs. The second module is the preference learning module, which is recommended based on the results of the embedded module. Refs. [8,16,17,35,36] use the node2vec algorithm to embed users and POIs. It establishes different user networks and POI networks for various users and POIs by considering different factors. These networks are then embedded by the node2vec algorithm. However, the embedding methods based on node2vec cannot consider the intrinsic link between multiple factors. Refs. [9,10] treat the POI as a word and the POI sequence as a sentence. These methods use the word2vec algorithm to embed POI. They embed the POI by analyzing the POI sequence. Ref. [15] uses a convolutional neural network to embed documents and then apply the results to the MF. However, these methods make it difficult to consider other factors.

6. Conclusions

In this paper, we propose an event-based embedding for POI recommendation called TARE. Different from the existing embedding methods, TARE not only considers the influence of various factors on user behavior in the process of embedding but also considers the intrinsic relationship between various factors. In addition, TARE integrates the time and location of the event into the embedded processor to improve the accuracy of the recommendation. Experimental results show that TARE achieves up to 12.39% performance enhancement in recommendation accuracy compared to up-to-date solutions.