An Inter-Frame Forgery Detection Algorithm for Surveillance Video

Abstract

Surveillance systems are ubiquitous in our lives, and surveillance videos are often used as significant evidence for judicial forensics. However, the authenticity of surveillance videos is difficult to guarantee. Ascertaining the authenticity of surveillance video is an urgent problem. Inter-frame forgery is one of the most common ways for video tampering. The forgery will reduce the correlation between adjacent frames at tampering position. Therefore, the correlation can be used to detect tamper operation. The algorithm is composed of feature extraction and abnormal point localization. During feature extraction, we extract the 2-D phase congruency of each frame, since it is a good image characteristic. Then calculate the correlation between the adjacent frames. In the second phase, the abnormal points were detected by using k-means clustering algorithm. The normal and abnormal points were clustered into two categories. Experimental results demonstrate that the scheme has high detection and localization accuracy.

1. Introduction

Video sequences are often believed to provide stronger forensic evidence than still images. Thus, surveillance video, as important evidence, is often used in the case investigation. However, the digitization feature makes surveillance video easy to be manipulated. Tampering with a digital video without leaving visible clues is easily accomplished by using a video editing software, such as Adobe Premiere. Therefore, digital video forensics, which is designed to verify the trustworthiness of digital video, has become an important and exciting field for recent research. Katsaounidou et al. [1] introduced a framework of cross-media authentication and verification, as well as the values of cross-media authentication in journalism and judicial. Arab et al. [2] proposed a detection method for surveillance systems by embedding robust watermark in video. The algorithm was proven efficiency at detecting a wider range of tampering. However, embedding watermarks is not feasible sometimes in the tested video. Therefore, the detection algorithm which does not depend on prior information, for instance, detecting traces of forgery, has caught much attention in recent research.

Forgery detection for surveillance video can be divided into source authenticity and content authenticity. Source authenticity [3,4] is actually the analysis of the video “where it came from” and “how it came from”. Many approaches have been developed to investigate each of steps of the acquisition process [5]. While content authenticity studies whether the video has experienced some kinds of tamper operation. Forgery detection of video content includes double compression detection [6,7,8], intra-frame forgery detection [9,10,11,12] and inter-frame forgery detection, because the videos may be tampered with in various ways, including spatial tampering, temporal tampering and spatio-temporal tampering [13]. Mizher et al. [14] made a detailed classification and introduction to video falsifying techniques and video forgery detection techniques. In this paper, we focus on detecting inter-frame forgery for surveillance video, which is one of the most common ways for video tampering. The purpose is to imitate or to conceal a specific event by inserting or deleting certain frames.

The existing inter-frame forgery detection methods can be divided into two categories [15].

(1) Methods based on the periodic effect of double compression

Su et al. [16] indicated that the power of high frequency region of DCT coefficients block in the inter-frame forgeries shows a clear periodic artifact. The weakness of the algorithm is that it may only apply to MPEG-2. Dong et al. [17] proposed a motion-compensated edge artifact (MCEA) scheme to detect frame-based video manipulation, by judging spikes in the Fourier transform domain after double MPEG compression. Due to the fact that frame deletion or insertion would result in the frames moving from one GOP to another, and gives rise to relatively larger motion estimation errors. A machine learning approach to detect frame deletion is put forward [18]. A number of discriminative features, such as prediction residuals, percentage of intra-coded macroblocks, quantization scales and reconstruction quality, are extracted from the video bit stream and its reconstructed images. Then, machine learning techniques were used to detect frame deletion. But the method cannot provide the exact localization of the deleted frames. Feng et al. [19] proposed a method which is applicable to video sequences with variable motion strengths. They analyzed the statistical characteristics of the most common interfering frames, then exploit a new fluctuation feature based on frame motion residuals to identify frame deletion points. However, the disadvantage of such methods is that they depend on the encoding parameters of the tested video. In addition, the forger can achieve anti-forensics by correcting the prediction error [20].

(2) Methods based on the discontinuity of content at tampering position. These methods are insensitive to encoding parameters and have more advantages in practical applications

Chao et al. [21] utilized optical flow consistency between adjacent frames to detect frame forgery, since inter-frame forgery will disturb the optical flow consistency. For frame insertion and frame deletion forgery, the authors select different detection methods. However, we could not know in advance what kind of forgery was involved. The methods have similar ideas in [22,23], which velocity field consistency and motion vector pyramid (MVP) consistency were used respectively. In [24], a method based on quotients of correlation coefficients between local binary patterns (LBPs) coded frames is proposed. The abnormal point detection is achieved by using chebyshev inequality twice. The weakness is that it fails to discuss the selection of multiple parameters while different parameters have different detection precision. Zhang et al. [25] also used chebyshev inequality to locate the tampering position. They used a three-dimensional tensor to describe the video features, then the tensor was factorized by Tucker non-negative decomposition method. Finally, they extracted time dimension matrix to calculate correlation to determine whether there is a frame insertion or deletion forgery. Zhao et al. [26] proposed an algorithm to detect the frame-deleting forgery. The feature extraction based on the normalized mutual information feature, and make use of generalized ESD test to localize the tampering point. However, the method assumes that there is only one discontinuity point and could not detect multiple tampering points.

Since content-based detection methods do not rely on encoding standards, the approach have a more widely application, and has attracted the attention of scholars in recent years. However, as mentioned above, there are still some shortcomings in the current methods. In this paper, we propose a novel scheme for inter-frame forgery detection based on 2D phase congruency and k-means clustering. By means of k-means clustering analysis, the abnormal points can be accurately located. There is no need to select multiple thresholds to avoid the impact of threshold selection on the detection results. And it is also effective for multiple tampering. Because the surveillance video has static background, inter-frame forgery operation will reduce the correlation between adjacent frames at tampering position. Then, the consistency of the consecutive correlation coefficients is disturbed. Tampering localization could be achieved by detecting these discontinuous points, i.e., abnormal points. When calculating the correlation of adjacent frames, we use 2D phase congruency as the feature of frame since it is a good image characteristic. Furthermore, we employ k-means clustering analysis to cluster the normal and abnormal points into two categories.

The rest of the paper is organized as follows. In Section 2, the concept of 2D phase congruency and its feasibility are introduced. Section 3 describes the k-means clustering algorithm and the detection procedure for abnormal points. Section 4 gives our experimental results and discussion. Finally conclusions are drawn in Section 5.

2. Feature Extraction

Digital video is composed of sequences of still images or frames, and thus, it is also referred to as motion pictures. Therefore, some descriptors for digital image are also applicable to video. Due to the limited storage space of the monitoring equipment, usually the resolution of surveillance video is low. The 2D phase congruency is very sensitive to the edge and texture of the image, which can be used to describe the content of the surveillance video.

2.1. 2-D Phase Congruency

The authors of [27] provide a detailed introduction of 2D phase congruency. The Local Energy Model developed by Morrone et al. [28] postulated that the sharp features are perceived at points of maximum phase congruency in an image. Phase congruency (PC) was first defined by Morrone [29] in terms of the Fourier series expansion of a signal at some location x as:

$$PC\left(x\right)=\underset{\overline{\varnothing }\left(x\right)\in \left[0,2\pi \right]}{\max }\frac{{{\displaystyle \sum }}_n{A}_n\mathrm{c}\mathrm{o}\mathrm{s}\left({\varnothing }_n\left(x\right)-\overline{\varnothing }\left(x\right)\right)}{{{\displaystyle \sum }}_n{A}_n}$$

(1)

where $A_n$ is the amplitude of the nth Fourier component, ${\varnothing }_n\left(x\right)$ is the local phase of the nth Fourier component at position x, and $\overline{\varnothing }\left(x\right)$ is the weighted mean local phase angle at position x. If PC equals to a maximal value of 1, indicating that a noticeable signal change information is detected. As in the case of step edge in square wave. Otherwise, PC takes on values between 0 and 1, which denotes that the signal of each harmonic phase begins to be inconsistent. In this case, signal changes begin to relax, even no characteristics change. The principle can be extended to a two-dimensional (2D) image signal. When each harmonic phase has a high degree consistency, the image contains sharp features, such as edge and line.

Kovesi [30] extended the 1D PC to allow for the calculation of 2D PC of image by applying 1D analysis over several orientations and combining the results in some way. To calculate 2D PC of a given image, the image is first convolved with a bank of log-Gabor filters. Let the image denoted by I(x, y), the even-symmetric filter and odd-symmetric filter at scale s and orientation o denoted by $M_{so}^e$ and $M_{so}^o$, respectively. The responses of each quadrature pair of filters is a vector.

$$\left[e_{so}\left(x,y\right),o_{so}\left(x,y\right)\right]=\left[I\left(x,y\right)\ast M_{so}^e,I\left(x,y\right)\ast M_{so}^o\right]$$

(2)

where * is the convolution operator. From Equation (2), the amplitude and phase of this response is given by Equations (3) and (4):

$$A_{so}\left(x,y\right)=\sqrt{e_{so}^2\left(x,y\right)+o_{so}^2\left(x,y\right)}$$

(3)

$${\varnothing }_{so}\left(x,y\right)=\arctan \left(e_{so}\left(x,y\right),o_{so}\left(x,y\right)\right)$$

(4)

The 2D phase congruency is then calculated by:

$$PC\left(x,y\right)=\frac{{{\displaystyle \sum }}_o{{\displaystyle \sum }}_s{W}_o\left(x,y\right){\left(A_{so}\left(x,y\right)\Delta {\varnothing }_{so}\left(x,y\right)-T_o\right)}^{+}}{{{\displaystyle \sum }}_o{{\displaystyle \sum }}_s{A}_{so}\left(x,y\right)+\epsilon }$$

(5)

where ${\left(\right)}^{+}$ denotes that the enclosed quantity is equal to itself if it is positive, and equal to zero otherwise, $W_o\left(x,y\right)$ is a measure of significance of frequency spread, $\epsilon$ is a small positive constant used to prevent division of zero, $T_o$ is a quantity introduced to compensate image noise, and $\Delta {\varnothing }_{so}\left(x,y\right)$ is a sensitive phase deviation function defined as:

$$\Delta {\varnothing }_{so}\left(x,y\right)=\cos \left({\varnothing }_{so}\left(x,y\right)-\overline{{\varnothing }_o}\left(x,y\right)\right)-\left|\sin \left({\varnothing }_{so}\left(x,y\right)-\overline{{\varnothing }_o}\left(x,y\right)\right)\right|$$

(6)

The results of 2D phase consistency processing for video frame is shown in Figure 1.

2.2. The Correlation of Adjacent Frames

The contents of adjacent frames in video are usually very close, while the contents of distant frames may be very different. Whether the video is captured by a static camera or hand-held camera. But there should be no shot change. Therefore, we will use the correlation coefficient as a measure of the continuity of the inter frame content. The video frames were first processed through 2D PC, and the correlation coefficient between adjacent frames is defined in Equation (7):

$$r_k=\frac{{{\displaystyle \sum }}_i{{\displaystyle \sum }}_j\left(P{C}_k\left(i,j\right)-\overline{P{C}_k}\right)\left(P{C}_{k+1}\left(i,j\right)-\overline{P{C}_{k+1}}\right)}{\sqrt{\left({{\displaystyle \sum }}_i{{\displaystyle \sum }}_j{\left(P{C}_k\left(i,j\right)-\overline{P{C}_k}\right)}^2\right)({{\displaystyle \sum }}_i{{\displaystyle \sum }}_j{\left(P{C}_{k+1}\left(i,j\right)-\overline{P{C}_{k+1}}\right)}^2)}}k=1,2,\cdots ,n-1$$

(7)

where $r_k$ indicates the correlation coefficient between kth and (k+1)th frame, $P{C}_k\left(i,j\right)$ represents the 2-D PC value of kth frame at location $\left(i,j\right)$, n is the total number of video frames, $\overline{P{C}_k}$ is the average of 2-D PC for the kth frame. Assuming the frame width of video is w pixels and the height is h pixels, then $\overline{P{C}_k}$ can be calculated by using Equation (8):

$$\overline{P{C}_k}=\frac{1}{w\times h}{\displaystyle \sum }_{i,j}P{C}_k\left(i,j\right)$$

(8)

As shown in Figure 2, we can get a sequence of inter-frame correlation coefficients. If the total number of video frame is n, the length of the sequence would be n − 1.

In the original video, the value of correlation coefficients are close to each other, which means the curve of r is consistent. Figure 3 shows an illustration of frames 143–202 of test video “person15_jogging”. When the original video is subjected to frame insertion tamper, the value of the correlation coefficient will decrease at the tamper position. The two frames calculating the correlation coefficient are not the original adjacent relation, regardless of whether the inserted frame is from the same video or another video. The same is true of frame deletion. Figure 4 shows the correlation coefficients of forged video. In Figure 4a, ten frames from the same video were inserted into original video. And the result of frame deletion (delete thirteen frames) was shown in Figure 4b. The red line indicates the tampering position, we can see that the correlation coefficient at tamper position is much lower than others.

2.3. The Variation of Consecutive Correlation Coefficients

Some characteristics of video, such as the complexity of the texture or the speed of movement, will affect the value of the correlation coefficient. For instance, in Figure 3, the correlation coefficient is larger when the video content change slowly. Therefore, the detection is not accurate if we simply use the value of correlation coefficient. To restrain this phenomenon caused by the diversification of video content, some scholars have put forward the solution of calculate the variation of consecutive correlation coefficients. Such as the quotients [24] or absolute difference [31] of consecutive correlation coefficients. The definitions are given in Equations (9) and (10):

$$\Delta r{1}_k=\{\begin{array}{c}\raisebox{1ex}{$r_k$}\!\left/ \!\raisebox{-1ex}{$r_{k+1}$}\right., if r_k\ge r_{k+1}\\ \raisebox{1ex}{$r_{k+1}$}\!\left/ \!\raisebox{-1ex}{$r_k$}\right., if r_k<r_{k+1}\end{array},k=1,2,\cdots ,n-2$$

(9)

$$\Delta r{2}_k=\left|r_{k+1}-r_k\right|,k=1,2,\cdots ,n-2$$

(10)

where $\Delta r1$ and $\Delta r2$ represent the quotients and absolute difference of consecutive correlation coefficients, respectively. By construction, $\Delta r1\ge 1$ and $\Delta r2\ge 0$. We selected 60 frames from video “person15_jogging” for testing. The curves of $\Delta r1$ and $\Delta r2$ of the test video are shown in Figure 5, the left is $\Delta r1$ and the right is $\Delta r2$. The red line represents the tamper position. Moreover, there appears a pair of peaks, which are called the abnormal points. Comparing to r and $\Delta r2$, $\Delta r1$ is more credible as video changes in content have little impact on it. Therefore, the quotients of consecutive correlation coefficients ($\Delta r1$) are more suitable for inter-frame forgery detection. We set $\Delta r1$ as the measure of the variation of consecutive correlation coefficients.

3. Detection Scheme for Abnormal Points

From Section 2.3, we can conclude that frame insertion and deletion will influence the consistency of the variation of consecutive correlation coefficients. The peaks at the tampering position are the abnormal points that we need to detect. From Figure 5, we can find that the normal value of $\Delta r1$ is near 1, this feature will have good clustering effect.

We hope to use clustering algorithm to detect outliers in sample points. K-means clustering is sensitive to outliers and may converge to a local minimum. It may help us to cluster the outliers into one category accurately.

In this section, we will describe the k-means clustering algorithm and the detection procedure for abnormal points.

3.1. The k-Means Clustering Algorithm

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). The k-means (KM) clustering [32] is the most widely used clustering algorithm due to its simplicity and efficiency. The objective of the algorithm is to minimize an objective function in order to assign a group of data to its centroid.

Given a set of observations $X(x_1,x_2,x_3,\cdots x_n)$, where each observation is a d-dimensional real vector, k-means clustering aims to partition the n observations into k $\left(k\le n\right)$ sets $S=\left\{S_1,S_2,S_3,\cdots S_k\right\},$ so as to minimize the within-cluster sum of squares. In other words, its objective is to find:

$$\underset{s}{\mathrm{argmin}}{\displaystyle \sum }_{i=1}^k{\displaystyle \sum }_{x_j\in S_i}\Vert x_j-{\mu }_i\Vert {}^2,j=1,2,\cdots ,n$$

(11)

where ${\mu }_i$ is the mean of points in $S_i$.

Combined with the characteristics of this paper, we introduce the step of k-means clustering algorithm as below.

Step 1. Load $\Delta r1$ as observations X.

The variation of consecutive correlation coefficients $\Delta r1$ can be obtained by applying Equation (9). Where inter-frame correlation coefficients r and 2-D PC for each frame were calculated according to Section 2.1 and Section 2.2, respectively.

Step 2. Initialize clustering parameters.

Since our purpose is to classify the sample points into two categories, one cluster is normal points and another is abnormal, so we set cluster number k = 2. K-means algorithm intensively depends on the selection of initial clustering centers. While the outliers to be detected are the several largest values in the sample points, so we select the two largest values of X as cluster $S_1$, and the centroid is the mean of $S_1$. While the other values of X as cluster $S_2$, the centroid is the minimum of X. The storage in cluster is the location of value.

Step 3. Assign each observation $x_p$ to the cluster whose mean yields the minimum distance between $x_p$ and centroid of cluster.

Because $\Delta r1$ is one-dimensional, the distance is defined as the absolute difference between $x_p$ and centroid m:

$$S_i^{\left(t\right)}=\left\{x_p:\left|x_p-m_i^{\left(t\right)}\right|\le \left|x_p-m_j^{\left(t\right)}\right|\right\},1\le i,j\le k$$

(12)

where (t) represents iterations. Each $x_p$ is assigned to exactly one $S^{\left(t\right)}$, even if it could be assigned to two or more of them.

Step 4. Calculate the new means to be the centroids of the observations in the new clusters.

$$m_i^{\left(t+1\right)}=\frac{1}{N}{\displaystyle \sum }_{j=1}^N{x}_j,\forall x_j\in S_i^{\left(t\right)}$$

(13)

Step 5. Steps 3 and 4 are repeated until convergence has been reached.

3.2. Abnormal Points Detection Based on KM

In this section, we will illustrate the effectiveness of the feature and the feasibility of the detection method in detail.

3.2.1. Clustering Results of Original Video

Figure 6a shows sixteen sequential frames of an original video, the resolution is 640 × 480. $\Delta r1$ was calculated using Equation (9), and the clustering results of $\Delta r1$ were shown in Figure 6c. ‘*’ indicates the cluster $S_1$, ‘☆’ represents the cluster $S_2$, ‘+’ denotes the centroids of the clusters. The centroids of two clusters were 1.0389 and 1.0118, respectively. We find that both of the values of centroids were very close to 1, if the video is not forged.

3.2.2. Clustering Results of Forged Video by Frame Insertion

In Figure 7a, the last eight adjacent frames are the insertion frames from the same video with the first eight frames. In Figure 7b, there is a pair of peaks at the 7th and 8th frames because of the low correlation between the 8th and 9th frame. If we detect the abnormal points, we can prove that the video has been forged. The location of abnormal point is where tampering happens. In clustering results, cluster $S_1$ represents the detected outliers. The centroids of two clusters were 2.5144 and 1.0413, respectively. We find that the centroid of $S_1$ is deviate from 1, due to the tampered operation.

3.2.3. Clustering Results of Forged Video by Frame Deletion

In Figure 8a, the 11th and the 12th frame are not adjacent, because we delete several frames between them. The consistency of $\Delta r1$ was destroyed at the delete position, and the frame number 10, 11 were clustered in $S_1$. The centroids of $S_1$ and $S_2$ were 1.7757 and 1.0359, respectively.

Sometimes, in order to falsify evidence, the forger will delete all frames which contain characters in the video. For example, in Figure 9a, we entirely remove a group of frames where a subject passes. There looks no discontinuities, but in fact, inconsistency appeared in the delete position. Shown in Figure 9b.

3.2.4. Clustering Results of Forged Video by Multiple Tampering

Sometimes video suffers from multiple tampering issues. For example, the combination of frame insertion and deletion. As shown in Figure 10a, the video suffered three forgeries. Due to the difference of tampering positions and the number of frames, the values of peaks are different. However, all the tampering positions were precisely detected as shown in Figure 10b. The centroids of $S_1$ and $S_2$ were 1.6289 and 1.0489, respectively.

From the above examples and analysis, we can conclude that the selected feature can effectively reflect the inter-frame manipulation. Moreover, the KM algorithm is able to locate the tampering position. From the clustering results, it is not difficult to see that if the video is forged, the value of ${S_1}^{\prime }$ centroid will be larger, which is deviated from 1, corresponding to the peaks of the tampering. Thus, the points in cluster $S_1$ are the abnormal points. But when the video is original, both of the two clustering centers are very close to 1. Therefore, the value of the cluster center is the basis for judging whether the video has been tampered with or not. Moreover, generally the value of the cluster center in frame insertion is larger than that of frame deletion, but it is not true in all cases.

3.3. Threshold Decision

According to the description in Section 3.2, the normal and abnormal points are clustered into two categories by k-means clustering, the value of centroid is the basis for judging whether the video has been tampered with or not.

We tested the proposed method on the original sub-database, and analyzed the cluster center of $S_1$ of 599 original videos. The histogram of the value of centroid is shown in Figure 11. The x-axis indicates the value of centroid of the cluster, the y-axis indicates the frequency of occurrence of each value.

From Figure 11, we conclude that most of the clustering center values are less than 1.25. So we set T = 1.25 as a threshold to distinguish original video and forged video. When centroid of $S_1$ is greater than T, the video is detected as forged. Otherwise, the video is detected as original.

4. Experimental Results and Discussion

4.1. Dataset

In our experiments, two datasets of different sources and resolutions are selected. The first dataset selects from public Kungliga Tekniska Högskolan (KTH) [33], include one original sub-dataset, and four forgeries sub-datasets with different number of tampered frames. The videos contain six types of human actions, namely walking, jogging, running, boxing, hand clapping, and hand waving. The test videos compressed by MPEG, and were taken with a static background with 50fps frame rate, the resolution is 180 × 144. The number of tested videos of the five sub-datasets is 599, 599, 599, 599, and 598, respectively.

The second dataset has 480 videos, half of them are original videos, and the other are forged videos, include frame insertion and deletion. The number of tampered frames is more than 20. The composition of the second video dataset is shown in the

Table 1. The composition of the second video dataset.

Source	Frame Rate	Resolution	Number of Original Videos	Number of Forged Videos
SULFA [34]	30fps	320 × 240	120	120
Camera	30fps	640 × 480	120	120

.

4.2. Evaluation Metrics and Method Assessment Procedure

In order to evaluate the validity of the scheme, we consider six performance indices: TPR (True Positive Rate), also known as recall, TNR (True Negative Rate), PPV (positive predictive value), also known as precision, Accuracy, F1 score, and Location Precision. Accuracy is the average detection accuracy. F1 score can be interpreted as the weighted average of the precision and recall. Location Precision is the percentage of correct localization among all the correct detected forgery videos:

$$TPR \left(recall\right)=\frac{TP}{TP+FN}$$

(14)

$$TNR=\frac{TN}{TN+FP}$$

(15)

$$PPV \left(precision\right)=\frac{TP}{TP+FP}$$

(16)

$$Accuracy\left(ACC\right)=\frac{TP+TN}{TP+FP+TN+FN}$$

(17)

$$F1=2\ast \frac{precision\ast recall}{precision+recall}$$

(18)

$$LocationPrecision\left(LP\right)=\frac{TPL}{TP}$$

(19)

where $TP$ is the number of true positive, means that the forged video was detected as forged, $TN$ is the number of true negative, means that the original video was detected as original;$FP$ is the number of false positive, means that the original video was detected as forged; $FN$ is the number of false negative, means that the forged video was detected as original, $TPL$ is the number of correct localization, $\left(TP+FN\right)$ is the total number of forged videos, $\left(TN+FP\right)$ is the total number of original videos, $\left(TP+FP+TN+FN\right)$ is the total number of database.

In order to improve LP, the location results will be post-processed. We reject the single suspected abnormal points, since in the tamper position there is always a pair of peaks.

4.3. Experimental Results

We tested the proposed method on the two dataset. The statistical data of test results were given in

Table 2. Test results of the first dataset.

Source	$\mathit{T}\mathit{P}$	$\mathit{T}\mathit{N}$	$\mathit{F}\mathit{P}$	$\mathit{F}\mathit{N}$	TPL	$\mathit{T}\mathit{P}\mathit{R}$	$\mathit{T}\mathit{N}\mathit{R}$	Precision	$\mathit{A}\mathit{C}\mathit{C}$	F1	$\mathit{L}\mathit{P}$
Original	-	579	20	-	-	-	0.9666	-	-	-	-
25 frames inserted	599	-	-	0	582	1.00	-	-	-	-	0.9716
100 frames inserted	599	-	-	0	581	1.00	-	-	-	-	0.9699
25 frames deleted	550	-	-	49	500	0.9182	-	-	-	-	0.9091
100 frames deleted	586	-	-	12	560	0.9799	-	-	-	-	0.9556
Average	584	579	20	15	556	0.9750	0.9666	0.9669	0.9708	0.9724	0.9520

and

Table 3. Test results of the second dataset.

Source	$\mathit{T}\mathit{P}$	$\mathit{T}\mathit{N}$	$\mathit{F}\mathit{P}$	$\mathit{F}\mathit{N}$	TPL	$\mathit{T}\mathit{P}\mathit{R}$	$\mathit{T}\mathit{N}\mathit{R}$	Precision	$\mathit{A}\mathit{C}\mathit{C}$	F1	LP
SULFA	114	111	9	6	109	0.95	0.925	0.9268	0.9375	0.9383	0.9561
Camera	112	110	10	8	105	0.9333	0.9167	0.9180	0.925	0.9256	0.9375
All	226	221	19	14	214	0.9417	0.9208	0.9224	0.9313	0.9320	0.9469

, where ‘-’ indicates that the data does not exist.

We give a comparison of our results against the results reported in [24,25]. To ensure fairness, we tested all methods on the same database, which is a mixture of the two datasets mentioned above. And used the parameters recommended from the method. The results are shown in

Table 4. Comparison of three methods on mixed database.

Method	Recall	Precision	F1	LP
Reference [24]	0.8673	0.8954	0.8811	0.8896
Reference [25]	0.9272	0.9455	0.9363	0.9361
Proposed method	0.9584	0.9447	0.9522	0.9495

. We can see that the proposed method is better than reference [24,25].

We also tested our algorithm on another dataset, the 5 frames-deleted and the 5 frames-inserted database, which are also generated from the original database. In the 5 frames-deleted database, the $TPR$ and $LP$ are 55.09% and 64.85%, respectively. In the 5 frames-inserted database, the $TPR$ and $LP$ are 100% and 96.83%, respectively. The reason is that, deleting 5 frames doesn’t efficiently change the video contents, and the correlation of adjacent frames changes a little after the tamper. Inserting a few frames may have a great impact on video content. Therefore, the TPR and LP are higher in frame insertion detection than frame deletion.

4.4. Time Complexity Analysis

The proposed method consists of feature extraction and abnormal point localization. During feature extraction, we extract the 2-D phase congruency of each frame. The computational complexity is related to the video resolution and the frame number. The higher the resolution and the frame number, the more time consuming the process. The k-means clustering algorithm was used to localize the abnormal points. The time complexity is $O\left(n\times k\times t\right)$, where n is the number of objects in dataset, t represents the number of iterations of the algorithm, and k is the number of clusters. We test two different resolutions with the frame number is 500, and the results are shown in

Table 5. The time consuming of the algorithm.

Video Resolution	Frame Number	Time of Feature Extract (s)	Time of Clustering (s)
320 × 240	500	185.56	0.0545
640 × 480	500	373.15	0.0632

.

5. Conclusions

In this paper, an inter-frame forgery detection scheme based on 2D phase congruency and k-means clustering was proposed for surveillance video. We calculate 2D PC for each frame firstly. Then, the correlation coefficients of adjacent frames and the variation of consecutive correlation coefficients are obtained. Finally, the discontinuous points caused by tampering are detected by using k-means clustering algorithm. Experimental results show that our approach can detect and localize the tampering positions efficiently. The shortcoming is that when deleting frames appear at the beginning or the end of the video, the detection method is impossible. Moreover, in this scheme, the TPR and LP are higher in frame insertion detection than frame deletion. Therefore, in the future work, we will focus on finding a better solution to improve the precision of detecting frame deletion.

In addition, the method locates the inter-frame tampering operations without distinguishing whether the inserted frames are copied from the same video or are spliced from another video. In future work, we will try to distinguish different frame insertion forgery operations.