*The Image Dataset*

To help us study the false negative using Stegdetect to analyse steganography content automatically, the tool require images that contain embedded data. This research is based upon hiding bits of messages into 2000 JPEG images files using the embedding tool, JPHide. We searched and selected images from Sam Houston State University, University of Washington and Google image databases. Unfortunately, with our initial google images, there was a problem with the size of the images which affected the stego-object, which made statically modified after embedding obvious. To resolve the issue the following parameters were set for the downloading from google.


However, we also activated both the search ON/OFF for the downloading of 300 images from Google to get the effect of this parameter on the outcome of the analysis. In addition to this, we also downloaded 700 clean JPEG images from University of Washington (Department of Computer science and Engineering) and 1000 images from Sam Houston State University image, 500 untouched and 500 manipulated with 75 bot quality.

Figure 1 shows samples of images hidden with messages using jphide. An automated utility, Stegdetect, which analyses bulk images with a hidden message with JPHide has been chosen to study its false negatives. For this purpose, we obtained JPHide version 0.5 as well as the Windows version of Stegdetect. We regulated the sensitivity value of Stegdetect against 2000 stego-object (obtained from different image databases such as google, Sam Houston State University and University of Washington). It was installed on a Windows 7 enterprise core i5 with 8 GB RAM.

**Figure 1.** Sample results of the jphide method.

### **4. Results**

All the results were analysed and interpreted in different phases deepening on the image dataset. Phase one analysed a total of 500 images manipulated by seam-carve from SAM Houston university image database, both at 75 quality before embedding using jphide with randomly generated bits. The table below gives a summary of the overall detection during the analysis.

We noted that detection of jphide method in the images was based on the changes in the sensitivity values. However, other algorithms detected by the tool are the circumstances in which stegdetect during the analysis identified other steganographic methods which during the embedding process we did not use. Table 1 shows that the highest ratio of detection with sensitivity results is 67.13% of the manipulated images by seam-carve which considering the level of the ratio is very high. Meanwhile, detection results for jphide were very low.


**Table 1.** Rate of sensitivity results from 500 images manipulated by seam-carve.

All results for false negative, jphide and other algorithm keep changing with change in sensitivity as shown in Table 2. The beginning of the analysis with low sensitivity value (0.1) the false negative ratio was very high (98%). However, a systematic drop was realised in the false negative ratio between sensitivity values 0.1 and 0.7, furthermore, the false negative ratio with sensitivity values 5.2–10.0 had a drastic drop as shown in Table 2 below. Here it becomes clear that the tool became more effective in detecting steganographic method used in embedding the secret messages.

**Table 2.** Results from manipulated images by seam-carve based on sensitivity values.


As shown in Figure 2 above, between sensitivity values 0.1 and 0.5 there were no changes in the results for jphide. Meanwhile, detection of jphide increased substantially between 0.7 and 10.0 with their related confidence levels (\*, \*\*, \*\*\*). Between 0.1 and 0.5 jphide (\*) was stable until it got to the range 0.7–3.4 when there was fluctuation in the detection ratio, it then had a sharp increased with 5.2 sensitivity, after which it experienced another sharp decrease between (7.3 and 10.0). For jphide (\*\*) between 1.5 and 10.0 there was a constant increase except with sensitivity of 3.4 which experience some drop. However, jphide (\*\*\*) maintained the increasing of its ratio.

**Figure 2.** Changes in the jphide rate with different sensitivities for seam carve manipulated images.

As per the analysis above, the level of confidence in detection by stegdetect is directly proportional to the sensitivity values. Meaning, the higher the sensitivity value the higher the confidence in detecting jphide. Furthermore, the high increase of confidence in detecting jphide was between (3.4 and 10.0). During the analysis, stegdetect detect other steganographic methods in the images other than jphide which we used. Figure 3 below shows that 0.2% of the detection was for other algorithms between 0.3 and 0.7 sensitivity which stegdetect claims was used in embedding secret messages in those images. Meanwhile, the percentage of other algorithm detected increased to 0.4% between (1.5 and 10.0). Finally, the images from the database were already manipulated before jphide method was used to embed the messages. It is therefore possible that the images were manipulated using any of the algorithms detected during the analysis.

**Figure 3.** Changes in other algorithms detected with different sensitivities.

Phase two of the analysis was focused on 500 Seam-carve untouched (clean) images from SAM Houston university image database which were embedded with a secret message using jphide. Compared to the detection results of the manipoulated images, there was slight incease in the detection for the false negative ratio, skipped (false positive likely) and jphide (\*) while other algorithms and jphide (\*\*, \*\*\*) experience a slight decreased with different sensitivity as shown in Table 3 below.


**Table 3.** Rate of sensitivity results from 500 seam-carve untouched images.

As show in Table 3 above, 67.78% of the overall detection was false negative which is very high. However, with an increase in sensitivity, the detection ratio for false negative, jphide and other algorithm all changed. Furthermore, as shown in Table 4 below, there was a significant increase in the confidence detection of steganographic method jphide with changes in sensitivity values. We observe slight changes in the detection between the manipulated and the untouched Seam-carving images. Detection of jphide in the untouched images embedded with bits of messages started with 0.5 sensitivity while detection for jphide in the manipulated images started with 0.7 sensitivity, after which there was a continuous increase in the confidence in detection of jphide method.


**Table 4.** Results of 500 images from seam carve untouched images with different sensitivity values.

The false negative results for untouched seam-carving images at the beginning were high 97.60% as shown in Figure 4 with 0.1 sensitivity value, this result is not different from the manipulated images, however there was slight decrease between 0.1 and 3.4, then there was massive fall in the false negative between 5.2 and 10.0 with increase in sensitivity value.

**Figure 4.** Overall false negative rate seam-carving untouched images with different sensitivity values.

The detection results for jphide (\*, \*\*, \*\*\*) between 0.5 and 3.4 was very marginal until the sensitivity was increased to 5.2 when jphide (\*) had sharp increase meanwhile, with continuous increase in the sensitivity value between 7.3 and 10.0 the detection of jphide (\*) experience a continuous decline, at the same time between 5.2 and 10.0 the level of confidence in detecting jphide (\*\*) had a continuous increase while jphide (\*\*\*) maintained its steady increase as shown in Figure 5 below.

Figure 6 shows that there was no effect of the sensitivity between 0.1 and 0.7 on the results for other algorithm detected, then between 1.5 and 10.0 there was a minor increase in the detection of other algorithms by the tool. However, between 3.4 and 10.0 the tool (stegdetect) maintain a constant detection ratio for other algorithms.

**Figure 5.** Changes in the jphide rate with different sensitivities for seam carve untouched images.

**Figure 6.** Changes in other algorithms detected with different sensitivities.

Phase three of the experiment analysis 700 images from the Department of Computer and Engineering, university of Washington image database. Each image was embedded with a different generated bits of a message using jphide. During the analysis of the 700 stego-images, 3.71% resulted in error between 0.1 and 10.0 sensitivity which compared to the volume of the images involved is quite small. In the case of the error images, stegdetect could not analysis because of the following stated reason. 1. Bogus DQT index 6, 2. Invalid JPEG file structure: SOS before SOF, and the last 3. Quantization Table 0 × 00 and 0 × 01 was not defined. The error rate can be seen in Table 5 below. It wealth noting that all the images analysed were subject to frequency counts. In other words, the analysis of any detection (false negative or jphide) was added to find the highest detection ratio (i.e., a number of times a specific detection occur). After which they were quantified as shown in Table 5 below.



The false negative result between 0.1 and 1.5 sensitivity was 78.29% which is a bit high, then when the sensitivity was change between 3.4 and 10.0 there was a sharp drop and a continuous decline until it reaches 17.71%. Moreover, comparing the false negative results of the previous seam-carving images (both manipulated and untouched images) we realised that with the previous experiment between 0.1 and 3.4 they had a significantly higher false negative ratio which was 80% to 98% before it had a sharp decline. Though the images from Washington University seem to have had a low false negative ratio compared to the seam-carving images, they all seem to have had a sharp decrease at some point, then when the sensitivity was set to 5.2 it maintained a slow but steady decrease as shown in Figure 7 below.

**Figure 7.** Overall false negative rate of Washington university image database with different sensitivity values.

The detection results of jphide (\*, \*\*, \*\*\*) started between sensitivity values (0.3 and 1.5), then there was a significant increase in the detection between (3.4 and 10.0). The detection for jphide (\*) was consistently increasing until 3.4–5.4 sensitivity when there was a height jump, meanwhile, between 7.3 and 10.0 sensitivity the detection for jphide (\*) started to decrease and jphide (\*\*) also had similar result like in the case of jphide (\*) where it experience a stable increase then a slight decrease with 0.7 sensitivity before it started to increase in detection again between 1.5 and 10.0 sensitivity. Finally, jphide (\*\*\*) maintain a continuous steady increase in detection between 0.5 and 5.2 then a height jump in the detection between 7.3 and 10.0 as shown in Figure 8 graph below.

**Figure 8.** Changes in the jphide rate with different sensitivities for Washington university image database.

Phase four analysis 300 image from google (SAFE ON/OFF), the results for skipped false negative likely, and errors were changed with different sensitivity, other algorithms detection was constant between 0.7 and 10.0. The detection results for false negative was still between (0.1 and 3.4). However, with (5.2–10.0) sensitivity just like the previous experiment, there was a significant fall in the false negative ratio as shown in Figure 9 graph below.

**Figure 9.** Overall false negative rate of google image database (SAFE ON) with different sensitivity values.

Again comparing the results with the other experiments conducted earlier the confidence level in jphide detection ratio keep change with changes in the sensitivity value as shown in Figure 10 below. For this set of images jphide (\*) had similar results we acquired from the images from seam carve and Washington university image databases, respectively. For all those experiment there was sharp increase in detection ratio and then another sharp decline in detection for jphide (\*) with different sensitivity values. However, jphide (\*\* and \*\*\*) had a different results from all the other experiments performed, for this experiment we realised a continuous increment in the detection ratio for both jphide (\*\* and \*\*\*) with increasing sensitivity value as shown in Figure 10 below.

**Figure 10.** Changes in the jphide rate with different sensitivities for google image database (SAFE ON).

We realised that there were different results especially for the jphide and false negative from all previous experiments. For instance, between (0.5 and 10.0) sensitivity there was continuous and significantly higher confidence in detecting jphide (\*\*\*) from the previous experiments. However, Google safe (OFF) as shown in Table 6 below gives slightly different results considering the confidence in detecting jphide (\*\*\*).


**Table 6.** Results of 150 images from google image database (SAFE OFF) with different sensitivity values.

The highest was again at the beginning of the experiment was the false negative ratio 90.67%, which is much different from the previous experiment, and had a further drop with increasing sensitivity. Figure 11 shows that the curve is not different from the previous experiment.

The detection results for jphide (\*\*\*) from google safe (OFF) is different from the results from the safe (NO) results. With the safe (off) detection of jphide (\*\*\*) started and continuous to increase between (1.5 and 10), but detection for jphide (\*\*\*) in safe (ON) started between (0.5 and 10.0), and jphide (\*) continuous to increase in detection between 0.5 and 5.2 before the detection started to fall has sensitivity increase between 7.3 and 10.0. Finally, jphide (\*\*) results at 1.5–5.2 sensitivity there was a steady increase before a quick and continues increase between 7.3 and 10.0. The two image groups were compared to show how the properties of images can affect the detection of Jphide method in images. Figure 12 gives a graphical representation of the jphide results.

**Figure 11.** Overall false negative rate of google image database (SAFE OFF) with different sensitivity values.

**Figure 12.** Changes in the jphide rate with different sensitivities for google image database (SAFE OFF).

The final phase, analysis the overall false negative ratio of the tool, this is to help forensic analyst during an investigation by providing accurate statistics of stegdetect false negative ratio, because in the court of law the forensic analyst must prove beyond every reasonable doubt that the results of the tool can be relied upon as evidence. This analysis was done using the results from all the different image databases, note that all the images had different properties, because there were some that had been manipulated with dotted at a quality of 75 and there were those that were untouched. The overall false negative results for all the different images it is very high between (0.1 and 3.4) but had a quick fall between (5.2 and 10.0), and as the false negative results drop the confidence in detecting jphide (\*, \*\*, \*\*\*) increases, this is an important information for the analyst investigating images from different sources. Especially noting that false negative ratio of the tool and how the higher the sensitivity between (5.2 and 10.0) influences the results of bulk images under investigation. Table 7 shows all the false negative values for the different datasets used.

Figure 13 below present the overall false negative ratio which was very high, but there is very important information about the graph the forensic analyst need to know. We set our acceptable false negative ratio to be 21%, which intersect with the mean of all the false negative at some point on the sensitivity. All the different image at 5.2 sensitivity had a quick fall in the false negative ratio but with a continuous increase in the sensitivity gave a stable and slow decline in the false negative ratio. Note, with our acceptable 21% false negative its correspondent sensitivity is 6.2. This will inform the analyst on the kind of sensitivity they can use depending on their acceptable false negative ratio during an investigation. During the analysis, the following observations were noted,


**Table 7.** Overall false negative rates of ALL the different image databases with different sensitivity values.


**Figure 13.** Overall false negative ratio from all different image databases.
