As digital evidence has a highly influential role in proving the innocence of suspects, methods for integrity verification of such digital evidence have become essential in the digital forensic field. Most surveillance camera systems are not equipped with proper built-in integrity protection functions. Because digital forgery techniques are becoming increasingly sophisticated, manually determining whether digital content has been falsified is becoming extremely difficult for investigators. Hence, systematic approaches to forensic integrity verification are essential for ascertaining truth or falsehood. We propose an integrity determination method that utilizes the structure of the video content in a Video Event Data Recorder (VEDR). The proposed method identifies the difference in frame index fields between a forged file and an original file. Experiments conducted using real VEDRs in the market and video files forged by a video editing tool demonstrate that the proposed integrity verification scheme can detect broken integrity in video content.
he increased use of surveillance cameras such as Video Event Data Recorders (VEDRs) and Closed Circuit Television (CCTV) cameras in the recent past has resulted in their rapid worldwide deployment
. Even considering that surveillance devices could infringe on privacy, this distribution trend is unavoidable because recorded videos play a key role as evidence in disputatious situations
. Further, surveillance devices assist criminal investigators by providing significant evidence.
Nevertheless, video evidence is susceptible to tampering. With the advent of easy-to-use video editing tools, video files are increasingly at risk of being forged
. Consequently, it is entirely possible that a forged video could be submitted to a court as evidence
. In such a scenario, because there is no guarantee that the video is genuine, the evidence could result in an incorrect verdict in a criminal matter. Further, even when the video is not modified intentionally, the video data stored in the surveillance devices’ storage can be damaged by external crashes. Thus, it is essential that the integrity of video data be verified. Consequently, the reliability of video data integrity verification techniques is a critical issue for digital forensics investigators. For example, recently Garfinkel et al.
introduced a digital media carving technique by using sector hashing and hashdb to verify the integrity of media files.
In this paper, we propose an integrity verification scheme for a VEDR video that uses the frame index data in video files. By checking for anomalies in the index artifact of the video files stored in the File Allocation Table 32 (FAT32) file system, the scheme verifies the integrity of Audio Video Interleave (AVI) video data. (AVI is one of the most popular video file formats used in VEDRs.)
Pre-employed integrity check codes or watermarks of video frames reserved at the time of recording have been proposed for video integrity verification in previous studies
. However, these pre-employment approaches have deployment problems because they must be embedded in the target VEDR devices by the manufacturers. Post-processing approaches involving image analysis have also been proposed to detect image manipulation
. However, they do not address frame-editing cases, where no changes occur in the remaining images, such as frame deletion without re-encoding. Our previous work
used frame-based verification to rectify the limitation of previous image-processing techniques. However, frame-based verification has limitations detecting cases where video frames are deleted at the end of a frame sequence. In this study, we achieve video file integrity verification without using any pre-employment schemes and overcome the limitations identified in previous proposals.
We evaluate the proposed method using video files recorded by VEDR devices and modified using a video editing tool in several scenarios. The evaluation results confirm that the proposed scheme can identify compromised integrity wherever video frames are deleted in a frame sequence, overcoming the limitation of our previous work. Of the eight forged samples, all eight were identified as being modified when various numbers of frames were deleted from the original video file.
The remainder of this paper is organized as follows. In Sections 2 and 3, we briefly explain VEDR file systems and video file attributes and provide an overview of previous studies related to video file integrity verification, respectively. In Section 4, we outline the proposed method. In Section 5, the efficacy of the proposed verification method is evaluated. In Section 6, we discuss several topics associated with frame data recovery and the compatibility of the proposed method with other file systems and video formats. Finally, we conclude this paper in Section 7.
- 2.1 Video Frames
Video files consist of sequential static images called video frames.
illustrates an example of video frame composition in a compressed video. The frames comprise a sequential Group of Pictures (GOPs), with each GOP containing a key frame followed by delta frames and an encoding profile. In a compressed video, only the key frames have the entire image information; the delta frames contain only information regarding the bits that have changed compared to the previous frame.
Example of video frame composition
- 2.2 File Format of AVI Videos
The AVI video format
is designed following the RIFF standard presented by Microsoft. It contains both video and audio streams. The general model of the AVI format is depicted in
. The actual frame data cover a field starting with LIST movi. The ensuing field, starting with idx1, contains frame index information including the offset and size of each actual set of frame data.
Basic Architecture of AVI Files
- 2.3 Allocation of Video Content in FAT32
Video files stored in storage media such as SD and NAND flash memory are managed by file system rules that provide a mechanism to store, categorize, and access data
. We adopted FAT32 as our target file system because FAT32 is the most compatible file system across all embedded systems including VEDRs. Basically, the relative positions of files stored in FAT32 file system is correlated with their creation order
. With the property of the file allocation, we can set several possible scenarios.
presents three cases of a file allocation scenario in the FAT32 file system. A cell in the areas indicated by C1 to CX represents
, which is composed of sectors, the minimum unit logically addressable in the FAT32 file system.
Examples of file allocations in a FAT32 file system
(a) is a case where three files are continuously allocated in the file system. We suppose that File A, B, and C have taken possession of two, three, and two clusters, respectively. At the end of each file, there remains a space called
. Slack space is the unused space in the last cluster when the size of a file is not an exact multiple of the cluster size.
(b) is a case where another video data File B’, depicted as a dark grey square, is written after deleting the original File B. File B’ is smaller than File B because data frames have been deleted using a video editing tool. The three clusters originally occupied by File B are referred to as
. Unallocated space refers to the unused region of a file allocation system. This space is present when either no files have yet been allocated or previously allocated files have been deleted from the file system. Because parts of the remaining frame data found in the original File B may correspond to frame data comprising File B’, we assume that both files have a
(c) describes a situation where File B, with a smaller size, overwrites the area of the original File B. This can happen because file systems such as FAT32 allocate a smaller sized file to the same position if the file fits into the unallocated sectors between two other files. In this case, there are both a slack space and unallocated space in the residual area because File B is not sufficiently large to occupy all three clusters. For ease of explanation, we will call both slack space and unallocated space
. The duplicated area of both the original File B and the secondary File B can be found in the allocated space of secondary File B.
This paper focuses on case (c), identifying broken integrity when there is no suspicious factor in the metadata of the file system. In case (b), the area where the video content has been deleted is obvious as we can analyze metadata such as the directory entry in FAT32; however, it is not possible to verify integrity with only metadata information in case (c). Note that the proposed scheme can also be applied to case (b).
3. Related Work
Integrity verification of video content can be studied from either of the following two perspectives: pre-processing or post-processing. The pre-processing method operates when the video content is being stored. An example of this approach is the integrity verification method proposed by Jayamalar
, where digital watermarking is embedded in the video content
. This method enables investigators to observe illegal copying or manipulation while playing the video in real time.
also proposed a data integrity scheme
for preventing falsification. The proposed scheme stores video content in two secure channels for original files and encrypts each file. Then, to verify the integrity of the video files, the original video files are encrypted and compared with the previously encrypted stored videos. However, these pre-processing approaches are difficult to incorporate into legacy surveillance camera systems because they must be applied in the design phase while the systems are being manufactured. The scheme proposed in this paper is applicable to surveillance cameras that are not equipped with a built-in protection module. The scheme verifies integrity only with the disk storage itself.
Similar to the scheme proposed in this paper, post-processing methods have been applied to investigations after an incident has occurred. Previous proposals detect digital forgeries using the video noise data caused by the instability of camera sensors or inter-frame correlation. Wang
proposed a forgery detection method that analyzes the noise data of images in video files. Wang
proposed a method that detects duplicated frames using inter-frame correlation. However, the previous methods only operate in image modification cases. Conversely, the scheme proposed in this paper achieves integrity verification in cases of frame editing by skimming the frame indices of the AVI files and disk space. Searching for the frame index area is not costly because, in a video file, the size of the index area that must be parsed is significantly smaller than that of the frame areas.
4. Integrity Verification Scheme for Disk Storage in VEDR
In this section, we examine a scenario where a video editing tool is used to modify a video. Verification of video file integrity with metadata in video frames is applicable to the majority of practical video file formats. By setting cases that address the most practical scenarios, we primarily target the AVI file format, which is one of the most popular formats used by VEDRs in practice. Next, we introduce the proposed scheme using the AVI file structure as an example. For the AVI file, we verify integrity by comparing the frame index information retrieved from both the allocated and residual space.
presents the overall flow of the proposed VEDR video file-integrity verification scheme. The scheme extracts the frame indices from the logical disk storage of a VEDR device using residual space and allocated space parsers. Then, forged video files are detected by comparing the frame indices extracted from each space.
Process flow of proposed VEDR video file-integrity verification scheme
- 4.1 Video Content Fabrication Scenarios
Video file fabrication can be performed with two kinds of techniques: image editing and frame editing
Image editing techniques can be used to change images and encode frames on a video file. However, the modifications made can be detected from the changes in the image quality. Conversely, frame-editing techniques do not affect the image itself; rather, they are used to add, delete, and reorder the image frames. The proposed scheme primarily focuses on broken integrity of the frame-editing techniques, an area that is not sufficiently covered by conventional image investigation techniques. In particular, frame editing with frame deletion is critical because the original data does not remain on the forged video file, unlike addition and reorder.
depicts a frame-editing scenario with deletion. In this scenario, an array of video frames is composed of frame sets
. Let us assume that frame set
contains an unfavorable scene for the suspect (#1). Therefore, the suspect deletes frames in frame set
and uses “direct stream copy” not to re-encode the video file and hide the intentional tampering of the video. Then, the following frame set,
, is attached to the end of frame set
(#2). The suspect then overwrites the original video file with a forged video file that has the same name as the original video (#3). In accordance with the file allocation property in FAT32, artifact data will remain in the residual space (#4).
Image editing:In cases where frame images contain crucial evidence, a suspect may attempt to manipulate the image. For instance, it is possible to change a traffic light from red to green using a video editing tool such as Adobe Premiere or Sony Vegas.
Frame editing:If suspicious frames are present in a video file, a suspect may delete parts of the unfavorable frames. After merging previous frames with subsequent frames, the suspect may then overwrite an original video file with the forged file.
Process of frame deletion in video files resulting in artifacts in the residual space
Depending on the location of the deleted frames in a frame sequence, there are two possible scenarios: Overlapping and Tail cut. An overlapping scenario arises when frames are not deleted at the end of the frame sequence in a video file. Conversely, a tail scenario arises when the frames are deleted at the end of the sequence. Both scenarios follow the simple video editing process illustrated in
• Scenario 1: Overlapping
An overlapping scenario arises when the frame deletion is not conducted at the end of a frame sequence. As illustrated in
, the deletion of frames results in partial matching of frames between the stored video file and residual space. The matching frame areas are indicated as (A) and (B). The remaining area, (C), is not matched with any frame in the stored video file. The mismatched area becomes larger as the size of the deleted frames becomes greater.
Frame deletion scenario (Overlapping)
• Scenario 2: Tail cut
In this scenario, the deleted frames are located at the end of a frame sequence. As indicated in
, there is no matching frame between stored video file (A) and residual space (B). When the frame deletion is conducted following our scenario, the new AVI frames index area idx1’ is located at the end of the new frame sequence.
Frame deletion scenario (Tail cut)
- 4.2 Integrity Verification of AVI Video File in FAT 32
In the case of AVI video files, it is possible to use the index information to verify integrity. According to Microsoft
, the AVI format comprises LIST hdrl, LIST movi, and idx1. Byte string idx1 indicates a starting point to a field listing AVI frame indices. Because idx1 follows LIST movi, it is located at the tail portion of the AVI video file.
graphically illustrates the concept of index-based verification. If a suspect overwrites a forged video file with an original video file, two idx1 fields will remain in the file allocation area. Because the forged video usually has fewer frame indices than the original video
, another idx1 field remains in the residual space. We call the idx1 field stored in the residual space idx1’.
Comparison of frame indices in both idx1 and idx1’
To verify the integrity of an AVI file, we refer to the size data of every frame index retrieved from idx1 and idx1’. Let us suppose that the suspect deletes part of the frames. The frame size data in all the frame indices from idx1’ are partially identical to those from idx1. In this situation, any mismatching frame between idx1 and idx1’ means the existence of frame manipulation in the video file. From this principle, we are able to ascertain that the integrity of the AVI is broken.
displays the comparison of integrity verification capabilities among pre-deployment approaches, image processing-based schemes, our previously proposed scheme, and the proposed index-based scheme. As a proof of concept, we evaluated the proposed scheme with a self-implemented prototype and AVI video files. To add forgery to the videos, we used a commercial video editing tool that supports a frame-editing technique. In the experiment, the proposed scheme successfully detected videos forged in both overlapping and tail cut scenarios. Note that it does not require any pre-deployment of the algorithm, even though it detects forged videos in both scenarios.
Comparison of integrity verification capabilities of various proposed schemes
Comparison of integrity verification capabilities of various proposed schemes
The proposed scheme was applied to video contents in two commercial automobile VEDRs,
iPass Black ITB-100HD
. Both of the VEDRs are equipped with a FAT32 formatted 8 GB flash memory card.
The recorded video files were stored on the SD flash memory card in AVI format. Note that the proposed integrity verification method is not restricted to AVI format only. We tested 20 video samples, which included four original videos and 16 manipulated videos with each frame-editing scenario. The four original videos are listed in
. Two of the original videos were one-minute duration, 1920 × 1080 resolution, 30 fps, H.264 encoding format, and consisted of approximately 1,800 video frames. The other two original videos had the same attributes as the former two original videos except that they had a 1280 × 720 resolution. We labeled the original video samples
. The manipulated samples were labeled
with frame deletion scenarios
ail cut, respectively. The subscripts “10” and “50” on the forged samples indicate the percentage of deleted frames.
VEDR video files used in the detection test
VEDR video files used in the detection test
The detection mechanism used to identify the manipulated video file is simple. When the verification tool found both matched frame indices and mismatched frames, it determined that the target file was a manipulated sample. Conversely, if only matched frames were identified, it was determined that the sample was cloned from the original, however, not forged. The original video samples,
, were found to be genuine when the detection mechanism was applied.
Overlapping scenario:In the overlapping scenario, we deleted 180 frames (10% of total) and 900 frames (50% of total) from a random position ofV1toV4and reorganized the video file to convert them into playable videosO1toO4. From the detection results inTable 3, it is clear that the identical frame indices of the manipulated samples,O1toO4, were detected from the residual space and the indices of the deleted or overlapped frames were estimated as the mismatched frames. According to the detection results, we detected all eight overlapping deleted video files (O1toO4).
Tail cut scenario:In the tail cut scenario, we deleted 180 frames (10% of total) and 900 frames (50% of total) from the end ofV1toV4, then constructedT1toT4. According to the detection results, we detected all eight tail-cut deleted video files (T1toT4).
Result of Integrity verification
Result of Integrity verification
- 6.1 Time Consumption and Scalability
For digital forensics investigators, the time consumption of an integrity verification scheme is one of the important criteria in terms of practicality
. The size of a target video file can be large, up to several gigabytes. To demonstrate the scalability of the proposed verification scheme against an increase of the target video file size, we analyzed the time consumption of the proposed scheme with larger video samples and more deleted frames. The experiments are performed on a desktop PC with an Intel i5 3.4 GHz CPU, 10 GB RAM, Windows 8, and USB 3.0 interface for reading the SD card storages.
According to empirical experiments, the proposed index-based integrity verification scheme had a linear time complexity increase with the target video file size and the number of deleted frames. In a larger file, file I/O time requires a more significant portion of the entire time consumption and the file I/O time increases linearly with the increase of target file size. The verification time consumption
the residual space search time Tr, the index comparison time Tc
file read and parsing I/O time Tio
is only affected by the number of deleted frames. As the size of the video file increases,
increase linearly, and the portions of
occupy a more dominant portion. In 10 MB, 100 MB, and 300 MB files, the proportions of
were 66%, 87%, and 93%, respectively. The overall manner of
is indicated in
Time consumption for integrity verification as increase of target video file size and the number of deleted frames
- 6.2. How Many Frames Can Be Recovered?
For digital forensics investigators, in addition to integrity verification, determining how many frames and the location in disk storage from which they can be recovered are also interesting issues
. In our scenarios, we created a scheme associated with frame recovery by calculating the size of the recoverable frame area.
The size of recoverable area
can be determined using
j, k, lo
, denote the number of frames from the deletion starting point to the end of the file, the number of deleted frames, and the size of the frame index area on the original (
) and forged file (
), respectively. Further,
) denotes the size of
For example, if 300 frames are deleted (
= 300) at 420 frames from the end of a frame sequence (
= 420), exactly 120 frames
(S(j – k) = S(120))
will remain in the residual space. However, the size of the recoverable area in the residual space is not exact,
, and a small number of the frames are not recoverable. Because two frame index areas remain at the end of the frame sequences of both an original and a forged video, the size of the area containing the recoverable frames is less than the 120 frames calculated. Therefore, the recoverable area
in this case is calculated as
. In practice,
has a length less than
and the number of recoverable frames is 118 to 119 frames out of the deleted 120 frames.
- 6.3 Is the Integrity Verification Scheme Applicable to Other Video Files?
The index-based integrity verification scheme is applicable to video files depending on the video file structure. An MP4 video file, for example, is composed of sequential media information boxes. The actual frame data is located in mdat (media data box), as indicated in
. Because there is no index list area in the MP4 file structure, the index-based verification method is not applicable to MP4 video files. Thus, a frame-based verification method, such as that proposed in our previous work
, would be able to verify the integrity of MP4 video files.
Basic architecture of the MP4 file format
Windows Media Video (WMV) and Advanced Systems Format (ASF) have file structures that are similar to that of AVI. They are composed of various objects, i.e., header, data, and index objects sequentially. Because WMV and ASF have an index field area at the last part of the file structure, an index-based verification method is applicable to these video formats. Further, the index-based verification method can be applied to other video files where the video formats contain an index field that remains in the residual space after frame deletion.
- 6.4 Does the Proposed Integrity Verification Method Work in File Systems Other Than FAT32?
Storage systems such as SD cards and USB interface storage used by the majority of VEDR devices use the FAT32 file system. FAT32 stores files in sequential sectors. This sequential property is beneficial to index-based verification schemes. Index-based verification schemes that use the information in residual spaces are limited to file systems that have the non-sequential property. In other file systems such as Extended File system (EXT), because a file is stored fragmented into blocks with relative references, the location and content of the index field in disk storage is not trustworthy. That is, a pair of frames in residual space and the related video file stored in the disk space are not determinable.
This study proposed a video file-integrity verification scheme for investigating VEDR records. The proposed scheme detects image frame editing involving frame deletion by checking anomalies in the frame index fields. The proposed index-based scheme is a post-processing approach that operates without the requirement for pre-deployment on the VEDR device. Further, it detects video file manipulation performed using a frame-editing technique, which is difficult using conventional image-based approaches. We demonstrated the verification capability of the proposed scheme in two frame manipulation scenarios with AVI video files recorded using real-world VEDR devices. The evaluation results confirmed that the proposed scheme could be useful in scenarios where maintaining reliable video records is essential for legal evidence.
Choongin Lee received a B.S. degree in Computer Science from Korea University, Korea, in 2015. He is currently pursuing a doctorate degree in Computer and Communication Security at Korea University, Korea. His research interests are digital forensics, software vulnerability analysis, and protocol vulnerability detection.
Jehyun Lee is a researcher at the Cyber Security Research Center (CSRC), Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea. He received his B.S., M.S., and Ph.D degrees in Computer Science and Engineering at Korea University, Korea in 2007, 2009, and 2015, respectively. His research interests include network security and malware.
Youngbin Pyo received a B.S. degree in Computer Science and Engineering from Korea University, Korea, in 2007 and 2014. He is currently pursuing an M.S. degree with the Department of Computer and Radio Communication Engineering, Korea University, Korea. His research interests include digital forensics and image processing.
Heejo Lee is a professor at the Department of Computer Science and Engineering, Korea University, Seoul, Korea. Before joining Korea University, he was at AhnLab, Inc. as CTO from 2001 to 2003. From 2000 to 2001, he was a postdoctorate at the Department of Computer Sciences and security center CERIAS, Purdue University. He received his B.S., M.S., and Ph.D. degrees in Computer Science and Engineering from POSTECH, Pohang, Korea. He serves as an editor of both the Journal of Communications and Networks and the International Journal of Network Management.
Grand View Research,North America car DVR market analysis by product (single channel, dual channel) and segment forecasts to 2016
Evening Standard,How car's black box trapped speeding Rich List heir who left baby paralysed in Range Rover crash
Expert: Digital evidence just as important as DNA in solving crimes
“Forensics Investigations of Multimedia Data: A Review of the State-of-the-Art,”
in Proc. of 6th IEEE International Conference on IT Security Incident Management and IT Forensics (IMF)
May 10-12, 2011
Fox News,Proposed new federal rule could put ‘big brother’ in your driver’s seat
“Roadmap to approaches for carving of fragmented multimedia file,”
in Proc. of 6th IEEE International Conference on Availability, Reliability and Security (ARES)
August 22-26, 2011
Digital Evidence and Computer Crime: Forensic Science, Computers, and the Internet.
Garfinkel S. L.
"Hash-based carving: Searching media for complete files and file fragments with sector hashing and hashdb,"
DOI : 10.1016/j.diin.2015.05.001
“Data Forgery Detection for Vehicle Black Box,”
in Proc. of IEEE Information and Communication Technology Convergence (ICTC)
October 22-24, 2014
“Survey on digital video watermarking techniques and attacks on watermarks,”
International Journal of Engineering Science and Technology
“Detection of forgery in digital video based on pattern noise,”
Journal of Southeast University (Natural Science Edition)
“Video Forgery Detection Using Correlation of Noise,”
in Proc. of 10th IEEE Workshop,of Multimedia Signal Processing
October 8-10, 2008
“Integrity Verification Scheme of Video Contents in Surveillance,”
IEICE TRANSACTIONS on Information and Systems
DOI : 10.1587/transinf.2014MUL0001
Microsoft,AVI RIFF file reference.
File System Forensic Analysis, Addison-Wesley Professional
Lee W. Y.
“Comments on the Linux FAT32 allocator and file creation order reconstruction [Digit Investig 11 (4), 224–233],”
DOI : 10.1016/j.diin.2015.09.003
“Exposing digital forgeries in video by detecting duplication,”
in Proc. of the ACM 9th workshop on Multimedia & Security
September 20-21, 2007
Towards an engineering approach to component adaptation
“Making sense of unstructured flash-memory dumps,”
in Proc. of the ACM Symposium on Applied Computing
March 22-26, 2010
Thing V. L. L.
Chua T. W.
Cheong M. L.
“Design of a digital forensics evidence reconstruction system for complex and obscure fragmented file carving,”
in Proc. of the 7th International Conference on Computational Intelligence and Security
December 3-4. 2011
“Forensic video reconstruction,”
in Proc. of the ACM 2nd International Workshop on Video Surveillance & Sensor Networks
October 10-16, 2004
Na G. H.
Sim K. S.
Moon K. W.
Kong S. G.
Kim E. S.
“Frame-based recovery of corrupted video files using video codec specifications,”
IEEE Transactions on Image Processing
DOI : 10.1109/TIP.2013.2285625