Storage Solutions for Video Surveillance

Video surveillance storage is different from standard IT data storage. First, it is important to realise that video surveillance data is fundamentally different from IT data and yet we use the same technology to store both. This technology is based on Hard Disk Drives (HDD) often set within Digital Video Recorders (DVRs), just the same in as in a standard networked data server or PC.

How Different Is Video Data from IT Data?
This is clear from looking at what actually happens in the process of storing and retrieving the different types of data. Video data is written to the storage device sequentially in a relentless, never-ending 24/7 process. Video data is 97% input (i.e. recording) and only 3% output (playback). Video is only reviewed if an event is spotted by the security monitoring team, an alert sent by video analytics software or somebody reports an incident.

IT data, by contrast, is typically processed in a ‘read-modify-write' manner via random access Input/Output (I/O) to a database, usually involving high data rates but small data volumes each time a document modification is made.
Modern HDDs are generally very reliable for typical data storage applications. However in more data-intensive storage applications such as video surveillance in DVR systems, HDDs are frequently stretched to their limit. So much so that the HDD is today commonly considered to be the weakest link in any disk-based DVR system. When the volume of data generated by a video surveillance system is large and the required storage period before overwriting is extended (i.e. up to 30 days) then typically ‘arrays' of HDDs are used to store the data. But when quantities of HDDs are used in a disk array the likelihood of a drive failure increases in direct proportion to the number of drives employed.

This is worrying enough for enterprise video surveillance systems but it gets worse. As the recorded data can be critically important, perhaps even a regulatory requirement, so vendors have built-in redundancy using fault tolerant disk array schemes, again developed for the IT industry (e.g. RAID 1, RAID 5 and RAID 6).

RAID
RAID (Redundant Array of Inexpensive Disks) allows computer users to achieve high levels of storage reliability from low-cost and intrinsically less reliable PC-class disk-drive components. RAID 5 is the most commonly used RAID system in IT applications today and is now in wide use for video surveillance systems. The basic concept is a good one. Data (and a copy of the data also called ‘parity data') is spread over all the disks in the array such that any one disk can fail without causing significant data loss. In the event of a failure, the RAID controller regenerates the data on the failed disk onto a new disk from the main data and the parity data which spread across all the surviving disks.

The problem with this approach is that reconstruction of the lost data on the new disk is both processor- and disk-intensive and complex. If you put this stress into a real-time recording application it is further complicated by the need to carry on writing new data (across all the disks including the new one) whilst simultaneously reconstructing the lost data.

For large RAID 5 arrays in DVRs today (i.e. with disks holding several thousand GBs each) the reconstruction process for a failed disk can take many hours or even days. During this period the disk array is under considerably increased read/write stress, especially if the incoming data streams (for recording) are high bandwidth. The fundamental problem here is that if another drive fails during the process (or indeed before the failed drive has been replaced) then ALL the data is lost across the entire array - a catastrophic data loss.

Very Real Risk of Disk Failure Cccentuated in RAID Devices
Mean Time To Failure (MTTF) figures quoted by drive manufacturers appear to understate disk failure rates as reported by real Annual Return Rates (ARRs) by more than three times. One extensive study, carried out by Bianca Schroder and Garth Gisbo at Carnegie Mellon University, which tracked 100,000 drives, estimated that manufacturers' MTTF average was 0.88% (that is the likelihood of failure within the first one million hours of service) whereas real analysis of failed disks returned to manufacturer in a single year (the ARR) found the numbers to average 3.01%, with some batches of drives running at 13% ARR. Worrying findings indeed, particularly when you think disks storing video surveillance data are under much more stress and usage than the normal IT usage they were designed for.

Three key reasons for failure are temperature, vibration and wear. Conventional JBOD (Just a Bunch of Disks) and especially RAID arrays present hard disks with a dangerous combination of these three factors, exacerbating problems and shortening disk lifetimes, especially when low cost PC-class components are used. Increased disk capacity, as I explained earlier, extends disk drive rebuilds on failure and increase the risk of catastrophic failure.

In addition both RAID 5, and its successor RAID 6, are energy hungry and complex. They really require heavy duty HDDs which are totally inappropriate for mass transportation of evidential data. These constraints have been compounded by the explosion in use of megapixel cameras which require far higher storage capacity levels.

Electricity supply issues such as power spikes and surges can also cause disks to fail, as can ‘power droops' (sometimes called brown-outs) when lower voltages are delivered to disk drives, often for a fairly short time periods. Individually and with light use, such as in a standard PC network, disk drives are extremely reliable. However in tightly-packed arrays, running hot with 100% duty cycles and constant vibration from the activity of all the other disks in the array, the hard-working disks do occasionally and understandably fail. This is what is required of them today in DVR systems up and down the land and essentially the technology is not suited for it.

Creating a New Storage Solution: Coldstore
To create Coldstore we looked at the problem of video surveillance storage from first principles. We studied how video is recorded and how it is retrieved and analysed in this market. We then designed a storage solution which recognises the specific needs of video surveillance systems. We designed a solution which offered four key elements which, our knowledge of the video surveillance market, told us were key: simplicity, reliability, energy efficiency and scalability, all at a reasonable price.

Simplicity, Reliability and Energy Efficiency

Simplicity
Simplicity was achieved by building a Network Attached Storage (NAS) array which can use any make and size of SATA disk and indeed any mix of disks. Disks can be added ‘on the fly' and will automatically be incorporated into an array. Disks may also be extracted at any time (if key evidence needs to be transported and shown to the police rapidly for example). And because they record in strictly sequential pattern (i.e. fill up disk in bay number one before moving to disk in bay number 2) the recorded time span of any disk can be displayed on the front panel of each disk. This means that we can extract disks at any time and also that individual disks may be played on a standard PC via a USB cradle. Simplicity does not end there. Veracity provides a simple Software Developers Kit (SDK) to DVR/NVR manufacturers who wish to provide their end users with the benefits of Coldstore. Direct control of Coldstore via defined network protocol is an alternative method of integration. Coldstore even supports browser-based array configuration and management remotely. The idea is that disk management is now simple enough for the end user to manage it themselves, without calling in their installation partner and incurring call out charges.

Reliability
Much lower risk of disk failure was achieved by developing our own Linear Array of Idle Disks (LAID TM) and a unique Sequential disk Filing System (SFSTM) so that video data records sequentially through the array, in a unique overlapping mirrored pair scheme, which provides redundancy without requiring twice the number of drives. The result of this for Coldstore is that all disks are off (on average) for 87% of the time. SFS controls the disk read/write heads, moving them across the disk very much like a vinyl record player, virtually eliminating disk vibration caused by the normal random movement of the read/write arm.

Energy Efficiency
The additional advantage of the sequential approach is that much less power is used because most disks are not running and disks' read/write arms are working much less hard. Taking both LAID and SFS, together with a custom-designed low-power main CPU board, results in a unit which hardly needs any cooling at all (more energy saved on driving fans), we have found Coldstore uses no more power than a standard light bulb at 56 watts (and often less than even that figure) even with a fully loaded 30TB array.

Scalability
Scalability is achieved by enabling multiple DVRs to access a single Coldstore, or vice-versa. Further, multiple Coldstores can be linked in series, or in parallel, for very large camera counts or very long archive periods.

Summary
We are not expecting RAID5/6 systems to disappear overnight, although we believe that it is now an inappropriate and outdated technology for modern video surveillance. Coldstore has been designed to meet the specific requirements of the surveillance market and is ready for a future dominated by megapixel video surveillance recording, where storing and easy transportation of vast volumes of video data is essential. All this has been achieved with extremely low running costs and very high reliability.