Space Wars. A Guide for the Ultimate Hard Drive Configuration!

Introduction
Since we began creating Machinima, configuring hard drives and finding space for projects has been an endless battle. Capturing video can stack up terabytes of data on your computer and chances are you’re going to need to plan ahead so you don’t run out. Before I started a career in Machinima, I was an Information Technology consultant and am A+ and Microsoft Certified. My experience with computers and technology has been valuable in helping to decide how to plan ahead for our growing technology needs. I hope this guide helps you to understand and explore new ways to maximize your storage solutions on a small or moderate budget.

All Configurations are Not Equal in Performance
Most devices can be configured in multiple ways that make it optimal for specific tasks. You will need to consider what tasks you'll be using each volume for before adding hard drives or other storage devices to your computer. Depending on whether you’re capturing, editing, or just archiving your projects can make a huge difference in what kind of devices you should choose and how to configure them. To begin, here are some terms you should know:

RPM – Rotations per minute.
Cache – A buffer that data is stored in between the physical disk and computer.
Latency – The amount of time it takes to access the drive from the initial request.
Interface – The method which the drive is used to connect to your computer.
SATA Interface – Serial ATA (currently there is SATA I & II, II being faster)
Volume - A single accessible storage area on one or multiple storage devices.
RAID - Redundant Array of Inexpensive Disks

1.    Reading & Writing Data
In most cases hard drives can read data much faster than they can write. With a single drive you will likely be able to read data fast enough for basic editing and capturing. If you’re running a single drive setup, I’d recommend a minimum 7200 RPM, 32MB ~5ms latency, and the SATA 2 interface.

2.    Storing Data
Sometimes it’s not important how fast you can access your data, but how much you can store. For the purpose of backing up or archiving past projects you want drives that can hold a lot more data. If you have several hard drives in your system and plan to upgrade, you can use your older drives to backup and save files that you don’t need access to very often. A good storage drive is 500GB to 1TB (1000GB). I know this will be outdated soon, but currently a 1TB-1.5TB drive offers the best value per gigabyte. Whenever in doubt, divide the number of gigabytes by the price and compare various sizes to find the best value.

3.    Sharing Data
If you have multiple computers or are on a network you may wish to share your data. Having a designated place to store your files is important if you plan on sharing it later; for instance capturing video on one computer and editing it on another. Not all external hard drives are as fast as internal, but they allow you to move your data easily by connecting it to a USB, Firewire, or eSATA port. eSATA ports allow you to connect external drives through the same interface that most computers use internally (SATA) offering improved performance and speed.

4.    Running your operating system and programs
A designated drive for your operating system and programs is important because this drive can be under constant stress. If you’re trying to edit or capture video on your system drive any interruption can slow you down or even cause a crash. Typically you want a fast system drive that’s a medium size so you don't store too much data on it. By keeping only operating system files and programs on your system drive it makes it much easier to manage and backup your system in case it crashes or fails.

How many drives?
As a minimum, try to install at least two storage devices: One specifically as a system drive and the other for saving your projects or capturing audio and video. Having too many drives can be as problematic as not having enough, so don’t get carried away. For an optimal configuration I would recommend the following:

•    A volume for your system (small to medium size, medium to fast speed).
•    A volume for video captures (medium size, fastest speed).
•    A volume for projects & editing (largest size, medium to fast speed with redundancy).
•    An external volume for storage (large size, slow to medium speed).

You can create multiple partitions on a single drive so each volume doesn't necessarily need a separate storage device. However, you will get the best performance if you are only using each drive for one purpose at a time. For example, try not to use your system volume to capture video.

Internal or External Storage?
If you’re looking to pack as much power into your box as possible, internal storage is a tidy way to do it. It’s usually a bit cheaper than external, but you may end up spending more on additional components like a more robust power supply and case to fit it all in. If you don’t feel comfortable installing components, plan to swap devices frequently, or work on multiple computers, external storage gives you a lot more freedom to move your data around. It’s much easier to swap drives if you don’t have to break out the toolbox. Since it really comes down to your personal preference, I’ll just recommend that you do a lot of research and crunch the numbers to find what fits in your budget and gets the job done.

Internal Configurations for Reading & Writing Data
Capturing and editing video is an intensive process and there are several ways you can optimize a single or multiple drives to read and write data faster.

Single Drive Configuration
When picking a single drive you want a fast RPM, large buffer, and short access time. Standard drives come in 5,400, 7,200, and 10,000 RPM. The faster the rotation, the quicker data can be written to the drive. The size of the drive is up to you, but I always consider how it will be used because faster drives are much more expensive per gigabyte. If the drive has a single purpose like capturing video it may not be as important to have a lot of space because once the data has been captured it can be moved to another volume. However, if I were choosing a single drive for capturing video and editing projects I’d recommend a drive with a good balance of speed and size.

Configuring a RAID
Raids combine multiple disks to create single volumes for increased performance and/or redundancy. To create a raid you will need at least two identical hard drives and a raid controller. Some computers come with a raid controller, but if you don’t have one you can pick one up and install it yourself. The cost is about $50 to $300 for the controller depending on the brand, configuration, and number of drives you want to install.

RAID 0
For capturing large video files you may also consider RAID 0. This configuration allows two or more hard drives to work as one logical disk. If you have two 500GB hard drives you can combine them to make a single 1000GB partition. When data is written it is split between each volume so your write speed will improve for each physical disk you add to the raid.  The downside to this configuration is that if a single drive fails or becomes corrupt you will lose all your data. While an excellent way to capture high resolution video, I highly recommend that you relocate or backup your files on another drive once you’ve captured everything you need.

RAID 1
Another common configuration is RAID 1. This works similar to RAID 0 except that your physical disks become mirrored and work as one. This means your two 500GB drives would create a single volume of 500GB. While this may seem less attractive, RAID 1 allows one of your drives to fail without losing any data. If you value the projects you’ve worked so hard on, RAID 1 is an inexpensive way to add a first line of defense against a hard drive crash. Despite that you’re running two drives, RAID 1 is not faster for reading and writing. In fact, the performance can be worse than a single drive. I would still recommend RAID 1 if you only have two drives and no way to backup your data.

RAID 1+0
If you’re looking for a great combination of performance and redundancy, RAID 1+0 offers the best value and reliability. The downside is that you’re going to need at least 4 hard drives (expandable in pairs) for this configuration. This setup works by creating mirrored sets and then striping them together, potentially allowing multiple drive failures without data loss and improved performance. If you’re planning on capturing and editing from a single (logical) storage device, this is probably the best overall configuration.

Other RAID configurations
There are dozens of ways to configure a raid and a few searches will bring up all the information you’ve ever wanted to know. For the sake of simplicity I’m only recommending the above configurations because they’d be the most likely to be used for Machinima production. If you do happen to come across a stack of hard drives and really want to burn them up you might look into RAID 5 or 6. Just keep in mind that in most cases you’re not going to gain much performance.

External Configurations for Reading & Writing Data
There have been a lot of technological improvements over the years and external storage is looking more attractive than ever. Besides that it’s much cheaper now, the options available go well beyond just plugging in a USB hard drive. NAS devices or Network-attached storage allows you to set up additional volumes or even a raid through your LAN or wireless network. You can also buy devices that plug in through high speed firewire and eSATA ports. Practically anything that you can install internally can be purchased as an external device, just make sure you research everything before you buy. Not all external devices are created equal!

USB Storage
Hard drives have been available as hot-swappable USB storage for a long time. They’re usually a reliable and easy way to add huge amounts of data for immediate access at reasonable prices. It’s important to know before you buy a USB hard drive that you may not be able to capture and edit Machinima on one. These drives offer limited read and write speeds that vary by chipset, manufacture, and the actual speed of your USB port. If you plan on using USB drives for Machinima tasks, do your research and look for reputable brands with lots of user reviews. Sometimes the specs on the box aren’t what the end user gets, so be really careful or you might end up with a door stop. If you’re just looking for a big tank to throw old projects on and don’t mind letting files transfer all night, you might browse the clearance rack.

eSATA & Firewire Storage & RAIDs
If you’re looking for an external alternative to internal storage devices, nothing beats the value of an external multiple bay hard drive enclosure. These come in a wide variety of configurations, from diskless (install your own drives) enclosures to pre-configured RAIDs with 2, 4, or more hard drives installed. All you need is a firewire or eSATA interface on your computer which can be purchased pretty cheap if you don’t already have one. Single drives with this interface are a good alternative for USB storage, just remember to plan ahead and buy an eSATA interface card that has enough ports for the amount of devices you plan to install.

NAS – Network attached storage
If you have a fast wired or wireless connection linking your computers together you may also consider adding network attached storage (NAS). While I wouldn’t recommend using a NAS device to edit or capture video, they’re great to backup, archive, or share data between multiple computers. These generally plug into your router or switch using a network cable, but some can also be configured alternatively to connect through wireless or directly to your computer. Since NAS devices are connected through your network, you can keep that stack of noisy drives and cooling fans in a separate room and expand it easily. You can also get a NAS with hot swap bays so you can add or replace your drives as needed without a toolbox.

Links to example devices
These links are provided as examples and not necessarily items you should run out and buy without a fair amount of research first. Even though I've had a lot of experience with various configurations, I can't promise these will all work well for you. Some devices also require the knowledge to configure them properly, so please make sure you know what you're buying and how to install it!

System Volume:
Western Digital 500GB 7200 RPM Hard Drive

Video Capture Volume:
Western Digital VelociRaptor 300GB 10,000 RPM Hard Drive

Projects & Editing:
Western Digital 1TB 7200 RPM Hard Drive

USB Storage:
Western Digital My Book 1TB Hard Drive
Lacie 1TB Hard Drive
Sans Digital 4-Bay SATA to USB Enclosure

eSATA & Firewire Storage:
Fantom Drive 1TB USB/eSATA Hard Drive
Iomega 1TB USB/eSATA Hard Drive

External RAIDs
Sans Digital 8-Bay eSATA Enclosure
Sans Digital 2-Bay SATA to USB/eSATA Enclosure
StarTech eSATA 4-Bay Enclosure
StarTech eSATA 2-Bay Enclosure

NAS Devices
D-Link 2-Bay Diskless Enclosure
Netgear 4-Bay Diskless Enclosure

In Conclusion
There's not a single configuration that's perfect for every task and there are many ways to get the same results in a different package. It's easy to end up with stacks of hard drives that quickly become full with nowhere to plug them in so it's important to plan ahead and create a strategy that can grow with your storage needs. Consider moving your old drives to hot swappable USB enclosures so you can save space for newer drives but still have access to that data.

Also, don't forget to plan a way to backup your projects and archive them externally to make space for new projects. Always do your homework to make sure you're using the right kind of drives and configuring them for the right tasks. And don't waste your money on expensive components unless you have a specific purpose for them and know for sure that it'll provide a clear benefit to your workflow. Good luck!

Credits
Written by: Clint Hackleman, Producer