Storage Challenge: The Degree of Content Value
Living in Silicon Valley it is very hard to remind myself that everyone doesn’t live on three – five devices constantly…24×7, 365 days of the year.
It’s so cool! It’s such a great distortion field I feel sorry for those who don’t eat, sleep, live it.
But it’s possible everyone is getting with it:
– Pew Research reported that 84 percent of the adults in the U.S. are on the Internet and what the other 16 percent does, I have no idea.
– Statista reports there are nearly 5B mobile phones in use globally (some have 2+) so they can text and occasionally even talk to each other.
– We’re inching closer to having everything/anything connected, which is why we ran out of IP(Internet Protocol) Version 4 addresses and moved up to IPv6 so the 50B things can chat.
No wonder attendees at Flash Memory Summit (FMS) 2015 were positively giddy.
Microprocessor speeds double every two years and device prices get lower, so we can create more things, stuff faster/better to be stored.
This year, we’ll pack away more than 3.5ZB (Zetabytes) of data.
By 2020, they’ll be hunting for places to put 44ZB.
Yes, 44 000 000 000 000 000 000 000 bytes!
More than Ever – We’ve created more content in the past two years than in the previous 5,000 and the volume of data, pictures, movies and things continues to grow at an accelerated rate. No wonder you’re constantly buying storage devices.
There are basically two parts of the storage market – devices we all use and the boss’ big IT (information technology) departments plus big storage chambers theoretically in the sky – The Cloud. There’s more, but those are put in the “other” bucket.
Spinning disks have been around since the 1950s when IBM’s RAMAC 305 held 5MB of data at $10,000 a megabyte – big as two refrigerators with 50 24-inch platters.
They squeezed down into a more manageable 2.5-in size for portable computing in the mid-1980s with capacity doubling and prices dropping about 50 percent every two years. The brutal capacity/price push shrunk the industry from 200 plus producers down to today’s three – Seagate, Toshiba, WD (Western Digital) and their subsidiaries.
They were doing great until 1984 when Toshiba introduced the first flash memory (electronic non-volatile computer storage) that can be electrically erased and reprogrammed. RAM (random access memory) SSD (solid state drives) started to appear in 1979 but you and I didn’t care because the cost was outta’ sight.
But by this year, flash was feeling its oats and was out to crush the marvelous spinning disk Flash is a lot like a computer’s processor; it doesn’t make a sound, just stores stuff.
There are just a few who make the chips but hundreds of other big/little, different/unique players are scrambling to get their part of the estimated $45B market.
The storage market is huge … and ultra-competitive.
Up and Up – Businesses, industries, governments and individuals are producing/storing content everywhere. With prices coming down for flash-based storage, sales have increased significantly. But, according to IDC, as good as the numbers and features are, flash continues to be a fraction of the total storage market. About 75 percent of our content is still stored on hard drives.
The little devices are being designed into damn near everything because of their flash fast (pun intended) data transfer rates, ultra-low latency (delay between input to outcome) and fast access time (time between asking/getting the answer).
Flash manufacturers don’t just make more storage space, now they want to make it cheaper and tougher for rough environments.
At FMS, they had more than 100 sessions crammed into three days with maybe 6,000 plus engineers talking acronyms and chippie stuff. They saw chips everywhere – 50B IoT (Internet of Things) sensors, transportation (an estimated 60 percent of today’s car cost is electronics), billions of smartphones and more.
Normal people don’t care about chips or disks though.
They simply want reliable capacity to handle the stuff their device captures/produces.
Some of us move between our desktop, notebook, tablet and smartphone.
A boomerish lady friend in SoCal recently explained her systems are work tools so she adds power and speed to conserve her budget.
“RAM is the energy of my system,” she said, “so I put as much as possible in my five-year-old systems and they’re better than new ones.”
Juiced Up – With the increasing demand for more creation/communications devices, there could be a very good reason desktop/notebook sales are off. More and more consumers choose to upgrade/enhance their older systems with more RAM and SSD in place of hard drives to extend the performance and use life of the systems at a fraction of the cost.
She swapped out the HD for an SSD, pulled the ancient DVD burner and threw in second SSD and BAM! 2TB of fast, responsive, reliable storage. And she carries three 1TB USB 3.0 SSDs in the backpack, just in case.
Most of the folks we talked to at FMS voted for flash for performance and HD for capacity.
At home, we share a 32TB Thunderbolt 2 software RAID home cloud (spinning disks); and at the office, three of the hummers – 2M filing cabinets of text or about 29,000 hours of video – taking up less space than a microwave.
That didn’t stop speakers from proclaiming HD would go the way of tape and optical discs in the enterprise.
And there are good reasons. Flash:
– is inherently fast, easy to manage
– compresses data more efficiently, removes dup files
– takes up less space in data center, requires less electricity/cooling
– jobs get done faster, better
All-flash arrays for enterprises have grown faster as prices decline. That’s probably why IDC has projected the all-flash systems market to reach $2.24B this year.
Shrinking Difference – Historically, there has been a significant price difference between flash-based storage and hard drives; but thanks to engineering breakthroughs and production scale increases, the price differential has become less and less a factor. The only differential continues to be capacity and availability.
Alex Grossman, of Quantum, which specializes in studio/station production solutions, noted during one of the M&E sessions that their customers see tangible benefits with faster workflow and lower admin costs.
“These people are at the tipping point,” he commented. “They know what days of saved production/post time does for their bottomline. Until the project is wrapped, they don’t make money. So all-flash solutions are easy to justify.”
Beyond the entertainment market, he said that Quantum’s enterprise customers often use a two-tier approach with a mix of about 20 percent all-flash and 80 percent spinning disk storage.
Why the split?
According to a recent EMC report, 20 percent of the stored data is active and the other 80 percent is seldom, if ever, touched.
Wikibon is wildly optimistic about flash sweeping HDs aside but noted that last year; about 1,000 times more disks were sold than flash.
It didn’t seem to bother FMS attendees, which were why Gabriel said, “Don’t confuse kindness with weakness.”
While the suppliers at FMS love Wikibon’s projections and love to show them to their customers and investors, they gloss over two important facts – we’re producing data/content faster than we can store it and there isn’t enough flash storage production capacity –globally – to take over the storage world.
HDs are still the main storage medium used but flash is important for the growing enterprise big-data analysis arena.
Today, they’re just trying to keep up with a place to store the stuff.
Outta’ Sight – Individual and organization creation of content has become overwhelming for data centers to capture, identify/categorize and retain. It’s less a matter of which storage technology you’re using but do you have enough capacity and ultimately, can you access and retrieve specific structured and unstructured data.
So why not simply crank up production?
Rob Peglar, vice president of Micron’s Advanced Storage, said his firm would love to see increased use of dense flash and reduced reliance on HDDs.
“Just like the film/content people you talk with, it’s a matter of budget – money and time,” he commented. “We’re squeezing every bit of production possible out of our present plants and planning expansion. Significantly expanding a fab requires $4 billion or more of investment. And it takes around two years to get it up and running smoothly.
“Given the long timeframes and capital expense required to expand production, there are always periods of supply/demand imbalance,” he added. “But this is a good problem to have – I have no doubt the use of flash will increase strongly over time. Flash density in a drive form factor is now beyond where HDD is – this is the disruptive effect of 3D NAND”.
Peglar said he does see major flash growth in the hyperscale cloud services but believes that will be dominated over time by dense flash, not necessarily leading-edge high-performance solutions.
Hard drives will continue to be used when price, capacity and availability are the primary considerations.
Flash will win where speed, ruggedness, form factor and fragmentation are important factors.
As hungry as people are for creating stuff and storing it, we just don’t believe the doomsday warning Seagate issued last year will ever come to pass…“The world could run out of storage capacity in two years.”
If it gets even close, people logic says prices would go up because the more in demand you are, the more you can charge.
At that point, folks might heed Juhana Venhalainen’s, University of East London, suggestions.
In a presentation he noted, “Our reluctance to throw away anything has made our storage infrastructure into something approximating a kitchen junk drawer. Instead of focusing narrowly on capacity allocation efficiency, planners need to consider capacity planning efficiency.”
I guess he means that just as we have a line at the end of our emails that says, “Think before you print,” we should have an image on our screen that says, “Think before you save.”
It’s a good thing half of that stuff is “white noise” and will simply disappear on its own.
A lot of executives at company and government hacked locations as well as folks who checked into Ashley Madison would like to see their online content gone as well.
But as Gabriel said, “You disappoint me, it’s the greatest good.”