Top Data Storage Challenges

Big data capacity, performance, and management challenges require that companies rethink their storage options. Consider software-defined storage’s benefits.
Big data just keeps getting bigger. According to IDC, by 2020 the digital universe will expand from 4.4 zettabytes to 44 zettabytes. Technology trends like mobility and the internet of things (IoT) are generating tons of data that can fuel crucial business decisions.
For instance, every time a consumer uses a mobile device to interact with your company, they reveal information about their needs and preferences. And every IoT device armed with a sensor provides information about how consumers use products.
The volume of data entering companies from these devices overwhelms storage capacities. Not only that, but this data needs to be organized properly so that it can be retrieved and processed in a timely manner.
Traditional hardware-based storage solutions may not be up to the challenge. Hard disk drives take time to provision. They also lack the performance and data management capabilities required to process and place information where it needs to be.
Data Variety Challenges
Companies are challenged by the amount of data streaming into their systems and the multiple varieties of data they need to contend with. Traditional storage is prepared to deal with structured data. Structured data comes from spreadsheets and database tables, so it is easy to classify and organize. But when it comes to unstructured data, which makes up 80% of new data according to Forbes, management becomes difficult.
And the growth of unstructured data shows no sign of stopping. IDG estimates unstructured data grows at a rate of 62% per year and predicts it will make up 93% of data by 2022.
Unstructured data is made up of text, video, images, and audio that won’t fit into the columns and rows in a relational database. This data comes in the form of Word documents, PowerPoints, instant messages, emails, social media posts, and many more. Within the healthcare industry, medical records are largely made up of unstructured data in the form of doctor’s notes and diagnostic scans. Companies rely on surveillance videos for security.
While structured data may be important for tracking trends, unstructured data explains why these trends are occurring. For instance, comments in social media conversations reveal the sentiments behind decisions your customers make, as well as their thoughts about your products and services. To take advantage of these insights, your business needs storage capable of organizing unstructured data so it can be accessed quickly.
Data Velocity Challenges
Unstructured data is being generated at a rapid rate. For instance, 300 hours of video are uploaded to YouTube every minute.
With the rise of eCommerce, customers generate data during every interaction, not just when they make a purchase. As consumers browse and research products before making a decision, they provide information that can help your business understand them and their needs.
Connected IoT devices have sensors and beacons that monitor the status and location of equipment, providing a constant stream of information. If your storage infrastructure doesn’t have the performance power to handle this increasingly rapid influx of data, your system can develop crippling bottlenecks. High velocity data demands super low latency and lightning-fast input/output rates.
Rethinking Storage for Big Data
To meet the challenges of big data volume, variety, and velocity, your business needs to adopt a new mindset towards storage. Provisioning more hard disk drives to handle rising capacity needs isn’t going to help your company store unstructured data more efficiently or solve the problem of performance demands. The resulting server sprawl ends up eating up your budget with operating expenses like energy and cooling.
Software-defined (SDS) storage is the answer to the big data capacity, performance, and management problems your company faces. SDS separates storage from the hardware layer so you can use more economical commodity hardware while achieving your high-performance and capacity goals.
SDS offers data management capabilities that can detect bottlenecks and transfer mission-critical data to new flash storage. Built-in deduplication and replication helps to optimize your storage.
IBM Spectrum Storage provides intelligent data tiering that can reduce infrastructure costs up to 90% by automatically moving data to more economical storage. Spectrum Storage also enables your business to organize unstructured data effectively by providing a bridge to object-based storage. Object storage quickly identifies and locates unstructured data through metadata.
As an IBM partner, Vicom can help your business make the transition to software-defined storage so you can harness your big data and create a competitive edge.


If you’re coming up on a storage refresh, you may be consider upgrading to a new type of storage. Your hard disk drive storage may be strained by rapidly increasing data loads. Moving to high-capacity, high-performance storage like flash can solve this problem. While the end results are worth it, the process of moving to a new type of storage can be daunting.

Any storage migration can be disruptive and risky. All your systems are connected. Any change to one part can interfere with the rest. Planning ahead and conducting the transition in stages can streamline the process. Following 5 tips for planning a flash storage migration will eliminate some of the stumbling blocks you might face.

The top 5 ways to ease your transition to flash storage are:

1) Determine whether you want to move to an all-flash array (AFA) or a hybrid solution.
Are you all-in, or do you want to combine flash storage with hard disk drives?

AFAs offer tier 1 storage for applications that demand continuous high-level performance. They also offer data management tools like deduplication and compression.

Hybrid flash allows you to use a combination of disk-based and flash storage to prioritize workloads. Workloads that demand a high IOPS rate can be handled by flash storage, while backup and archiving can be handled by lower performing disks.

2) Work with your partner to minimize the amount of business time used to make the transition.
While the migration to an AFA can take less than a day, your company needs to account for the time needed to move applications to the array.

Planned downtime during a migration is inevitable, but you can minimize the amount of time required. Consult with your individual departments to figure out optimal times when applications can be offline. Each department may have an ideal time when they won’t be affected by a shutdown.

3) Make sure your network can work well with a flash storage array.
Oftentimes, decisions about the network were made with slower disk storage in mind, so you need to make sure your network is prepared for the challenge. Flash storage eliminates bottlenecks from backend storage with its high rate of IOPS and super-low latency. As great as these levels of performance are, flash can shift bottlenecks to the network.

You can plan ahead by identifying potential bottlenecks before upgrading to flash. Prevent traffic jams by upgrading your network and preparing for higher speed storage.

To get the network performance you want with flash, you need a minimum of 8 gigabytes per second (Gbps) on fiber channel. If possible, 16 Gbps is ideal to prevent bottlenecks.

4) Determine that you have adequate bandwidth by investing in performance monitoring tools.
AFAs were once reserved for specialized applications to use. Today, they may use extra network bandwidth to run multiple applications and databases. Before moving to flash, you want to be prepared for the network requirements of a higher performing storage solution.

Network performance monitoring can help you establish a baseline to see if you will have enough bandwidth once the transition to flash takes place. Simulating and testing the effects of incorporating flash will help identify which applications may be disabled.

5) Take time to deploy, test, and train your employees on the new system.
Conduct a dry run of the new storage solution to work out the kinks before putting other parts of your system at risk. Once you are confident the system is in working order, then you can move on to installing and testing the network and backup systems.

Working with a new storage solution takes some time to get used to. After deployment, your staff needs to familiarize themselves with the new system. Training sessions will help get everyone up to speed.

A little foresight goes a long way when making a major transition like upgrading your storage. Choosing the right partner to help you through the transition can make all the difference. Consulting with a partner that has a depth of experience with flash migration can work with you to develop the best plan.

How to Leverage Open Technologies with Linux on Power

Implementing Linux on Power is the first step to joining the open source community that collaborates on developing hardware and software innovations.

What if your business could be part of an exclusive think tank made up of the top technology experts and software developers? As a member of this group, you could move to the forefront of IT innovation by accessing the latest applications.

This goal can be your reality if you implement Linux running on IBM Power Systems. By using Linux on Power, you become part of the open source movement that rejects proprietary, closed software in favor of a collaborative approach. By working as a team, open source developers can speed applications to market while providing superior functionality and security.

Once you have Linux on Power on board, your application developers can become part of the open source team.

Embracing the Open Source Philosophy
Open source software is developed according to “the open source way” that expresses a willingness to share, a commitment to transparency, and a desire to play an active role in projects. The open source methodology enables anyone to examine the application’s source code. Users can also modify and improve the application by adding features or fixing flaws.

Benefits of Open Source
When your business uses open source software, you gain control over your applications. Your application developers become empowered to collaborate and innovate. Instead of being forced to use applications out-of-the-box, your team can customize them. Your developers can add features depending on your particular business needs.

Participation in the collaboration process also gives your developers the knowledge and experience needed to become even better programmers. They get an insider’s view of techniques used by top application developers and can use them as a springboard for their own ideas.

Additional sets of eyes inspecting the software also ensure more effective security. Instead of waiting for patches or upgrades, you get real-time fixes. Whitehat hackers work to make sure that operating systems are free from the kind of flaws cybercriminals exploit to infiltrate companies’ systems.

Collaboration leads to faster, more reliable deployments for open source software. The applications are pre-tested and evaluated by a jury of experts before release. Your business benefits from higher quality applications at a lower cost because there is no need for licensing fees.

Linux and Open Source
The Linux operating system was built on open principles. As an open source technology, Linux is customizable. Many distributions of Linux are available that offer a range of software and the ability to switch out key components, like user interfaces.

In 2013, IBM joined the open source movement by investing $1 billion in Linux. Since then, more than 600 developers have created over 400 new software products.

Open Technology Resources
Running Linux on Power makes a variety of open technology resources available to your business. These include:

DeveloperWorks Open
DeveloperWorks Open develops and shares software projects designed for cloud, analytics, mobile, social, and security. They contribute tools and platforms that help developers design applications. In their first year, the group completed over 100 open source projects.

OpenStack Foundation
The OpenStack Foundation shares resources designed for public and private OpenStack cloud. While membership is free, members are expected to provide technical contributions or help build the community. The Foundation serves 30,000 Individual Members in 170 countries as a way of expanding the OpenStack ecosystem. Their goal is to make OpenStack the leading cloud platform. Software developers are encouraged to produce innovative applications for the cloud while keeping community feedback in mind.

OpenPOWER Foundation

As a member of OpenPOWER Foundation, your business contributes to the evolution of Power architecture. Projects include the development of cognitive applications that apply machine learning and natural language processing to formulate hypotheses based on data. To encourage innovation, the foundation creates challenges to encourage the development of particular kinds of applications.

Open the Door to Technological Possibilities
Just by implementing Linux on Power, your business is already benefitting from open technology. But this is only the beginning. Once you begin taking advantage of the foundations, the collaborative world of open source will begin to unfold. Not only can you access cutting-edge applications and software developed by the best minds in the industry, but you can also become a part of the process.

What is the Value of IT?

IT provides a lot of value to the organizations it serves. We all know that; we see and live it each and every day. Most of the organizations we work with could not function effectively without what IT provides. That includes hardware, software, services, and resources. It’s so integral in so many aspects that it’s hard to image it not existing within organizations. But yet, are we truly more productive as organizations (generally) and individuals (debatable) because of IT? That can be debated in many different ways, and each could probably be correct. But it does lend itself to a question: What is the value of IT? That’s what I want to explore this month.

We work with a lot of different types of organizations. Our customers are in media, finance, insurance, state and local government, healthcare, charities, higher education, and many others. Many different types of organizations in many different types of industries. But all have commonality: They are all dependent upon IT. That dependency helps dictate IT’s value overall to the organization. And that value can vary widely between organizations.

IT, in all of its forms, (hardware, software, process, resources, services, etc.) is simply tools. Tools to help an organization do what they do better. Yes, there certainly are organizations that exist solely because of IT, but it’s still a tool that is subservient to the needs of the business. And being tools, their purpose is to serve the organization by providing value. To help a manufacturer manufacture better. To help an insurance company be a better insurance company. To help a university be a better university. If IT doesn’t help to accomplish that, then what’s its purpose? That’s a conversation that my colleagues and I routinely have with our customers. Since we are in IT, we admit we like IT and technology. That a reason we all chose this profession. But for most organizations, they don’t really care about the technology. They care about how that technology services the needs of their organization. Very simple and we all know it, but it’s so true. We routinely speak with our customers on it: If your IT doesn’t provide value, then why invest in it?

When it comes to value and the perception of value, what does that really mean? For each organization that might mean something different. But what is most important? Email, Wi-Fi, network, servers & storage, availability of business applications; all of them; some of them; which? It depends. For some it’s all out how using IT will differentiate them from their competitors to provide them with a competitive advantage. For others it’s just making sure the technology is up and running. So the value pendulum can swing wildly.

Some value is what I can “forced value” versus voluntary or planned value. Organizations that are heavily regulated, such and financial and healthcare institution are required to have their IT provide value just to remain compliant. That forces value. In how they use and protect data, in how they interact with their customers, and how they project their corporate image and brand to the world. This is important value, to be sure, but in many cases it’s something that they have to do.

There are organizations that still see IT as a liability and cost center versus something strategic that can help drive value and differentiate themselves out in the market place. They see it as money being spent versus invested smartly. Again, a different perspective around value. I strongly believe that money well spent on IT can produce dividends and provide strong value. Perhaps we are bit biased given what we do, but we see firsthand the value received through IT. Conversely we also see the challenges that organizations face when IT does not provide expected value. When it doesn’t do what it’s expected to do.

A trend we continue to see is where CEO’s are tasking CIO’s with driving value through IT. CEO’s are asking (and demanding) how IT is going to help make money and drive strategic value, versus just being a cost center. I believe it’s great move. It causes people to truly reflect on what IT can do, versus just being technology that is just there to fill a need. IT can be much more valuable and it requires leadership and vision to help drive it to being strategically important. It’s can also require a change in culture as well. But a good change.

How can organizations do that and why would they? Value, of course, can be found in different types of hardware, software, services and resources. Some see more value in strong hardware or cloud platforms to run their business applications. Some need software tools and efficient processes to help drive value within their environment. Some need services and resources to tie it all together to ensure optimum value. Most need a combination to find the value that they need.

You’ve heard me say before but this absolutely where good partners can help. A good partner can help you find the value in your IT. In the hardware, in the software, in the services and resources to help you understand and realize the value that can be expected. Good partners bring the experience of working with lots of different types of organizations who have similar challenges. As the saying goes: they’ve been there and done it before. There is enormous value in that. There is undeniable value in hearing from and working with a partner that has helped many organizations with similar challenges and issues. I believe they also should be proactive in wanting to tell you about where they’ve done it before. That is the mark of a good partner. Not necessarily what they can do (but that is of course, important), but where they’ve done it before. That’s powerful. That brings the value out of IT. In a way that is valuable specifically to them.

Getting value out of your IT (in all forms: HW, SW, services, resources) is critical, otherwise why invest in it? Also recognize that that value can change and most likely will grow over time. That’s a good thing, which means the organization is growing and its needs evolving. And IT can be there to help if it’s approached and done right. It’s our job as a partner to ensure our customers are doing it right and seeing value, and it’s up to our customers to embrace and see the value of IT. That value will lead to business success. And there’s lots of value in that.

Until next month.

Andy Jonak