Investment in big data technologies continues to expand, according to a recent market study by Gartner, which found that 73 percent of survey respondents have invested or plan to invest in big data in the next 24 months -- that's up from 64 percent in 2013.
Organizations are starting to move on their big data investment strategy. Moreover, the number stating they had no plans for big data investment fell from 31 percent in 2013 to 24 percent in 2014.
"Big data investment continues to be led by North America, with 47 percent of organizations reporting investment, which is up from 37.8 percent in 2013," said Nick Heudecker, research director at Gartner. "All other regions experienced increases in investment over the last year."
However, this increased investment has not led to an associated increase in organizations reporting deployed big data projects. Like 2013, much of the work today revolves around strategy development and the creation of pilots and experimental projects.
Gartner believes that 2013 was a year of big data experimentation and early deployment, and so is 2014. In 2013, only eight percent of organizations reported having big data projects deployed to production. This has increased to 13 percent in 2014.
However, the six percent drop in organizations still gathering knowledge about big data and the seven percent increase in pilots and experiments indicate that organizations are evolving in their understanding and willingness to explore big data opportunities.
According to the Gartner assessment, big data can help address a wide range of business problems across many industries and for the third year in their study, both enhancing the customer experience and improving process efficiency are the top areas to address.
The most dramatic changes are in enhancing customer experience, especially in transportation, healthcare, insurance, media and communications, retail, and banking. Another area where they see an increase is using big data to develop information products, where organizations are looking to monetize their data. This is especially true among IT vendors, government and manufacturing.
Gartner continues to see planned investments across all vertical industries as communications and media continuing to lead the pack, with 53 percent of organizations surveyed having already invested and a further 33 percent planning investments in big data technology.
The other year-to-year changes in the survey findings are a function of the adoption stage. As organizations move beyond knowledge gathering and developing a strategy to making investments, piloting and deploying, the challenges they face become more practical.
Those with no big data plans feel the big hurdles are determining how to get value from big data, defining a strategy, leadership or organizational issues, and some are still trying to understand big data.
In the planning stages, beyond determining value, the top challenges are obtaining skills and capabilities needed, defining strategy, obtaining funding, and beginning to think about infrastructure issues. Companies that are further along with investments must begin to address risk and governance issues, data integration and infrastructure.
When it comes to the volume, variety and velocity aspects of big data, volume received most of the focus. Increasing data volume is easily understandable: you're getting the same data you had before, but at massive scale. Volume is also the easiest to deal with by increasing storage and compute capacity.
On the other hand, data variety is far more challenging. Getting value from a variety of data sources, such as social media feeds, machine and sensor data, as well as free-form text, requires not only increased storage capacity, but also different tools and the skills to use them.
The challenges introduced by analyzing a variety of data sources may explain why most organizations are studying traditional data sources for their big data projects. Those organizations analyzing transactions increased from 70 percent in 2013 to 79 percent in 2014, while those analyzing log data fell slightly by two percent.
Organizations are starting to move on their big data investment strategy. Moreover, the number stating they had no plans for big data investment fell from 31 percent in 2013 to 24 percent in 2014.
"Big data investment continues to be led by North America, with 47 percent of organizations reporting investment, which is up from 37.8 percent in 2013," said Nick Heudecker, research director at Gartner. "All other regions experienced increases in investment over the last year."
However, this increased investment has not led to an associated increase in organizations reporting deployed big data projects. Like 2013, much of the work today revolves around strategy development and the creation of pilots and experimental projects.
Gartner believes that 2013 was a year of big data experimentation and early deployment, and so is 2014. In 2013, only eight percent of organizations reported having big data projects deployed to production. This has increased to 13 percent in 2014.
However, the six percent drop in organizations still gathering knowledge about big data and the seven percent increase in pilots and experiments indicate that organizations are evolving in their understanding and willingness to explore big data opportunities.
According to the Gartner assessment, big data can help address a wide range of business problems across many industries and for the third year in their study, both enhancing the customer experience and improving process efficiency are the top areas to address.
The most dramatic changes are in enhancing customer experience, especially in transportation, healthcare, insurance, media and communications, retail, and banking. Another area where they see an increase is using big data to develop information products, where organizations are looking to monetize their data. This is especially true among IT vendors, government and manufacturing.
Gartner continues to see planned investments across all vertical industries as communications and media continuing to lead the pack, with 53 percent of organizations surveyed having already invested and a further 33 percent planning investments in big data technology.
The other year-to-year changes in the survey findings are a function of the adoption stage. As organizations move beyond knowledge gathering and developing a strategy to making investments, piloting and deploying, the challenges they face become more practical.
Those with no big data plans feel the big hurdles are determining how to get value from big data, defining a strategy, leadership or organizational issues, and some are still trying to understand big data.
In the planning stages, beyond determining value, the top challenges are obtaining skills and capabilities needed, defining strategy, obtaining funding, and beginning to think about infrastructure issues. Companies that are further along with investments must begin to address risk and governance issues, data integration and infrastructure.
When it comes to the volume, variety and velocity aspects of big data, volume received most of the focus. Increasing data volume is easily understandable: you're getting the same data you had before, but at massive scale. Volume is also the easiest to deal with by increasing storage and compute capacity.
On the other hand, data variety is far more challenging. Getting value from a variety of data sources, such as social media feeds, machine and sensor data, as well as free-form text, requires not only increased storage capacity, but also different tools and the skills to use them.
The challenges introduced by analyzing a variety of data sources may explain why most organizations are studying traditional data sources for their big data projects. Those organizations analyzing transactions increased from 70 percent in 2013 to 79 percent in 2014, while those analyzing log data fell slightly by two percent.