These days it is all about Big Data. More organizations are storing, processing, and extracting value from data of all forms and sizes as it becomes increasingly available through various data-gathering devices. As such, systems that support large volumes of both structured and unstructured data will continue to rise. The market will demand platforms that help data custodians govern and secure big data while empowering end users to analyze that data. These systems will mature to operate well inside of enterprise IT systems and standards. Here are three trends you can expect to see in Big Data in the coming year as determined by Tableau in their latest whitepaper.
1. Big data becomes fast and approachable: Options expand to speed up Hadoop
Sure, you can perform machine learning and conduct sentiment analysis on Hadoop, but the first question people often ask is: How fast is the interactive SQL? SQL, after all, is the conduit to business users who want to use Hadoop data for faster, more repeatable KPI dashboards as well as exploratory analysis. This need for speed has fueled the adoption of faster databases like Exasol and MemSQL, Hadoop-based stores like Kudu, and technologies that enable faster queries. Using SQL-onHadoop engines (Apache Impala, Hive LLAP, Presto, Phoenix, and Drill) and OLAP-on-Hadoop technologies (AtScale, Jethro Data, and Kyvos Insights), these query accelerators are further blurring the lines between traditional warehouses and the world of big data.
2. The convergence of IoT, cloud, and big data create new opportunities for self-service analytics
Everything will have a sensor that sends information back to the mothership. IoT is generating massive volumes of structured and unstructured data, and an increasing share of this data is being deployed on cloud services. The data is often heterogeneous and lives across multiple relational and non-relational systems, from Hadoop clusters to NoSQL databases. While innovations in storage and managed services have sped up the capture process, accessing and understanding the data itself still pose a significant last-mile challenge. As a result, demand is growing for analytical tools that seamlessly connect to and combine a wide variety of cloudhosted data sources. Such tools enable businesses to explore and visualize any type of data stored anywhere, helping them discover hidden opportunity in their IoT investment.
3. Variety, not volume or velocity, drives big-data investments
Gartner defines big data as the three Vs: high-volume, high velocity, high-variety information assets. While all three Vs are growing, variety is becoming the single biggest driver of big-data investments, as seen in the results of a recent survey by New Vantage Partners. This trend will continue to grow as firms seek to integrate more sources and focus on the “long tail” of big data. From schema-free JSON to nested types in other databases (relational and NoSQL), to non-flat data (Avro, Parquet, XML), data formats are multiplying and connectors are becoming crucial. Companies will continue to evaluate analytics platforms based on their ability to provide live direct connectivity to these disparate sources.
About Oliver Parks
Oliver Parks offers search-based recruitment solutions to the technology sector, specialising in the ERP, CRM, CMS, ECM and Business Intelligence spaces. We add real value through our relationship-driven, consultative approach, which we tailor to each of our clients’ individual needs. We focus on mid to senior level requirements, from experienced functional and technical IT specialists, through to senior level management. Our team of consultants currently cover twelve countries from our offices in London, Newcastle, New York, Frankfurt and Helsinki. To find out more, speak to one of our consultants who will understand your specific sector, and the employment opportunities within it.