#polariis

Along with social, mobile and cloud, analytics and associated data technologies have earned a place as one of the core disruptors of the digital age. 2015 saw big data initiatives moving from test to production and a strong push to leverage new data technologies to power business intelligence. As 2016 gets underway, five insiders share their predictions for what 2016 holds in store for the data and analytics space.

Scott Gnau, CTO of Hadoop distribution vendor Hortonworks, predicts the following trends will dominate data and analytics in 2016:

Internet of Anything. In 2016, businesses will look at deriving value from all data, Gnau says. "It's not just the Internet of Things but rather Internet of Anything that can provide insights," he says. "Getting value from data extends beyond devices, sensors and machines and includes all data including that produced by server logs, geo location and data from the Internet."

Data at the jagged edge. Businesses must look beyond the edge of their data centers all the way out to the jagged edge of data, Gnau says. He notes that data flows now originate outside the data from many devices, sensors and servers on — for example, an oil rig in the ocean or a satellite in space. There is a huge opportunity to manage the security perimeter as well as to provide complete data provenance across the ecosystem. Gnau says IoAT creates a new paradigm that requires new thinking and new data management systems, and these solutions will mature and permeate the enterprise in 2016.

Data in motion platform.The industry will see the evolution of data in motion platforms in 2016. "There is a need for a higher-level platform to handle the many device protocols and bring all of the data flows into Hadoop," Gnau says. "The platform needs to facilitate communications in multiple protocol languages. There combination of data in motion and data at rest is a big opportunity for the year."

Big data made easy. There is a market need to simplify big data technologies, and opportunities for this exist at all levels: technical, consumption and so on. Gnau says that in 2016 there will be significant progress toward simplification. "It doesn't matter who you are — cluster operator, security administrator, data analyst — everyone wants Hadoop and related big data technologies to be straightforward," he says. "Things like a single integrated developer experience or a reduced number of settings or profiles will start to appear across the board."Hadoop for mission critical workloads. In 2016, Hadoop will be used to deliver more mission critical workloads — beyond the "web scale" companies, Gnau predicts. "While companies like Yahoo, Spotify and TrueCar all have built businesses which significantly leverage Hadoop, we will see Hadoop used by more traditional enterprises to extract valuable insights from the vast quantity of data under management and deliver net new mission critical analytic applications which simply weren't possible without Hadoop," he says.

Deepak Kumar, founder and CTO of IT systems management solutions provider Adaptiva, predicts:

This is the year data gets limits. "Data usage will become more regulated, as providers won't be able to keep up with the data demand and businesses won't be able to keep up with the rising cost," Kumar says. "As a result, companies will begin to leverage technologies that monitor this data."

Systems management will get smart about big data analytics. "Integration of big data analytics solutions will continue to fall short, leaving perishable business insights undiscovered in disconnected silos of data — systems management will step in to help," he says.

Badri Raghavan, chief data scientist at energy analytics specialist FirstFuel Software, says 2016 will see:

The democratization of data. Raghavan says that thanks to solutions like Amazon Mechanical Turk, businesses and individuals will be able to much more easily collect data from around the world that to which they previously did not have access. "Not only will data be easier to find, but the emergence of user-friendly tools will enable people without extensive data knowledge to parse the information so as to secure meaningful value," he says.

Increased concerns around data privacy. Europe recently set strict regulations around data, which means that organizations will need to be strategic about how they're tackling data security issues. "Rather than considering data privacy an afterthought item, people will need to be proactive in explaining exactly how they will use the data and ensuring compliance with local and global laws," he says.

New applications for data insights. In 2016, Raghavan says organizations and individuals will tap data and analytics to deliver personalized and engaging experiences across industries including energy, sports, social good and music. "For instance, people will be able to use data to change a song based on their personal preferences (e.g., lots of drum)," he says.

Dan Kogan, director of product marketing at business intelligence and analytics company Tableau Software predicts a slew of trends in the big data space for 2016, including:

The NoSQL takeover Kogan says 2016 will see the shift to NoSQL databases as a leading piece of the Enterprise IT landscape as the benefits of schema-less database concepts become more pronounced. "Nothing shows the picture more starkly than looking at Gartner's Magic Quadrant for Operational Database Management Systems, which in the past was dominated by Oracle, IBM, Microsoft and SAP," he says. "In contrast, in the most recent Magic Quadrant, we see the NoSQL companies, including MongoDB, DataStax, Redis Labs, MarkLogic and Amazon Web Services (with DynamoDB), outnumbering the traditional database vendors in Gartner's Leaders quadrant of the report.

Apache Spark lights up big data. Apache Spark has moved from a being a component of the Hadoop ecosystem to the big data platform of choice for a number of enterprises. "Spark provides dramatically increased data processing speed compared to Hadoop and is now the largest big data open source project, according to Spark originator and Databricks co-founder, Matei Zaharia," Kogan says. "We see more and more compelling enterprise use cases around Spark, such as at Goldman Sachs, where Spark has become the "lingua franca" of big data analytics.

Big data grows up: Hadoop adds to enterprise standards. Kogan says the enterprise capabilities of Hadoop will mature in 2016. "As further evidence to the growing trend of Hadoop becoming a core part of the enterprise IT landscape, we'll see investment grow in the components surrounding enterprise systems such as security," he says. "Apache Sentry project provides a system for enforcing fine-grained, role-based authorization to data and metadata stored on a Hadoop cluster. These are the types of capabilities that customers expect from their enterprise-grade RDBMS platforms and are now coming to the forefront of the emerging big data technologies, thus eliminating one more barrier to enterprise adoption."

Big data gets fast: Options expand to add speed to Hadoop. Kogan says 2016 will see Hadoop gain the sort of performance that has traditionally been associated with data warehouses. "With Hadoop gaining more traction in the enterprise, we see a growing demand from end users for the same fast data exploration capabilities they've come to expect from traditional data warehouses," he says. "To meet that end user demand, we see growing adoption of technologies such as Cloudera Impala, AtScale, Actian Vector and Jethro Data that enable the business user's old friend, the OLAP cube, for Hadoop — further blurring the lines behind the "traditional" BI concepts and the world of 'big data'."

The number of options for "preparing" end users to discover all forms of data grows. Self-service data preparation tools are exploding in popularity. Kogan says this is in part due to the shift toward business-user-generated data discovery tools such as Tableau that reduce time to analyze data. "Business users now want to also be able to reduce the time and complexity of preparing data for analysis, something that is especially important in the world of big data when dealing with a variety of data types and formats," he says. "We've seen a host of innovation in this space from companies focused on end user data preparation for big data such as Alteryx, Trifacta, Paxata and Lavastorm while even seeing long established ETL leaders such as Informatica with their Rev product make heavy investments here."

MPP Data Warehouse growth is heating up…in the cloud! Kogan says the "death" of the data warehouse has been overhyped for some time now, but it's no secret that growth in this segment of the market has been slowing. "But we now see a major shift in the application of this technology to the cloud where Amazon led the way with an on-demand cloud data warehouse in Redshift," he says. "Redshift was AWS's fastest-growing service, but it now has competition from Google with BigQuery, offerings from long-time data warehouse power players such as Microsoft (with Azure SQL Data Warehouse) and Teradata, along with new start-ups such as Snowflake, winner of Strata + Hadoop World 2015 Startup Showcase, also gaining adoption in this space. Analysts cite 90 percent of companies who have adopted Hadoop will also keep their data warehouses, and with these new cloud offerings, those customers can dynamically scale up or down the amount of storage and compute resources in the data warehouse relative to the larger amounts of information stored in their Hadoop data lake."

The buzzwords converge: IoT, cloud and big data come together. The technology is still in its early days, but the data from devices in the Internet of Things will become one of the "killer apps" for the cloud and a driver of a petabyte-scale data explosion, Kogan says. "For this reason, we see leading cloud and data companies such as Google, Amazon Web Services and Microsoft bringing Internet of Things services to life where the data can move seamlessly to their cloud based analytics engines," he says.

Dan Graham, general manager of enterprise systems at data warehousing and big data analytics specialist Teradata, predicts that in 2016:

Organizations will hit reset on Hadoop. Graham believes 2016 will see enterprises use lessons learned from past deployments to rearchitect their approaches. "As Hadoop and related open source technologies move beyond knowledge gathering and the hype abates, enterprises will hit the reset button on (not abandon) their Hadoop deployments to address lessons learned — particularly around governance, data integration, security and reliability," he says.

Algorithms will enter the boardroom. "Algorithms heat up in the data ingest and preparation processes for house holding and profiling," he says. "As a result, CEOs and Investors will start talking deep analytics as core business goals."

Data lakes will finally discover a few killer apps. Data lakes will be the most common repository for staging raw IoT data, driven by volume and costs, Graham says. "The size of IoT M2M data will over run in-memory capacity by orders of magnitude, driving implementers to data lake technologies for low cost storage," he says.

IoT data captured at the data center will erode in value faster than transaction data. "Lacking monetary data fields, most sensor data will become low value in hours, days or weeks as it is replaced by fresh collections of the same sensor data," Graham says. "Architectures and systems will be forced to compensate for this rapid decline to cope with retention and processing costs."

In the future, we may look back on 2015 as the year that machine data analytics emerged as a true market disruptor, emanating from the mega market trends of Big Data, Cloud, DevOps, and the Internet of Things. Ironically, these mega trends have played a dual role in this market disruption: both as the creator of the problem — the volume, velocity, and variety of data — and source of the solution — a cloud-native, service platform to provide the scale, security, and rich analytics to transform real-time log streams into powerful operational and customer insights for business growth and success.

Although the big data market will be nearly $50B by 2019 according to analysts, what’s most exciting is that the disruptive power of machine data analytics is only in its infancy. Machine analytics will be the fastest growing area of big data, which will have CAGR greater than 1000%. As digital transformation grows, so does the reliance on new software and architectures. Today, software is not only driving business processes, but entire business models, and the need to manage, monitor, and troubleshoot applications in real-time has never been more critical. Thus, the need for speed, full-stack visibility, and agility — all in real-time — is the true business demand underpinning the growth of machine data analytics.

From this vantage point, below are five trends that will continue into 2016 that will have significant impact to the world of machine data analytics:

1. The maturation of DevOps and DevOps tools

It’s no secret that cloud will continue to grow at an unabated pace as organizations look to new technologies to increase their business speed, agility, and competitive edge. This is also resulting in the need for new tools that help them break down traditional silos between developers and IT operations teams to innovate more continuously and scale as fast as their business. As organizations embrace the DevOps approach to application development, they face new challenges that simply can’t be met with traditional on-premise and antiquated monitoring tools.

In 2016, we’ll see DevOps adopt a new breed of next-generation log and machine data analytics services that run at cloud-scale, employ predictive algorithms, and can be seamlessly integrated with a host of DevOps tools across the entire pipeline — not just server container or infrastructure data — in order to dramatically improve the continuous integration and continuous deployment processes.

2. CISO and security operations to invest more in system intelligence

For years, companies have understood the value of using big data to gain actionable insights for business decisions. Now, through advancements in technology such as machine learning, data analytics are providing business insights at a granular layer in the systems infrastructure not yet experienced by most companies.

For security teams, analytics are giving rise to new, faster intelligence around system and user anomalies, threat detection, and breach alerts that will not only improve mean time to investigate (MTTI) / mean time to recovery (MTTR) speed, but change the way security leaders think about security systems architectures for years to come. Moreover CISO’s and security teams will join forces to partner with DevOps teams to help secure new application architectures with embedded security capabilities leveraging integrated machine analytics.

3. Log management will be a huge opportunity for IT operations and customer support teams

Thinking about the value of log data may seem too far in the weeds for most tech industry professionals. However, using analytics to monitor, manage, and gather insights from user, application, or infrastructure logs will be the only near-perfect way to address the growing complexity of cloud and hybrid-cloud infrastructures. We’ve already seen vendor consolidation and are likely to see new vendors try and move into the logs analytics space. As such, we anticipate that this will increase organizational awareness of the value of log management to support application development, security and IT operational success. This will be the first sign of the “democratization of analytics.”

4. The rise of “Extreme Architectures”

Historically, advancements in the capacity and processing power of microprocessors has served as a common metric for software advancement (e.g., Moore’s Law, in which microprocessor power doubles every two years.) However with today’s cloud infrastructure, the ability to “string together” thousands of microprocessors via virtualized servers obviates Moore’s Law.

Thus, the most innovative CTOs will push for new, “extreme” software-based architectures to harness the processing power inherent in public or private cloud infrastructures to lessen traditional in-house datacenter constraints (performance, management and maintenance) — while increasing focus at the application layer to drive the functionality needed for differentiated customer experiences.

5. Business intelligence value shifts from rear-view to continuous

As technology infrastructures shift to cloud-based platforms to increase business speed and agility, it begs the question, “Wouldn’t you want your business intelligence to do the same?” Therefore, 2016 will be the year that the definition for business intelligence shifts, from on-premise solutions delivering rear-window insights to solutions that deliver continuous intelligence in real-time.

By harnessing the insights inherent in real-time log data analytics, companies will have faster access to operational and customer data that can enable 24/7 innovation and sustain their competitive edge. Hence, for companies betting their business on software/application platforms, continuous intelligence will not be a nice-to-have, but a must-have.

It will be interesting to look back 12 months from now to see what unfolds, but no matter what happens, one thing is certain: we’re in a whole new world and data is in the driver’s seat.

Nobody uses those exact words, of course. Yet the expectation is clear: choosing the right degree to earn, or skills to acquire, will ensure a bright future and a generous salary. Often, people believe so strongly in the power of education that they are prepared to invest an awful lot of money and time to get it.

Analytics is a broad field with many real opportunities for satisfying work at decent pay. And a good education has value, not just for the salaries it returns, but also for the wider benefits of being a well-educated person. Still, the costs of some analytics programs are significant. A full-time master’s program in analytics may require a year away from work (and a year of income lost), plus tuition of $40-50,000. At the opposite end of the cost spectrum, some classes are available online, free, and can be completed on your own schedule. You can even take advantage of that great traditional resource, the library, and find books and other resources for learning there. But even if you spend no cash, self-study still takes time, perhaps more time than you’d need for conventional classes. How can you make sure that the training you choose is worth it?

Before you invest in analytics training, take these steps to make sure you get real value :

Define your goals. It’s hard to get value when you haven’t decided what’s valuable to you.

Identify others who have jobs or accomplishments similar to your own goals, and learn what training they found necessary or helpful.

Use the information you’ve gathered to select training options that help you achieve your goals without wasting time and money.

The fundamental question to ask about data analytics training is: what do you want it to do for you? Are you most interested in getting a new job, being able to do more in your current job, understanding new things which fascinate you, or do you have some other goal? Decide what you want and define your personal goals.

Be specific when defining goals. If it’s employment you’re after, do you want to work in business, government, academia or nonprofits? Do you primarily want to be a hands-on data analyst or do you want to grow in your current profession by adding analytics skills? Where do you want to live? Outline as much as you can about the job you want to have. Reflect on whether the job you want realistically exists. If so, the next step is to examine the job requirements and figure out what, if any, education gaps you must fill to get it.

Right now, jobs with “data science” titles are hot. So, take that as an example goal: a job with a “Data Scientist” title, and a hands on analytics role, at a tech company, in the town where you live now. I took that as a starting point, used LinkedIn to find some local people who had jobs like that, and reviewed their profiles to learn what education might be appropriate for those jobs. (Wait a minute. Why not just look at job ads and see what skills are required? Because job ads often ask for an unrealistic set of desired skills.)  Here’s what I discovered: some of those people were graduates of one of the best-known Masters in Analytics programs, some had Ph.D.s in fields such as applied mathematics or computer science, and the rest were simply college-educated people.

That simple result shows why it’s important to do your own research and not get all of your information from educational institutions. It’s great to hear that students who earn a Master of Analytics degree can get a good job that interests you. But no university is likely to tell you that you might be able to get the same job without earning that degree. It took me only a few minutes to find several people who had landed the sort of jobs I was investigating, without graduate degrees.

A good next step would be to ask for information interviews with some people who are working in the kind of role that you want. You might try to meet a few people who have taken the special training that interests you, and a few who have not. This would give you the opportunity to determine the real minimum qualifications for the job, and the potential benefits of additional training. If an additional degree would ensure a big boost in salary, for example, it might be worth paying hefty tuition. If the pay’s the same, well, you do the math.

Will more analytics training help you reach your business and personal goals? If so, what’s the right training to choose? The answers are as unique as you are. Don’t invest first and hope it will all work out. Do the research and choose the analytics training that’s worth it to you.

Nearly half of executives worldwide anticipate that data analytics will result in an across-the-board shift in their ability to deliver a superior customer experience within the next two years, February 2016 research found. More than a quarter said it has already done so.

Data from Forbes Insights and SAS revealed that 29% of executives worldwide who currently use data analytics have seen a significant shift across the board in their ability to deliver superior customer experiences. More than a third have seen shifts in many areas, and more than a quarter of respondents have seen moderate shifts. That adds up to 90% of respondents who say they’ve noticed at least some change thanks to data analytics. 

However, in two years, respondents expect changes to be more noticeable. For example, 42% of respondents said they anticipate their use of data analytics to significantly shift their ability to deliver better customer experiences, while just 17% say they expect the changes will be only moderate. And again, only 9% of respondents expect there to be no changes at all to customer experiences.

Data and analytics capabilities have become a top priority for businesses, with many organizations using the tools to help gain a competitive advantage and improve the customer experience. 

Separate research from Black Ink ROI looked at the changes in analytics budgets for 2016 among US marketing managers. Some 80% of respondents expected their 2016 budgets to either increase or stay the same, reiterating the discipline’s importance.

Written by Lisa Lambert

As someone who began my career as a software developer, but ended up as a venture capitalist, I would offer the following advice:   

Have a plan but be prepared to change it

Your plan will put you on a path of discovery, and what you uncover will either validate your plan or challenge you to change course. Case in point: I thought I would be a software engineer for my entire career. Although I enjoyed writing code, I realized after several years on the job that I didn’t want to do it forever. So, I raised my hand for a new assignment, which pointed me in the direction of business school. There I found that what I truly enjoyed was the business of technology. I wanted to work for a technology company in a general management role: making decisions, owning decisions, and watching them play out. The path to self-discovery was more important than sticking to my original plan; I’m happier and more successful because I allowed myself the journey.

Anticipate obstacles but focus on the long-term

Your career is going to last anywhere from 40 to 50 years, so don’t expect to be at the top of your game after the first five. You’re going to hit some rough patches, and when you do, keep going. There may be times in your life when you feel you’re making more progress outside of work (marriage, a baby, etc.) and that’s okay; zigs and zags are expected. I joined Intel as a product-marketing engineer, despite my aspirations to be in general management. Instead of feeling discouraged, I used the opportunity to strengthen my technical skills and expand my network. This position ultimately led to me to my dream job within Intel Capital. What could have been a setback actually helped land me right where I needed to be.

Build a community of colleagues along the way

Don’t wait until you need a new job to start making connections. From day one, build a network of professionals whose work you admire and opinions you respect. Ask for honest feedback – it will broaden your perspective and make you more valuable. Your career journey shouldn’t be a solo mission. Find fellow-travelers, make yourself known, and nurture these relationships. Your next job will likely come from this community.

Have inward and outward conviction

Don’t forget why you started this journey; and don’t forget the conviction you developed in pursuit of your goals. Conviction will give you confidence even when you’re operating outside of your comfort zone and give you the courage to speak up, demonstrate your knowledge, and get noticed. But expect to be challenged; people are going to have opinions about every decision you make. Practice, prepare, and learn your craft — then have the conviction to stand up and be counted.

Written by Brad Heidemann

The stigma of "career hopping" is waning. Employers no longer shy away from job seekers who are looking to change their role, or even industry.

My own career path has been an interesting and varied journey. I began in the shoe department of Nordstrom, where I started working on the displays and quickly rose through the ranks, turning my focus to developing the brand’s now-famous reputation for customer service. Then I got a job at Microsoft, overseeing sales and guiding the technology buying process for customers. After Microsoft, I worked with two interactive marketing agencies before founding my own customer experience (CX) agency, Tahzoo.

Through this journey, I’ve learned the importance of thoughtful research, self-evaluation, and persistence. If you are looking to make a career change, here are some tips for how to make it happen.

1. Understand What Makes You Tick

Introspection is key to a successful career shift. Before you even start a job search, take a look at your current role and identify what elements most interest and excite you and where you excel. Can you run a team well? Do you always secure new business? Are you a "customer whisperer"? Then, look at the areas of the business where you wish you had a more active role, or where you would like more expertise. Don’t take a narrow view of your role, but look at where your strengths play into the overall business.      

Determining your strengths, weaknesses, and goals will help you decide what the next step in your career should be. For example, I knew I wanted to work for a tech company when I realized just how much technology was impacting the entire retail process, from the manufacturers all the way to my role in the display department. I wanted to get to the source of the technology—I wanted to work for a company that made the technology. And I made that happen.

2. Set Yourself Apart

Once you’ve identified your strengths, find a job that will play into those skills. Figure out what your strongest skill set is and play that up. These need to be the skills that make you a valuable employee across the board. You need to be confident in the skills you identify—if you’re in an interview and put on the spot regarding your lack of industry experience, your ability to convey confidence in your unique skill set will be crucial.

In any company, it takes multiple perspectives to solve a business problem. As a job candidate, you can show your value by displaying the characteristics that will make you an efficient problem-solving employee, and show how an outside perspective can benefit the business. Perhaps your solutions and approaches are non-conventional to the industry. Even better. Bring the next company you work for a fresh look and new thinking. If you can demonstrate your commitment and confidence in the value that you know you will add to a company, that will make you an attractive candidate to hire.

It’s also important to go the extra mile. Think of new and different ways to leverage the knowledge you have to add a different perspective to a new industry. For example, if you’re looking to move from sales to marketing, think of how your experience dealing with potential customers can help you in your new role. Anticipate what the next trends and changes in the industry will be and apply your skill set to those changes. In order to commit to the career change you want to make, you need to always be looking for the most impactful and relevant skill you can bring to the new industry.

3. Beat The Learning Curve

If you’re making a big jump and switching industries, you will lack the knowledge that coworkers who started in that field have built over the course of their careers. This can seem daunting, but my best advice is to read up! Gather all the information you can and get to know the industry lingo, jargon, and terms. Tap old colleagues who might know more. Prepare to be a fast learner and a continuous learner. When I made the move from Nordstrom to Microsoft, I read every computer magazine I could find. And when I decided to take on agency life, I made sure I understood the intricacies behind how agencies were run, and practiced the language my new teams would speak.

I made a huge shift in my own career—from shoes to CX. Pivoting isn’t always easy, but it’s certainly worth it to take risks and to not be afraid of change in pursuit of your passion. By following the above tips, you’ll be able to ease the transition as you grow in your career.

Ever wonder how to secure that promotion, what other opportunities out there for the skills that you have, or how to move into another line of business? These are also burning questions of several early-career professionals. That is the period of our life that we start pondering about deeper questions surrounding “what’s next” in our career. These are not simple matters. It requires experience, knowledge and techniques, which not all of us have.

If this sounds like the challenges you are facing, Polariis (www.polariis.com) might be the answer for you. Polariis is a career advisory app, which lets working professionals find the right mentors for their career advancement. It will help connect highly experienced senior advisors and junior professionals within the same industry. 

Polariis handpicked highly qualified experts into its advisor/mentor pool. For the launch, Polariis will focus on financial industry in Chicago. Specializing in one niche market allows Polariis to work more closely with every sub-segment of the user groups. 

Recognizing that career advancement and career satisfaction play a major role in everyone’s professional and (indirectly or directly) personal life, Polariis sets out to solve these questions for working adults alike:

1) How to transition to the next stage of your career?

2) What skill sets are required to work in your desired job?

3) How to be promoted to a higher position in your existing company?

4) How to develop and manage a healthy relationship with your coworkers and managers?

5) How to increase job satisfaction? Etc.

No one can have better answers to these questions than the people who had achieved them in their career. That is the reason why discussing these challenges with a mentor who knows the “ins and outs” of your industry could help you unlock the secrets to career success. Polariis, therefore, is the best platform to connect you with a right mentor to unleash your potential faster and achieve your professional goals.

What sets Polariis different from other career service providers? Indeed, advisors/mentors on Polariis platform are not general career coaches. They are the people who have intensive experience in your field of work, highly successful in their career and are passionate about sharing them with motivated individuals like you. By that approach, you can be assured that the advice given to you is utmost personalized, extremely relevant and specific.

As you will meet your mentors face to face, your questions will also be addressed immediately. It will be a very interactive process in which you can receive instant answers to your questions.  And because it is only between you and your mentor, you can discuss any sensitive matters that are personal to you and your growth.