“Reskill” yourself as a Big Data-Literate and get yourself a promotion and salary bump
Get yourself ahead in the Big Data Talent Pool
The overwhelming success of the Internet economy finally gave way to its successor, Big Data & AI (Artificial Intelligence). Whether you are in retail/service, information technology, telecommunications or financial industry, the application of big data has become indispensable in your business.
1. Emergence of Big Data
- Databanker (數邦客) conducted a survey on Big Data application with 3,000+ enterprises in May 2017. The result indicated that:
- 5% of the companies are building their big data platforms;
- 5% have already applied big data in production environment with successful cases/products;
- 5% have acquired sufficient understanding and are ready to apply the knowledge in their business;
- Only 13.5% of these enterprise have no knowledge of Big Data.
Rapid development of Artificial Intelligence (AI) also requires huge data base to support its full function. Big Data has become the ‘trend’ for all industries.
- What is “big data”? McKinsey Global Institute defines big data as a data set that is large enough to significantly exceed the capabilities of traditional database software tools in terms of acquisition, storage, management and analysis. The four major characteristics of Big Data are: massive data scale, rapid data flow, diverse data type and low value density.
- We are now at an era where big data development is advancing fast. Everything we do, whether it is via the Internet or not, leave digital traces behind. For example, credit card shopping, Internet search, mobile Internet access or simply a click on any browser creates a digital trace that will be recorded. Therefore, the magnitude (size and complexity) of these big data has made themselves becoming impossible to capture, manage and process through the prevailing technologies at a reasonable cost within an acceptable time frame. Big Data, therefore, is the technique necessary to obtain valuable information quickly from a wide range of massive data.
2. Global influence of Apache
Apache Hadoop forms a core part of big data technology. Apache is the largest and most influential open-source community in the global big data industry. The Apache Software Foundation (also known as the Apache Software Foundation, or ASF for short) is a non-profit organization dedicated to supporting open-source software projects. The big data technicians believe that cutting-edge technologies should be widely used and become available for open-source sharing. With this belief, they established ASF in 1999 offering free projects for sharing to the public. ASF has developed more than 300 projects after 20 years of development, and most of the world-class data corporations are using its software.
Hadoop is one of ASF’s open-source projects, which is an open-source software specifically responsible for processing big data storage and analysis. Driven by low-cost storage and fast processing capabilities, Hadoop has been widely used in enterprises and is considered the most widely used big data development tool in the market per se. Hadoop has become a core component of big data and artificial intelligence solution until Hadoop 2.0 released in October 2013. Survey indicates that close to 40% of companies tend to choose Hadoop in the technology architecture for their big data platform.
Current users of Hadoop include global technology giants such as social networking sites Facebook, LinkedIn, Twitter and e-commerce company like Alibaba. These companies process massive amounts of data every day. As a testimony of Hadoop’s value in commercial applications, there are over 6,000 reposts per second on twitter, and Alibaba’s turnover on November last year was high at ¥16.8 billion. In light of the speed of innovation and the level of requirement, more and more SMEs will have its own Hadoop architecture over the next five years and Hadoop is expected to be widely used by enterprises for commercial benefits in all areas. Forbes predicted global Hadoop market will grow at a compound annual growth rate of 42% to reach US$ 99 billion by 2022, with over one third of this growth will be coming from the Asia-Pacific region.
3. Talent Gap in Big Data
As the use of big data technology continues to expand, demand for big data talents well exceeds its talent pool and thus result in a huge talent gap. According to the “Big Data Talents Report” released by HR Tech (數據尋英) , the current 460,000 big data talents in China will render a shortfall of 1.5 million over the next 3-5 years.
According to “2016 China Internet Hottest Jobs Talent Report”(2016年中國互聯網最熱門職位人才報告) released by Linkedin, social media platform for professional, Research and Development Engineers, Product Managers, Human Resources, Marketing, Operations and Data Analysis are 6 hottest job positions in China’s internet industry. LinkedIn report also indicated that the supply index for data analysis talent is the lowest at 0.05, which means the supply of data analytics talent is very scarce. The high rate of job transfer within the Data Analysis labour force also aggravated the problem, with an average job transfer rate at 19.8 months. Furthermore, the salary premium continues to rise. The annual salary of newcomers is 134% higher than the average technician, while people with 5-8 years of experience receive 78.04% higher salary then the average technicians in internet industry.
According to statistics conducted by Data Analysis Professional Committee of the China Business Association (中國商業聯合會數據分析專業委員會), the talent gap for basic data analysis in China will reach 14 million, while over 60% of the positions recruited by BAT are big data talents.
4. Acquiring big data knowledge through professional certification
The demand for new types of big data analysis and forecasting technology talents has far exceeded that for traditional business intelligence and information management talents. The big data talent gap is expected to further widen in the next five years. More and more people come to realize the possibility of landing a high pay job through mastering big data technology.
Dr. Toa Charm, Associate Professor of the Chinese University of Hong Kong Business School, Visiting Associate Professor of the SPACE China Business School of the University of Hong Kong and Chairman of OpenCertHub, has dedicated his time in the research of big data and artificial intelligence for commercial application. With over 30 years of experience in management, marketing, consulting, entrepreneurship and training on innovation and technology industry, Dr Charm was ex-Chief Public Mission Officer in Hong Kong Cyberport and held different senior management positions in multinational, Hong Kong and Mainland technology and financial services companies.
Dr. Tao opined the development of big data in Mainland China and other developed countries has entered into a “fever” stage. “The development of big data over the next few years will mark an important milestone in Big Data history. The most important thing for enterprise is to fill the talent gap. For people who are in the workforce, they should not be worried about becoming obsolete under this development. Instead, they should be actively upgrading their skills through obtaining relevant professional certifications as a demonstration of their competence for promotion. The general environment remains optimistic for people who are willing to reskill themselves. “
Most people believe they do not need to acquaint themselves with high-end artificial intelligence technology as they are no scientists. There is also a lack of basic knowledge and understanding of the concepts and applications of big data among the general public. But Dr Toa Charm believes that “No matter you are in finance, retail, real estate, logistics and tourism, to frontline, back-end or management positions, big data will exert a profound impact on your business. Mastering of Big data technology has become indispensable.”
To conclude, big data literacy has become an important factor in one’s career success equation. Whether you are after promotions, salary increase or a change of career path, big data competency is definitely worth the investment in enhancing your competitiveness in the market.
Notes：
Apache Software Foundation (ASF) is the world’s largest open-source software organization since 1999 and managing over 300 open-source software projects. Some of its world class open-source projects are widely adopted by renowned technology corporations such as Facebook, Twitter, Alibaba, Tencent, Baidu, etc.
OpenCertHub is a proprietary Online Examination Platform integrated with Big Data and AI technology，and also the only official authorized organization in the Asia-Pacific region delivering CPFA™ (Certified Professionals For Apache projects) professional examination and certification. OpenCertHub has developed its intelligent examination platform so that everyone can access its “Role-Based” examinations and gain official authentication of his/her open-source Apache big data knowledge and relevant skill sets with professional qualifications such as the CPFA™ Citizen Data Scientist, CPFA™ Data Architect, CPFA™ Data Scientist, and CPFA™ AI Engineer.
