Data is an important tool in the decision-making process, and enterprises have taken the data-oriented approach to improve efficiency and reduce costs. Taking data as the key factor to drive industrial transformation and upgrading has not only become an industry consensus at the macro level but also brings actual benefits to enterprises at the micro level. The application of big data technology has begun to bring practical benefits to many enterprises. The role of data in industrial R&D design, process optimization, equipment maintenance, quality control, energy conservation, and emission reduction is becoming increasingly prominent.
The innovative use of industrial big data has led to the development of sophisticated machine-learning algorithms that can predict failures before they happen. Such algorithms analyze data like temperature, vibration, and data concerning industrial processes. By analyzing these streams in real-time, modern machine-learning algorithms can uncover trends that predict failures and alert technicians to potential problems.
However, the development of industrial big data still faces four major challenges: lack of data resources, lagging data management, widespread silos, and insufficient application depth. To this end, it is necessary to consolidate the data foundation at the enterprise level, seize the opportunity of technological innovation, and establish standards and rules for data interoperability and circulation at the industry level.
Lack of adequate data resources
The industrial big data application landscape is still emerging, and with it, new challenges are constantly being identified. One of the most significant challenges facing industrial big data applications is the difficulty of data acquisition.
In order to develop an effective big data application, companies need to have access to accurate and reliable data. However, this can be difficult to obtain, especially for companies that operate in industries with high levels of regulation. Additionally, data acquisition can be expensive and time-consuming, which can limit the feasibility of developing big data applications.
Another challenge is that enterprise data (i.e., data warehouses, Hadoop, data lakes, etc.) and data from IoT sensors, mobile devices, and social media are scattered across the organization. Data scientists who need to access this data most often have to depend on IT to help.
Even when data is accessible, compared with general-purpose data, which are readily available, industrial data, which comes from production systems is often in formats that are difficult to use, making it time-consuming and expensive to convert into a usable form. It requires substantial investment to collect, organize, and analyze due to the complexities involved.
The challenge of data acquisition is compounded by the fact that data is constantly changing. As organizations update their systems and processes, the data required for industrial big data applications changes as well. This requires constant vigilance on the part of data scientists and application developers to ensure that the data they need is still available and still accurate.
In theory, the data in the industrial field should be very rich but in practice, it is scarce. A 2019 McKinsey report shows that America’s discrete manufacturing sector has the largest volume of data stored, but that data is not always valuable. This is because industrial data is often polluted with “bad” samples, because machines in industrial sectors often need to be analyzed by capturing their “faulty” states. So, it is hard to get effective samples that are not cthe ostly. Also, many industrial scenarios demand collection of sampling data in a very short period of time (such as 100 million measurements per second) so the subtle conditions of machinery and equipment be captured, which requires a new generation of dedicated time series databases and stream processing platforms
High-performance databases that can ingest, process, and analyze millions of measurements per second, even in real-time, hold the key to future success for automated and connected physical systems.
Industrial data asset management and governance lags behind
As industrial companies increasingly rely on big data to drive business decisions, the need for effective data management and governance becomes more critical. Unfortunately, many companies struggle with both aspects of industrial big data.
Surveys show that less than one-third of industrial enterprises have carried out data governance, and 41% of enterprises are still using documents or more primitive methods for data management. Industrial enterprises should regard data as an asset that is as important or even more valuable as machinery and equipment, and strengthen data asset management.
Poor data management can lead to decreased efficiency, inaccurate decision-making, wasted time and resources and lost revenue. It is estimated that poor data quality costs businesses around 20% annually.
In order to properly manage industrial big data, businesses need to have a clear understanding of their data governance policies and procedures. They also need to have a plan in place for managing and storing data. Without these things in place, it can be very difficult to properly utilize big data applications.
The good news is that more and more industrial companies have started to manage data assets from master data or metadata. Moreover, with the development of machine learning technology, intelligent data asset management tools are becoming more and more perfect, and the management of industrial data assets can be completed efficiently by relying more on artificial intelligence. However, compared with industries such as finance, and telecommunications with a higher degree of informatization, the management of industrial data still has a lot of debts to make up.
Industrial data silos are common
Data silos are a dilemma faced by nearly all businesses. In an enterprise, there are many IT systems developed by different vendors at different times, such as customer management, production management, sales procurement, order warehousing, financial manpower, etc. To further promote intelligent manufacturing, not only do the above-mentioned IT systems need to be interconnected but also further be extended to communicate data with the OT systems to optimize production in a variety of ways. Moreover, the larger the enterprise, the heavier the management and technical burden.
According to a survey of over 200 enterprises by International Data Corporation (IDC), nearly two-thirds of them believe they will need a vendor partner to access external data or to externally provide data in the future. However, over half of those companies worry about regulatory compliance and security while only a mere 10 percent do not think there will be any risks or obstacles involved. There are still many barriers to overcome regarding commercial models and technical standards, which greatly restrict the flow of data between enterprises. From the perspective of the overall industry development, the trend for integrating data from a single enterprise across the whole supply chain to achieve overall optimization requires the full end-to-end flow of data across the value chain to be secured to guarantee compliance, safety, and other issues.
The German Industry 4.0 plan has taken data circulation as a key topic and has carried out model exploration in the construction of industrial data space. At the same time, technologies such as homomorphic encryption, secure multi-party computing, zero-knowledge proof, blockchain and smart contracts are becoming practical, and they also provide a promising route for using technology to break the deadlock of data sharing.
Industrial data applications are not yet widespread
The use of big data in the industrial sector can be seen from three levels. At the most, basic level data can be used to describe the history and current situation of industrial production lines, marketing, and enterprise operations. On a more sophisticated level, data can be used to predict the future conditions of equipment, workshop, and the whole enterprise based on data. The highest level is to automatically guide the operation of enterprises and form a smart data loop according to the analysis results of data, bypassing artificial intervention. The role of big data in the industrial field can also cross the whole chain of design, production, sales and service. However, the data analysis application of industrial enterprises is still in the shallow stage. 40% of platform applications are focused on the detection, diagnosis and predictive analysis of product or equipment data, while in business management optimization and resource matching coordination scenes involving broader data scope and higher analysis complexity, most existing data analysis capabilities of platforms cannot meet the application needs. and it is necessary to further promote the innovation of data analysis technology and realize the accumulation of long-term industrial knowledge.
In the future, industrial data analysis will still need to be problem-oriented and closely combine industrial mechanisms with data science methods working closely to take data application to the next level thereby producing greater value.
What can be done to promote the development of industrial big data
The long-term goal of the industrial internet is to build digital twins. Only when industrial data is increasingly rich and comprehensive, and of high quality, can the twins resemble each other and be “in sync”. This is the only way that physical objects can be replicated in the digital world, and the optimal operation of the physical world can be guided by digital calculations, analysis, predictions, and optimization, thus opening up new growth prospects. To this end, we need to face up to challenges and do a good job in several areas:
First, consolidate the data foundation and attach great importance to the strategic value of data asset management. Enterprises should not only focus on the explicit value of final data analysis but also attach importance to data collection, asset management, governance, interoperability and standardization. Only when the foundation is solid, can industrial big data be credible and usable, and become a valued source.
The second is to seize the opportunity of technological innovation. Data technology is entering a new stage of development. Time-series databases, knowledge graphs, deep learning, and secure multi-party computing breed new impetus for industrial big data collection, integration, and analysis. Combining specific application scenarios with new technologies is expected to bring new impetus breakthroughs.
The third is to establish industry standards and rules. At the industry level, the role of industry alliances can be brought into play to establish industry standards in terms of data collection protocols, data models, etc., to remove obstacles to interoperability at the technical level. At the same time, it is necessary to promote the formation of industry rules for data sharing among industrial enterprises, create a safe, credible, and balanced data circulation ecology, and pave the way for breaking the industry-wide data island.
