In 2021, with the further development of vehicle intelligence, L2 and L2+ advanced assisted driving functions will be accelerated to full integration, and autonomous driving level L3 was also in the process of implementation. At present, the ‘fully autonomous driving’ is still to be realized. Therefore ‘human and machine co-driving’ is currently the optimal choice for the industry and the market.
The frequent occurrence of traffic accidents in the "human and machine co-driving" stage is closely related to bad driving behavior, such as fatigued driving, distracted driving, dangerous driving and other traffic safety "hidden killers". The "invisible killers" in the scene of traffic safety will continue to threaten the life safety of drivers and passengers for a long time.
DMS is the abbreviation for Driver Monitoring System. In the "human and machine co-driving" stage, when the driving scenario exceeds the system's capability, whether the driving right can be safely returned from the machine to the human driver, and how to ensure that the human taking over has the corresponding ability are of vital impotence. All of these processes require the system to sense and understand the current state of the human driver.
As for the domestic environment for the development of DMS, the DMS market in China started a little later compared with foreign markets, but with the release of the new policy in 2018, the Chinese DMS market has started to rise in the commercial vehicle sector. The validation of DMS function in the commercial vehicle field has also led to the explosive increase in DMS of the passenger car field. In the next few years, the DMS market in China will see exponential growth under the joint promotion of OEMs and suppliers.
How the stage of "human and machine co-driving" enhances driving safety needs, and how the DMS helps avoid driving risks
In 2022, with the further development of vehicle intelligence, intelligent driving functions of L2 and higher levels will be accelerated on board. In the current "human and machine co-driving" stage, subjective factors such as fatigued driving, distracted driving and aggressive driving have become the main causes of road accidents and deaths, accounting for up to 37%.
In the upcoming L3 state, when the active driving scenario is beyond the system's capability, whether the driving control can be safely returned from the machine to the human driver – and how to ensure that the human driver can take over – are questions of vital importance. These processes require a system to sense and understand the current state of the human driver.
How active technology is becoming mainstream, and how fatigue, distraction and dangerous behavior monitoring are the core functions of today's DMS
The DMS technology route is divided into passive technology and active technology. Due to the shortcomings of the passive technology such as high false alarm rate and low intelligent degree, and meanwhile, with the rapid development of AI vision technology, the non-contact, low-cost and highly intelligent DMS technology of active vision based on infrared technology, vision camera and facial recognition technology, has emerged and become the mainstream technology route nowadays.
Active vision DMS technology obtains images and video information of the driver's eye state, head posture, yawning, phone calls, smoking and other behaviors through optical cameras and infrared cameras deployed on the steering wheel, dashboard or A-pillar, and analyzes the acquired information through deep learning algorithms to determine the current state of the driver and watch for fatigue, distraction and dangerous behavior.
Fatigue Monitoring: During the driving process, the camera samples the driver's eyes closed and yawning behavior. Compared to driving time, driving speed and other factors, the DMS is used to determine whether the driver is fatigued and his fatigue level. Meanwhile, according to the fatigue level, the system initiates corresponding warnings to the driver, such as audible alerts, voice alerts, seat belt tightening instrument alerts and so on.
Distraction Monitoring: During the driving process, the camera samples the driver's line of sight offset as well as the facial angle offset, and the system makes a judgment based on the offset angle threshold. And the offset threshold will trigger timing and give the corresponding warnings according to the distraction level decided by the length of time, such as sound alarm, voice alarm, seat belt tightening, instrument alarm and so on.
Dangerous Behavior Monitoring: During the driving process, the camera samples the driver's hand movements. Behaviors such as smoking, talking on the phone, eating and drinking are defined as dangerous behavior, and appropriate alerts are given, such as audible alerts, voice alerts, seat belt tightening, and instrument alerts.
The operation chain of DMS is complex and involves three main links: camera/sensor for image capture and input; chip board for decoding and algorithm; and human-machine interaction of infotainment system (IVI) for monitoring prompt and feedback, which finally enables the driver to get the corresponding safety warning feedback. As the key link that affects the accuracy of DMS function, the algorithm logic of "decision" is from the initial configuration parameters to the signal release involved in the complex and diverse links and modules, in order to build the competitiveness of the key technology.
Decision-making is the key part affecting the accuracy of DMS implementation, and its full algorithm logic, from the initial configuration parameters to the signal release, is crucial.
China's DMS Market Explodes Since 2018 as DMS Meets Safety Needs under Normalization of Travel
With the development of electrification and intelligence, cars are transforming from traditional walking tools to mobile intelligent service spaces, and the time people spend in the car is increasing significantly. Comfort becomes an urgent need. In 2018, with the promulgation of China's new policy, most commercial vehicles need to be equipped with DMS compulsorily. Therefore, the explosion of China's DMS market started with the commercial vehicle sector.
The application of in-vehicle DMS function in commercial vehicles has been labeled as " needed". Research shows that 31% of traffic accidents on highways are caused by fatigue driving. China's new policy in 2018 has become the trigger for in-vehicle DMS to burst into increase in the Chinese commercial vehicle market.
The success of rear-mounted DMS for commercial vehicles verifies the feasibility of DMS in commercial vehicles. With the iteration of the supply chain and product of DMS for commercial vehicles, the end product of DMS for commercial vehicles will be upgraded from rear-mounted to front-mounted for the improvement of product power, cost performance and intelligence.
DMS hardware mainly relies on imports, and domestic hardware and software suppliers are striving to promoting the development of industry synergy
DMS components and hardware upstream sources are required to satisfy the specific automotive level, which mainly relies on imports at present. And the core components of DMS, the main control semiconductor, are imported from international suppliers mostly as well. The products from international companies are expensive, with low adaptability, and cannot be purchased separately. Meanwhile, under the impact of the global shortage of chips the supply of chips exceeds demand greatly. In recent years, influenced by some factors such as policy and capital market attention, domestic companies have also ushered their rapid development.
Companies in the DMS midstream are mainly software algorithm and solution providers. Although international DMS companies had first-mover advantage to seize part of the first opportunity, domestic enterprises also have the momentum to catch up. Moreover, with the development of domestic vehicle modules, sensors and other hardware manufacturers, algorithms of midstream and integrated system providers will also get their opportunity to occupy a place in the market.
The development of passenger car DMS is budding, and the product is redefined by the user side
The development of passenger car DMS is in the initial stage, and the policy is not yet perfect. Meanwhile, the passenger car market has been attributed to consumer mostly and discrete demand relatively, so users are very sensitive to the cost and cost performance of the product. The policy-driven logic of DMS for commercial vehicles cannot be reused in the passenger car market, and users will not pay for DMS if it does not become a practical feature in passenger cars.
For passenger car users, DMS functions are low-frequency applications, and its necessity and practicality are not easy to perceive, while most DMS products of the current passenger car have poor consumer experience due to the cost. For this reason, the DMS in passenger cars with obvious consumer attributes needs to be redefined, such as integration with intelligent cabin, human-machine interaction, intelligent driving and other functions to achieve high-frequency and intelligent application.
Zhao Rui, Head of Intelligent Automotive Vision Product Line of ThunderSoft (Chinese: 中科创达), also shares his opinions on passenger car DMS in the front end with us:
“Firstly, the attributes of the vehicle determine the design of the product, and ultimately affect the user’s experience. The attributes of a commercial vehicle are completely different from those of a passenger car. The main value of a commercial vehicle is that it is a tool of production, which ensures the safety of the goods and the efficiency of transportation. That is to say, commercial vehicles are more about transporting goods to their destinations safely and efficiently. But for passenger cars, it has actually changed to the attributes of the consumer. Although both passenger cars and commercial vehicles are driven by humankind, commercial vehicles do not place as much importance on the preferences of the drivers as passenger cars.
Secondly, the market now favors front-mounted DMS. and with the increase in computing power, it is obvious that supporting only dual-screen dashboards and central control systems will be a waste. So, the other perception modules such as DMS can be integrated into the entire intelligent cockpit domain control.
Thirdly, the car production supply chain is of far length, and the traditional car manufacturing is still dominated by hardware. But nowadays, commercial vehicles have realized the industrial ecology of separating hardware and software. OEMs will dominate the whole vehicle production structure, and then suppliers like ThunderSoft can work with OEMs to do the design of front-end solutions.”
As DMS development faces technical challenges, Chinese policies and industry standards have not yet been implemented across the board
Chinese suppliers have the ability to provide complete solutions for DMS, such as DMS products from ThunderSoft (Chinese: 中科创达), SenseTime(Chinese: 商汤科技),TSINGTECH (Chinese: 轻研微视), Zhihua Tech (Chinese:智华科技) and Kankan Tech (Chinese:瞰瞰智能), which have also been mass-produced and installed on models of various brands. However, the commercialization and mass production of front-mounted DMS systems are still facing many difficulties, most notably difficulties in vision technology, lack of software computing power, lack of data collection/annotation and challenges caused by unstable driver status.
The mass production of front-mounted DMS in China requires not only the cooperation between the industry and enterprises to solve the technical difficulties, but also academia and the government to work together to develop sound industry norms and standards to guide and regulate the industry from the power side.
According to the opinions from Xingwen Chen, co-founder and general manager of product division of Kankan Tech(Chinese:瞰瞰智能), the issue of data privacy is an issue that the entire smart car industry is facing. On the first point, the state power will firstly legislate and regulate the relevant behavior from the industry perspective.
On the second point, from the perspective of the industry and practitioners, first, it’s of vital importance that whether the users have clearly indicated their acceptance when collecting data, and whether the users' right to know is under protection. For example, what data is collected, the specific use and the expected value, and what value is expected to be brought. Second, there should be certain limits on data collection. Kankan Tech is dedicated to preventing excessive data collection and misuse of data. In the case of DMS, we do not need a complete image of the face data, that is to say, capturing some facial features and extracting the face data for analysis are sufficient to analyze the driver's current state. This requires us to collect a limited amount of feature data from the algorithm and then use it in a limited scope.