People and animals perceive and learn through visual, listening, speaking, etc., and are essentially multimodal learning. In recent years, due to the development of deep learning, multimodal machine learning has become a research hotspot of artificial intelligence. This article briefly introduces the content and challenges of multimodal machine learning, and some of them are based on CVPR2016 and ACL2016's Tutorial Multimodal Learning and Reasoning [1], Tutorial on Multimodal Machine Learning [2].
People's perceptions in life are diverse, including sight, hearing, touch, taste, smell and so on. Any lack of perceptual ability can cause abnormalities in intelligence or ability.
Based on this, MultimodalMachineLearning provides multimodal data processing capabilities for machines. For example, look at the picture and watch the movie translation. The long-term goal of multimodal learning is to make the machine fully aware of the environment, such as perceiving human emotions, words, expressions, and interacting more intelligently with the environment.
At present, academically mature is multimodal learning between visual and semantic. For example, generate a text description for an image, or answer the corresponding text question for the content of a picture. Visual information is usually processed by CNN, and text information is processed smoothly using RNN. The way to align multi-dimensional data is the attention mechanism, for example, to see which object in the figure corresponds to the noun in the picture. Moreover, multi-modal learning is superior to single-modal machine learning in many traditional machine learning tasks. For example, the text translation effect of assisting visual information is better than using only text information.
Heat-shrinkable Tubing For Cable Outer Protection
Heat-shrinkable tubing for cable outer protection
Heat-shrinkable tubing for cable outer protection,Heat-shrink tube,Heat shrinkable tubing,thermal contraction pipe,Shrink tube
Mianyang Dongyao New Material Co. , https://www.mydyxc.com