Introduction
Video structure, that is, the structured processing of video data, is to intelligently analyze the original video, extract key information, and perform semantic description of the text. What are the key information that needs to be extracted in a video? At present, there are three main categories: the first category is the identification of sports targets, that is, the identification of sports objects in the picture, whether they are people or cars; the second category is the identification of sports target characteristics, that is, what characteristics are the people, cars, and objects in the picture, if they are people, men or women, do you wear glasses, what color of clothes you wear, if it is a car, what license plate number is, what color and what model is, etc.; the third category is the trajectory analysis of sports targets, that is, whether the person or car turns left, right or hovering in the picture, etc.
Video structure can be divided into the following 4 steps:
Background modeling, target background separation
Target tracking,Classification
Extract features (vehicle, personnel and item features)
Event detection
Vehicle structured description: license plate, body color, car type, brand, sub-brand, car stickers, car accessories, etc.;
Structured description of people: appearance, face, dress, details, height, age range, gender, etc.
The challenge of video structure
In addition to application models, technology is also another mountain currently lying on the road to the development of video big data. There are many aspects, involving many areas such as acquisition, storage, and management. The biggest technical obstacle lies in the structure of video. Most of the data in commercial applications areStructured data, Each data consists of a series of clear descriptive attributes, and the big data processing system can classify different attributes according to the user's requirements, thereby discovering and mastering the objective laws of the development of things. However, videos are not the case. There are no other tags except the attributes of time and space. In addition to finding the corresponding videos by time and place, most videos can only be slowly identified by people, which is far from big data applications.
To doBig Data Application, each video must be labeled with more attribute tags, which is what the industry calls the structured process. The author believes that this is the commanding heights of future video application technology, and its core isPattern recognitionalgorithm, it is necessary to automatically identify the features in the video and label them before entering the library. In this way, when needed in the future, rapid query and collision analysis of massive videos can be achieved, and even classified statistics can be achieved like commercial big data.
Difficulties in video structure
1. Research key technologies and break through application problems. Based on the study of the video application rules of business departments, establish a structured description of surveillance videosModel, conquer a number of key technologies involving video segmentation, content extraction, and content description. Research on the description database management technology, image and video semantic retrieval technology and corresponding data service technology involving key applications.
2. Strengthen top-level design and build a standard system simultaneously. Standardization is the basis of information sharing. Through the study of the characteristics and application modes of video structured technology, a standard system model is established for video structured descriptions, a standardized system covering the implementation and application system of technology, and relevant standards are formulated step by step to standardize technical research and equipment development, guide all aspects of system construction, operation and evaluation, and lay a solid foundation for the comprehensive development of video information intelligence application from the source.
3. Carry out the construction of the video information intelligence system platform step by step and gradually promote the integration and application of * information resources. Carry out research on application service models for structured video description data, formulate systems and solutions for structured video applications, and build application demonstration systems for one to two typical application environments. Through the construction and operation of the system, verify solutions for the video structured description system, explore data interaction and service interaction issues between the video surveillance network and the service network, and try to integrate resources with other information systems.
The significance of structure is not difficult to understand, but it is difficult to implement it for several reasons:
1. What characteristics are identified? An image or a video can be described with countless angles of label attributes. What are the attributes we need? This is closely related to the purpose we need to get.
2. It is difficult to develop the recognition algorithm. Since it is a flat image, the main principle of feature recognition is to compare the outline, color, texture in the image area with the feature library. However, the contours displayed on cameras of different monitoring angles are different, so they cannot be recognized.
3. Large-scale data processing is difficult. Even if the identification algorithm is achieved, if large-scale videos are structured through the form of data processing servers, the construction cost is huge, and its energy consumption is not practical in China's summer when power is required.
In this way, the path of video structure seems to be impossible, but there are many ways to "save the country through curves" in the industry. for example:
1. Vigorously develop the construction of electric police checkpoints: At present, the application needs and frequency of electric police checkpoints in image investigation have long surpassed the traffic police, because cases basically have to be connected with vehicles, which can find many clues. The snap angle of the vehicle of the checkpoint is relatively fixed, and corresponding vehicle feature recognition technology can be developed. The electric warning lockpoint is a good matching point for business needs and technical implementation.
2. Structural recognition forward movement: When the camera collects images, structured work must be done well. For example, bayonet cameras should integrate intelligent recognition algorithms. At present, many manufacturers have launched corresponding smart card cameras, and it is recommended that the government should vigorously promote it and use this type of smart card camera to replace it when the old card camera is updated and replaced, so as to prepare for large-scale video structure in the future.
3. The development of special cameras such as binoculars breaks through the limitations of plane image characteristics and obtains more accurate three-dimensional information, such as the number of human bodies, height, object length, etc. Similar products are suitable for use in key areas and are in line with the current severe anti-terrorism situation in China.
4. The application of more perception technologies such as the Internet of Things will be integrated into more IoT perception technologies, such as RFID technology, as an effective supplement to video structured information.
Summary of key points:
1. The concept of structured video storage: it is to intelligently analyze the original video, extract key information, and perform semantic description of the text.
2. 4 steps to structuring video:
① Background modeling, target background separation;
② Target tracking and classification;
③ Extract features (vehicle, personnel and item characteristics);
④ Event detection.
3. Examples of video structure:
① Vehicle structured description: license plate, body color, car type, brand, sub-brand, car stickers, car accessories, etc.;
② Structured description of people: appearance, face, dress, details, height, age range, gender, etc.
4. Challenges of video structure:
① The data in commercial applications is mostly structured data, and each data is composed of a series of clear descriptive attributes. The big data processing system can classify different attributes according to the user's requirements, thereby discovering and mastering the objective laws of the development of things.
② Videos are not the case. There are no other tags except the attributes of time and space. In addition to finding the corresponding videos by time and place, most videos can only be slowly identified by people, which is far from big data applications.
5. Solutions to structuring video:
① Method: Apply more attribute tags to each video, that is, the process of video structure.
② Effect: Automatically identify the features in the video and label them and enter the library;
6. Difficulties in video structure:
① Establish a structured description model for surveillance videos, conquer a number of key technologies involving video segmentation, content extraction, and content description, and study description database management technology, image and video semantic retrieval technology and corresponding data service technology involving key applications.
② Establish a standard system model for structured video description and formulate a standardized system covering the implementation and application system of technology.
7. Reasons for the difficulty of video structure:
① An image or video can be described with countless angles of label attributes;
② Since it is a flat image, it is difficult to develop recognition algorithms; the main principle of feature recognition is to compare the outline, color, texture in the image area with the feature library.
③ It is difficult to process large-scale data. If large-scale videos are structured through the form of data processing servers, it consumes electricity costs and other costs;
8. Application:
①Electric police checkpoint: At present, the application needs and frequency of electric police checkpoints in image investigation have long surpassed the traffic police, because cases basically have to be connected with vehicles, which can find many clues. The snap angle of the vehicle of the checkpoint is relatively fixed, and corresponding vehicle feature recognition technology can be developed. The electric warning lockpoint is a good matching point for business needs and technical implementation.
2. Structural recognition forward movement: When the camera collects images, structured work must be done well. For example, bayonet cameras should integrate intelligent recognition algorithms. At present, many manufacturers have launched corresponding smart card cameras, and it is recommended that the government should vigorously promote it and use this type of smart card camera to replace it when the old card camera is updated and replaced, so as to prepare for large-scale video structure in the future.
3. The development of special cameras such as binoculars breaks through the limitations of plane image characteristics and obtains more accurate three-dimensional information, such as the number of human bodies, height, object length, etc. Similar products are suitable for use in key areas and are in line with the current severe anti-terrorism situation in China.