Abstract At present, the accuracy of real-time moving video image recognition methods are poor. Also energy consumption is high and fault tolerance is not ideal. Consequently this paper proposes a method of moving video image recognition based on BP neural networks. The moving video image is divided into two parts: the key content and the background by binary gray image. By collecting training cubes. The D-SFA algorithm is used to extract moving video image features and to construct feature representation. The image features are extracted by collecting training cubes. The BP neural network is constructed to get the error function. The error signal is returned continuously along the original path. By modifying the weights of neurons in each layer, the weights propagate to the input layer step by step, and then propagates forward. The two processes are repeated to minimize the error signal. The result of image feature extraction is regarded as the input of BP neural network, and the result of moving video image recognition is output. And fault tolerance in real-time is better than the current method. Also the recognition energy consumption is low, and our method is more practical.