Overview: Tracking the target in video under different conditions is not only the basis of video analysis, but also one of the key research topics in machine vision. Some emergency information can be gotten effectively by analyzing the video. At present, the intelligent processing of video monitoring information has received more and more attention in the fields of computer vision, pattern recognition and machine learning. Some traditional algorithms are introduced into this field for target detection and tracking, such as particle filter algorithm and ViBe algorithm. These algorithms are mature and effective, but these algorithms are less traceable under complex conditions. Other methods, such as Target tracking algorithm based on kernel function, the CodeBook based on clustering algorithm have great performance under complex conditions, but these algorithms' calculations are complex and have difficulties in realizing. Inter-frame differential method is a common algorithm for tracking the moving targets with the advantage of high execution speed, but its disadvantage is that it is easily affected by the environment. Optical flow method is another important way in image processing. It can overcome the shortage of the inter-frame differential method, but the optical flow algorithm is more time-consuming. To solve the problem of multiple targets' detection and tracking under the complex environment, an improved moving objects detection method is proposed based on inter-frame differential method(four inter-frame differential method) and optical flow algorithm(pyramid LK optical flow). In this paper, by discussing the advantages and disadvantages of four inter-frame differential method and optical flow algorithm, a improved combination method is put forward. Firstly, four inter-frame difference method is used to process the of video sequences. The computed area is reduced and therefore the detection speed is improved. Then objects in the video is detected accurately by the optical flow algorithm used on light streaming video sequences. This improved method enhances the processing speed of optical flow method and reduces the effects of environment's illumination. Finally, the paper compares the proposed algorithm with four-frame difference method, optical flow algorithm, particle filter, ViBe algorithm under different scenarios with different moving targets and individual number. Experimental results show that this improved method is proved not only with good robustness, but also can work more quickly and accurately on the target detection and tracking.