The mechanism of locating multiple objects over a sequence of frames (videos) is referred as multi object tracking. The task of detecting and tracking of moving objects in surveillance video sequence forms a basis for the top level intelligence application and has many challenges. The proposed system portrays detection and tracking. Detection is done with the help of foreground detector which involves background subtraction. kalman filter is also used for prediction(tracking) of the object’s motion. By combining the background subtraction model with the kalman filter, the algorithm is not only used for object tracking but also for the purpose of predicting the motion of the tracked object in case of partially occluded object if the object is disappeared in the scene because of full occluded.