Tracking free form objects by fitting algebraic curve models to their boundaries in real-time is not feasible due to the computational burden of fitting algorithms. In this paper, we propose to do fitting once offline and calculate an algebraic curve space. Then, in every frame, algebraic curves from the search region of curve space are evaluated with the extracted edge points. The curve that has the smallest error according to some error metric is chosen to be the fit for that frame. The algorithm presented is for tracking a free-form shaped object, moving along an unknown trajectory, within the camera’s field of view (FOV). A discrete steady-state Kalman filter estimates the future position and orientation of the target object and provides the search area of curve space for the next frame. For initialization of the Kalman filter we used the “related points” extracted from the decomposition of algebraic curves which represent the target’s boundary and measured position of target’s centroid. Related points undergo the same motion with the curve, hence can be used to initialize the orientation of the target. Proposed algorithm is verified with simulations.