The quantity of digital videos is huge, due to technological advances in video capture, storage and compression. However, the usefulness of these enormous volumes is limited by the effectiveness of content-based video retrieval systems (CBVR). Video matching for the retrieval purpose is the core of these CBVR systems where videos are matched based on their respective visual features and their evolvement across video frames. Also, it acts as an essential foundational layer to infer semantic similarity at advanced stage, in collaboration with metadata. This chapter presents and discusses the core field concepts, problems and recent trends. This will provide the reader with the required amount of knowledge to select suitable features’ set and adequate techniques to develop robust research in this field.