Case Studies of Performance Characterization in Computer Vision


Pilot
European
Image
Processing
Archive


The PCCV Project:
Benchmarking Vision Systems

Overview
Tutorials
Methodology
Case studies
Test datasets
Our image file format
HATE test harness



Information
General links
Conferences
Mailing lists
Research groups
Societies

Techniques (CVonline)
Software
Image databases


Other stuff
Linux on ThinkPad

  • Voxel Based Analysis of Tissue Volume MRI Data: 16pp report, available as PDF (370 Kbytes), PostScript (228 Kbytes)
    ABSTRACT: This document has been written to illustrate the role that assumptions play in the design of image analysis algorithms. We present several common methods for the segmentation of MR data on the basis of underlying tissue. These methods, which may appear disparate at first sight, are discussed and related in terms of the assumptions regarding the data fornmation process needed to derive them. We summarise the use of these techniques using a flow diagram which makes explicit the questions which need to be addressed in order that they are used appropriately.

  • Computing Covariances for Mutual Information Co-registration: 5pp paper, available as PDF (59 Kbytes), PostScript (166 Kbytes)
    ABSTRACT: This paper identifies the important role that covariance estimation has to play in the construction of analysis systems. The problem of co-registration for inter-modality clinical volumes is often solved by maximising the so-called mutual information measure. This paper extends the existing theory in this area and suggests a viable way of constructing covariances for mutual information approaches by treating this algorithm as a bootstrapped likelihood based approach. We provide both theoretical and practical tests of the validity of this method. In doing so we identify important subtleties in the current use of these measures for coregistration. These issues suggest potential improvements in the way that such measures might be constructed and used.

  • Characterisation of a Stereo Matching and Object Location System: 35pp, available as PostScript (7.4 Mbytes)
    ABSTRACT: In this report we demonstrate and characterise a 3D wireframe model matcher, applied to a stereo object location problem typical of many industrial machine vision applications, and relate its design to the methodology we proposed as an earlier component of this project. We begin by outlining the stereo requirements of a 3D model matcher, and in this context discuss the general characteristics of two approaches to solving the stereo correspondence problem: Feature Based Stereo (FBS) and Area Based Stereo (ABS). We show that FBS has significant advantages in terms of the accuracy of the recovered stereo data and of reduced susceptibility to image variations, but that ABS generally has a simpler flow of control and is therefore suitable for parallel processor or hardware implementations. We go on to describe the FBS and ABS algorithms implemented in our TINA vision system: PMF, a classical feature-based solution, and the Stretch Correlator (SC), a re-formulation of the algorithmic constraints of PMF into a feature driven, but fundamentally area-based, solution. We present a theoretical analysis of the errors involved in stereo calculations, and use it to show how the results from both stereo algorithms can be characterised in terms of correspondence mismatch and 3D reconstruction error. This leads to the conclusion that geometric fitting to 3D data should be done in disparity space, to exploit the property of isotropic errors. We also summarise an analysis of the mismatch probability of a stereo correspondence algorithm, and show that mismatches are inevitable and must be accounted for in later algorithm stages. We discuss the consequences of our analysis for the design of practical, modular 3D vision systems and describe the components of TINA relevant to this project. We then provide the results of evaluations of the performance of the TINA stereo algorithms, based on our analyses. In particular, we have found that SC has a mismatch probability of typically 1%-2% of the stereo data and in the worst case up to 8%. Finally, we give a practical demonstration of TINA's 3D Wireframe Model Matcher, using both PMF and SC to reconstruct the 3D structure from stereo views of a brake shoe assembly and then locating a wireframe model of the assembly in the recovered 3D scenes. This is done with the object placed in a variety of orientations in a general scene with no special lighting. For similar object views, the number of features recovered by PMF and SC is shown to be identical, within statistical limits. For a set of 33 views of the object in different orientations, PMF provides matches in 18 cases and SC in 17 (rising to 21 and 20 matches respectively when the Canny double thresholds are reduced). A breakdown into sub-groups with similar object orientations shows that the numbers of matches remain identical within statistical limits in each sub-group. On the basis of these results, we conclude that our methodology can successfully be used to design and incrementally refine a modular vision system capable of stereo matching and 3D object location for "difficult" stereo reconstruction tasks. Moreover, the performance of our PMF and SC algorithms is to all intents and purposes identical, which given the computational simplicity of the latter may have positive implications for developing parallel or hardware implementations suitable for temporal stereo tasks.

  • Noise Filtering and Testing for MR Using a Multi-Dimensional Partial Volume Model: 8pp paper, available as PDF (299 Kbytes), PostScript (2.1 Mbytes)
    ABSTRACT: One of the most common problems in image analysis is the estimation and removal of noise or other artefacts (e.g., grey level quantization) using spatial filters. Common techniques include Gaussian Filtering, Median Filtering and Anisotropic Filtering. Though these techniques are quite common in the image processing literature they must be used with great care on medical data, as it is very easy to introduce artifact into images due to spatial smoothing. The use of such techniques is further restricted by the absence of gold standard data against which to test the behaviour of the filters. Following a general discussion of the equivalence of filtering techniques to likelihood based estimation using an assumed model, this paper describes an approach to noise filtering in multi-dimensional data using a partial volume data density model. The resulting data sets can then be taken as gold standard data for spatial filtering techniques which use the information from single images. We explain how such data has advantages over data generated purely by simulation when testing alternative algorithms.

  • The identification of constituents of remotely-sensed spectra (web pages)
    ABSTRACT: Multi-spectral imaging sensors are now routinely in use. With this increasing utilization of imaging spectrometer data, automatic identification of spectral signatures emanating from this imagery would be an invaluable facility as a precursor to classifying each pixel. Existing methods for identifying constituent spectra typically rely on spectra that are selected either manually or involve manual intervention. The aim of our research work is to assess the viability of linear and non-linear approaches for spectral recognition and to devise techniques suitable for fully-automatic analysis.


Last updated on 16-Jan-2004 by Adrian F. Clark. [Comment on this page]