Interest within the automatic speech recognition research community has recently focused on the recognition of speech where the microphone is located in the medium field, rather than being mounted on a headset and positioned next to the speakers mouth to realize the long-term goal of ubiquitous computing. This is a natural application for beamforming techniques using a microphone array. A crucial ingredient for optimal performance of beamforming techniques is the speaker location. Hence, to apply such techniques, a source localization algorithm is required. In prior work, we proposed using an extended Kalman filter to directly update position estimates in a speaker localization system based on time delays of arrival.We also have enhanced our audio localizer with video information. In this work, we investigate the influence of the speaker position on the word error rate of an automatic speech recognition system operating on the output of a beamformer, and compare this error rate with that obtained with a close talking microphone. Moreover, we compare the effectiveness of different localization algorithms. We tested our algorithm on a data set consisting of seminars held by actual speakers. Our experiments revealed that accurate speaker tracking is crucial for minimizing the errors of a farfield speech recognition system.