From
the BAS Speaker Aug.-Sept. 1984
The
Digital Challenge: A Report
by Stanley P. Lipshitz
University of Waterloo
Waterloo, Ontario Canada
Some readers may be unaware of the background
to the "digital test" in which Ivor Tiefenbrun participated
on February 23, 1984 [1-4]. To summarize briefly: Tiefenbrun
has been quite outspoken about the inadequacies of digital audio
recordings and the systems on which they are made, and his organization,
Linn
Products Ltd., was instrumental in publishing an
analysis of the Sony consumer PCM-F1 digital audio adapter [5]
outlining their objections. I, on the other hand, have been
using this very system for a number of years now and have made
close to one hundred recordings with it with superb results
and not the "execrable results" reported by Tiefenbrun.
I and my colleague John Vanderkooy have moreover conducted blind
listening comparisons between the PCM-F1's input and its reconstructed
output signals, and had yet to find anyone who can reliably
distinguish between them on musical program material.
I therefore challenged the "anti-digital"
community in general, and Tiefenbrun in particular, to participate
in a blind listening test of the PCM-F1 to give them the opportunity
to substantiate their claims of poor sound. The challenge details
were spelled out in [2]. When I learned in February 1984 of
Tiefenbrun's impending visit to Toronto, I reissued this challenge
and was pleased to have him accept.
The test took place in the home of the local
Linn distributor, Mr. Michael Remington, using his all-Linn/Naim
system (Linn LP-12 turntable, arm and cartridge, Naim NAC 32
pre- and NAP 250 power amplifiers and Linn Isobarik loudspeakers)
and his choice of program material (all LP records). Vanderkooy
and Alan Lofft, editor of Sound Canada magazine, were also present.
The atmosphere throughout was cordial and more relaxed than
I expected.
The day began with two brief tests of the
Tiefenbrun claim that undriven transducers (digital alarm watches,
telephones, headphones, or other loudspeakers) in the same room
audibly degrade the sound quality - a claim which forms the
rationale behind their "single speaker" demonstration
demand. Firstly, a digital alarm watch with piezoelectric "beeper"
was held about 500 mm behind Tiefenbrun's head while he listened
to the loudspeaker reproduction from his stereo seat on the
couch, with the watch either fully exposed or clasped firmly
between the palms of my hands. We were assured that the latter
artifice would muffle any deleterious effects. This was thus
a single-blind test: The testee did not know the covered/uncovered
status of the watch at each trial, but the tester did know.
A random series of 20 trials was conducted while Remington cued
up the turntable (playing a female vocalist) on each occasion,
as he did throughout the day. Tiefenbrun's result: 10 correct
responses in 20 trials, an outcome which shows no ability to
discriminate between the two situations.
The second test, also single-blind, used
a Linn "Kan" loudspeaker as the undriven transducer.
Again the female vocalist was used as source material. The loudspeaker
lay on the thickly-carpeted floor behind the listening couch.
It was placed either on its side (the "uncovered"
condition) or on its face (the "covered" condition)
according to a random series of choices. Ten trials were conducted
during which Tiefenbrun achieved a score of 5 correct out of
10. Again, this demonstrates no discrimination ability beyond
what one would expect purely on the basis of chance.
With these preliminaries out of the way
(but just for safety all watches and headphones were left in
another room throughout the entire day's proceedings), we proceeded
to the main test of the day - that of the audibility of the
Sony PCM-F1 digital audio processor in 16-bit mode when inserted
into the audio chain, as proposed in my original challenge.
The PCM-F1's analog-to-digital (D/A) converter
output was looped straight through to the digital-to-analog
(D/A) converter input by connecting "video out" to
video in". The complete encode-decode chain including the
two low-pass anti-aliasing filters as well as the sample-and-hold
circuits and the A/D and D/A converters was thus subject to
the test. In order to remove the F1's polarity reversal it had
been internally fitted with a digital inverter chip (as used
in the later non-inverting Sony PCM-701 model) to interchange
the ones and zeroes and hence perform a polarity correction.
|
Fig. 1
|
Schematic of the A/B/X switching
arrangement used to insert the PCM-F1 (set for precise unity
gain) into the pre-to-power-amplifier chain on a double-blind
basis. Only one of the stereo pair of channels is shown. |
The PCM-F1 was inserted into the signal
path between the preamplifier and the power amplifier by means
of an A/B/X switchbox of our design (see [6, 7]). The setup
is shown in Fig. 1, above. When the switchbox was in the "A"
position the PCM-F1 was inserted into the signal path, while
the "B" position represented the "straight-through"
configuration. The "X" position was a randomly selected
choice made by the switchbox. For each trial "X" was
chosen by the box to be either "A" or "B",
but its identity was unknown to any of the participants, thus
making the experiment double blind. The subject could at his
leisure compare any of all of "A", "B" and
"X" with each other. What he had to do was identify
whether "X" was "A" or "B". Once
his decision was made, the switchbox was interrogated to discover
the true identity of "X", and then "X" was
re-randomized in preparation for the next trial.
The gains of the "A" and "B"
paths were matched in both left and right channels to within
0.05 dB at 1 kHz using the PCM-F1's gain controls. This was
done by measuring across the amplifier output terminals. The
match was then confirmed to be within ± 0.25 dB across
the whole audio band. The PCM-F1's "peak hold" feature
was used to keep a record of the peak signal levels passing
through it during the test, especially in view of the relatively
high sensitivity of the Naim power amplifier (<1 Vrms at
clipping) and the relatively low listening levels chosen by
the participants. More about this shortly.
After an acclimatization period, a set of
10 trials was conducted in an unhurried fashion before breaking
for lunch, after which a further set of 10 trials was conducted.
Tiefenbrun's score for the series was 11 correct decisions out
of 20, a result which shows no statistically significant ability
to discriminate between "A" and "B" any
more accurately than would be expected on the basis of random
guessing.
At this point I thought that I could reliably
distinguish between the "A" and "B" paths
on the basis of the slight noise level increase which occurred
when the PCM-F1 was inserted into the chain, and which was marginally
audible due to the high gain of the Naim MAP 250 power amplifier
combined with the low peak signal levels through the F1, which
the peak-hold meters showed to have risen no higher than -20
dB. (0 dB is the digital clip point, and these peak levels were
somewhat unfair to the digital processor since 20 dB of its
signal-to-noise ratio was being thrown away.) [In other words,
for this segment of the test the F1 was in effect a 13-bit processor.
- Ed.]
I expressed my desire to try the test, and
Remington went to cue up the record again, but I requested to
be allowed to undertake the test with no signal passing though
the system. Before realizing the import of what he was saying,
Vanderkooy interjected: "Ah! You're going to listen to
the sound of the relays." Yes, there is indeed a slight
audible difference between the acoustic "click" made
when the "A" and "B" relays pull in. This
is due to the unavoidable differences in the mounting positions
of the relay on the A/B/X box chassis and, although slight,
it can be heard if one listens for it. I replied that I was
going to listen to the difference in background hiss, and the
subsequent series of blind trials showed conclusively that the
two signal paths could be reliably distinguished on this basis
alone.
What conclusion can we draw from this? Tiefenbrun's
random results show that he had not been aware of either the
background noise difference - masked as it most likely was during
his trials by the record surface noise - or the relay click
difference. The null result of this first PCM-F1 listening test
is thus valid in spite of the potential for non-blindness due
to the subliminal noise differences. I had been aware of the
hiss difference from the outset from listening while the stylus
was off the record, but had decided to proceed with the test
in the knowledge that if the test produced a null result, the
hiss difference could not have affected the outcome - a decision
vindicated by the actual data.
|
Fig.2
|
Schematic of the A/B/X switching
arrangement for the second series of PCM-F1 listening tests.
This time the digital system is inserted into the preamplifier's
tape monitor loop on a double-blind basis. Only one channel
is shown in the diagram. |
The "secret" about the slight
relay noise difference was, however, now out in the open, and
if listened for could from then on influence (albeit subconsciously)
the blindness of any succeeding tests made using the switchbox.
Nevertheless we decided to proceed with a second series of PCM-F1
listening tests, this time with the digital system inserted
into the preamplifier's tape monitor loop, as shown in Fig.
2, above. Once again levels were carefully equalized and frequency
responses checked to be very closely matched between "A"
and "B". This insertion point precedes the preamplifier's
volume control, where the signal levels are significantly higher,
thus more fully exercising the F1's full dynamic range while
at the same time rendering its output noise level inaudible
at the replay volume setting then being used. This configuration
removed the hiss difference as a potential factor.
A set of 23 double-blind trials was conducted
listening to music through the system, with Tiefenbrun voting
for the identity of "X" at each trial. It was then
that Vanderkooy pointed out that, although the relays in the
A/B/X box were switching normally during this series, no tiny
electrical clicks were audible from the loudspeakers during
their operation, as normally would be the case. Investigation
revealed that the preamplifier's "source/tape" monitor
switch had inadvertently been left in the "source"
position, and as Fig. 2 reveals, the A/B/X box and digital system
were thus not being inserted into the chain at all during this
run!
Because the "A" and "B"
positions were completely identical due to this error, an unbiased
decision for "X" should have produced a 50:50 split
of "A" and "B" votes. Interestingly, it
turned out that this was not the case, Tiefenbrun's votes being
14 for "A" and 9 for "B". Could something
have been influencing his voting, or did he just prefer the
letter "A"? A comparison of the actual "X"
choices showed that during this series Tiefenbrun voted incorrectly
in 16 out of 23 trials, although the sound from the loudspeakers
never changed. [This proportion of incorrect guesses would occur
in a truly random situation less than 5% of the time. - Ed.]
What does this mean? We cannot say for certain, but the most
logical explanation is that he was (perhaps subconsciously)
voting on the basis of the relays' acoustic click difference
which we had now revealed. If so, his remembrance of the sound
of the A and B clicks was inverted. Be this as it may, the error
we had made by leaving the tape monitor switch in the "source'
position turned out to provide an interesting sidelight on the
question of personal bias. It also illustrates one of the potential
pitfalls of high-resolution blind testing, and the danger of
jumping to conclusions before very carefully checking the test
setup.
With the tape monitor switch now correctly
set, a rather rapid series of 37 trials took place with Tiefenbrun
voting. The results: 10 correct decisions out of 37, a result
far worse than would have been expected by chance alone. What
was influencing the voting? Was it the relay sound? We do not
know, but this seems likely. It should be remembered that using
the A/B/X box, a direct comparison between "A" or
"B" or "X" is always available, so a series
of trials in which there is a tendency as above to produce a
consistent reverse identification seems to implicate some extraneous
factor unrelated to, but correlated with, the loudspeaker sound.
Tiefenbrun did very little switching during this series, leaving
the A/B/X box set to the "X" position most of the
time. It does indeed seem possible that the identifications
were being made on the basis of the relay sound incorrectly
remembered.
|
Fig. 3
|
Schematic of the setup used
to test for audibility of the relay contacts in the A/B/X
box. The relay contacts can be inserted into the tape monitor
loop using the source/tape switch on the preamplifier. This
is a single-blind experiment. |
To wrap up the day's work, we conducted
a final series of tests to assess whether the relay contacts
in the A/B/X box could have been electrically affecting the
sound in any audible way. For this single-blind test the configuration
was as in Fig. 3, above. The A/B/X box was powered up and left
permanently in one position, and the relay contacts inserted
into the signal path by switching the preamplifier's monitor
switch to the "tape" position. The switching sequence
(relay contacts "in" or "out") was determined
by a random number sequence. Thirty trials were conducted, interrupted
three times to provide Tiefenbrun with a reference comparison
of the "direct" sound versus the relay contacts. His
score in identifying the presence of the relay contacts was
12 correct out of 30 trials. This of course also shows no statistically
significant ability to identify correctly when the relay contacts
were in circuit.
In summary, then, no evidence was provided
by Tiefenbrun during this series of tests that indicates ability
to identify reliably:
(a) the presence of an undriven
transducer in the room,
(b) the presence of the Sony PCM-F1 digital processor
in the audio chain, or
(c) the presence of the relay contacts of the A/B/X
switchbox in the circuit.
The tests were conducted in an amicable
rather than confrontational atmosphere, and the parties departed
feeling that the day's work had been worthwhile. Further carefully-conducted
blind tests will be necessary if these conclusions are felt
to be in error.
I would like to acknowledge the friendly
cooperation and assistance of both Michael Remington and John
Vanderkooy in carrying out these experiments, and would like
to take this opportunity to express to Ivor Tiefenbrun, the
guinea pig of the day's experiments, our genuine admiration
for having the sincerity and guts to put his professed beliefs
on the line.
References:
[1] S. P. Lipshitz, "Views", HFN/RR,
Aug. 1984, p.15
[2] S. P. Lipshitz, "Views", HFN/RR,
Sept. 1983, p.19
[3] I. S. Tiefenbrun, "Views",
HFN/RR, Jan. 1984, pp. 19, 21.
[4] I. S. Tiefenbrun, "Views",
HFN/RR, June 1984, pp. 13, 15.
[5] A. Orlowski, "Digital Sound: The
View from Scotland", HFN/RR, June 1983, pp. 34-37.
[6] S. P. Lipshitz and J. Vanderkooy, "The
Great Debate: Subjective Evaluation", J. Audio Eng.
Soc., Vol. 29, pp. 482-491 (1981 July/Aug.).
[7] D. Clark, "High-Resolution Subjective
Testing Using a Double-Blind Comparator", J. Audio Eng.
Soc., Vol. 30, pp. 330-338 (1982 May).
|