Subject: Mathematical methods (Was: Re: Sampling and Vulgate)

From: [email-masked] (Robert B. Waltz)
Date: Wed, 1 May 96 07:07 CDT

On Tue, 30 Apr 96, [email-masked] (Vincent Broman)

>
>If your sampling method is biased, increasing the sample size won't help,
>the larger sample will still be biased, it just has a smaller variance.

I must disagree with both halves of this statement. First, increasing
the sample size DOES help, at least somewhat (to argue by analogy --
if you take a public opinion poll, and ask 90% of the people, you
don't have to use an unbiased sample; you will still be close. Not
that I am claiming to sample anything like 90% of the variants).

Also, we should not mistake the meaning of variance. This is REALLY
important, folks. I am a mathematician; I KNOW.

Everything you ever hear about variance, standard deviation, and
other measures of "precision" is based on the so-called "Normal
Distribution" or "Bell Curve."

But not everything follows normal distributions. I could cite
quite a few examples if I went to my math library (stellar
radiation patterns spring quickly to mind).

There is no reason to believe that textual relationships follow
a normal distribution. It may be so, but the fact needs to be
proved. (And until it is, you cannot draw ANY conclusions from
variance.) There is, in fact, reason to believe that the distribution
is NOT normal, but instead has a bunch of spikes. We call the spikes
"Text-types."

In any case, let's remember what variance and standard deviation
are meant to measure. They measure the SPREAD of a curve around the
mean. Thus, once the shape of a curve is established, increasing
the sample size does NOT significantly reduce the variance.

In any case, we are not measuring a distribution here. In any
given reading, two manuscripts can do only two things. They can
agree, or they can disagree. They cannot have "63% agreement," or
some such. There is NO curve for the variance to measure. Just
two spikes, with "truth values" 0 and 1.

Please, folks, let's get our mathematics right.

>The RELATIVE statistics are =exactly= what gets botched up
>by bias in your sampling.  What you need is a sampling method
>that you can convince your readers is totally independent of
>what you're trying to measure.

"Totally" independent? No such thing.

I'm not saying my sample is perfect. I would prefer one that
is at once larger and less biased. But I have yet to see something
better offered. Certainly the statistical basis used by the Alands
is worse; the sample is smaller AND conforms to their peculiar
baises.

>And you would have a hard time convincing me that the selection
>of variants in the UBS3 apparatus is =independent= of the attestation
>by texttypes.

That last statement, at least, I agree with. That's the exact reason
why I increased the sample.

Bob Waltz
[email-masked]

---

From: [email-masked] (Vincent Broman)
Date: Wed, 1 May 96 09:53:00 PDT

-----BEGIN PGP SIGNED MESSAGE-----

[email-masked] replied:
>                                            ...First, increasing

Elementary counterexample:
Try to estimate the number of homeless by taking a telephone poll,
contacting 95% of the phone subscribers in your town.  Your sample
is huge, but it excludes nearly all the homeless, making the bias
catastrophic.

> important, folks. I am a mathematician; I KNOW.

Perhaps.

> Distribution" or "Bell Curve."

A variance is defined for any random variable X with known distribution
and finite mean (not just Normal distributions) as

$  Var(X) = E( |X - E(X)|^2 )  $

if this quantity is finite.  I think there is some confusion here
between the $ \sigma^2 $ parameter of Normal distributions
and the Variance defined for general distributions.

> the sample size does NOT significantly reduce the variance.

The whole point of the Weak Law of Large Numbers and of the Central
Limit Theorem (for general distributions) is to show that under
certain conditions certain variances decrease to zero at a certain speed
as you take more and more samples.  Now these conditions require
repeated independent samples from a random variable, and the
New Testament textual tradition is not a random variable,
but we treat it as such as long as we don't observe all the data
out there but only subsets of the data.

If larger samples don't decrease the uncertainty or variance,
why would we ever take more than one sample?

> Please, folks, let's get our mathematics right.

Yes, please.

Whenever historic data is discussed in terms of probabilities,
we are not saying the past events themselves are still random,
we are describing our uncertainties about the future event
of our encountering the rest of the data.  For example, we examine
the MS in the even-numbered chapters and then predict the
trends we expect to see in the odd-numbered chapters, not yet examined.

When we talk about two MSS agreeing in, say, 58% of the units of variation
listed in Tischendorf, then we mean one of two things, generally.
   1. We have collated all of both MSS and counted the 58% agreement.
or 2. We have compared samples from the MSS and _estimate_ that 58%
      of the sampled and unsampled units of variation would be agreements.
Case 2 involves a model (perhaps unconscious) of the text being a
sequence of Bernoulli trials, each having a probability, p, of showing
an agreement and a probability 1-p of disagreement.  Under certain
conditions, the "best" estimator of this parameter, p, is the ratio
of the number of sample agreements to the number of samples.

> "Totally" independent? No such thing.

Pardon the absolutistic language.
"Independent enough for the present purpose" was meant.
The one who judges the "enough" being the reader, of course.

Vincent Broman,  code 783 Bayside                        Email: [email-masked]
Naval Command Control and Ocean Surveillance Center
Research Development Test and Evaluation Division
San Diego, CA  92152-6147,  USA                          Phone: +1 619 553 1641
=== PGP protected mail preferred.  For public key finger [email-masked] ===