Subject: Re: Replication of strings

From: "Vinton A. Dearing" <[email-masked]>
Date: Mon, 25 Aug 1997 14:47:21 PST

Thanks to Jim Deardorf for his explanations. I'll have to examine the 
replications in detail before contributing further to the discussion. 
Three questions, though, for Jim. 1. How do you determine the odds 
in question? You cite 0.7 as an example, but do not say what 
circumstance would lead you to choose 0.7. And 2, you speak of 
expected numbers becoming significantly less than 1. How do 
you decide what is "significantly less"? And 3, you mention that 
translations or editings may be independent. Do you have a 
statistical test for independent events? In my book Principles and 
Practice of Textual Analysis I have a chapter on "Probabilistic 
Methods in Textual Criticism" in which I examine some proposals for 
statistical reasoning and make one of my own. If you have such a 
test, I should be glad to have more details.
    Vinton Dearing

---

From: Jim Deardorff <[email-masked]>
Date: Mon, 25 Aug 1997 17:21:37 -0700 (PDT)

On Mon, 25 Aug 1997, Vinton A. Dearing wrote:

> circumstance would lead you to choose 0.7.

Vincent,

What I did was work it backwards.  First you go through the comparison of
duplicated word strings between the parallel passages and get their
distribution.  I call it Y(I) -- the number of duplicate strings found of
length "I" words each.  Then you fit a two-constant exponential curve to
it, using the data points at I=4, say, and I = 9 or 10.  That allows you
to determine the two constants involved, call them A and b, in:

        Y(I) = A*exp(-b*I)

where * denotes multiplication and exp is the exponential.

Then you take the ratio Y(I+1)/Y(I), which by definition is that factor
like f = 0.7 we used as the example.  "A" cancels out and you get

   f = exp(-b)  .

For the gospel parallels I examined, I found "b" to be 0.45, which means
then that f was 0.64.  

> And 2, you speak of 
> you decide what is "significantly less"?

One way was using the test case; there a string was observed for I=11 for
which the exponential predicted 1.20 strings; another string was observed
for I=12 for which the exponential predicted 0.69 strings; and a final
string was observed for I=13 for which the exponential predicted 0.40
strings.  There was no string for I=14 for which the exponential predicted
0.23 strings, and none for any greater values of I.  So this test-case
data set appeared to have independently worded texts (though parallel in
content). 

However, if you had been "unlucky," you still might have run into a string
as long as I=15, say, for which the exponential predicted only 0.044
strings.  This would mean that if you had 23 (=1/.044) very similar cases,
with parallels containing the same number of words, etc., and they had
been independently edited or translated, then you could still expect by
chance to see a duplicate string 15 words long in one out of those 23
cases.  So this is around the usual 5% level where it is customary to draw
the line. But if one had observed two strings of this length, and another
string of I = 16 or 17 words, then one would have to conclude the data
weren't independent, or random.

Another way to look at it is to see the similarity between this and the
simple conceptual problem of drawing a ball, colored either black or red,
out of a mixed bag of 1000 balls, 577 black and 423 red.  Then keep track
of its color, replace and mix it, and draw another, etc., doing this as
many times as necessary until your data base is as large as with the test
case.  Then you'll find that the number of strings (Y) of black balls you
drew in a row (I) before getting a red ball follows the same exponential
distribution.  (The number of black balls is chosen here such that f & b
would be the same as in the test case, where b=0.55.)  You can do this on
the computer scores of times and examine just how infrequent it is to
observe a string (Y=1) after I has become so large that the predicted
number of strings is as small as 0.1, say.  And it comes out the way I
discussed above.

> And 3, you mention that 
> statistical test for independent events?

If the distribution Y(I) indeed turns out exponential, within the expected
limits of sampling error, then the preceding analysis indicates the
editings were either independent, or done in a manner that imitated
independence.  Fortunately, with the gospel parallels one doesn't have to
distinguish between these two cases, as the distribution deviates very far
from exponential for large "I".  For "I" as small as 2 or 3 one doesn't
necessarily expect a very good fit to the exponential curve, since there
are so many nouns with articles (I=2) that an editor can't avoid
replicating, and so many 3-word prepositional phrases that are hard to
edit or translate into anything else.  But for I=4 and on, the statistics
of large numbers seems to take over, and the idiosyncracies of the
particular language involved doesn't seem to matter, so that the
exponential curve prevails unless purposeful editorial replications of
strings occurs. 

I've written this up, along with results for the gospel parallels, and
submitted it to a journal.  Hope to hear before too long if they accept it
or not.

> In my book Principles and 

Does any of your chapter on this relate to the problem at hand?

Jim Deardorff