To make a valid statistical argument, you shouldn't start with the names in the tomb you have already , observe that some of them resemble names from the NT, and then conclude, based on statistics, that this exact combination of names is very unlikely and must be the very same individuals. Instead, you need to start with the whole basket of names that we have for Jesus's family (of which Mary Magdalene is not one). This list includes: Jesus, Joseph Sr., Mary, Zacharias, Elizabeth, John, James, Joseph Jr., and Simon. This is 9 individuals. And if you really want to speculate about non-relatives (like Mary Magdalene) named in the Bible who might under some circumstance have been buried in a tomb with them, the list would only lengthen considerably, because objectivity would demand that Mary #2 not be the only name considered. Then it would have to be calculated statistically, based on the known frequency of names and known number of family tombs that would have existed at the time (not merely those we have found), just how many such tombs would have collocations of each permutation of these names. I must believe that a tomb with 10 ossuaries that only contained 3 out of the initial 9 names--3 very common names--along with one outside the 9, cannot be that unlikely.Also in comments, Matt Page writes:
I'm not sure you are quite right about the statistics, your comments have more to do with the way those statistics are taken and interpreted.Not quite. My comments are more to do with the evidentiary basis for the statistics, i.e. the information that was fed to Feuerverger, which I regard as (a) incomplete (Matia and Judas son of Jesus are not neutral data) and (b) misleading (Mariamne Mara is not a name given to Mary Magdalene, or Mary Anyone Else from the Gospels, nor -- if it were -- is anyone in Jesus' family given that name). So my complaint is not about what they did with the statistics but what they did before they had started with the statistics, i.e. the information fed to the statistician, hence my use of the term "cherry picking". Matt continues:
The probability of a certain cluster occurring (Jesus, Joseph, Mary, Joses, James) within a group of 10 is not affected by other names that may or may not also be present. What the filmmakers are trying to say is that this combination alone is so unlikely that this must be the tomb of THE Jesus. The rest really is trying to explain the unusual data within that given frame.I think this accurately conveys what the film-makers are claiming, but it is flawed, not only because there is no James, but also because clustering becomes increasingly less impressive relative to the size of the sample. So it is clearly more likely to get the names in question occurring in a group of seven named ossuaries than it is in a group of three -- and so on. Michael Turton expresses the problem, also in comments, in the following useful way:
the real question is: what are the odds that a tomb with ten ossuaries is going to contain a half dozen names that might be construed as significant in an NT context? In other words, if they had found Joseph, Joses, James, Andrew, and Peter, it would have been just as suggestive. So would Mary, Barabbas, Cleophas, John, and Saul. Or James, Andrew,.... there must be tens of thousands of such combinations -- especially, as Mark points out, if you get to cherry pick your data set.Meanwhile on Deinde, Danny Zacharias suggests that the film-makers are aware of the problem over "Judah son of Jesus" and so suggest that this is the beloved disciple, with some spurious exegesis to get one there.
James Tabor, the scholar most closely associated with the documentary, has a useful post on the Flawed Statistics & Ossuary Names and news of the Ted Koppel show to air on Sunday after the documentary, and featuring James Tabor as well as Darrell Bock, who also mentions it. A quick word about the tone of James Tabor's posts on The Jesus Dynasty Blog: he is setting a high standard for civil and respectful discourse. It is a pleasure to see the non-polemical tone and others could learn from it.
If you are getting a bit fed up of the whole thing, there is some great humour around. Jon Steward on the Daily Show is a must see -- go to The Daily Show website. Thanks to several people who have drawn attention to this, e.g. Joe Weaks on the Macintosh Biblioblog. Or on Paleojudaica, Jim Davila mentions Scott ("Dilbert") Adams' humour on the same issue. And one of my favourite comments in my blog was the person who remarked, "What is the likelihood that Jesus would have named his son Judas, of all names . . . "!
There are several more things I would like to mention. My email inbox is choc-a-bloc. So please be patient. More later.
9 comments:
The more I hear of the statistical "reasoning" behind the various assertions, I am reminded of a long-ago professor's take on a Hippocratic Oath for statisticians: "At least tell no lies..." :-)
-- Ishmael
Greetings Dr. Goodacre,
Since your e-mail is choc-a-bloc, I thought I'd use the comments-board to let you know that my own essay on the claims in "The Lost Tomb of Jesus" -- including the supposed connection between the name "Mariamne" and Mary Magdalene, and the misrepresentation of the meaning of the statistics, and some interesting statements by James Tabor -- is online, at
www.curtisvillechristian.org/TombOne.html .
From an interpretive standpoint you have to understand that the final number the documentary folks are attempting to calculate is an expected value, not a probability, as they incorrectly suggest. The number represents the average number of tombs you would expect to find with those names if you could investigate every tomb if there were 1000 total tombs and each tomb had the same number of names. (This is known as an "expected value" or "average value.") They attempt to calculate the probability of a given tomb having a particular set of names, then multiply that probability by the total number of tombs. This tells you how many tombs you expect to find on average.
Therefore, the number they come up with is highly dependent on the number of tombs during that time period. The higher or lower the total number of tombs, the higher or lower number of tombs with those four names you would expect to find.
Thus the number they are trying to calculate (even if they got it right, which they don't) does not tell you the likelihood of finding a family during that time with those four or five names in them. In other words, there might be many, many families with people of those names in them, but because only a small percentage of families had tombs, the number of tombs you expect to find is small.
Even if you ignore the various interpretive problems as well as the many problematic mathematical issues such as the existence of other ossuaries and names in the Talpiot tomb, the number of total tombs, the variation in sizes and numbers of names in those tombs, there is one very simple issue which even a first year student would recognize.
Their calculation depends on finding those five names in the listed order. It doesn't take into account the probability of finding those names in a different order.
If you adjust the calculation just to include all possible orders (4!=24) you increase the expected value to 1/25, or 0.04 tombs.
They list five names and throw out one because it is of questionable relevance, even to the documentary makers. But you can't just throw it out. You have to then say, what is the probability of finding four names out of five to be those four names, plus a fifth name which could be anything. That increases the expected value to 1/5, or 0.2 tombs. (Five choose four, times 1/25)
There were actually six names found in the Talpiot tomb. That increases the expected value to 3/5. In other words, you'd expect to find 0.6 tombs, given our assumptions.
There were four ossuaries without names. If you assume a thousand identical tombs (which is what the Cameron documentary essentially does) and treat those four unnamed ossuaries as having any possible name that increases the expected value to 8.4 tombs. In other words, out of a thousand tombs with ten ossuaries, you’d expect to find 8.4 tombs with those four names out of ten. Admittedly this involves a particular interpretation of how to deal with the unnamed ossuaries. It is the harshest against the Cameron documentary claims, but that is not unreasonable given that the documentary has the burden of proof, so to speak. Archaeological and historical knowledge would help one have a better idea of how to deal with the four unnamed ossuaries. Ignoring them is a possibility, but would seem to me to be less likely to be correct than including them somehow.
Again, let me repeat that the number they are trying to calculate (even if they got it right, which they don't) does not tell you the likelihood of finding a family during that time with those four or five names in them. It attempts to give you the expected value for the number of tombs you would expect to find on average given certain assumptions.
The bottom line is that the simplicity of the methodology and the fundamental nature of these errors that I described makes me question the seriousness and integrity of this entire project. I expected to find some questions of interpretation and issues of data bias due to archaeological assumptions, but instead I found glaring basic mathematical flaws.
BTW, the expected value of 0.6 comes from "six choose four", times 1/25.
The expected value of 8.4 comes from "ten choose four" times 1/25.
Post a Comment