Wednesday, February 28, 2007

The Statistical Case for the Identity of the "Jesus Family Tomb"

I wish to preface my remarks here by making clear that I am not a trained statistician and my concerns about the way that the statistical tests have been set up here are the concerns of a layman. They are therefore subject to correction from real experts.

Before beginning, let's get the key information at our fingertips. The Statistics claim is discussed at Monday's Press Conference (Discovery Channel Website, links on left), in Simcha Jacobovici's interview (especially Parts 3-4), in the Tomb Evidence PDF from the Discovery site (go to p. 13), and in three pages on the Jesus Family Tomb website headed Probability, The Football Field and Probability: Principles Adopted.

The major part of the case that the Talpiot tomb is Jesus' family tomb is based on a statistical claim. It is thought to be so unlikely that this cluster of names, so familiar from the New Testament record, would show up by accident that the identification of this tomb with the family of Jesus is on firm ground. What are the chances, they ask, that one would find a Jesus son of Joseph together with a Maria, a Mariamne and a Jose? Their answer is that the chances are something like 600:1 on a conservative estimate. The identification between this tomb and Jesus' family is all but certain.

I think this case is severely flawed. The essential problem, as I see it, is that the matches between the Talpiot tomb and the early Christian literary record are factored into the calculations in a positive way, but the non-matches are simply ignored, or treated as neutral. This will not do. If a case is built up on the notion that a remarkable cluster of names in a given places matches with a known cluster of names in another place, it is essential that the non-matches are taken seriously too, all the more so when some of the non-matches are not only non-matches but also contradict the literary record. The non-matches are simply absent from the statistical calculations here. The non matches in question are three, and the first of these needs to be underlined because it is being treated not only as a match but as one of the key matches:
  • There is no reliable historical tradition that Jesus was married to a woman called Mariamne (or for that matter Mary, Salome, Joanna or anyone else). It is important to underline this. It is an unexamined assumption that lies behind all the film-makers' discussion of the "family tomb". The ultimate source of this is, I am afraid, popular fiction like The Da Vinci Code. I would not want to assume that the film-makers' research here was deficient by suggesting that The Da Vinci Code was the source of their information but remarkably, they are actually citing it in their remarks in favour of the identification, as if The Da Vinci Code is here giving shared knowledge.* Now given that no reputable historian of Christian origins seriously thinks that Jesus was married to Mary Magdalene (or anyone else, as far as we know), the presence of a Mariamne in the tomb can in no way be allowed to be a part of the statistical calculations here. We cannot assume unevidenced data in setting up the calculation. If the statistical calculation is to have any validity at all, we must work only with the known quantities.

  • There is no reliable historical information that anyone called Matia was related to Jesus' family. The film-makers appear to be aware of this, and talk about the possibility that he might be a relative by marriage, perhaps one of Jesus' sister's husbands. (At the press conference, it is even suggested that he might have been the Gospel writer). One cannot allow negatives like this to be left out of consideration. The Matia ossuary is a non match with any of the data we have about Jesus' family and it cannot be left out of the calculations. In other words, this is not simply a piece of neutral information that one can leave to one side. It needs to be given negative weight, to detract from the probability that this is Jesus' family tomb.

  • There is no reliable historical information that a character called Judas son of Jesus was connected with the Jesus movement. Indeed, this is evidence that contradicts the literary record in a striking way. Let us be clear about how important the appearance of this character is. There is no record of Jesus having any children, and so the evidence here contradicts the identification of the tomb as Jesus' family tomb. It will not do to say that our evidence is incomplete, or that this is an argument from silence, or that we should not rule out the possibility that Jesus had children. The point is that the case being made by the film makers is a case built up on the basis of an alleged remarkable match between one set of data (the names on the ossuaries) and another set of data (the early Christian record). Where that is the basis of the case, it is essential that non matches between the sets of data are taken as seriously as the matches, all the more so where non-matches actually contradict elements in the early Christian record.
Perhaps some will respond by pointing out that professional statisticians have been consulted. Were they not the ones responsible for these calculations? One of Simcha Jacobovici's major claims about the years of research that have gone into this television programme is that previously archaeologists were not talking to statisticians, and so the archaeologists simply did not realize how remarkable this cluster of names was. Jacobovici consulted four statisticians. I have not been able to find out the identities of the other three, or what they said, but there is one who is prominent in the publicity for the film, Dr Andrey Feuerverger, Professor of Mathematics and Statistics at the University of Toronto. He was on stage with James Cameron, Simcha Jacobovici, James Tabor and James Charlesworth at Monday's press conference and he is featured on the Discovery website on the programme. Clearly he knows a lot more about statistics than most of us, and I would not dream of trying to second guess him. But he revealed a very important piece of information at the press conference, that he is not an expert on the New Testament or archaeological data, so he was working with the data given to him by the programme makers. The relevance of this is that a significant and fatal bias was introduced into the analysis before it had even begun.

One can view the data that was given to Feuerverger on the Discovery website, in the PDF packet of documentation, where the grounds for the statistical analysis are given. It is clear from this that the task he was given was to work out the probability of a certain cluster of names occurring, where in each case all known examples of the given name in the given period were divided into all known naming possibilities in the given period. And the names he worked with were Jesus son of Joseph, Mariamne, Maria and Joseph. The name Matia was initially factored in too, and then removed "since he is not explicatively [sic] mentioned in the Gospels". But the problem is not just that Matia is not mentioned as a family member in the Gospels, it is that the greater the number of non-matches, the less impressive the cluster becomes. Or, to put it another way, it stops being a cluster of striking names when the cluster is diluted with non-matches. Mariamne needs to be taken out of the positive calculation and instead treated as a non-match; Matia needs to be treated as a second non-match; Judas son of Jesus needs to be treated as contradictory evidence. These three pieces of data together detract radically from the impressiveness of the given cluster.

At the risk of labouring the point, let me attempt to explain my concerns by using the analogy of which the film-makers are so fond, the Beatles analogy. This analogy works by saying that if in 2,000 years a tomb was discovered in Liverpool that featured the names John, Paul and George, we would not immediately conclude that we had found the tomb of the Beatles. But if we also found so distinctive a name as Ringo, then we would be interested. Jacobovici claims that the "Ringo" in this tomb is Mariamene, whom he interprets as Mary Magdalene and as Jesus's wife, which is problematic (see Mariamne and the "Jesus Family Tomb" and below). What we actually have is the equivalent of a tomb with the names John, Paul, George, Martin, Alan and Ziggy. We might well say, "Perhaps the 'Martin' is George Martin, and so this is a match!" or "Perhaps John Lennon had a son called Ziggy we have not previously heard about" but this would be special pleading and we would rightly reject such claims. A cluster of names is only impressive when it is a cluster that is uncontaminated by non-matches and contradictory evidence.

In short, including Mariamne and leaving out Matia and Judas son of Jesus is problematic for any claim to be made about the remaining cluster. All data must be included. You cannot cherry pick or manipulate your data before doing your statistical analysis.

* One example of this is Jacobovici's interview, Part 4, where he says: "There are two Marys in Jesus' life, as everybody knows, one is his mother, you know, the Virgin Mary, and the other is, Mary Magdalene, you know, post Da Vinci Code everybody knows Mary Magdalene." There are actually at least four Marys in the Gospels, not two: (1) Mary mother of Jesus, (2) Mary Magdalene, (3) Mary sister of Martha, (4) Mary of James and Joses, and (5) Mary of Clopas, though (1) and (4) may be the same person, or (4) and (5) may be the same person. None of these is described as Jesus' wife.


Anonymous said...

There do appear to be problems with the show's statistics and DNA claims. Since burial niches were ordinarily reused and ossuaries often held more than one set
of remains, and since, apparently, the tomb was disturbed in ancient and in modern times, there is no secure link between a DNA sample and a particular inscription.

Though various sets of statistics have been presented, the assumptions on which they are based are questionable. For one example, it biases the results to
emphasize that the Talpiot ossuary with Jesus son of Joseph (if that's the correct reading) is from a controlled dig while the earlier found ossuary with Jesus son of Joseph was not (though the earlier one was in a museum and apparently involved no money-making or hoaxing and is quite likely genuine and
likely from Jerusalem) while, simultaneously, claiming that the "James" ossuary came from Talpiot, an ossuary whose inscription and provenance are, to say the least, questioned.

Stephen Goranson

Anonymous said...

For a non-statistician, you pose a cogent argument in that a perfectly valid statistical analysis of a biased sample will produce a biased result.
It's a real challenge you face when analyzing data provided by someone else particularly when the data concerns as arcane (to the statistician) an area as the distribution of personal names in first century Judea, the circumstances of the excavation, common burial practices, etc.
Sometimes all the statistician can do is perform a valid technical analysis while publicly disclosing the underlying data so that it can be critiqued by the subject matter experts.
I think I personally would have declined to perform this analysis because there's just too much risk of the results of the analysis being interpreted/used beyond the reasonable limits of its validity but I don't think the statistician involved violated any standards of professional integrity.
-- Ishmael

Eric Rowe said...

I have been thinking along the same lines. To make a valid statistical argument, you shouldn't start with the names in the tomb you have already , observe that some of them resemble names from the NT, and then conclude, based on statistics, that this exact combination of names is very unlikely and must be the very same individuals. Instead, you need to start with the whole basket of names that we have for Jesus's family (of which Mary Magdalene is not one). This list includes: Jesus, Joseph Sr., Mary, Zacharias, Elizabeth, John, James, Joseph Jr., and Simon. This is 9 individuals. And if you really want to speculate about non-relatives (like Mary Magdalene) named in the Bible who might under some circumstance have been buried in a tomb with them, the list would only lengthen considerably, because objectivity would demand that Mary #2 not be the only name considered. Then it would have to be calculated statistically, based on the known frequency of names and known number of family tombs that would have existed at the time (not merely those we have found), just how many such tombs would have collocations of each permutation of these names. I must believe that a tomb with 10 ossuaries that only contained 3 out of the initial 9 names--3 very common names--along with one outside the 9, cannot be that unlikely.

Their whole argument as it is reminds me of people I have spoken to who, on statistical grounds, are completely convinced that Shakespeare MUST have been involved in translating Ps 46 in the KJV, since the 46th word from the beginning is "shake" and the 46th word from the end is "spear."

Matt Page said...

Hi Mark, two quick comments. (Not sure why this didn't post first time)

Firstly I'm not sure you are quite right about the statistics, your comments have more to do with the way those statisitcs are taken and interpreted. (I should add I'm no expert either although there was some stats in my degree course - Chemical Engineering)

The probability of a certain cluster occuring (Jesus, Joseph, Mary, Joses, James) within a group of 10 is not affected by other names that may or may not also be present. What the filmmakers are trying to say is that this combination alone is so unlikely that this must be the tomb of THE Jesus. The rest really is trying to explain the unusual data within that given frame.

In other words P(Jesus, Joseph, Mary, Joses, James) = P(Jesus, Joseph, Mary, Joses, James & Jude son of Jesus) + P(Jesus, Joseph, Mary, Joses, James & NOT Jude son of Jesus).

So, in a sense, they are arguing that this five way combination is so unlikley that everything else is irrelevant.

The problem is, however, that

a - probability, is only probability it does not equate to a hard fact. Bauckham's tomb names are indicative, but they are not like having a census. (There may have been temporary surges in cetain names, or pockets missed by people who couldn't afford tombs. or just unlikely coincidences). So whilst the proposed model can state it is unlikely that this combination existed in more than one family, it can't state it as fact. If I have 4 Kings in a hand of poker, it is highly, highly unlikely that someone has a better hand, but that doesn't mean that my opponent at that point doesn't.

This is best illustrated by running Tabor's stadium analogy backwards. The maths turns out the same, but it highlights the difference between probability and reality. Before you ask the last question ("stay standing if you are called Jesus") you are left with 20-30 men standing. Now probability would dictate that there would either be one man in that group who was called Jesus, or none. The thing is that we know this isn't how things work out in reality. For example I was in a group on Saturday, and out of 8 men, four of us were called Matt. Statistically unlikey, but that was how it happened. And the thing is that in real life we wouldn't know how his ould have turned out. Its only a probabilty model.

b - The the association of the last two names is more tenuous and radically alters the possibilities. Firstly the evidence seems to be against the name "James" being present. Secondly the chances of there being another Joses is above the background level due to their being another Joseph in the family already (see my post at rejesus)

c - The names probability doesn't reflect clustering within sub-groups in a society. As with the famous "Birthday Paradox" sub-clustering increases the probabilities of particular combinations, whilst decreasing others. For example in Ireland in the last century, the likelihood of finding a cluster of Seumas, Mary and Patrick would be greater than the product of their individual probabilities because they are all Catholic names. In fairness this might be fairly trivial in this case.

d - as you say they can't really pick and choose how they apply the historical record. You can't say we'll accept the historical record on Joseph, but not on Jesus having a son. This doesn't alter the statistical likeliness of those 5 names clustering, but it calls into question the basis of those five names in the first place.

Secondly, what no-one seems to have pointed out with "The Beatles" analogy is that is more analogous than the filmmakers have realised. If in 2000 years time we did find such a tomb, whilst people would be excited by the cluster and the probability of it happening, someone would (hopefully sooner or later) say, but wasn't John Lennon killed in New York (and wasn't he cremated not buried). As with Jesus, you may have an unusual cluster of names, but you've still got some ecxplaining to do as to how they all ended up in the same place, particuarly given the historical accounts.

Hmm, might turn that into a blog post myself.

Steve Walton said...

Well argued Mark! There's now a fine guest post by Richard Bauckham on Chris Tilling's blog which utilises Bauckham's knowledge of names and tombs to drive a proverbial bus through the arguments of the TV show. See

Michael Turton said...

Eric Rowe has hit upon the same analysis I did. It's worse than he says, however, since the real question is: what are the odds that a tomb with ten ossuaries is going to contain a half dozen names that might be construed as significant in an NT context? In other words, if they had found Joseph, Joses, James, Andrew, and Peter, it would have been just as suggestive. So would Mary, Barabbas, Cleophas, John, and Saul. Or James, Andrew,.... there must be tens of thousands of such combinations -- especially, as Mark points out, if you get to cherry pick your data set.


Bill said...

A more accurate "Beatles analogy" would be a tomb found not in Liverpool, but Hamburg.

Anonymous said...

Also, regarding the football stadium claim, Jesus was not part of the year-round population of Jerusalem, but a visitor, for instance during the pilgrimage festival Passover, when the population perhaps doubled. How likely would it be for the US to have two George Bush presidents and two John Adams and two Roosevelts? Names are not randomly distributed, for various reasons. If there had been a Jesus family tomb in Jerusalem, why then would they use that offered by Joseph of Arimathea?

Stephen Goranson