The Talpiot Tomb and the Bloggers I: An Early Success

On Thursday this week I am giving a short presentation at the Duke Symposium on Archaeology, Politics and the Media: Re-Visioning the Middle East (Flyer). My topic is "The Talpiot Tomb and the Bloggers".  Naturally, I will be outlining my presentation here, with some hyperlinks to the relevant material.  In this first post, I would like to begin with an example of how blogging successfully held the programme makers to account and resulted in changes to the claims made.   Most of the links below are to the NT Blog where you can find further links to the relevant information, and you can get a feel for the original timeline, which on this occasion is important.

The Lost Tomb of Jesus was first broadcast across a two hour slot on Discovery Channel at 9pm on March 4 2007.  Several of us live blogged the event.  But by this point, Discovery's publicity machine had been in full force for several days;  there was a press conference, a snazzy "official" website and Discovery's own website.  The bloggers got to work on this informtion straight away and by the time the documentary had aired, there were already major question marks against the claims being made by Simcha Jacobovici and the other programme makers.

The case for the identification between the Talpiot Tomb and Jesus of Nazareth is based largely on statistics.  The cluster of names found in this tomb is said to correspond to a remarkable degree with the names of Jesus and his family.   Before the documentary had aired, I was highly sceptical of the statistical case, not least because it appeared to rely on a dubious identification between Mariamene and Mary Magdalene while at the same time failing to take seriously important contrary evidence, Judas son of Jesus, and ignoring the non-match Matia.

Simcha Jacobovici had hired a top statistician, though, and surely, he argued, his expertise should be taken seriously. The statistician in question was Dr Andrey Feuerverger, Professor of Mathematics and Statistics at the University of Toronto. I wrote the following:

Clearly he knows a lot more about statistics than most of us, and I would not dream of trying to second guess him. But he revealed a very important piece of information at the press conference, that he is not an expert on the New Testament or archaeological data, so he was working with the data given to him by the programme makers. The relevance of this is that a significant and fatal bias was introduced into the analysis before it had even begun.

One can view the data that was given to Feuerverger on the Discovery website, in the PDF packet of documentation, where the grounds for the statistical analysis are given. It is clear from this that the task he was given was to work out the probability of a certain cluster of names occurring, where in each case all known examples of the given name in the given period were divided into all known naming possibilities in the given period. And the names he worked with were Jesus son of Joseph, Mariamne, Maria and Joseph. The name Matia was initially factored in too, and then removed "since he is not explicatively [sic] mentioned in the Gospels". But the problem is not just that Matia is not mentioned as a family member in the Gospels, it is that the greater the number of non-matches, the less impressive the cluster becomes. Or, to put it another way, it stops being a cluster of striking names when the cluster is diluted with non-matches. Mariamne needs to be taken out of the positive calculation and instead treated as a non-match; Matia needs to be treated as a second non-match; Judas son of Jesus needs to be treated as contradictory evidence. These three pieces of data together detract radically from the impressiveness of the given cluster.

In an attempt to make the point by extending and reapplying an analogy that Simcha Jacobovici was fond of, I continued:

At the risk of labouring the point, let me attempt to explain my concerns by using the analogy of which the film-makers are so fond, the Beatles analogy. This analogy works by saying that if in 2,000 years a tomb was discovered in Liverpool that featured the names John, Paul and George, we would not immediately conclude that we had found the tomb of the Beatles. But if we also found so distinctive a name as Ringo, then we would be interested. Jacobovici claims that the "Ringo" in this tomb is Mariamene, whom he interprets as Mary Magdalene and as Jesus's wife, which is problematic (see Mariamne and the "Jesus Family Tomb" and below). What we actually have is the equivalent of a tomb with the names John, Paul, George, Martin, Alan and Ziggy. We might well say, "Perhaps the 'Martin' is George Martin, and so this is a match!" or "Perhaps John Lennon had a son called Ziggy we have not previously heard about" but this would be special pleading and we would rightly reject such claims. A cluster of names is only impressive when it is a cluster that is uncontaminated by non-matches and contradictory evidence.

In short, including Mariamne and leaving out Matia and Judas son of Jesus is problematic for any claim to be made about the remaining cluster. All data must be included. You cannot cherry pick or manipulate your data before doing your statistical analysis.

That post appeared on Thursday 1 March.  (Actually I remember being up late that night to write it, and the time stamp of 1.45am confirms that memory).  Within 24 hours, I was able to publish a follow-up based on a helpful but technical email from Joe D'Mello who was concerned about some of the claims being made on the Discovery Channel website.  D'Mello was able to go much further than I, and others like me, were able to go.  We were largely questioning the data that had been fed to Feuerverger, but D'Mello could see that there were problems also in the interpretation of the statistical calculations.  D'Mello was disputing the following claim that appeared prominently on the Discovery Website:

A statistical study commissioned by the broadcasters (Discovery Channel/Vision Canada/C4 UK) concludes that the probability factor is 600 to 1 in favor of this tomb being the tomb of Jesus of Nazareth and his family.

D'Mello was clear that this conclusion was not justified by the data.  I invited him to write a guest post for me;  and he wrote to Feuerverger and Discovery.  Within two days, now the day of the broadcast itself, D'Mello had secured important corrections from Feuerverger, including the following:

In this respect I now believe that I should not assert any conclusions connecting this tomb with any hypothetical one of the NT family. The interpretation of the computation should be that it is estimating the probability of there having been another family at the time whose tomb this might be, under certain specified assumptions.

Again, I published the material here and again it was not the end of the story. By March 10, D'Mello had secured an agreement that there should be an adjustment on the Discovery website itself, a correction that duly appeared three days later, on 13 March, and then throughout the site by the end of the week, on 16 March. Perhaps the most significant of the changes was this one:

Dr. Andrey Feuerverger, professor of statistics & mathematics at the University of Toronto, has concluded a high statistical probability that the Talpiot tomb is the JESUS FAMILY TOMB.

changed to

Dr. Andrey Feuerverger, professor of statistics at the University of Toronto, has concluded (subject to the stated historical assumptions) that it is unlikely that an equally "surprising” cluster of names would have arisen by chance under purely random sampling.

It is easy to see that the second statement is significantly weaker than the first.

The discussion of the statistics continued for some weeks and months after this initial flurry of emails and posts, and I should make special mention of the work of Randy Ingermanson, who was involved in the discussions right from the beginning and went on to write what I think of as the definitive piece on the subject, Analysis of Andrey Feuerverger's Article on the Jesus Family Tomb. But I have homed in in this post on the early contributions of Joe D'Mello, and the discussions of the statistics in this blog, because it illustrates one of the upsides of the blogs. By providing informed comment in an up-to-the-minute way, the blogs can, on occasions like this, hold the media to account, exposing problematic claims and faulty logic. It was, I think, the combination between speed and accuracy that made the impact. The reactions were speedy, at the very time that the eye of the media was upon us, and when Discovery wanted to avoid criticism. The reactions were informed and accurate, the blogging revolution allowing connections to be made between Biblical scholars and statisticians.

In the next part, I will turn to the broader picture of the blogging of the Talpiot Tomb, and how it had success in changing the scene.