Reviews


I have never been able to argue with authorial authority

I have never been able to argue with authorial authority

In a recent discussion with my regular role-playing group one player was complaining about the plethora of super-hero movies being released recently, and her increasing exhaustion with this genre. Another defended it partially on the basis that he has always really enjoyed superhero comics so seeing good movies of them is fun, but yeah maybe there are a few too many. I chimed in to this essential conversation to observe that I’ve never been able to get into super hero comics by Marvel and DC (and I guess Vertigo too) because I find the text so incredibly frustrating to read. The way they put bold/italic emphasis on almost random words in the text – in almost every piece of text – really distracts me from what I’m actually reading and drives me crazy. The original complainer agreed that she, too has always found this off-putting.

As an example, consider this blog post at Lawyers, Guns and Money about what a superb comic artist some guy is. It gives a long, detailed dissertation about how the action within the frame is juxtaposed with the flow of the panels to inculcate in the reader the same sense of discomfort and challenge experienced by the character the panels are about. This seems like a fairly plausible interpretation of the effect of this particular set of panels but I just can’t care about how great this makes the artist because the entire scene is so devastatingly annoying. What is with all that emphasis in all the text? Why emphasize the word “lightning bolt” and the names? It’s distracting and annoying.

I’ve felt this way for years of course but never really investigated, so I tried a bit of googling to see if I could find anything on the topic, and a brief search revealed nothing – possibly because including the words “Marvel”, “DC” or anything similar in a search term drowns out the rest, but possibly also because no one writes about this stuff. So what is going on? Why do they have to put emphasis in comic book text at all, let alone randomly throughout every second speech bubble? Is it something about the reading age of the audience? Is it meant to add dramatic tension? Is there no one in either of these quite large companies who reads this stuff, finds it annoying, and occasionally considers maybe not doing it? Are there two types of people in the world? As far as I know the method isn’t used in manga, at least not in Japanese and I don’t remember it in English either. Why do these comics do it? And is there a legion of haters of this stuff out there? If you do hate it, is it possible to enjoy the comics at all or are is it always overwhelming?

Inquiring MINDS want to KNOW.

Those bastards!!

Those bastards!!

I ran into more problems trying to run OpenBUGS on a Bayesian spatial regression model today. The problems are laid out pretty simply in the picture above: I ran into an illegal memory write. I know this is not the fault of my model, as it runs smoothly for 50, 500 or 1000 iterations. This illegal memory write happens somewhere between 6000 and 6100 iterations.

This model is a simple version of a larger model I’m building. It has data on about 1800 small areas collected over four time points, with age and sex covariates, with a Poisson distribution for the outcome and an adjacency matrix for the 1800 areas. The data are laid out with the time points and age- and sex-values as separate records, so for one municipality I have 4*6*2=24 observations. I can’t use other packages for this because they assume one observation per municipality (i.e a very simple relationship between data structure and adjacency matrix). So I’m using OpenBUGS or WinBUGS. The ultimate goal is to extend to 30 time points and 18 age categories, with a more complex time and age structure, but I want to get this simple model running first because I’m running into major computational issues.

Today I identified that these computational issues can’t be solved. The reason for the illegal memory write is that OpenBUGS has a 1.6Gb RAM limit. People say this but it’s hard to confirm; however, in digging through critiques of BUGS I discovered it is written in Component Pascal and compiled in BlackBox, and a bit more digging reveals BlackBox has a 1.6Gb RAM limit.

This is frankly ridiculous. It’s understandable that there would be a RAM limit in a 32-bit system, but OpenBUGS has been around for about 9 years now and no one has upgraded it to handle either multi-core computers or 64 bit architecture. Given that OpenBUGS is designed to handle complex, processor-demanding simulation-heavy Bayesian models, the decision to write it in such a restrictive framework is ridiculous. I’m a strong critic of open source systems but if it had been written for R it would at least be able to use the 64-bit architecture, even though R has been late to the multi-core party. I bought a PC with 128 Gb of RAM specifically for this big project, and I might as well have bought a laptop with the minimum RAM. For the model I ultimately aim to build I expect I will need about 100,000 iterations to get stability, which means that OpenBUGS will never get there. The only way to run this model without a workaround is to either 1) write the full adjacency matrix by hand and implement it directly [something I am considering] or 2) recompile the source code (which I think might not be available) in a different Pascal compiler. I have no idea how to even start with that.

I have considered two workarounds, however, though I don’t know whether either of them might work.

  1. Save and rerun: the reason that OpenBUGS hits its RAM limit is that it saves all iterations in RAM, but I think this is possibly bad design. So one option could be to run the model to 5000 iterations, then save the results, shut down OpenBUGS, reopen OpenBUGS, load the model file, and then use the update functions to run another 5000 iterations on what has already been run. I think this can be done (running additional updates on a past model) but I’m not sure. If this works I just need to run one set of runs a night for about 3 weeks, and I’ll get my model.
  2. Rerun with sequential initial values: If method 1 doesn’t work, another option that I am very sure will work is to run the model for 5000 iterations, extract all estimated values from the model, then start a new model of 5000 runs with the past estimated values as the initial values for the next model. I’m pretty sure it will start off where it left off, assuming I correctly specify them all, although there might be small jumps in the trace, but ultimately it’ll get where it needs to go. But restarting the model is going to take a lot of time unless I can find a way to loop through and extract values automatically (I think I can’t). So probably a month to run the model.

Ridiculous! And even if I had a supercomputer I couldn’t speed it up …

Today Stata 14 was released, and it finally has built in functionality for a wide range of Bayesian statistical tasks. I’m hoping that they’ll introduce conditional autoregression and the BYM model in version 15. Really, if you have a grant that can afford a single Stata license, it’s really worth getting. You just can’t rely on open source statistical packages. Only use them when you must!

Another perfect moment in British colonial development

Another perfect moment in British colonial development

I am in London for a week doing some research with small area analysis, and on the weekend had a brief opportunity to actually see the city. As is traditional by now on my annual trips to London, I visited the World Wildlife Photography Exhibition (which was a bit weak this year, I thought), and having a bit of time to kill wandered up the road to the Science Museum. Here I stumbled on a small and interesting exhibition entitled Churchill’s Scientists, about the people that worked with Winston Churchill before, during and after the war on various projects, and Churchill’s powerful influence on British science.

This year will see the 70th anniversary of the end of the war, and you would think by now that popular culture of the victorious countries would finally have got to the point where it is able to handle a more nuanced analysis of the politics of that time than mere hagiography. It’s clear that the allied powers were uncomfortable about some of their actions during the war: the careful elision of Arthur “Bomber” Harris and his fliers from peacetime awards is an example of British squeamishness about the morality of the bomber war, but this squeamishness doesn’t seem to have manifested itself in any kind of clear critical reevaluation of the behavior of the allies at war, at least in popular culture. This silence is starting to be broken by, for example, Antony Beevor’s uncomfortable discussion of rape in Berlin, or his discussion of the treatment of collaborator women in Normandy; but it is generally absent from public discussion. Churchill’s Scientists is, sadly, another example of this careful and deliberate overlooking of the flaws of wartime leaders and their politics when presented in popular culture.

The exhibition itself is small and interesting, walking us through various aspects of the scientific endeavours of the pre-war and post-war eras. It describes the scientists who worked with Churchill, their relationship with him and the public service, and how science was conducted during the war. Churchill was very close friends with a statistician who advised him on all aspects of war endeavours, and also was very supportive of operational research, which was basically an attempt to revise wartime strategy on the basis of evidence. The achievements of these scientists given their technological limitations are quite amazing: drawing graphs by hand on graph paper to attempt to explain every aspect of the statistics and epidemiology of rationing, conducting experiments on themselves to understand the effects of low-calorie diets, and feverishly working to improve tactics and technologies that were valuable to the war. The post-war efforts were also very interesting: there is a life-size installation showing the original model of myoglobin, which was studied using x-ray crystallography and then built by hand using cane rods and beads to create the three-dimensional structure. There is a telling quote about how scientists became used to asking not “how much will it cost” but “how quickly can we get it done and what do we need?” There are also some interesting examples of how the wartime expectations of scientists translated into peacetime success: they had contacts in the ministries from their wartime work, they were used to having funds and knew how to raise money, and they had access to hugely increased resources as the ministries dumped wartime surplus in universities and research institutes. In the 1950s this translated into rapid advances in medicine, genetics, nuclear power and astronomy, all of which are documented in the exhibition.

There are, however, some political aspects that are overlooked. Currently in the UK there is an ongoing debate about whether to stop conducting the Census because it costs too much, and it is clear that since the war there has been a shift in funding priorities and a move away from the idea that science should be funded at any cost. I would have been interested to find out how this happened: did Churchill change his attitude towards funding for science or was this a post-Churchill trend? Was Churchill the last of the Great Investors? What did subsequent conservative party leaders make of his legacy and how do they talk about it? Why is it that the country that invented radar, that perfected antibiotic production, and that contributed more than any other to modern geographical statistics and demography, can no longer “afford” the census? Was the war a high point and an aberration in the history of British science funding? Did its successes distort the post-war scientific landscape and expectations? None of this is really described in the exhibition, which limits itself to Churchill’s positive legacy, and doesn’t seem to want to explore how it was undone. There is also a bit of attention paid to female scientists in Churchill’s war efforts, including women who developed X-ray crystallography and did important nutritional epidemiology research. But we know that much of the computational work done in the war and immediately after was also done by women, but they were slowly squeezed out of the industry after the war. I would have been interested in some description of what happened to all those female scientists and ancillary staff after the war – were they forced out of science the way women were forced out of factory work, or did Churchill’s support for women in science during the war permanently change the landscape for women in science? It seems clear that Watson and Crick’s work – initially sparked by x-ray images of the DNA that are shown in this exhibition – must have been built on the work of crystallography’s pioneers, who were women. But where did those women end up when the war effort wound down?

The other aspect of this exhibition that is sadly missing is a discussion of Churchill and his scientists’ darker sides. We are introduced to the exhibition through Churchill’s love of flying; the website for the exhibition quotes him talking about new technologies in aerial bombing; and the exhibition itself talks about his support for a British nuclear weapon. But nowhere in the exhibition is his enthusiasm for terror bombing discussed, nor the unsavoury way in which he developed this enthusiasm, running terror bombing campaigns against Iraqi tribespeople in the 1920s. Arthur Harris is only presented once in the exhibition, dismissing a biologist who proposed a campaign of tactical bombing of railway junctions (he “wasted his time studying the sexual proclivities of apes,” was the dismissal); but nowhere is the corollary of this position – Harris’s lust for destroying cities – mentioned, or the extensive scientific work that went into developing the best techniques for burning civilians alive. In the year that western governments will demand Japan apologize for its wartime atrocities (again!), one would think they could at least mention in an exhibition on wartime science the extensive research that went into perfecting the practice of burning Japanese civilians alive.

In case one thinks this might have been just an oversight on the part of the curators, later we see a more direct example of this careful elision, when the exhibit focuses on Britain’s post-war nuclear weapons program. Again, we have been presented with Churchill’s direct interest in blowing stuff up; here we are shown video of a nuclear test, and discussion of the research that scientists were able to do on the environmental and physical effects of the bombs. The exhibition doesn’t mention that many of these tests, conducted in Maralinga in Australia, were conducted on land that Aborigines had been expelled from and were unable to return to. It also doesn’t mention the contamination of Aboriginal customary lands, any possible harmful health effects for Aborigines living in the area, and the controversies of the Maralinga inquiries and subsequent compensation for soldiers and workers. Not even a one sentence reference.

Given that we know Churchill was a deeply racist man who supported colonialism and had no interest in the rights of non-white British, it seems hardly surprising that he might have had a slightly cavalier attitude towards ethics in research and military tactics where it was directed against Iraqi tribesmen or Australian Aborigines. It seems like 70 years after the end of the war it might be possible to start talking about this stuff honestly outside of academia, and to publicly reevaluate the legacy of men like Churchill, and many of his senior scientists, in the light of everything we know now, rather than simply portraying all their efforts through only the lens of wartime heroism. Churchill was undoubtedly a great man and a powerful leader, and the world owes him a debt of gratitude. He was also a racist and a colonialist, and some of the decisions he made before, during and after the war may not have been either right or the best decisions for the time. It also appears that despite his greatness, the legacy of his interest in science and education was soon undone, and the reasons for this are important for us to consider now. What does it say about Britain that 70 years after the end of the war it is still not possible to honestly assess Churchill’s wartime efforts but only to extol his great contribution to science; yet 70 years later his contributions to science have been so far wound back that the government is considering abolishing the Census? Does such hagiography benefit Britain, or British science? I would suggest not.

This year is the 70th anniversary of the end of the war, and we are going to see a lot more public discussion of the actions and contributions of the great people of that time. I fear that this discussion is going to be very shallow, and sadly empty of any attempt to critically reassess the contributions of the people involved, and how they shaped our post-war culture. This exhibition is a good example of how the war will be presented this year: stripped of moral context, all uncomfortable truths banished from discussion, and all long-term ramifications for post-war politics and culture carefully sanitized to ensure that no difficult questions are asked, or answered. Perhaps we aren’t doomed to repeat history, but I think this year at the very least we are going to be bored stiff by it.

Only what you see man, only what you see

Only what you see man, only what you see

Today a friend took me, without explanation, to see Sophie Calle’s The Unsold (売り残し) at Koyanagi Gallery in Ginza. I don’t often attend art shows – let alone modern art installations – and I almost never visit Ginza, so this was a real novelty for me, but despite my initial misgivings it was definitely worth it. Here is my review.

When I entered the gallery my first glance revealed an installation of everyday objects, including two dresses, that to my jaundiced and cynical eye immediately resembled Tracey Emin’s execrable bed-type stuff, and I was immediately disappointed. However, right at the door there is an introductory explanation (in Japanese and English) of the premise of the work, which changed my mind. Basically, three artists set up a flea market in the grounds of Yasukuni Jinja. They laid out their wares on three squares of cloth, as shown in the picture. One (I don’t recall which) sold worthless every day items, to each of which was attached a story that actually happened (i.e. a real story) with some relationship to the item but in which the item itself was not directly involved (so e.g. the typewriter on sale is not necessarily the typewriter from the story). Another sold a mixture of semi-antiques (cutely mis-spelled as “semi-antics” in this exhibition) and ordinary items, to which were attached completely fake stories with apparent emotional content[1]. The third sold actual antiques, and one of his original photos. For example one person was selling a completely normal bra for about 25,000 yen, and another person was selling a picture of a psycho-analyst (freud?) for 38,000 yen. One of the antiques was an ancient ceramic hot water bottle, and the picture was a pretty cool sea/sky thing. Each artist catalogued what they sold and the amount of money they sold it for – which was surprisingly large. Apparently an American tour guide passed by as this sale was going on and told his charges “there is nothing here, ignore it.” (Cute). The explanation finishes with the simple, curt phrase “These are the unsold.” So the exhibition consists of the material that was not sold.

This exhibition consists of three pieces of cloth on which the remaining items are laid out, attached to each of which is a tag with the price and the story. Behind each installation, on the wall, is a photo of the original setup, so you can see what was sold. On the opposite wall are the tags for the sold items, with their corresponding story. These tags have no information about the item to which they correspond, so you have to wander across to the original picture and guess. The stories are really interesting and believable, though whether they are actually true or not I have no clue. Investigating on wikipedia I discovered that the Eiffel tower story is true, and just as unbelievable as it sounds – Sophie Calle certainly knows how to do crazy things (I can’t remember if the item attached to this story was sold or not).

I’m an uncultured barbarian, so I have no idea what this installation was trying to tell me about whatever, but I thought it was really cool. Trying to understand why people bought these ludicrously overpriced objects because of their vague stories, or didn’t buy some object even though its story was cool, was an exercise in intruding into someone else’s private life. The stories themselves were fascinating, disconnected monologues, none of which I believed (but some of which I have subsequently learnt are real!) I can’t speak for the Japanese but the English used in the broader narrative descriptions – what the exhibition is about, how the artists met – is clear, sparse and strong. The structure of the main introductory sign and its finishing statement, “These are the Unsold” is particularly powerful, and suits the style of the exhibition. It’s a simple idea done well, and it holds your attention. Why did the passersby leave the charred bedspring and buy the useless typewriter? This, I cannot fathom. I wouldn’t buy the red bucket some guy pissed in, but why would someone else buy the bottle. Also the story of the horn is acutely sad and the horn is quite cheap, but apparently un-sellable. What does that mean?

I didn’t know anything about Sophie Calle before this exhibition, but reading her Wikipedia page I get the impression that she is a powerful, prodigious and generally unethical talent. My friend has also seen the exhibit Take Care of Yourself, which as the quoted reviewer says seems to be both shallow and deeply engaging. Her attempt to get blind people to define beauty sounds like it has the potential to be very powerful (I don’t draw any conclusions!) and the work where she gets a guy to shadow her and then presents pictures of herself sounds really interesting. Invading others’ privacy, not so much. How come medical researchers have to get ethics approval, but French artistes can pursue some guy across the world, or hijack a stolen diary for money?

Don’t answer that.

Anyway, I’d never heard of Sophie Calle before today and I think her work is a genuinely interesting and challenging example of modern art at its finest. I don’t know what she’s trying to say with this exhibition and I can’t really say what I think of it, but it’s really cool. It would be better if she followed it up with some kind of article in a peer-reviewed journal giving her conclusion about what the purchases and non-purchases mean, instead of leaving it to an ignorant rube like me to try and understand, and if she had found a way to summarize what was bought and wasn’t (e.g. rankings with stories, or a website where you can see all the objects with what was bought and what wasn’t, and its story) then the exhibition would have been even cooler. But despite these missed opportunities this exhibition is very cool, and in general I have to say Sophie Calle’s work seems pretty interesting. I hope more of her stuff comes to Japan, and I recommend visiting it if you are in Japan, or keeping an eye out for her work if you are not.

 

 

 

fn1: I may be mis-remembering the exact nature of what these items were, but I hope you get the general gist.

I’ve recently been building a fairly complex series of Bayesian spatial regression models in BUGS, and thought I’d share some tips based on hard won experience with the models. The various BUGS packages have the most cryptic and incoherent error messages of any stats software I have ever worked with, and although various Bayesian boosters claim that their modeling approach is intuitive, in my opinion it is the exact opposite of intuitive, and it is extremely hard to configure data for use in the packages. Furthermore, online help is hard to find – google an error message and you will find multiple websites with people asking questions that have never been answered, which is rare in the modern world. I take this as a sign that most people don’t understand the error message, and indeed the BUGS manual includes a list of errors with “possible interpretations” that reads more like the I Ching than a software guide. But Confucius say Enlightenment is not to be found in Black Box Pascal, so here is my experience of BUGS.

The models I’m running are complex, with nested conditional autoregressive structures and the higher level having more than 1000 areas with complex neighbour relationships, and millions of observations. I originally ran them on a completely hideous Hewlett Packard laptop, with 4 cores and 8Gb of RAM. I subsequently upgraded to a Dell Workstation (joy in comparison to HP’s clunky root-kitted horror) with 8 cores and 16Gb of RAM; I’m not sure that hardware is the main barrier to performance here though …

The HP machine had a secret administrator account (arseholes!) so I couldn’t install winBUGS[1], so I started off running OpenBUGS called through R’s R2OpenBUGS package running in RStudio. I use R to set up the data and initial values, because I can’t think of any other way to load millions of observations into a text file without going stir crazy. But when I call OpenBUGS it just hangs … no error messages or any other kind of indication of what is going on. I also can’t tell if it is happening at the data loading or compiling or inits stage.

Some digging around online and I found an old post by Andrew Gelman, observing that BUGS does not work well with “large datasets, multivariate structures, and regression coefficients.”

i.e. pretty much every statistical problem worth doing. Gelman also notes that “efficiently-programmed models can get really long, ugly, and bug-prone,” which seems like a contradiction in terms.

Anyway, noting that my data was large, with multivariate structures and regression coefficients, I thought maybe I should tone it down a bit so I tried using a higher level of spatial heirarchy, which reduces the adjacency matrix by an order of magnitude. Still no dice. It was at this point that I upgraded to the bigger computer.

On the bigger computer the smaller model actually worked! But it didn’t work in the sense that anything meaningful came out of it … It worked in the sense that it reported a completely incomprehensible bug, something like a node having an invalid value. I tried multiple different values and nothing worked, but somewhere on the internet I found someone hinting that you should try running BUGS directly rather than calling through R, so I tried this … having created the data in R, I killed OpenBUGS then opened the OpenBUGS interface directly and input the model, then the data, using the text files created by R[2].

When I did this I could step through the process – model was syntatically correct, then model failed to compile! Given that loading inits comes after compilation, an error telling me that I had the wrong initial value seems a bit misleading… in fact I had an “index out of range” error, and when I investigated I found I had made a mistake preparing one part of the data. So where the actual error was “the model can’t compile because you have provided the wrong data,” when called through R the problem was “you have the wrong initial values” (even though I haven’t actually loaded initial values yet).

WTF?! But let’s step back and look at this process for a moment, because it is seven shades of wrong. When you run R2OpenBUGS in R, it first turns the data and inits into a form that OpenBUGS can read; then it dumps these into a directory; then it opens OpenBUGS and gets OpenBUGS to access those files in a stepwise process – at least, that’s what I see R doing. If I decide to do the model directly in the OpenBUGS graphical interface, then what I do is I get R to make the data, then I use the task manager to kill OpenBUGS, then I call OpenBUGS directly, and get OpenBUGS to access the files R made in a stepwise process. i.e. I do exactly the same thing that R does, but I get completely different error messages.

There are various places on the internet where you might stumble on this advice, but I want to stress it: you get different error messages in OpenBUGS run natively than you do in OpenBUGS called through R. Those error messages are so different that you will get a completely different idea of what is wrong with your program.

Anyway, I fixed the index but then I ran into problems after I tried to load my initial values. Nothing seemed to work, and the errors were really cryptic. “Invalid initial value” is not very useful. But further digging on the internet showed me that OpenBUGS and WinBUGS have different approaches to initial values, and winBUGS is not as strict about the values that it accepts. Hmmm … so I installed winBUGS, and reran the model… and it worked! OpenBUGS apparently has some kind of condition on certain variables that they must sum to 0, while winBUGS doesn’t check that condition. A free tip for beginners: setting your initial values so they sum to 0 doesn’t help, but running the same model, unchanged, in winBUGS, works.

So either OpenBUGS is too strict, or winBUGS lets through a whole bunch of dodgy stuff. I am inclined to believe the former, because initial values shouldn’t be a major obstacle to a good model, but as others[3] have observed, BUGS is programmed in a completely opaque system so no one knows what it is doing.

So, multiple misleading errors, and a complex weirdness about calling external software through R, and I have a functioning model. Today I expanded that model back to the original order of magnitude of small areas, and it also worked, though there was an interesting weirdness here. When I tried to compile the model it took about three hours, and produced a Trap. But the weird thing is the Trap contained no warnings about BUGS at all, they were all warnings about windows (something called Windows.AddInteger or similar), and after I killed the Trap my model updated fine. So I think the compile problems I previously experienced may have had something to do with memory problems in Windows (I had no problems with badly designed adjacency matrices in the larger model), but OpenBUGS just doesn’t tell you what’s going on, so you have no idea …

I should also add, for intrepid readers who have got this far, that this dude provides an excellent catalogue of OpenBUGS errors with his plain English explanations of what they actually meant. He’s like this mystical interpreter of the I Ching for Bayesian spatial regressives. Also I want to add that I think the CAR spatial correlation model is super dodgy. I found this article (pdf) by Melanie Wall from the Journal of Statistical Planning and Inference (what a read!) that shows that the way we construct the spatial adjacency matrix is the primary determinant of the correlation structure, and that the correlation structure determined by this adjacency matrix is nothing like what we think we are getting. Today on my whiteboard and with the help of R I imagined a simple industrial process where each stage in the process is correlated with the one before and after it, and I showed very easily based on Wall’s work that the adjacency matrix required to describe this process is completely different to the one that you would naively set up under the framework described for CAR modeling. So I think most of the “spatial correlation” structures described using CAR models have no relationship to what the programmer thinks they’re entering into the model. But I have no proof of this, so I guess like everyone else I’ll just press on, using the adjacency matrix I think works …

So there you have it. Next time you see an opinion formed on the basis of a spatial regression model built in BUGS, remember the problems I had getting to the output, and ask yourself – do you trust that model? Really?

fn1: Well, I could copy winBUGS into the program files folder but I couldn’t patch it or install the immortality key, which, wtf? When I bought Time Series Modelling and Forecasting by Brockwell and Davis, ITSM came as a free disk with the book. When I buy the BUGS book I get to install software that comes with a big message telling me to get stuffed, and years later they finally provide a key that enables you to use it for free …

fn2: If you aren’t aware of how this works, basically when you call OpenBUGS in R, providing data from inside R, R first dumps the data into text files in the directory of your choosing, then OpenBUGS opens those files. So if you aren’t comfortable preparing data for BUGS yourself, use the list and structure commands in R, then kill OpenBUGS and go to OpenBUGS directly … the text files will remain in the directory you chose.

fn3: Barry Rowlingson does a pretty good job of showing how interesting and useful spatial analysis can be: see e.g. his post on mapping the Zaatari refugee camp in Syria.

Galadriel goes to market

Galadriel goes to market

One of the English loan-words that Japanese people misuse slightly in a really cute way is gorgeous (ゴージャス). In Japanese gorgeous refers not to something really nice, but to something that is overdone or just a bit too much – not necessarily unappealing or unattractive, but just a bit too much. I’ve heard the word applied to appearance, food and even writing (e.g. scientific writing should not be gorgeous). It’s often associated with the stylistic choices of young women of a certain social class, and also with hostesses. It’s not necessarily a marker of class or taste, and not deployed in a particularly judgmental way, but it suggests a certain immaturity or inelegance in taste, something that’s acceptable in young women but not for example something one would respect in an adult[1].

The Hobbit: Battle of Five Armies is the cinematic showcase for this word. It’s too long, the battle scenes especially are unnecessarily embellished, and the heroism is over the top and over-frequent. Almost every moment of it is also great fun. These battle scenes are the kind of battles where you can imagine seven impossible maneouvres before breakfast, where enormous and terrifying trolls are killed with a single knife stroke, and where a war pig can be more terrifying than a giant. There are even sand worms! As battles go it’s a tour de force, the entire movie is basically one long series of battles, with maybe two brief pauses to discuss the importance of family and tasteless jewellery. The centerpiece battles – between the Uruk Hai and the dwarven heroes – are masterfully done and very enjoyable, but they’re so over the top as to be ridiculous. They’re also good examples of exactly what gorgeous means: for example, Legolas’s prancing up the collapsing tower is precisely how I imagine an elf to be able to move against the laws of nature, it’s the right thing to be in this kind of movie, but it is dropped into the middle of such a long-running series of epic-level feats that instead of being stunning and impressive, it’s just another blister of impossibilities on the back of your retina.

In this regard the movie can be contrasted very effectively with other works from the same series. The final battle between the fellowship and the Uruk Hai in The Fellowship of the Ring, for example, is a masterclass in how to turn a classic role-playing battle into believable cinema. It depicts a group of high-level characters at the peak of their power pulling themselves out of what is basically a lethal ambush by overwhelming numbers, with minimal losses. They do things we know are physically impossible, but they aren’t so far from impossible that we are lifted out of the feeling of the battle by them, and they aren’t so fast-flowing that they become overwhelming in their fantasticality. That battle is heroic fantasy at its finest, patently unrealistic but completely believable in the context of the world, and really engaging. The battles in the Battle of Five Armies are so full of over-the-top heroics and impossibilities that they become less an exercise in story-telling and heroic fantasy and more of an exercise in braggadocio by everyone involved. Yes, I want to see my fantasy heroes do impossible things; I want to see victories against overwhelming odds; I want to know that these people are not normal, not like me, doing things I can’t do. I don’t want this experience to be transformed into marveling at the ingenuity of the movie’s creator’s rather than its characters.

Just as a young hostess’s style can be so gorgeous that it becomes a self-evident performance of beauty rather than beauty itself, so this movie has turned heroic fantasy into a performance of itself, rather than a performance for its fans.

And don’t get me mistaken, I am a fan. The Hobbit is not a particularly interesting or enjoyable book, and Peter Jackson had pretty thin gruel to work with in making this part of the epic; he also had to please a group of tantrum-prone true-believers with an immature and shallow approach to the work. Given how dark and grim the later Lord of the Rings movies turned, he also had to find a way to leaven the silly boys-own-adventure style of the main plot with some kind of nod to the growing shadows. By choosing to work in the unwritten parts of the original story – Gandalf’s exploration of Mirkwood and the battle with the necromancer, for example – I think he has made the story more engrossing and enjoyable. He has also managed to present us with a breathtaking and splendid vision of Middle Earth, carved out of New Zealand, that has been more or less consistent across six diverse movies, and has stuck very closely to the aesthetic vision of Tolkien’s main visual interpreters. He managed to lift the dwarves from their shallow representation in the book and Snow White-style triviality in popular culture into serious, adult figures without falling on the cheap Jewish or Scottish stereotypes that often get attached to them, and for this all Tolkien fans should be eternally grateful. The dwarves are excellent, and as dwarves should be – dour, hard working, tough, narrow-minded and loyal. They look like adults and adventurers, and unlike Gimli (or Dwain in this movie) they don’t get turned into comedy sideshows. The Hobbit would have been an utter disaster if it had been made by someone trying to be loyal to the original book and the needs of the fans, it would have been a single stupid movie involving 12 characterless technicolor idiots and a dude in a pointy hat, cocking up everything they do.

Furthermore, The Hobbit is a rare example of a movie that manages to make a dragon a central part of it without cocking it up monumentally, which every other movie except Dragonslayer and Reign of Fire has managed to do. Smaug is an evil, cunning, wily and deeply sinister monster of terrifying power, and as soon as he is let loose on Dale you can see why armies of dwarves would fall before one of these things. His supreme arrogance, coupled with his incredible power and complete disregard for mortals and their feeble efforts, is a joy to behold. This is how a dragon should be! But even here we see Jackson falling for the gorgeous: the simple tale of Smaug’s death gets padded out with an unnecessary piece of sentimentality and impossibility, and a spot of slightly out of place (but nonetheless enjoyable) humour. Nothing in this movie just jumps, or just climbs, or just dies. Not even Smaug.

Still, I didn’t sign up for the last instalment in this epic so I could see a handful of orcs get their arses kicked by some woodland sprites and a few technicolor stereotypes in a backwoods scrap. I signed up for a monumental battle between the noble forces of good and the deepest evil ever conceived, and that’s what I got – in spades. The Orc leaders and Uruk Hai champions were awesome, the dwarven and elven battle scenes were spectacular, the troll stormtroopers impressive and exciting (though like every other stormtrooper, remarkably easy to kill …), the desperation of the human defenders grim and hopeless. This is a two-plus hour rollercoaster of well-deserved death and slaughter, and though you will at times find yourself thinking “what were they thinking?” and marvelling more at the movie-makers’ ingenuity than the actual traits of the people on the screen, you’ll still love every minute of it.

But it is too gorgeous.

fn1: Remembering that in modern Japan the word “adult” is increasingly coming to mean a person over 30, and there is even a growing fashion trend for otona (大人) that is specifically aimed at offering classy but still pretty and sexy clothes to women aged in their 30s and 40s. This style is largely the opposite of gorgeous.

I have a dataset of about 40 million records covering rare events over a 30 year period, with about 600,000 events in total. The data are clustered in about 50 regions, and in the latter years of the file there are only about 4000 events a year (they increase in frequency in the early years). This means that in one year there may be only 80 events in any one region. I want to conduct a multi-level Poisson regression of the events, with about four or five categorical covariates plus year and a random effect for region. I’m doing this using GLLAMM in Stata on an Apple computer with 32Gb of RAM, 12 cores and Stata/MP optimized for all 12 cores (this is not a cheap setup!) The raw dataset takes about 16Gb but I can collapse it to a dataset of counts of events and populations at risk, which reduces it to (I think, from memory) about 170,000 records. I’m using a Poisson distribution, so essentially a loglinear model, but I should probably ideally use negative binomial for zero-inflated data – unfortunately GLLAMM doesn’t handle negative binomial data and I have read of problems with the xtnegbin procedure, so I just have to suck it up and use the Poisson family. I’m also not adjusting for serial dependence, because that would mean extra random effects and/or correlation structures and it would just get too nasty.

The problem here is that GLLAMM just won’t converge. If you set it running with relatively standard settings you can first run it with the NOEST command to get the coefficients, then feed them as starting values into a model with adaptive quadrature and 12 integration points, but it just won’t converge: after 3 or 4 days and 150 iterations (one estimation cycle takes about an hour) it still hasn’t converged, and the maximum likelihood estimate is not concave. The maximization process seems to be proceeding in some fashion, because every 10 iterations or so the likelihood decreases slightly, but it just won’t seem to stop. Interestingly you can’t set the tolerance in the estimation process in GLLAMM, so I can’t stop it when the MLE is decreasing by less than 0.01 (which it is doing now) but more than 0.00001.

This non-convergence is not a data issue – it works fine in a standard Poisson regression, there are no warnings about combinations of covariates with all zeros or all 1s, and this also seems to be the case when the data is divided up into regions (as best I can tell). I think the problem is just that the events are rare, there are a lot of dimensions over which the likelihood needs to change, and it’s just a very tough optimization problem. I have run into this problem with non-linear multi-level models before, and it appears to be a problem of big data with rare events and large numbers of regions. So what to do?

I have noticed in the past that estimation in GLLAMM is more likely to be successful if there are more integration points, which slow down the time to converge but improve stability. I’ve also noticed that the initial estimates can really kill the model, and also I’ve noticed that a big part of the estimation procedure is about getting standard errors, not estimates. Also, although you can’t control tolerance you can limit the number of iterations (a rough version of the same thing). Each iteration taking an hour, if one wants to get actual results it is helpful to run as few iterations as possible[1]. So today I tried a new method for getting convergence, which works in stages like this:

  • Start with the NOEST option to get initial coefficients for starting values
  • Feed these into a model with 1 integration point (Laplace Approximation) and 1 iteration
  • Feed the coefficients from this as starting values to a model with 2 integration points and 2 iterations
  • Keep running new models, slowly increasing integration points and iterations

If I did this a few times, I found that my models started converging after just two Newton-Raphson cycles. By the fifth model (with nip(12) and iterate(8)) the coefficients had all become stable to 4 decimal places, and the variances were largely fixed: all that remained was small changes in the variance of the random effect. At the fourth model some of the variances were out of whack, but the coefficients close to their final value. The fourth and fifth models converged on the second Newton-Raphson step. I set it running last night and it’s basically now fine-tuning, using a model with 24 integration points and a maximum of 32 iterations. That will be the final model, it’s probably going to be done by tomorrow morning.

Because this modeling process starts with the final coefficients from the previous model, it seems to bypass a lot of the convergence problems that arise from starting with a model that has coefficients very far from the final values and then waiting. Instead, you get rough coefficients halfway through the process and then restart. Since much of the challenge for the estimation process seems to be in calculating variances rather than coefficients, this saves you a lot of time waiting to get values. It seems to have shortcircuited a lot of convergence issues. I don’t know if it would work for every problem (GLLAMM seems to be a temperamental beast, and every time I use GLLAMM a new problem seems to rear its ugly head), but if you are having convergence problems and you are confident that they are not caused by bad data, then maybe give my method a try. And let me know if it works! (Or if it doesn’t!)

fn1: Actually this model process takes so long that there are some sensitivity analyses we just have to skip. There is one method we use in preparing the data that I would like to check through resampling and sensitivity analysis, but if every model is going to take 2 days, then you can’t even do a paltry 100-sample bootstrap and get a meaningful answer in half a year. Crazy!

Next Page »

Follow

Get every new post delivered to your Inbox.

Join 59 other followers