As a scientist in the later stages of my career, the managerial and mentorship load has increased, leaving less time for math and programming. These technical activities were also why I wanted to be a scientist in the first place, and I regretted losing the time for this hands-on research. Formerly the core of my scientific work, these activities require sustained attention, hours at a time, a resource now in short supply.

Also like many scientists, I’ve watched the coming of artificial intelligence over the last few years with growing interest. The technical predecessors and underlying substructure of these large language models (LLMs) are neural networks, a computer technology that has already begun to revolutionise many scientific fields, including cosmology and astrophysics. I wrote about LLMs in my recent book, The Random Universe, but hadn’t really used them for my own work.

So I started using Anthropic’s Claude (no particular reason for this choice, except that some colleagues had had positive coding experiences with it), figured out how to wire it up at the command line, and started talking to it. (After, yes, paying a subscription fee.)

My first project with one of these newly-capable AIs was a minor reanalysis of our data from the Planck satellite, seeing how the cosmological inferences respond to small changes in the data, part of the work for a recent paper. I knew exactly what I needed to do, but it was a lot of plumbing: getting disparate bits of software written by other people to work together in a way different from their authors had intended. I figured I could do it in a few days of solid work.

Instead, I pointed Claude to the draft of the paper, along with publicly available Planck data and code repositories, and asked it to implement the paper’s algorithms with Planck’s data. A few hours later, there was code, alongside tables, figures, and lots of tests to make sure I could trust — and understand — the results. It wasn’t (we weren’t) just able to write and debug the code quickly, it was able to run it, again and again, making small tweaks to the inputs and the code itself, and to the figures it generated, now part of our recent paper.

Next was something more involved: colleagues and I have created a program called Almanac to analyse specific kinds of cosmological data. We wanted to apply Almanac to some new results, in a regime in which it hadn’t really been tested (a very small patch, around 1% of the total sphere of the sky). Almanac helps us measure a curve called the power spectrum, which I’ve written about before.

I pointed Claude to our code, our papers, and to the new data, and explained the problem. Even ensuring that the (poorly documented) data was in a form that our code could understand would have taken me a few hours, but Claude suggested and implemented a series of tests to ensure that everything was self-consistent.

Almanac is a Monte Carlo sampler: because we are trying to understand the probability distribution of matter in the Universe, using noisy and incomplete data, the answer to our questions can only be given as probability distributions. Almanac is essentially a very complicated random number generator, and you can do self-consistency checks to see whether it is producing random numbers with the right properties.

Almanac’s results failed these tests. Could we understand why? Could we fix it? Now, rather than just plumbing, I needed Claude to help me diagnose the problem. It took a while.

Was it a simple bug? We did a series of tests showing that Almanac does work, essentially perfectly, on simpler datasets covering much more of the sky. In fact, this gave me the opportunity to ask Claude to write some new software, based on a paper and related code that I first wrote, with Dick Bond and Lloyd Knox, about 30 years ago. This older algorithm (“BJK”, from our initials) answered the same statistical question as Almanac, using a very different technique. On large areas of sky, Almanac and this older algorithm got the same answer — the code works.

We went on a long rabbit-hole modifying the details of Almanac’s setup, making it more similar to other state of the art samplers. This change also didn’t solve our problem, although it seems to help on the margins. I made one suggestion that I thought would help, based on our long-ago experience with the BJK algorithm, bundling up some of the numbers we were trying to determine into “bands”. I don’t know if Claude would have come up with this idea on its own, and it took a while to get the details right. In fact, Claude would sometimes declare premature victory, admitting its mistakes only when I pointed them out.

It worked, eventually. After a lot of iterations, we transformed a problem unsolvable with the previous version of the code to one that was, well, easy.

But it only worked because my knowledge and experience — literally decades working on problems of this sort — meshed with Claude’s own “talents” — quick turnaround, patience, and encyclopaedic, if not always discriminating, knowledge of computing and of at least some aspects of the underlying science, statistics, and mathematics.

And it was fun! I thought that I liked programming, but I am very happy to have Claude do most of the grunt-work for me. The quick turnaround, and not having to sweat the details of writing and running re-writing and re-running program after program, was a delight.

In many way, working with Claude was like working with a junior colleague. But Claude is not a colleague, but a machine. And, as David Hogg has advocated, the point of doing astrophysics, a beautiful but useless field of science, is exactly the training and fulfilment of the people doing it. That has at least two implications. First, given how much my own experience was necessary to getting good results, that means we had better make sure that we are training humans, not just better LLMs. Second, no matter how delightful the interactions, they mustn’t replace training our students and collaborating with our colleagues.

(This post was written by me, not by Claude, though I did ask it to suggest a title — and this sentence.)