As is typical with any advance in AI and robotics, the initial coverage of ChatGPT — OpenAI’s artificial intelligence “large language model” that generates conversational responses to question prompts — emphasizes that it’s pretty cool. Which it is. The following answer to my question about the post-1960s Great Stagnation is a not-terrible answer. Certainly a decent launching pad for further research:
Then come the stories about all the potential downsides, such as job loss. This from the Guardian: “Professors, programmers and journalists could all be out of a job in just a few years, after the latest chatbot from the Elon Musk-founded OpenAI foundation stunned onlookers with its writing ability, proficiency at complex tasks, and ease of use.” Another concern is students using ChatGPT to write essays and papers for them. This from NewScientist: “Schools and educational institutions in the US and elsewhere are announcing bans on the recently released AI-powered chatbot ChatGPT out of fear that students could use the technology to complete their assignments.”
Overall, however, it seems that media coverage falls far short of hysterics about technological unemployment and turning all our kids into cheaters. Perhaps that’s due to the obvious shortcomings of this iteration of ChatGPT. “Some ChatGPT answers have bias, circular logic, and inaccuracies, which are often disguised by very confident prose,” notes Bloomberg reporter Trung Phan.
I also wonder if the shortcomings of autonomous driving technology are perhaps playing a role. The emergence of self-driving tech in the late 2010s led to lots of bold predictions that have yet to pan out. There aren’t a million AVs on the roads. Nor do AVs appear close to causing mass riots among unemployed truckers. And maybe that disappointment has led to a bit more caution about predicting how fast ChatGPT and other such technologies will advance and what their impacts will be.
Then again, maybe more journalists are making more of an effort to think creatively. Here’s New York Times tech columnist Kevin Roose: “I believe schools should thoughtfully embrace ChatGPT as a teaching aid — one that could unlock student creativity, offer personalized tutoring, and better prepare students to work alongside A.I. systems as adults.”
More of that, please! I would like to see more coverage of potential upsides, especially the ability of such language models to aid academic research as a sort of “super research assistant.” That aspect of ChatGPT is the subject of new research from finance professors Michael Dowling (Dublin City University) and Brian Lucie (Trinity College Dublin), “ChatGPT for (Finance) Research: The Bananarama Conjecture.” The title makes reference to the 1980s English pop group whose biggest hits in America were “Cruel Summer” and “Venus.” (Well, at least those are the songs I’m most aware of. But the paper’s title was inspired by the lyrics to their (along with Fun Boy Three) 1982 song “It Ain’t What You Do (It’s The Way That You Do It)”: “It ain’t what you do, it’s the way that you do it / And that’s what gets results.”
The reason for the reference is that researchers argue, “Like all tools, ChatGPT is best in experienced hands.” (Not the strongest reasoning, perhaps, but clearly either Dowling or Lucie, perhaps both, are monster Bananarama fans. Love the fandom. I’ll allow it.) That conclusion comes from their analysis of ChatGPT’s ability to assist with writing a research study — in this case a research study on cryptocurrencies. ChatGPT had to generate four elements: a research idea, a condensed literature review, a description of suitable data for the research idea, and a suitable testing framework given the research idea and the proposed data. The first version used these prompts:
1. Research idea: “Write me a 100 word research idea suitable for a good finance academic journal on the topic of: cryptocurrencies, sentiment, and economic uncertainty”
2. Literature review: “Thank you. Now write me a 300-word literature review which justifies the research idea. Please only use real articles as references, do not make anything up.” [This last sentence was introduced after earlier trials showing that ChatGPT will generate ’fake’ references if not instructed otherwise]
3. Data summary: “Can you please now describe in about 100 words the data that would be suitable for investigating this research idea? The data should be suitable for publication in a good finance journal.”
4. Testing framework: “I would now like you to design a suitable testing approach based on this research idea and the data. Five tables in total of testing output are needed. Can you list what these five tables might be, with descriptions?”
This version used only public data already within ChatGPT. A second version incorporated private data, “abstracts and article identifiers for 188 articles identified as related to cryptocurrencies and published in Finance Research Letters (2021-2023) from the Elsevier Scopus database.”
The third version involved the researchers taking the results of the the second version and then telling ChatGPT how it might improve its suggested answers to the research idea that examines the “relationship between economic policy uncertainty, trading volume, and cryptocurrency returns.” For example:
Iterating the data summary: “For the research idea below and the literature review, can you suggest a suitable data approach that could be adopted to implement the research idea. This data should be suitable for a good finance journal, so might include the type of data discussed in abstracts in the bibtex files at the start of this thread. Please list the number of years of data to be collected for each data item. There should be a specific data item identified for: (1) cryptocurrencies – e.g. what cryptocurrencies; (2) market sentiment – how to measure specifically; and (3) market regulations – e.g. what countries regulations, and why. The research idea and literature review is as follows: …”
And their conclusion, which makes it clear why the Bananarama reference is used:
Comparing the different research versions we see a clear outperformance by our most advanced research study, V3: Private Data and Expertise. We were surprised to see that the version with added private data underperformed compared to the version with only public data. On reflection, this appears to be due to the private data model excessively relying on the provided private data and restricting the extent to which it accessed other beneficial public data. This could be improved by either instructing the platform to not ignore useful public data, or by providing a better-curated set of relevant private data. The outperformance of the V3 research study is notable, not just on an overall basis, but also in the extent to which it is also capable of producing acceptable literature reviews and testing frameworks where the other research studies have less success. … ChatGPT can generate, even in its basic state, plausible-seeming research studies for well-ranked journals. With the addition of private data and researcher expertise iterations to improve output, the results are, frankly, very impressive. Bear in mind, also, that these results are obtained without the advantages of GPT-4 as an underlying generative model, due to launch later in 2023 and which promises a truly revolutionary language model due to advances in algorithms and over 600 times greater testing parameters.
Dowling and Lucie do discuss whether there are ethical issues of human researchers presenting a research paper generated in this fashion as their own. But more interesting to me is this example of ChatGPT as a super RA or an “e-Research Assistant,” to use the authors’ nomenclature. It sure does seem like the future of research, especially for those scholars who don’t work in rich countries. Again, from the paper:
Indeed, under this perspective the platform might even be viewed as democratising access to research assistants, hitherto the reserved domain of wealthier universities in wealthier countries. Could ChatGPT help to flatten the disparities between the global south and wealthier nations in terms of research output? Maybe, now everyone can have access to such assistance, like the research-version of a dæmon from a Phillip Pullman novel following the researcher around and always available to offer pertinent advice.
Why is having Super RAs or e-Research Assistants so important? As ChatGPT failed to clearly note in its above answer, there’s good reason to believe that much of the “long-hanging fruit” of discovery and innovation has been picked. If so, we need to devote more researchers and resources to the effort. Or this: Making researchers more productive through new AI tools, from those that can aid drug discovery by predicting the shape of all human proteins to those that can predict the structure and properties of more than 31 million materials that don’t yet exist. Not only can AI like ChatGPT and its future iterations help us climb higher up the Tree of Knowledge, but it can find more trees in the orchard.
Micro Reads
▶ A.I. Turns Its Artistry to Creating New Human Proteins – Cade Metz, The New York Times | Using many of the same techniques that underpin DALL-E and other art generators, these scientists are generating blueprints for new proteins — tiny biological mechanisms that can change the way of our bodies behave. Our bodies naturally produce about 20,000 proteins, which handle everything from digesting food to moving oxygen through the bloodstream. Now, researchers are working to create proteins that are not found in nature, hoping to improve our ability to fight disease and do things that our bodies cannot on their own.
▶ Quantum computing is harder than herding kittens – John Thornhill, FT Opinion | Is quantum computing going to rewrite the rules of computation, cryptography, logistics and materials science, as enthusiastic supporters claim? Or are we heading into a “quantum winter”, where the diabolical difficulties of building a functional quantum computer lead to a collapse of confidence? That first question has re-emerged following the recent publication of a Chinese research paper outlining a theoretical way to crack the most common form of online encryption by combining existing quantum and classical computing techniques. If proved, this would be the stuff of security nightmares, hastening the arrival of the so-called Q-day, when users could “break the internet”.
▶ Regulatory rush job will be a disaster for advanced nuclear energy – Ted Nordhaus and Adam Stein, The Hill |
▶ Is Dynamism Doomed? – Brink Lindsey, The Permanent Problem |
▶ Electric Vehicles Keep Defying Almost Everyone’s Predictions – David Wallace-Wells, NYT |
▶ Has economics run out of big new ideas? – The Economist | In New Orleans economics’s biggest names offered ideas that were fresh and interesting, but hardly breakthroughs on the scale of, say, the Nash equilibrium or the idea of asymmetric information. Gita Gopinath, the imf’s chief economist, discussed research on how the economics of international finance has shifted since the seminal work of Robert Mundell and Marcus Fleming in the 1960s. In a seminar on economic growth, Thomas Philippon of New York University argued that growth follows linear trends, rather than being an exponential process. Daron Acemoglu of the Massachusetts Institute of Technology presented work on “distorted innovation”, arguing against the idea that markets tend to get innovation right.
▶ Exxon scientists in the 1970s accurately predicted climate change – Madeleine Cuff, NewScientist | Scientists working for Exxon between 1977 and 2003 accurately forecasted the rate at which global average temperatures would rise as a result of carbon emissions, correctly predicted that human-caused global warming would first be detectable by around 2000 and reasonably estimated how much carbon dioxide would lead to dangerous warming, according to the study. Leaked internal documents published in 2015 suggest Exxon, which became ExxonMobil in 1999, was aware of climate change in the 1970s and knew the threat could cause “dramatic environmental effects before the year 2050”. But this is the first time Exxon’s quantitative climate projections have been assessed. Geoffrey Supran at Harvard University and his colleagues analysed all publicly available internal documents and research publications disclosed by the company between 1977 and 2014 to assess the accuracy of Exxon’s scientific predictions against both contemporary models and subsequent real-world changes in temperature. The results were in line with scientific thinking at the time of writing, the team found. Meanwhile, 63 to 83 per cent of the projections were accurate in predicting subsequent rates of global warming.
▶ Last year marked the end of an era in spaceflight—here’s what we’re watching next – Eric Berger, Ars Technica |
▶ Embrace Dynamism: The Future and Its Enemies at 25 – Art Carden, AIER |
▶ Burning Man sues BLM over geothermal project – Jennifer Solis, Nevada Current |
▶ Science and Innovation in the 118th Congress – Matt Hourihan, Federation of American Scientists |
▶ Flying car, anyone? – John Gapper, FT | “A whole new era of aviation is about to take off. What looks like something out of a movie or a cartoon is really happening,” says Billy Nolen, acting head of the US Federal Aviation Administration. The aircraft could make short hops from airports to city centres, cross cities or link islands in Greece or south-east Asia. Lilium wants to go further, with a physical range of up to 250km. But flying prototypes is not the same as launching the real thing. Target dates have slipped and raising funding is harder as investors increasingly shun risk. The era of start-ups rushing ahead with little regard to regulation is over. Kittyhawk, a venture backed by Google’s co-founder Larry Page, shut in September, and Brian Foley, an aviation analyst, says time is getting short for other hopefuls: “For many, the flame of hope will flicker out.”
