The Economist July 10th 2021 71
Science & technology
Automatingprogramming
The software software engineers
G
pt-3 is quitea beast. The Generative
PreTrained Transformer 3, to give its
full name, is a language model developed
by Openai, a partcommercial, part not
forprofit artificialintelligence (ai) lab
oratory in San Francisco. gpt3 was trained
on an unprecedented mass of text to teach
it the probability that a given word will fol
low preceding words. When fed a short text
“prompt”, it cranks out astonishingly co
herent prose written in a similar style.
Access to gpt3 is restricted. For one
thing, says Jack Clark, former head of poli
cy at the organisation, it might otherwise
be used to mass produce fake news or flood
social media with “trolling and griefing”
messages. But Openai also knows that
gpt3 is commercially valuable. Last year
the laboratory started letting vetted firms
buy its output for approved uses. These in
clude producing answers to typed ques
tions about products, and powering the
speech of fictional characters in virtual
worlds. But perhaps most important, gpt3
can also be used to write computer code.
Several firms are already using gpt3
and its predecessor gpt2 to add aito the
software that their programmers use to
write code. Much of what these program
mers type out has already been written
elsewhere at some point in the past. This
means that by feeding oodles of preexist
ing code into such packages, they can be
trained to predict the lines a programmer
needs next. As a programmer types, poten
tial “code completions” of one or a few
lines pop up on the screen.
Predict and provide
One company that has created such an ai
completion feature is Tabnine, of Tel Aviv.
Tabnine used gpt2 to feed so much code
to its programming software, also named
Tabnine, that this software gained a sort of
“world knowledge”, says Eran Yahav, the
firm’s top technologist. Dr Yahav describes
this as “a pretty good notion of how the
world behaves”, at least when it comes to
programmingspeak. Tabnine software
may detect that a user has begun to type
code to handle, say, purchase orders. It will
then suggest code to display product
names and prices, as well as code to create
fields to be filled with quantities, payment
and delivery data. It works even though
Tabnine has never been specifically in
structed to do that.
Some coding sequences are rare. In
these cases, Tabnine lengthens its popup
list of suggested completions to increase
the likelihood of offering a useful one. By
clicking on one that is appropriate, the
programmer teaches Tabnine to perform
better. Tabnine’s professional version
seems “almost intelligent” in its ability to
understand a programmer’s intent, ac
cording to Dror Weiss, the firm’s boss.
Tabnine is not alone. On June 17th Mi
crosoft, an American software giant, re
leased a new version of an aicompletion
feature which it embeds in coding soft
ware called Visual Studio. The original ver
sion, released in 2018 and named Intelli
Code, was trained on a few thousand on
line repositories in which code for pro
gramming projects is stored. Microsoft
trained its upgraded system on more than
half a million such repositories. Amanda
Silver, one of the executives in charge of Vi
sual Studio, says these extra heaps of train
ing fodder allow the new version to glean
intent better from hints in code that a pro
Artificial intelligence is transforming the way computer programs are written
→Alsointhissection
72 Coppergushers
73 Theoriginofsongbirds
74 Ants,acaciasandbribery
74 How tea gets its flavour