WEBVTT

1
00:00:00.000 --> 00:00:02.550
Dominik Lukeš's lecture maps the large language models

2
00:00:02.670 --> 00:00:04.644
of autumn twenty twenty-six.

3
00:00:04.720 --> 00:00:07.720
It starts from one idea: all AI is

4
00:00:07.770 --> 00:00:10.084
a model plus orchestration.

5
00:00:10.134 --> 00:00:12.980
Without orchestration the models are useless, and without

6
00:00:13.030 --> 00:00:15.647
the models the orchestration is not very helpful.

7
00:00:15.700 --> 00:00:17.671
This session covers the models.

8
00:00:17.790 --> 00:00:19.558
Agents follow the week after.

9
00:00:19.680 --> 00:00:21.804
There are three kinds of model.

10
00:00:21.880 --> 00:00:24.960
Computational models are normal computing, built on

11
00:00:25.010 --> 00:00:27.700
string matching and rules like if this then

12
00:00:27.750 --> 00:00:28.281
that.

13
00:00:28.360 --> 00:00:31.510
Predictive models estimate fit, trained to

14
00:00:31.560 --> 00:00:34.255
match predefined features, which is data science.

15
00:00:34.305 --> 00:00:37.530
Semantic models understand content and learn their

16
00:00:37.580 --> 00:00:38.642
own features.

17
00:00:38.760 --> 00:00:40.690
These are the large language models.

18
00:00:40.810 --> 00:00:43.728
What large language models add is meaning.

19
00:00:43.780 --> 00:00:46.850
Older models needed labelled training data, and were

20
00:00:46.900 --> 00:00:48.360
precise but brittle.

21
00:00:48.480 --> 00:00:51.780
Large language models take instructions in plain language

22
00:00:51.830 --> 00:00:53.883
and handle messy input.

23
00:00:53.960 --> 00:00:56.200
They are less exact, so their output has

24
00:00:56.250 --> 00:00:58.251
to be checked, as a person's is.

25
00:00:58.370 --> 00:01:00.910
Take the sentence Paul Schuster was born in

26
00:01:00.960 --> 00:01:01.793
Dresden.

27
00:01:01.870 --> 00:01:04.260
Capturing its meaning in symbols or a graph

28
00:01:04.310 --> 00:01:05.918
took years of effort.

29
00:01:06.030 --> 00:01:08.500
A large language model can answer where he

30
00:01:08.550 --> 00:01:11.240
was born even when the text never says

31
00:01:11.290 --> 00:01:11.593
it.

32
00:01:11.670 --> 00:01:13.220
It can list the facts that can be

33
00:01:13.270 --> 00:01:16.402
inferred, or extract a knowledge graph.

34
00:01:16.520 --> 00:01:19.039
Semantics reaches other models too.

35
00:01:19.120 --> 00:01:21.950
An ordinary text-to-speech voice reads a script's

36
00:01:22.000 --> 00:01:25.339
names, scene setting and stage directions out loud.

37
00:01:25.460 --> 00:01:28.530
Gemini three point eight Flash text-to-speech recognises them

38
00:01:28.580 --> 00:01:29.720
and leaves them out.

39
00:01:29.800 --> 00:01:32.870
An open model, Qwen's text-to-speech, still reads some

40
00:01:32.920 --> 00:01:33.602
of them aloud.

41
00:01:33.720 --> 00:01:37.021
A customer's email shows the three kinds together.

42
00:01:37.071 --> 00:01:39.760
A computational model rejects the messy input, and

43
00:01:39.810 --> 00:01:41.900
a predictive model gives only a class and

44
00:01:41.950 --> 00:01:44.002
a probability, with no reason.

45
00:01:44.120 --> 00:01:46.890
A language model extracts the details because it

46
00:01:46.940 --> 00:01:48.366
understands how they fit.

47
00:01:48.480 --> 00:01:51.400
Combined, it extracts, a computational model

48
00:01:51.450 --> 00:01:54.382
calculates, and it then explains the result.

49
00:01:54.500 --> 00:01:57.190
Language models offer knowledge, cognition,

50
00:01:57.240 --> 00:02:00.320
perception, deliberation, conversation, code

51
00:02:00.370 --> 00:02:02.419
generation and tool use.

52
00:02:02.500 --> 00:02:05.190
Perception means pictures, audio and

53
00:02:05.240 --> 00:02:06.031
documents.

54
00:02:06.081 --> 00:02:08.770
Deliberation is reasoning through a hard problem in

55
00:02:08.820 --> 00:02:09.818
many steps.

56
00:02:09.940 --> 00:02:12.250
Deliberation and tool use are the two most

57
00:02:12.300 --> 00:02:14.613
important advances of the last year or two.

58
00:02:14.730 --> 00:02:16.889
Two training phases make a model.

59
00:02:16.940 --> 00:02:19.480
Pre-training gives general knowledge and skills such as

60
00:02:19.530 --> 00:02:22.600
language, from a curated web crawl, books, news,

61
00:02:22.650 --> 00:02:25.880
academic papers and Wikipedia, in trillions

62
00:02:25.930 --> 00:02:27.785
of words, twenty or more.

63
00:02:27.900 --> 00:02:31.200
Post-training gives the chat: holding a conversation, following

64
00:02:31.250 --> 00:02:33.870
instructions, using tools and showing a

65
00:02:33.920 --> 00:02:35.137
personality.

66
00:02:35.220 --> 00:02:37.990
Its data is selected examples of

67
00:02:38.040 --> 00:02:41.468
formats, expert-made instructions and human feedback.

68
00:02:41.590 --> 00:02:43.821
Llama 3.2 shows the difference.

69
00:02:43.871 --> 00:02:46.110
Asked the capital of the Czech Republic in

70
00:02:46.160 --> 00:02:49.230
one word, the pre-trained version only continues the

71
00:02:49.280 --> 00:02:52.186
question, and the post-trained one says Prague.

72
00:02:52.300 --> 00:02:55.380
Reasoning came with o1, previewed in September

73
00:02:55.430 --> 00:02:56.566
twenty twenty-four.

74
00:02:56.650 --> 00:03:00.210
Post-training for it produced large jumps over GPT-4o

75
00:03:00.910 --> 00:03:04.061
in mathematics, scientific reasoning and hard code.

76
00:03:04.180 --> 00:03:07.330
On the Artificial Analysis scale, ChatGPT scored

77
00:03:07.380 --> 00:03:09.930
about a hundred and twenty-five in November twenty

78
00:03:09.980 --> 00:03:10.498
twenty-two.

79
00:03:10.580 --> 00:03:12.550
The largest models now score about three hundred

80
00:03:12.600 --> 00:03:13.115
and fifty.

81
00:03:13.230 --> 00:03:15.700
Each lab makes a family of models, from

82
00:03:15.750 --> 00:03:18.287
larger and dearer to smaller and cheaper.

83
00:03:18.340 --> 00:03:21.790
Anthropic's tiers are Haiku, Sonnet, Opus

84
00:03:21.840 --> 00:03:22.890
and Fable.

85
00:03:22.990 --> 00:03:25.910
OpenAI's are Luna, Terra, Sol and

86
00:03:25.960 --> 00:03:26.663
Astra.

87
00:03:26.780 --> 00:03:30.080
Google's are Gemini four Carbon, Gemini four Argon,

88
00:03:30.130 --> 00:03:32.630
Gemini three point eight Flash and Gemini three

89
00:03:32.680 --> 00:03:34.251
point five Flash-Lite.

90
00:03:34.370 --> 00:03:36.980
The tiers sit on different

91
00:03:37.030 --> 00:03:38.051
generations.

92
00:03:38.101 --> 00:03:40.710
Haiku, Sonnet and Opus are on five point

93
00:03:40.760 --> 00:03:43.279
five while Fable is on five point one.

94
00:03:43.360 --> 00:03:45.590
Terra and Sol are on six point one

95
00:03:45.650 --> 00:03:48.003
while Luna and Astra are on six.

96
00:03:48.080 --> 00:03:51.299
Opus and Sonnet sometimes outperform Fable.

97
00:03:51.420 --> 00:03:54.640
Every model runs at several reasoning levels, from

98
00:03:54.690 --> 00:03:55.831
low to max.

99
00:03:55.910 --> 00:03:59.100
More reasoning usually scores higher: Opus goes from

100
00:03:59.150 --> 00:04:01.300
thirty-four point five at low to fifty-eight point

101
00:04:01.350 --> 00:04:03.790
four at max, at four dollars per million

102
00:04:03.840 --> 00:04:04.853
input tokens.

103
00:04:04.903 --> 00:04:07.290
At max a model can overthink and score

104
00:04:07.340 --> 00:04:08.486
slightly worse.

105
00:04:08.600 --> 00:04:12.060
GPT-6 Luna and Claude 5.5 Haiku

106
00:04:12.110 --> 00:04:14.420
are the cheapest, at twenty cents per million

107
00:04:14.470 --> 00:04:14.990
tokens.

108
00:04:15.110 --> 00:04:17.910
He sorts model makers into rings.

109
00:04:18.000 --> 00:04:21.150
In the United States the core is OpenAI

110
00:04:21.200 --> 00:04:23.440
and Anthropic, which began around twenty twenty as

111
00:04:23.490 --> 00:04:26.457
a splinter group of former OpenAI employees.

112
00:04:26.570 --> 00:04:28.890
Google, at the edge of the core, started

113
00:04:28.940 --> 00:04:31.930
the revolution with the twenty seventeen paper Attention

114
00:04:31.980 --> 00:04:34.600
Is All You Need, but jumps around in

115
00:04:34.650 --> 00:04:35.662
what it releases.

116
00:04:35.780 --> 00:04:39.000
Meta and xAI form the second ring, close

117
00:04:39.060 --> 00:04:41.215
to the frontier without the top models.

118
00:04:41.265 --> 00:04:43.730
The third ring holds smaller labs with specialised

119
00:04:43.780 --> 00:04:46.700
or open models, such as ElevenLabs for voice

120
00:04:46.750 --> 00:04:48.493
and Midjourney for images.

121
00:04:48.610 --> 00:04:51.214
Mistral is Europe's lab in the second ring.

122
00:04:51.330 --> 00:04:53.338
China complicates the picture.

123
00:04:53.390 --> 00:04:56.310
DeepSeek, Alibaba, Moonshot AI with Kimi,

124
00:04:56.360 --> 00:04:59.360
Z.ai with GLM, MiniMax, Xiaomi and

125
00:04:59.410 --> 00:05:02.250
ByteDance compete for the core, with no clear

126
00:05:02.300 --> 00:05:04.962
winner, and most of their models are open.

127
00:05:05.080 --> 00:05:07.310
By country, the United States has gone furthest,

128
00:05:07.370 --> 00:05:08.374
with China second.

129
00:05:08.490 --> 00:05:11.720
Proprietary models come only as an API

130
00:05:11.850 --> 00:05:12.784
or product.

131
00:05:12.834 --> 00:05:15.760
Open-weights models are published, and a

132
00:05:15.810 --> 00:05:18.500
useful one needs a server costing hundreds of

133
00:05:18.550 --> 00:05:20.374
thousands or millions.

134
00:05:20.450 --> 00:05:23.600
Local models are smaller open-weights models that run

135
00:05:23.650 --> 00:05:25.670
on your own computer, such as Gemma and

136
00:05:25.720 --> 00:05:26.321
Qwen.

137
00:05:26.440 --> 00:05:28.670
Open models run four to six months behind

138
00:05:28.720 --> 00:05:31.190
the frontier, but beat the previous generation of

139
00:05:31.240 --> 00:05:32.275
proprietary models.

140
00:05:32.390 --> 00:05:35.470
Labs publish model cards describing what a

141
00:05:35.520 --> 00:05:37.980
model is for and how it was evaluated.

142
00:05:38.030 --> 00:05:39.880
They come from the lab at launch, with

143
00:05:39.940 --> 00:05:41.651
vendor-run evaluations.

144
00:05:41.770 --> 00:05:44.770
Standard benchmarks allow comparison but

145
00:05:44.820 --> 00:05:46.213
saturate quickly.

146
00:05:46.263 --> 00:05:49.110
MMLU took thirty months to reach about ninety

147
00:05:49.160 --> 00:05:52.230
percent, and GPQA Diamond twenty-four months to

148
00:05:52.280 --> 00:05:54.171
reach about ninety-six.

149
00:05:54.290 --> 00:05:57.360
Some labs use public benchmarks as a training

150
00:05:57.410 --> 00:06:00.330
signal, so models do better on benchmarks than

151
00:06:00.380 --> 00:06:01.446
in real use.

152
00:06:01.530 --> 00:06:03.893
This is called benchmaxing.

153
00:06:04.010 --> 00:06:06.930
When Artificial Analysis released version five of its

154
00:06:06.980 --> 00:06:10.280
index, the Chinese open models fell a lot,

155
00:06:10.330 --> 00:06:13.710
while the leading American models held or improved.

156
00:06:13.830 --> 00:06:15.605
Intelligence is one measure.

157
00:06:15.655 --> 00:06:18.880
Artificial Analysis also tracks speed, latency and

158
00:06:18.930 --> 00:06:22.010
price: Haiku is very cheap, and Astra and

159
00:06:22.060 --> 00:06:23.754
Fable are very expensive.

160
00:06:23.870 --> 00:06:26.260
Benchmarks run by individuals fill in the picture,

161
00:06:26.310 --> 00:06:28.850
and so do reviewers on YouTube, in blogs

162
00:06:28.900 --> 00:06:30.289
and in newsletters.

163
00:06:30.410 --> 00:06:32.720
Most of the big models take pictures as

164
00:06:32.770 --> 00:06:35.380
well as text, and Gemini 4, Qwen and

165
00:06:35.440 --> 00:06:37.543
Nova take audio and video too.

166
00:06:37.660 --> 00:06:39.590
Voice can chain three models or use one

167
00:06:39.640 --> 00:06:42.260
end to end, as OpenAI's older Voice Mode

168
00:06:42.310 --> 00:06:43.994
and GPT-4o show.

169
00:06:44.110 --> 00:06:47.110
Claude 5.5 offers neither audio in nor speech

170
00:06:47.160 --> 00:06:47.472
out.

171
00:06:47.590 --> 00:06:50.120
Image models are now semantic.

172
00:06:50.180 --> 00:06:53.250
OpenAI's GPT Image 2.5 leads,

173
00:06:53.300 --> 00:06:55.031
ahead of Google's Nano Banana.

174
00:06:55.150 --> 00:06:58.680
ElevenLabs leads text-to-speech, scored in arenas

175
00:06:58.730 --> 00:07:01.314
where users compare models side by side.

176
00:07:01.430 --> 00:07:03.138
Suno leads in music.

177
00:07:03.250 --> 00:07:06.180
His short version: bigger models are better,

178
00:07:06.230 --> 00:07:08.840
reasoning models are better than non-reasoning ones, and

179
00:07:08.890 --> 00:07:11.788
semantic models are better than non-semantic ones.

180
00:07:11.900 --> 00:07:14.600
The sweet spot for reasoning is medium to

181
00:07:14.650 --> 00:07:15.177
high.

182
00:07:15.260 --> 00:07:17.800
If a small model fails, he suggests trying

183
00:07:17.850 --> 00:07:19.400
a bigger one or a model from a

184
00:07:19.450 --> 00:07:20.490
different lab.