WEBVTT

1
00:00:09.599 --> 00:00:13.400
Great to have you here.

2
00:00:15.280 --> 00:00:20.560
Hi everyone. I'm Chad Nelson. I'm a

3
00:00:16.960 --> 00:00:22.320
creative director here at OpenAI.

4
00:00:20.560 --> 00:00:24.240
Stop. Stop. Stop. We'll keep it casual,

5
00:00:22.320 --> 00:00:26.720
but no, this is going to be a fun talk.

6
00:00:24.240 --> 00:00:29.519
Uh so in my career I've made a lot of

7
00:00:26.720 --> 00:00:32.000
content, films, video games, television

8
00:00:29.519 --> 00:00:34.559
shows and in all my experiences there's

9
00:00:32.000 --> 00:00:36.079
been one common kind of through line

10
00:00:34.559 --> 00:00:38.000
which is how am I going to make it? How

11
00:00:36.079 --> 00:00:40.000
am I going to make this stuff ultimately

12
00:00:38.000 --> 00:00:41.520
like what are my creative workflows that

13
00:00:40.000 --> 00:00:44.160
I need? What are the tools I'm going to

14
00:00:41.520 --> 00:00:46.239
use? And so what happens you start a

15
00:00:44.160 --> 00:00:47.760
project

16
00:00:46.239 --> 00:00:49.200
and you basically look at the tool

17
00:00:47.760 --> 00:00:51.280
landscape that's out there and you say

18
00:00:49.200 --> 00:00:53.199
okay what are the tools I can use?

19
00:00:51.280 --> 00:00:55.520
Hopefully, they're not too generalized

20
00:00:53.199 --> 00:00:57.920
or or worse, too complicated for my

21
00:00:55.520 --> 00:00:59.680
project's needs. And it also doesn't

22
00:00:57.920 --> 00:01:02.239
help that in kind of the landscape

23
00:00:59.680 --> 00:01:03.840
today, these AI tools, I mean, literally

24
00:01:02.239 --> 00:01:05.360
new tools are coming out every single

25
00:01:03.840 --> 00:01:08.240
month.

26
00:01:05.360 --> 00:01:10.080
But kind of made us ask a question. What

27
00:01:08.240 --> 00:01:12.080
if you could actually develop a tool

28
00:01:10.080 --> 00:01:14.560
that was perfect for your individual

29
00:01:12.080 --> 00:01:17.040
project, your creative workflow, and not

30
00:01:14.560 --> 00:01:18.080
have it take months to develop? So

31
00:01:17.040 --> 00:01:20.479
that's something we've been thinking

32
00:01:18.080 --> 00:01:22.640
about here at OpenAI and we found a

33
00:01:20.479 --> 00:01:25.759
project that actually helped us test

34
00:01:22.640 --> 00:01:27.840
this theory. Now a few months ago or I

35
00:01:25.759 --> 00:01:30.159
should say actually it was two weeks ago

36
00:01:27.840 --> 00:01:32.000
uh Native Foreign and Vertigo Films out

37
00:01:30.159 --> 00:01:34.320
of the UK announced a feature film

38
00:01:32.000 --> 00:01:36.880
called Critters. It's an animated film

39
00:01:34.320 --> 00:01:39.280
and their team in London they basically

40
00:01:36.880 --> 00:01:41.200
wanted to power or partner their artists

41
00:01:39.280 --> 00:01:43.439
if you will with our latest open AI

42
00:01:41.200 --> 00:01:44.960
models. And what was interesting about

43
00:01:43.439 --> 00:01:46.240
this, they came to us and said, 'Well,

44
00:01:44.960 --> 00:01:48.799
we want everything to make sure that

45
00:01:46.240 --> 00:01:51.680
it's humanled. Like, we want our artists

46
00:01:48.799 --> 00:01:53.439
to be driving the AI tools itself and

47
00:01:51.680 --> 00:01:55.600
not have them kind of work the other way

48
00:01:53.439 --> 00:01:58.479
around, if you will. So, their goals

49
00:01:55.600 --> 00:02:00.240
were simple.

50
00:01:58.479 --> 00:02:02.479
Let's start with storyboarding. Now,

51
00:02:00.240 --> 00:02:05.119
storyboarding is this pro process where

52
00:02:02.479 --> 00:02:07.119
you sketch out each scene of the film to

53
00:02:05.119 --> 00:02:08.720
ultimately visualize the entire movie.

54
00:02:07.119 --> 00:02:09.840
And when we sat down with their team in

55
00:02:08.720 --> 00:02:11.280
the beginning, they said, "Well, this

56
00:02:09.840 --> 00:02:13.440
process can typically take about a

57
00:02:11.280 --> 00:02:15.599
year." Uh and and the one of the issues

58
00:02:13.440 --> 00:02:17.120
with that is it takes so long that you

59
00:02:15.599 --> 00:02:19.920
really only have a few chances or

60
00:02:17.120 --> 00:02:21.599
iterations to kind of get it right. And

61
00:02:19.920 --> 00:02:22.959
so we thought about that and we said hm

62
00:02:21.599 --> 00:02:26.000
that'd be interesting. What if we could

63
00:02:22.959 --> 00:02:27.920
design a tool for this very specific to

64
00:02:26.000 --> 00:02:31.840
storyboarding?

65
00:02:27.920 --> 00:02:34.640
So we took chat GPT5 and codeex and we

66
00:02:31.840 --> 00:02:38.000
said okay let's design a tool that was

67
00:02:34.640 --> 00:02:39.920
very very focused on storyboarding which

68
00:02:38.000 --> 00:02:41.360
would allow you go from sketches to

69
00:02:39.920 --> 00:02:42.879
something that's more of a highfidelity

70
00:02:41.360 --> 00:02:45.200
render and then potentially even to

71
00:02:42.879 --> 00:02:47.760
final frame and we internally dubbed

72
00:02:45.200 --> 00:02:49.519
this project storyboard and to

73
00:02:47.760 --> 00:02:52.660
demonstrate how it works I'd like to

74
00:02:49.519 --> 00:02:59.040
introduce to you my colleague Olivia

75
00:02:52.660 --> 00:03:01.519
[Applause]

76
00:02:59.040 --> 00:03:03.680
Hi everyone. My name is Olivia Morgan

77
00:03:01.519 --> 00:03:05.200
and I'm a solutions engineer here at

78
00:03:03.680 --> 00:03:08.319
OpenAI.

79
00:03:05.200 --> 00:03:10.879
Now, I want to use Critters as kind of

80
00:03:08.319 --> 00:03:13.760
the backdrop to show you this storyboard

81
00:03:10.879 --> 00:03:15.440
application that we built. And what I

82
00:03:13.760 --> 00:03:17.040
hope you all take away from this today

83
00:03:15.440 --> 00:03:19.360
is not just that we're able to use

84
00:03:17.040 --> 00:03:22.640
codecs to accelerate the build of a

85
00:03:19.360 --> 00:03:25.280
truly custom tool, but that by

86
00:03:22.640 --> 00:03:28.640
thoughtfully exposing the parameters in

87
00:03:25.280 --> 00:03:30.000
our image APIs and our video APIs, um

88
00:03:28.640 --> 00:03:31.840
you could really unlock a lot of

89
00:03:30.000 --> 00:03:34.080
creative freedom.

90
00:03:31.840 --> 00:03:35.360
So I'm going to show you a demo of

91
00:03:34.080 --> 00:03:37.440
storyboard and I'm going to use the

92
00:03:35.360 --> 00:03:40.319
persona of a storyboard artist, but you

93
00:03:37.440 --> 00:03:41.760
can imagine any creative field, right?

94
00:03:40.319 --> 00:03:44.239
somebody in design, somebody in

95
00:03:41.760 --> 00:03:46.400
marketing trying to generate an initial

96
00:03:44.239 --> 00:03:48.480
concept.

97
00:03:46.400 --> 00:03:51.440
So when I land on the storyboard page,

98
00:03:48.480 --> 00:03:53.040
you see this concept of projects. Now to

99
00:03:51.440 --> 00:03:56.000
start a new project, I'm going to go to

100
00:03:53.040 --> 00:03:59.280
the upper right hand corner and we'll

101
00:03:56.000 --> 00:04:00.640
call this project today dev day. And for

102
00:03:59.280 --> 00:04:03.040
the sake of the demo, I'm going to

103
00:04:00.640 --> 00:04:05.519
duplicate from an existing project. Now,

104
00:04:03.040 --> 00:04:07.840
all this is going to do is duplicate my

105
00:04:05.519 --> 00:04:09.439
project setup. And I'll explain more

106
00:04:07.840 --> 00:04:10.959
about the project setup in a little bit.

107
00:04:09.439 --> 00:04:12.959
Uh but that's essentially what's given

108
00:04:10.959 --> 00:04:14.799
me this population of characters,

109
00:04:12.959 --> 00:04:16.479
locations, and props. But you can kind

110
00:04:14.799 --> 00:04:18.720
of see the framework that we're starting

111
00:04:16.479 --> 00:04:22.400
with. On the lefth hand side, we bring

112
00:04:18.720 --> 00:04:25.199
in sk um some sketches and some prompts.

113
00:04:22.400 --> 00:04:27.520
Then we have a list of controls. Um now

114
00:04:25.199 --> 00:04:31.360
these are images that are fed into our

115
00:04:27.520 --> 00:04:33.360
edit endpoint on GPT image one. Um and

116
00:04:31.360 --> 00:04:36.080
helps us control uh the fidelity of

117
00:04:33.360 --> 00:04:37.280
those final outputs.

118
00:04:36.080 --> 00:04:39.440
Now you'll notice here that I can

119
00:04:37.280 --> 00:04:41.759
generate in two different presets. So I

120
00:04:39.440 --> 00:04:43.440
have kind of a turbo preset in the

121
00:04:41.759 --> 00:04:45.600
project setup. We have this set to

122
00:04:43.440 --> 00:04:47.680
generate in black and white sketches.

123
00:04:45.600 --> 00:04:49.600
And then we have a high fidelity preset.

124
00:04:47.680 --> 00:04:52.720
And so the this gives us really our

125
00:04:49.600 --> 00:04:56.000
final renderings if you will. So let's

126
00:04:52.720 --> 00:04:58.000
walk you through an example.

127
00:04:56.000 --> 00:05:01.600
So in the upper leftand corner I'm going

128
00:04:58.000 --> 00:05:03.680
to upload some frames and we'll go to

129
00:05:01.600 --> 00:05:07.639
our swimming critter, our volcano

130
00:05:03.680 --> 00:05:07.639
critter, and our waving critter.

131
00:05:08.400 --> 00:05:11.520
And you'll see when those hit the

132
00:05:10.240 --> 00:05:14.240
storyboard, they're here on the lefth

133
00:05:11.520 --> 00:05:16.639
hand side. Now, I have this critter

134
00:05:14.240 --> 00:05:18.560
swimming with a yellow fish here. And we

135
00:05:16.639 --> 00:05:20.320
need to add a prompt to describe the

136
00:05:18.560 --> 00:05:22.560
sketch. So, I'm going to jump into my

137
00:05:20.320 --> 00:05:26.560
notepad. And I'm just going to grab some

138
00:05:22.560 --> 00:05:28.160
of these prompts that I have.

139
00:05:26.560 --> 00:05:29.919
And we'll describe this one as a white

140
00:05:28.160 --> 00:05:31.759
furry critter swimming underwater as

141
00:05:29.919 --> 00:05:33.440
she's surrounded by bubbles floating up

142
00:05:31.759 --> 00:05:36.080
around her. A yellow and white fish with

143
00:05:33.440 --> 00:05:37.840
scales approaches.

144
00:05:36.080 --> 00:05:39.440
Now I want to select the controls,

145
00:05:37.840 --> 00:05:41.280
right? I want that fidelity in my final

146
00:05:39.440 --> 00:05:43.039
output of what I want my critter to look

147
00:05:41.280 --> 00:05:45.680
like. So here I have a list of

148
00:05:43.039 --> 00:05:47.520
characters. I'm going to select uh this

149
00:05:45.680 --> 00:05:49.120
critter actually for our character. But

150
00:05:47.520 --> 00:05:50.639
if I had a rendering where I wanted to

151
00:05:49.120 --> 00:05:53.840
use Grandpa Critter, I could select him,

152
00:05:50.639 --> 00:05:55.680
too. And then we move on to locations.

153
00:05:53.840 --> 00:05:57.440
So this critter is underwater. Um and

154
00:05:55.680 --> 00:06:00.240
this is just an an image we took with an

155
00:05:57.440 --> 00:06:01.840
iPhone. So, we'll select that underwater

156
00:06:00.240 --> 00:06:04.160
location, but we could also play with a

157
00:06:01.840 --> 00:06:07.759
desert or a forest location or bring

158
00:06:04.160 --> 00:06:09.440
more locations in in our project setup.

159
00:06:07.759 --> 00:06:10.720
Um, then we have this concept of props.

160
00:06:09.440 --> 00:06:12.160
So, if I wanted this critter to be

161
00:06:10.720 --> 00:06:14.080
wearing sunglasses, I'll show you this

162
00:06:12.160 --> 00:06:16.800
in a a later generation. We could bring

163
00:06:14.080 --> 00:06:18.720
that in, too. Now, let's kick off these

164
00:06:16.800 --> 00:06:20.400
generations,

165
00:06:18.720 --> 00:06:22.720
and I'll move on to these last two

166
00:06:20.400 --> 00:06:25.280
images. So, here we've got some critters

167
00:06:22.720 --> 00:06:27.680
in front of a volcano. And I'll grab

168
00:06:25.280 --> 00:06:30.080
that prompt

169
00:06:27.680 --> 00:06:31.840
to describe this sketch. We've got two

170
00:06:30.080 --> 00:06:34.240
furry critters looking at a distant

171
00:06:31.840 --> 00:06:35.840
volcano uh on the other side of a lake.

172
00:06:34.240 --> 00:06:39.120
There are berry bushes on the sides, and

173
00:06:35.840 --> 00:06:40.880
this is going to be early morning light.

174
00:06:39.120 --> 00:06:42.720
So, again, I'm going to use that initial

175
00:06:40.880 --> 00:06:44.639
critter. For the location, I don't need

176
00:06:42.720 --> 00:06:46.319
to bring in an asset. We described it in

177
00:06:44.639 --> 00:06:48.880
our prompt, and we'll kick off those

178
00:06:46.319 --> 00:06:50.720
generations as well. And then for this

179
00:06:48.880 --> 00:06:53.039
last guy, I'm actually going to show you

180
00:06:50.720 --> 00:06:56.039
an example of bringing in a prompt, a

181
00:06:53.039 --> 00:06:56.039
prop.

182
00:06:56.240 --> 00:07:02.800
And we'll describe this sketch as

183
00:07:00.800 --> 00:07:05.199
a white furry critter waving hello

184
00:07:02.800 --> 00:07:06.639
wearing sunglasses. So you'll see in the

185
00:07:05.199 --> 00:07:08.319
image he's not, but we'll bring in that

186
00:07:06.639 --> 00:07:10.400
prop for it. So here's our first

187
00:07:08.319 --> 00:07:12.400
critter, and we'll bring in those props,

188
00:07:10.400 --> 00:07:15.520
uh, those crystal sunglasses as a prop.

189
00:07:12.400 --> 00:07:17.680
So this will be what he's wearing in the

190
00:07:15.520 --> 00:07:20.000
final generation.

191
00:07:17.680 --> 00:07:21.840
All right. Now, before we look at these

192
00:07:20.000 --> 00:07:24.400
generations, I want to take you through

193
00:07:21.840 --> 00:07:26.240
the project setup.

194
00:07:24.400 --> 00:07:28.479
Um, so this is what was duplicated when

195
00:07:26.240 --> 00:07:29.759
I initially spun up the project. And

196
00:07:28.479 --> 00:07:31.599
when we were working with the creative

197
00:07:29.759 --> 00:07:33.039
team in London, they mentioned that this

198
00:07:31.599 --> 00:07:35.039
part of the image generation was really

199
00:07:33.039 --> 00:07:37.680
important, the ability to set an overall

200
00:07:35.039 --> 00:07:39.360
style in their storyboard. So, you'll

201
00:07:37.680 --> 00:07:41.199
notice up here at the top, we have that

202
00:07:39.360 --> 00:07:42.880
style guide where we can kind of set the

203
00:07:41.199 --> 00:07:45.199
aesthetic description that's applied to

204
00:07:42.880 --> 00:07:47.840
every generation. In our case, we want

205
00:07:45.199 --> 00:07:49.520
the sketch uh to uh we want to turn the

206
00:07:47.840 --> 00:07:51.599
sketch of a storyboard frame into a

207
00:07:49.520 --> 00:07:53.039
detailed scene. And we want to maintain

208
00:07:51.599 --> 00:07:55.120
the position and the composition of the

209
00:07:53.039 --> 00:07:57.680
initial sketch.

210
00:07:55.120 --> 00:07:59.599
Now, for the turbo generation prompt, uh

211
00:07:57.680 --> 00:08:01.759
we just want a storyboard style black

212
00:07:59.599 --> 00:08:04.080
and white marker and ink rendering, no

213
00:08:01.759 --> 00:08:06.479
color. And then for that highfidelity

214
00:08:04.080 --> 00:08:09.360
generation prompt, this is a full color

215
00:08:06.479 --> 00:08:11.599
highly realistic movie scene, 35mm

216
00:08:09.360 --> 00:08:16.479
photography.

217
00:08:11.599 --> 00:08:18.400
Now, speaking of controls or assets, um,

218
00:08:16.479 --> 00:08:21.199
down here at the bottom is where we can

219
00:08:18.400 --> 00:08:23.440
manage our locations, our characters,

220
00:08:21.199 --> 00:08:25.360
and our props.

221
00:08:23.440 --> 00:08:27.680
Now, for each image that we bring in, we

222
00:08:25.360 --> 00:08:29.039
can add a description. Um, for the

223
00:08:27.680 --> 00:08:31.520
characters, we actually added the

224
00:08:29.039 --> 00:08:33.279
ability for GPT5 to generate the

225
00:08:31.520 --> 00:08:34.560
description for us. We were kind of

226
00:08:33.279 --> 00:08:36.320
playing around with these controls. So,

227
00:08:34.560 --> 00:08:40.159
I'll show you an example. We'll bring in

228
00:08:36.320 --> 00:08:43.680
our yellow critter. And you'll notice uh

229
00:08:40.159 --> 00:08:46.000
very quickly it's fed into GPT5 and we

230
00:08:43.680 --> 00:08:47.279
get a short description, a small fluffy

231
00:08:46.000 --> 00:08:50.000
bright yellow critter with a round

232
00:08:47.279 --> 00:08:52.640
chubby body covered in software.

233
00:08:50.000 --> 00:08:54.800
Now again, these are all fed into the

234
00:08:52.640 --> 00:08:57.440
prompt for that final image generation.

235
00:08:54.800 --> 00:08:59.200
So both the text and those images. Um

236
00:08:57.440 --> 00:09:02.000
and this is important because that GPT

237
00:08:59.200 --> 00:09:03.839
image one model that we have has a um

238
00:09:02.000 --> 00:09:05.519
high input fidelity parameter that you

239
00:09:03.839 --> 00:09:08.959
can turn on and you'll kind of see how

240
00:09:05.519 --> 00:09:10.399
this comes to play here in a second.

241
00:09:08.959 --> 00:09:13.440
Now, the last thing I want to touch on

242
00:09:10.399 --> 00:09:15.040
is this history tab. Um, when working

243
00:09:13.440 --> 00:09:16.800
with the creative team in London, they

244
00:09:15.040 --> 00:09:18.320
mentioned that having this concept of a

245
00:09:16.800 --> 00:09:20.240
history tab was important. They wanted

246
00:09:18.320 --> 00:09:22.399
to be able to see the initial sketch,

247
00:09:20.240 --> 00:09:23.920
the generated images, whether or not we

248
00:09:22.399 --> 00:09:26.000
were referencing certain characters,

249
00:09:23.920 --> 00:09:27.920
props or locations, and then which

250
00:09:26.000 --> 00:09:30.080
prompt went with the image that was

251
00:09:27.920 --> 00:09:31.600
generated. And we gave them on the

252
00:09:30.080 --> 00:09:34.160
righthand corner the ability to export

253
00:09:31.600 --> 00:09:36.160
this all in a CSV so that they have full

254
00:09:34.160 --> 00:09:38.320
auditability on all of these storyboards

255
00:09:36.160 --> 00:09:40.560
that they're creating.

256
00:09:38.320 --> 00:09:42.720
Now, let's go back to the storyboard

257
00:09:40.560 --> 00:09:44.240
frame and see what was generated. So,

258
00:09:42.720 --> 00:09:46.240
you'll remember we had that initial

259
00:09:44.240 --> 00:09:47.920
sketch of the the critter underwater.

260
00:09:46.240 --> 00:09:50.560
And we've got our our black and white

261
00:09:47.920 --> 00:09:52.080
frames now. Um, so we have four

262
00:09:50.560 --> 00:09:53.519
different presets because when you're

263
00:09:52.080 --> 00:09:54.720
working with these image models, you

264
00:09:53.519 --> 00:09:56.399
know, there's a little bit of creative

265
00:09:54.720 --> 00:09:58.399
freedom that is to be had with these

266
00:09:56.399 --> 00:10:00.720
models. And so, it just gives the artist

267
00:09:58.399 --> 00:10:03.360
the ability to choose the best rendering

268
00:10:00.720 --> 00:10:06.000
that fits the vision. Um, I can confirm

269
00:10:03.360 --> 00:10:09.040
for download here in the uh bottom right

270
00:10:06.000 --> 00:10:10.240
hand corner. And um, I want to get some

271
00:10:09.040 --> 00:10:11.760
participation from the audience. I'm

272
00:10:10.240 --> 00:10:13.920
going to look at these last four and you

273
00:10:11.760 --> 00:10:16.160
cheer for me which one you like the

274
00:10:13.920 --> 00:10:21.160
best. Do we like this generation? We

275
00:10:16.160 --> 00:10:21.160
like this generation or this generation.

276
00:10:22.560 --> 00:10:27.519
>> First one. Okay.

277
00:10:25.519 --> 00:10:28.959
Okay. So, we got our our confirmed

278
00:10:27.519 --> 00:10:31.760
generations. I'll go through these last

279
00:10:28.959 --> 00:10:34.000
two just very quickly. Um, we now have

280
00:10:31.760 --> 00:10:36.560
also our critters in front of the

281
00:10:34.000 --> 00:10:37.920
volcano. So, if I can get a shout out

282
00:10:36.560 --> 00:10:40.160
which one we like best, I'll confirm

283
00:10:37.920 --> 00:10:42.880
that one for download.

284
00:10:40.160 --> 00:10:44.720
>> I heard two.

285
00:10:42.880 --> 00:10:46.560
Um, and then here at the bottom, we've

286
00:10:44.720 --> 00:10:48.880
got an example of those generations with

287
00:10:46.560 --> 00:10:50.160
the glasses. So, we're still waiting for

288
00:10:48.880 --> 00:10:52.320
the black and white sketch, but we got

289
00:10:50.160 --> 00:10:54.959
the full color generations. I kind of

290
00:10:52.320 --> 00:10:57.040
like either two or four, so we're going

291
00:10:54.959 --> 00:10:59.839
to go with number two. But why this is

292
00:10:57.040 --> 00:11:01.440
important is when you're working with uh

293
00:10:59.839 --> 00:11:02.880
you know storyboarding or you're

294
00:11:01.440 --> 00:11:04.000
generating initial sketches a lot of

295
00:11:02.880 --> 00:11:05.600
times in production you're not just

296
00:11:04.000 --> 00:11:07.680
working with one image you have hundreds

297
00:11:05.600 --> 00:11:09.440
of images and so the ability to do this

298
00:11:07.680 --> 00:11:11.680
in bulk stage it for download and then

299
00:11:09.440 --> 00:11:15.200
move it into your other creative tools

300
00:11:11.680 --> 00:11:17.920
is a really powerful concept.

301
00:11:15.200 --> 00:11:20.720
And when we were working with the

302
00:11:17.920 --> 00:11:23.440
creative uh team in London, what we

303
00:11:20.720 --> 00:11:25.519
found really fascinating is that um they

304
00:11:23.440 --> 00:11:28.160
started utilizing this storyboard

305
00:11:25.519 --> 00:11:30.000
concept for many different aspects

306
00:11:28.160 --> 00:11:33.360
including things like environmental

307
00:11:30.000 --> 00:11:35.200
design. So you'll see here uh the sketch

308
00:11:33.360 --> 00:11:37.440
of an initial environment and and

309
00:11:35.200 --> 00:11:39.920
storyboard's final rendering of it. uh

310
00:11:37.440 --> 00:11:41.440
but also things like character design

311
00:11:39.920 --> 00:11:42.880
where we're throwing an initial sketch

312
00:11:41.440 --> 00:11:44.560
of the character into storyboard and

313
00:11:42.880 --> 00:11:47.600
we're using storyboard to render out all

314
00:11:44.560 --> 00:11:50.480
of the final details. Um now this really

315
00:11:47.600 --> 00:11:52.480
opened our eyes to what a tool like this

316
00:11:50.480 --> 00:11:54.560
uh could be used for in uh you know

317
00:11:52.480 --> 00:11:57.600
larger enterprise applications. So

318
00:11:54.560 --> 00:11:59.519
imagine you are an athleisure brand and

319
00:11:57.600 --> 00:12:01.920
you're preparing for a seasonal launch

320
00:11:59.519 --> 00:12:04.399
or you're an automotive company and

321
00:12:01.920 --> 00:12:06.079
you're building a new campaign.

322
00:12:04.399 --> 00:12:07.920
Um and so that's really where storyboard

323
00:12:06.079 --> 00:12:10.880
comes in. turns these initial concepts

324
00:12:07.920 --> 00:12:12.480
into stunning production style shots so

325
00:12:10.880 --> 00:12:14.160
you can move fast and kind of stay

326
00:12:12.480 --> 00:12:18.079
confident that your first sketch to

327
00:12:14.160 --> 00:12:21.600
final frame is exactly how you want it.

328
00:12:18.079 --> 00:12:23.519
And with the new uh Sora 2 API, we are

329
00:12:21.600 --> 00:12:26.240
really excited because what this means

330
00:12:23.519 --> 00:12:30.160
now for our storyboard tool is we can go

331
00:12:26.240 --> 00:12:33.040
from initial sketch to a beautiful image

332
00:12:30.160 --> 00:12:34.560
now to full motion. Um, and I am so

333
00:12:33.040 --> 00:12:37.040
excited for you all to get your hands on

334
00:12:34.560 --> 00:12:39.360
this. I think what you'll find in Sora 2

335
00:12:37.040 --> 00:12:41.360
is that the motion understanding, the

336
00:12:39.360 --> 00:12:44.000
physics, the ability to guide the

337
00:12:41.360 --> 00:12:46.639
details through prompts and images um is

338
00:12:44.000 --> 00:12:49.279
is truly truly impressive. And so you

339
00:12:46.639 --> 00:12:53.079
can actually then go from visuals or

340
00:12:49.279 --> 00:12:53.079
images like this

341
00:12:53.760 --> 00:13:00.720
to uh full motion and sound like this.

342
00:12:58.320 --> 00:13:05.560
Hey there, shiny friend. You like to

343
00:13:00.720 --> 00:13:05.560
dance, huh? You're beautiful.

344
00:13:06.280 --> 00:13:12.720
[Applause]

345
00:13:10.639 --> 00:13:15.200
So, that's a little teaser of our Sora 2

346
00:13:12.720 --> 00:13:16.560
API. Um, I'm so excited for you all to

347
00:13:15.200 --> 00:13:19.360
get your hands on it. I'm certainly

348
00:13:16.560 --> 00:13:21.760
excited to continue to build this into

349
00:13:19.360 --> 00:13:23.680
storyboard. Um, and speaking of

350
00:13:21.760 --> 00:13:25.440
building, next I want to bring up my

351
00:13:23.680 --> 00:13:27.360
colleague Allison to actually walk you

352
00:13:25.440 --> 00:13:29.839
through how we built Storyboard. But

353
00:13:27.360 --> 00:13:32.320
before I bring her up here, um, Chad,

354
00:13:29.839 --> 00:13:34.800
are you still around?

355
00:13:32.320 --> 00:13:37.040
Um, what? I'm sorry.

356
00:13:34.800 --> 00:13:39.040
>> I wanted to bring you back up here. Um,

357
00:13:37.040 --> 00:13:40.480
do you think you could sketch something

358
00:13:39.040 --> 00:13:41.839
that we could actually throw into

359
00:13:40.480 --> 00:13:43.360
storyboard so we can actually show them

360
00:13:41.839 --> 00:13:45.279
how we've been working?

361
00:13:43.360 --> 00:13:47.360
>> Uh, I think you might have given me a

362
00:13:45.279 --> 00:13:47.600
challenge here. I need to get the iPad.

363
00:13:47.360 --> 00:13:49.120
>> Okay.

364
00:13:47.600 --> 00:13:49.360
>> Hold on a second. I'll be right back.

365
00:13:49.120 --> 00:13:51.519
Okay.

366
00:13:49.360 --> 00:13:53.200
>> But, uh, what do you want me to draw?

367
00:13:51.519 --> 00:13:54.639
>> So, I'm thinking we have the swimming

368
00:13:53.200 --> 00:13:56.320
critter. We got the volcanoes. What

369
00:13:54.639 --> 00:13:57.519
about the cave scene? Could you do like

370
00:13:56.320 --> 00:13:59.199
a critter in a cave?

371
00:13:57.519 --> 00:14:02.399
>> I got it. Let me get the iPad. Oh, I

372
00:13:59.199 --> 00:14:04.160
found it. Wow. Oh, look at that. Look at

373
00:14:02.399 --> 00:14:06.000
that. Okay. Hopefully this cord's long

374
00:14:04.160 --> 00:14:08.480
enough.

375
00:14:06.000 --> 00:14:10.079
>> Um, how many have like like two, three,

376
00:14:08.480 --> 00:14:11.680
cuz I'm not I'm just I'm a creative

377
00:14:10.079 --> 00:14:12.639
director, not a storyboard artist, but

378
00:14:11.680 --> 00:14:14.800
I'll I'll try.

379
00:14:12.639 --> 00:14:17.519
>> We'll give you a you got a few minutes.

380
00:14:14.800 --> 00:14:19.279
Um, so while he sketches, I want to

381
00:14:17.519 --> 00:14:23.399
bring up my colleague Allison to walk

382
00:14:19.279 --> 00:14:23.399
you through how we built storyboard.

383
00:14:23.620 --> 00:14:26.919
[Applause]

384
00:14:27.199 --> 00:14:31.680
>> Thanks, Olivia. Hi, everyone. I'm

385
00:14:29.680 --> 00:14:34.160
Allison August, a solutions engineering

386
00:14:31.680 --> 00:14:35.839
leader at OpenAI. Now, while Chad is

387
00:14:34.160 --> 00:14:37.680
going to be furiously drawing over

388
00:14:35.839 --> 00:14:40.320
there, I'm going to take a few minutes

389
00:14:37.680 --> 00:14:42.240
to explain how we built Storyboard in 48

390
00:14:40.320 --> 00:14:45.519
hours and how it's accelerated the

391
00:14:42.240 --> 00:14:47.360
design process for Critters.

392
00:14:45.519 --> 00:14:49.199
So, our team has worked with many

393
00:14:47.360 --> 00:14:51.279
different creatives across the industry,

394
00:14:49.199 --> 00:14:53.199
from some of the largest media companies

395
00:14:51.279 --> 00:14:55.680
to small teams who really need to move

396
00:14:53.199 --> 00:14:57.600
fast. And what we heard was clear. They

397
00:14:55.680 --> 00:15:00.399
want more control over the creative

398
00:14:57.600 --> 00:15:02.720
process with AI. Now, with a lot of

399
00:15:00.399 --> 00:15:04.800
tools today, you send a text prompt for

400
00:15:02.720 --> 00:15:06.480
an image and you get an output that can

401
00:15:04.800 --> 00:15:08.720
vary actually quite far from what you

402
00:15:06.480 --> 00:15:10.800
want.

403
00:15:08.720 --> 00:15:12.560
But creatives have a vision of how a

404
00:15:10.800 --> 00:15:14.160
shot should play out in their head. They

405
00:15:12.560 --> 00:15:16.079
want to be able to visualize the scene

406
00:15:14.160 --> 00:15:18.160
and the style to get an output that

407
00:15:16.079 --> 00:15:20.000
matches their concept rather than go

408
00:15:18.160 --> 00:15:21.519
straight to final generation. And in

409
00:15:20.000 --> 00:15:23.279
fact, Chad, that's your drawing there on

410
00:15:21.519 --> 00:15:26.720
the right, right? It's looking pretty

411
00:15:23.279 --> 00:15:29.040
good. judgment free zone.

412
00:15:26.720 --> 00:15:30.720
>> It looks great, Chad. Um, so with

413
00:15:29.040 --> 00:15:32.639
Storyboard, our goal was really to

414
00:15:30.720 --> 00:15:34.560
create a workflow that allows creatives

415
00:15:32.639 --> 00:15:36.639
to follow a process that gives them this

416
00:15:34.560 --> 00:15:39.920
level of control and fidelity to bring

417
00:15:36.639 --> 00:15:41.920
their light their shots to life with AI.

418
00:15:39.920 --> 00:15:44.320
So, at an internal company hackathon

419
00:15:41.920 --> 00:15:45.920
back in August, I worked with Olivia and

420
00:15:44.320 --> 00:15:47.920
three other solutions engineers to try

421
00:15:45.920 --> 00:15:49.519
to solve this problem. We had crossed

422
00:15:47.920 --> 00:15:51.199
paths with the critters team a few days

423
00:15:49.519 --> 00:15:52.959
before and thought we could work with

424
00:15:51.199 --> 00:15:55.199
them as a thought partner to ideulate

425
00:15:52.959 --> 00:15:57.519
towards a solution during this hackathon

426
00:15:55.199 --> 00:16:00.079
and we built a working prototype in just

427
00:15:57.519 --> 00:16:02.560
two days.

428
00:16:00.079 --> 00:16:06.480
So to build this MVP quickly we relied

429
00:16:02.560 --> 00:16:08.720
on codec cli and cursor powered by GPT5.

430
00:16:06.480 --> 00:16:11.120
We built the app in Nex.js JS with

431
00:16:08.720 --> 00:16:14.320
superbase as the back end which manages

432
00:16:11.120 --> 00:16:17.279
project state images and authentication

433
00:16:14.320 --> 00:16:19.600
and everything is deployed on Verscell.

434
00:16:17.279 --> 00:16:23.040
Now on Verscell we run functions that

435
00:16:19.600 --> 00:16:26.720
coordinate GPT5 image gen and now the

436
00:16:23.040 --> 00:16:29.519
new SOR 2 API and for every request we

437
00:16:26.720 --> 00:16:32.160
generate four image variations with GPT

438
00:16:29.519 --> 00:16:33.680
image one giving creatives the option to

439
00:16:32.160 --> 00:16:36.720
select the image that most closely

440
00:16:33.680 --> 00:16:39.440
matches their vision. Now, as Olivia

441
00:16:36.720 --> 00:16:41.120
mentioned a little earlier, GPT5 is also

442
00:16:39.440 --> 00:16:43.440
embedded throughout the application to

443
00:16:41.120 --> 00:16:45.759
improve our generated results from

444
00:16:43.440 --> 00:16:48.000
refining an artist sketch description to

445
00:16:45.759 --> 00:16:49.759
automatically generating rich character

446
00:16:48.000 --> 00:16:52.320
text that drives more detailed image

447
00:16:49.759 --> 00:16:53.839
outputs.

448
00:16:52.320 --> 00:16:56.240
So, let's take a peek at one of our

449
00:16:53.839 --> 00:16:57.759
original concepts for Storyboard. So, we

450
00:16:56.240 --> 00:17:00.240
started with an interface where you

451
00:16:57.759 --> 00:17:02.480
could upload one sketch at a time, add a

452
00:17:00.240 --> 00:17:05.039
prompt, and select a character, or in

453
00:17:02.480 --> 00:17:06.799
this case, a very cute little critter.

454
00:17:05.039 --> 00:17:09.039
It was pretty basic. There was no

455
00:17:06.799 --> 00:17:10.799
project setup, no generation history,

456
00:17:09.039 --> 00:17:12.720
and you couldn't even upload sketches in

457
00:17:10.799 --> 00:17:14.559
bulk.

458
00:17:12.720 --> 00:17:16.559
Now, as we worked, we realized that the

459
00:17:14.559 --> 00:17:18.640
concept of a project where you can bulk

460
00:17:16.559 --> 00:17:20.319
upload sketches from a scene and render

461
00:17:18.640 --> 00:17:22.480
different types of outputs based on a

462
00:17:20.319 --> 00:17:24.400
project level configuration would be so

463
00:17:22.480 --> 00:17:26.160
much more helpful for creatives. So,

464
00:17:24.400 --> 00:17:27.919
here's a mockup of what we actually sent

465
00:17:26.160 --> 00:17:29.679
over to the Critters team and looks a

466
00:17:27.919 --> 00:17:33.120
lot more similar to what you saw Olivia

467
00:17:29.679 --> 00:17:35.360
demo a little bit earlier.

468
00:17:33.120 --> 00:17:37.280
So when we sent this over, the Critters

469
00:17:35.360 --> 00:17:38.960
team started testing the app immediately

470
00:17:37.280 --> 00:17:41.919
and they had a lot of really valuable

471
00:17:38.960 --> 00:17:44.320
feedback from basic asks like adding

472
00:17:41.919 --> 00:17:46.080
file name visibility to more complex

473
00:17:44.320 --> 00:17:48.960
requests like adding the options to

474
00:17:46.080 --> 00:17:50.880
control inputs for props and locations.

475
00:17:48.960 --> 00:17:52.240
Now, initially Olivia and I hadn't

476
00:17:50.880 --> 00:17:54.400
planned to make any changes after our

477
00:17:52.240 --> 00:17:56.080
hackathon, but when we saw the requests

478
00:17:54.400 --> 00:17:58.160
and how valuable some of these changes

479
00:17:56.080 --> 00:18:00.240
could be for the team, we decided to

480
00:17:58.160 --> 00:18:04.799
test Codex, our coding agent, and the

481
00:18:00.240 --> 00:18:06.559
newest GPT5 codeex model.

482
00:18:04.799 --> 00:18:08.240
So, we would send tasks to Codeex in

483
00:18:06.559 --> 00:18:10.480
between meetings. We really easily

484
00:18:08.240 --> 00:18:12.160
reviewed and merged PRs into production,

485
00:18:10.480 --> 00:18:13.679
which Codex even allowed us to do from

486
00:18:12.160 --> 00:18:15.760
our phones. Right, Olivia?

487
00:18:13.679 --> 00:18:18.240
>> Yep. Now, right now you're seeing a

488
00:18:15.760 --> 00:18:19.840
screenshot of my actual codeex workspace

489
00:18:18.240 --> 00:18:22.080
and some of the various tasks that we

490
00:18:19.840 --> 00:18:24.799
shipped. We started with some quick wins

491
00:18:22.080 --> 00:18:26.640
like adding small elements to the UI,

492
00:18:24.799 --> 00:18:28.880
but soon found that it actually excelled

493
00:18:26.640 --> 00:18:30.960
with more complex tasks like compiling

494
00:18:28.880 --> 00:18:33.120
our generation prompt based on assets

495
00:18:30.960 --> 00:18:35.120
uploaded at both the project and the

496
00:18:33.120 --> 00:18:37.280
sketch level.

497
00:18:35.120 --> 00:18:38.799
Now, in my experience, once we built the

498
00:18:37.280 --> 00:18:40.480
foundation for the application and

499
00:18:38.799 --> 00:18:43.200
designed what we thought about the base

500
00:18:40.480 --> 00:18:44.960
UI, Codex was really incredible at

501
00:18:43.200 --> 00:18:46.320
adding the additional features and

502
00:18:44.960 --> 00:18:48.720
actually troubleshooting what we needed

503
00:18:46.320 --> 00:18:50.480
in record time. And every day, we

504
00:18:48.720 --> 00:18:52.240
completed nearly 10 feature requests

505
00:18:50.480 --> 00:18:54.240
from the Critters team, and they were

506
00:18:52.240 --> 00:18:56.880
simply blown away at how quickly we ship

507
00:18:54.240 --> 00:18:58.240
those improvements.

508
00:18:56.880 --> 00:18:59.440
All right, Chad, this is my last slide,

509
00:18:58.240 --> 00:18:59.760
so I hope you're wrapping it up over

510
00:18:59.440 --> 00:19:02.080
there.

511
00:18:59.760 --> 00:19:04.240
>> I'm Yes, I think we'll be ready.

512
00:19:02.080 --> 00:19:06.400
>> I'm sure it will be great.

513
00:19:04.240 --> 00:19:07.600
So to summarize on my piece here, we

514
00:19:06.400 --> 00:19:09.200
partnered really closely with the

515
00:19:07.600 --> 00:19:10.640
Critters team and stayed in a tight

516
00:19:09.200 --> 00:19:12.240
feedback loop, which meant that with

517
00:19:10.640 --> 00:19:14.000
help from Codeex, we were able to

518
00:19:12.240 --> 00:19:16.240
prototype faster than we ever have

519
00:19:14.000 --> 00:19:18.640
before and incorporate feedback on a

520
00:19:16.240 --> 00:19:20.400
daily basis. Typically, this kind of

521
00:19:18.640 --> 00:19:22.320
loop can truly take anywhere from one to

522
00:19:20.400 --> 00:19:25.440
two weeks. But with Codeex, we brought

523
00:19:22.320 --> 00:19:27.440
our ideation time truly down to a day.

524
00:19:25.440 --> 00:19:29.200
Now, the final storyboard tool helps the

525
00:19:27.440 --> 00:19:31.280
Critters team visualize thousands of

526
00:19:29.200 --> 00:19:33.600
sketches across different scenes in a

527
00:19:31.280 --> 00:19:35.520
matter of days. A process that normally

528
00:19:33.600 --> 00:19:37.760
would have taken months. All right,

529
00:19:35.520 --> 00:19:39.679
Chad, pencil's down. Can we take a look

530
00:19:37.760 --> 00:19:42.480
at what you drew?

531
00:19:39.679 --> 00:19:44.960
>> Yeah. So, okay.

532
00:19:42.480 --> 00:19:45.440
It's this little sketch here.

533
00:19:44.960 --> 00:19:46.960
>> Beautiful.

534
00:19:45.440 --> 00:19:49.120
>> Uh,

535
00:19:46.960 --> 00:19:51.520
you said a ca entering a cave. I gave it

536
00:19:49.120 --> 00:19:53.120
a little torch, like a little, you know,

537
00:19:51.520 --> 00:19:55.280
it's a dark cave.

538
00:19:53.120 --> 00:19:57.600
>> So, yeah. I I'm very curious to see what

539
00:19:55.280 --> 00:20:02.080
it did with it.

540
00:19:57.600 --> 00:20:04.559
>> All right, so I've got the generations.

541
00:20:02.080 --> 00:20:07.919
They just came in just in time. Um,

542
00:20:04.559 --> 00:20:09.840
we've got Chad's sketch. Uh, I gave it a

543
00:20:07.919 --> 00:20:11.919
little description, the front view of a

544
00:20:09.840 --> 00:20:14.720
furry critter who's entering a dark

545
00:20:11.919 --> 00:20:17.600
cave. We selected, you know, our critter

546
00:20:14.720 --> 00:20:20.559
character as that asset that we wanted

547
00:20:17.600 --> 00:20:22.480
to feed into our prop uh into our

548
00:20:20.559 --> 00:20:24.640
prompt. And then we've got our black and

549
00:20:22.480 --> 00:20:28.720
white generations of the adventurous

550
00:20:24.640 --> 00:20:30.320
critter and our final images here that

551
00:20:28.720 --> 00:20:32.240
we can stage to move into.

552
00:20:30.320 --> 00:20:34.720
>> I love the torch. The torch works really

553
00:20:32.240 --> 00:20:36.000
well. This is amazing. I actually really

554
00:20:34.720 --> 00:20:38.240
love the fact that I can just take a

555
00:20:36.000 --> 00:20:41.280
sketch like in barely what two minutes

556
00:20:38.240 --> 00:20:42.720
and get it um visualized this way. And I

557
00:20:41.280 --> 00:20:44.559
think that's what's been so amazing

558
00:20:42.720 --> 00:20:46.480
about seeing how their team in London is

559
00:20:44.559 --> 00:20:48.159
working with the tools. Uh, now we

560
00:20:46.480 --> 00:20:50.720
mentioned Sor 2 and I think we wanted to

561
00:20:48.159 --> 00:20:52.960
show I mean obviously I love the fish

562
00:20:50.720 --> 00:20:54.720
but I think Sor 2 made that audio. The

563
00:20:52.960 --> 00:20:56.080
Critters team in London put together a

564
00:20:54.720 --> 00:20:58.400
little demonstration. Now they've had

565
00:20:56.080 --> 00:21:00.559
access to the tool about a I don't know

566
00:20:58.400 --> 00:21:02.240
a few days u when we announced it last

567
00:21:00.559 --> 00:21:03.840
week. So let's take a look at something

568
00:21:02.240 --> 00:21:06.400
they put together to show us an example

569
00:21:03.840 --> 00:21:07.600
of what it can do there. Give it a quick

570
00:21:06.400 --> 00:21:10.799
comb. PERFECT.

571
00:21:07.600 --> 00:21:14.000
>> PICTURES UP. CAMERA STAND BY and you

572
00:21:10.799 --> 00:21:14.799
sorted. Good. Camera's rolling and ready

573
00:21:14.000 --> 00:21:16.559
on the big land.

574
00:21:14.799 --> 00:21:17.600
>> Have the antenna tidy. Thank you.

575
00:21:16.559 --> 00:21:19.039
>> All right, EVERYONE. SET

576
00:21:17.600 --> 00:21:23.200
>> ROLLING ACTION.

577
00:21:19.039 --> 00:21:26.600
>> HI THERE. Uh um what's my line again?

578
00:21:23.200 --> 00:21:26.600
Blast it.

579
00:21:26.640 --> 00:21:30.840
>> I love it. I love it.

580
00:21:32.080 --> 00:21:36.240
>> I think Sora too. I mean the visual

581
00:21:34.320 --> 00:21:38.159
quality is amazing. I love the sound. I

582
00:21:36.240 --> 00:21:40.559
mean all the voices and all the ambient

583
00:21:38.159 --> 00:21:42.159
and the music. It's amazing. But I think

584
00:21:40.559 --> 00:21:43.840
what's so exciting about this time is

585
00:21:42.159 --> 00:21:45.919
that we're seeing this period where

586
00:21:43.840 --> 00:21:47.919
artists and developers can now work more

587
00:21:45.919 --> 00:21:49.360
closely than ever before. And we've seen

588
00:21:47.919 --> 00:21:50.799
what happens when you take artists and

589
00:21:49.360 --> 00:21:52.480
couple them with AI. I mean that

590
00:21:50.799 --> 00:21:54.720
produces, you know, amazing work. But

591
00:21:52.480 --> 00:21:56.720
now how they're producing that work can

592
00:21:54.720 --> 00:21:59.440
be truly transformed with these custom

593
00:21:56.720 --> 00:22:01.760
tools. So I think Critters has become an

594
00:21:59.440 --> 00:22:03.200
ex exceptional example of this. Uh I

595
00:22:01.760 --> 00:22:05.120
can't wait to see what they build next

596
00:22:03.200 --> 00:22:06.880
with the tools. Uh but when I look out

597
00:22:05.120 --> 00:22:09.120
in this room, I see a whole room full of

598
00:22:06.880 --> 00:22:10.880
developers. I mean, what excites us is

599
00:22:09.120 --> 00:22:12.640
what you all will build. What creative

600
00:22:10.880 --> 00:22:14.559
tools might you bring to the industry

601
00:22:12.640 --> 00:22:18.080
and to artists to let them explore their

602
00:22:14.559 --> 00:22:20.880
imaginations? So, with that, I'll say on

603
00:22:18.080 --> 00:22:22.880
behalf of uh Olivia and Allison uh and

604
00:22:20.880 --> 00:22:25.039
myself, thank you so much for coming.

605
00:22:22.880 --> 00:22:28.200
Have a wonderful dev day. Thanks again.

606
00:22:25.039 --> 00:22:28.200
Thank you.


