吳振和：多模態與RAG碰撞AI新視界

1. In one sentence

Getting AI to reliably produce the advertising image you want is hard when a human writes the prompt, so his answer is to let a multimodal model describe good images itself — translating them into the detailed language AI understands — and then use RAG to retrieve those "good prompt structures" and apply them, so different image models all produce consistent results.

2. The session

The speaker and the company, checked afterwards

The host introduced him as VP of technology at cacaFly (聖洋科技) and head of its cloud AI business, having won Google Cloud Partner All Stars awards in the past two years for sales and for AI capability respectively. Checking the company afterwards — it is a Taiwanese digital media agency founded in 2009, working in social, creative planning, media integration, data analysis and trend forecasting. He said on stage himself that "cacaFly is an advertising industry company", consistent with the company site's positioning. His English name is CH Wu, and the formal title on cacaFly's blog is "Vice President, Cloud Intelligence Center, cacaFly 聖洋科技"; the same article renders that position in English as "VP of Technology" — which does line up with the host's "VP of technology", one being the full Chinese title and the other a direct English rendering, so both are kept here for comparison. On the Google Cloud Partner All Stars awards, no public award information or years could be found from either cacaFly or Google Cloud, so that is unverified.

Generating an ad image has more gates than you would think

He said producing a decent advertising image with current AI is genuinely hard, because there are several gates to pass. The first is the prompt: they looked at the literature and found that getting the right result out of multimedia generation takes roughly three times as many prompt attempts as a text-only large language model (an estimate he drew from the literature; the corresponding original paper could not be found, so it is recorded as his own account). A non-expert might go fifteen to thirty attempts before giving up — because they lack "the language for talking to a designer" and do not know how to make the AI understand the instruction; even an industry expert takes about five to ten (likewise the speaker's own account, no external source).

Audience exercise: the same image, wildly different prompts

He put an image on screen and gave the room three seconds to imagine "how would you prompt to generate exactly this image", then collated the prompts his colleagues actually wrote — young woman with black curly hair, light beige boots, sitting on a log in a forest. That description already sounds quite specific, but put into different image models it produced completely different scenes; and even the same prompt in the same model produced a different result each of two or three runs.

Where the problem is: humans fill in the gaps, AI does not

His analysis: describing an image well takes an enormous amount of detail, but human-to-human communication usually only manages "a good-looking guy", "Asian", "roughly this age", "doing this action" — the light, the detail, the expression, the setting, the pose never get said out loud, because in human-to-human communication the other person fills all of that in automatically. AI cannot yet fill in like that, so without a clear instruction it has nothing to go on, and that is the communication gap between people and AI. His direction for a fix is to bring "how humans describe things" and "the world as the AI understands it" closer together — guess what the AI does and does not understand, and fill in the difference.

Using a multimodal model as a translator

Their experiment: since today's multimodal models can see images (he noted Google, GPT and Claude had all just released new models with noticeably stronger multimodal capability), let the AI look at a well-composed image itself and write a detailed description, then use that AI-written description to generate a new image — so the human never has to rack their brain describing the picture. In effect it reduces "how do you describe this picture" to a purely textual structure problem. They tested with an example of Mario playing a video game in bed with Princess Peach beside him: the AI's description looked thorough, but the results across different image models were wildly different — some models did not recognise Mario, some could not work out where the character's feet should go, some simply generated two Marios to bluff their way through; one model outright refused to generate on policy grounds. Half-mocking himself, he said this proves language models hallucinate and image models evidently do too.

Using RAG to find the structure of a good prompt

Since "the description an AI writes from an image" does not itself reliably produce a good image either, their approach was to take the generated images, compare them against the original with mature image similarity techniques, tag the close ones one way and the distant ones another, and thereby filter out which prompt structures really do produce good images. Once that tag library accumulates, you can build a very simple text-based RAG (retrieval-augmented generation): the user writes a simple prompt, the system looks up which prompt structures describe a similar scene well, and then recombines what the user originally wanted with the retrieved reference structure and re-prompts — producing a good image reliably. They wired that flow up and tested it: the same AI-produced prompt across different image models gave consistent quality, validating the approach.

In practice: an advertising image generator

They packaged this flow into an advertising application: enter the product attributes, choose ethnicity and style, and the system uses a built-in base prompt to generate a representative image; to change it, a multimodal model can adjust from the original image — switching from cartoon style to photoreal, changing hair colour, adding an action. The creative flow runs roughly: produce a draft, generate a base image from the draft, work on the base image (switching to photoreal, say), then extend the same concept into new variations from that image.

The biggest commercial pain point: consistency of people and products

He named the real-world problem this flow hits commercially: the person generated is different every time, so you cannot have the same "person" recur across a series of assets; and if you need to lock onto a specific product, there is no guarantee the same item comes out each time. The traditional answer is to train a separate LoRA model to lock a style or object, but that requires model training skills, compute, and enough training data in the first place. He mentioned that Google had released Imagen 4) two days before this talk (the formal release date is 20 May 2025, at Google I/O 2025), but Imagen 4 did not yet have the feature he was about to discuss, so he was talking about the previous generation, Imagen 3.

Imagen 3 lets you add one to four reference images on top of the base model (Google officially calls this subject customization), with the reference subject split into people, products and animals (though as he noted himself, people are arguably animals too). Following Google's official guidance for a "multiple angles" reference prompt, he generated a composite creature he described as "unsettling the more you think about it", with features somewhat like his own; using Imagen 3's prompt enhancement to expand the prompt automatically produced something equally unrecognisable and equally alarming. But once he had spent the time to work out the right prompt patterns, he managed to lock both a product and a face at once, compositing two reference photos into a good new image and extending it into multiple angles and poses. He thinks this is particularly well suited to virtual humans: previously, once you had finally generated one good virtual-human photo with AI you could never reproduce a second, and this flow is what makes a virtual human actually viable.

Motion assets: Veo 2 takes over

The logic on video is similar: he mentioned Veo 3 had also just been released, but he would still be talking about Veo 2) (released December 2024, generating clips up to 8 seconds; Veo 3 arrived in May 2025, differing by adding synchronised audio generation). Their approach is to produce a base image they like, then ask Veo for a motion clip with a simple action prompt; but motion itself is hard to describe precisely in text — they are not directors and cannot articulate professional camera moves or blocking — so they also reference an existing clip with good motion (a hair-flick, say), feeding the base image plus that reference clip to generate the corresponding motion asset. He mentioned these motion assets are already running on outdoor billboards.

Closing: all that's left are the ideas we haven't said out loud

He closed the session on one line: if prompting can be made easier, then what is left to do is only "the ideas we haven't said out loud yet".

3. Figures and cases

• Multimedia generation takes about three times as many prompt attempts as a text-only large language model (his rendering of the literature; the corresponding original paper could not be found — speaker's own account, no external source).
• A non-expert takes about fifteen to thirty attempts before giving up; an industry expert about five to ten (speaker's own account, no external source).
• Imagen 3's subject customization takes one to four reference images across the people, product and animal categories (the Google Cloud Vertex AI documentation confirms the feature exists; the counts and categories are his on-stage account).
• Imagen 4 was released on 20 May 2025 (Google I/O 2025), two days before he spoke (Wikipedia Imagen entry)).
• Veo 2 was released in December 2024, generating clips up to 8 seconds; Veo 3 arrived in May 2025 adding synchronised audio (Wikipedia Veo entry)).
• cacaFly won Google Cloud Partner All Stars awards in the past two years for sales and AI capability (background given by the host on stage; no public award information found — unverified).

4. Lines worth keeping

• "I've decided to sacrifice my self-introduction and the company plug so everyone can finish on time."
• "I give the AI every detail it wants; I guess what the AI understands and doesn't understand, so my problem and its world get closer together."
• "I have no idea what image is even producible."
• "Prompt from this reference and it still won't come out right."
• "The more you think about it the creepier it gets — that face looks a bit like mine, and the longer I look the less comfortable I am."
• "If prompting can be made easier, all that's left is the ideas we haven't said out loud yet."

5. Tools and terms mentioned

• **RAG (retrieval-augmented generation)**: retrieving relevant material before handing it to a generative model as reference; he treats "good prompt structures" as the retrieved knowledge base and applies it to image generation prompts.
• **Imagen 3)**: Google's text-to-image model, the version carrying the subject customization (reference image) feature he demonstrated.
• **Imagen 4)**: Google's next-generation text-to-image model released on 20 May 2025, which did not yet have the reference image feature when he spoke.
• Subject customization: the Google Vertex AI documentation's formal name for the "upload a reference image to lock a person or product" feature.
• **Veo 2)**: Google DeepMind's text- and image-to-video model, released December 2024, generating clips up to 8 seconds.
• **Veo 3)**: the next generation of Veo, released May 2025, adding synchronised audio generation.
• **LoRA (low-rank adaptation)**: a low-cost fine-tuning technique; he mentioned that locking consistency for a specific style or object traditionally means training a LoRA model.
• Ghibli-style transfer: he mentioned this as an obvious demonstration of multimodal models (the image-style-transfer craze in the community at the time), but it was not the focus here.

6. Wider observations

The most elegant move in this session was redefining "the communication gap between people and AI" as a problem solvable by engineering: use a multimodal model's image-reading to translate a picture into the detailed language AI understands, use image similarity comparison to filter out which prompt structures actually work, and put a very basic text RAG on top — turning the whole thing from "try things by feel" into "look it up and assemble". That shares its logic with the earlier code-generation sessions: find out what the AI does not understand and level the gap systematically, rather than expecting the user to become a prompt engineering expert. His swipe at the official reference-image guidance — "prompt from this reference and it still won't come out right" — is refreshingly honest too; watching a demo, it is easy to assume that following the official tutorial makes everything work, when in fact there is a great deal of hard-won detail in between.

7. Sources