黃仕鎮：玉山銀行生成式AI服務的革新之路

1. In one sentence

E.SUN Bank chief engineer 黃仕鎮 walked through a full year of the bank's in-house generative AI chat platform GENIE — from poor early usage to 200% growth in monthly users and six-fold growth in usage volume after GENIE 2.0 launched in 2024 — and showed how purpose-built AI applications such as meeting minutes, OCR and financial news aggregation land inside a bank's compliance framework.

2. The session

The speaker and E.SUN's Intelligent Banking Division

黃仕鎮 is currently chief engineer at E.SUN Bank, in the Intelligent Banking Division; he holds a computer science doctorate from New York University, was a researcher at AT&T Bell Labs, teaches part-time at National Central University, and has held roles inside E.SUN including chief architect and director of data science (verified: the conference's official Facebook speaker introduction, consistent with his self-introduction on the slides). He jokes that he is "an IT grease monkey and an AI novice", with over a third of a century of development experience behind him.

E.SUN's organisational culture sets a goal every ten years: the last decade's (the third) goal was "the technological E.SUN", which is when the Intelligent Banking Division was founded; this year's (the fourth decade) goal is "E.SUN of Taiwan, E.SUN of the world", shifting weight toward overseas markets. The Intelligent Banking Division currently has an AI team of about a hundred people, building models and delivering solutions in-house (speaker's own account; later press coverage likewise mentions an Intelligent Banking Division of about a hundred people operating over two hundred and eighty models or process robots daily, which counts as external corroboration).

GENIE's painful early days

The story goes back to November 2022, when ChatGPT's arrival started E.SUN thinking about how a large language model could be used inside the bank. But banking is a licensed industry, and he put it plainly: "customers' data is our life, and we can't just throw our life outside the wall." So E.SUN could not simply wire in an external ChatGPT, and instead decided to build its own in-house conversational platform. The underlying models are still OpenAI's, but before data is sent out it passes through a DLP (data loss prevention) mechanism that masks or substitutes sensitive information. After about a year of preparation, GENIE launched formally in January 2024.

The early results were poor: thirty-seven days of statistics showed that the number of unique people who had used it was only 17.8% of everyone it was open to, of whom 4.1% had asked only one or two questions and 6.6% only three to ten; only 2.7% reached the intensity of "a question a day". Daily unique users were only 2.1% of everyone it was open to.

He took "why is nobody using it" apart dichotomously: first, used versus never used. Those who used it and stopped split into three: cannot use it, can use it but there is no function they want, and can use it but are unhappy with the output. Those who never used it split into: unaware the service exists, aware but do not know how to use it, and aware but with no desire to (often because a manager did not let their reports use it, since a monthly bill is charged to each department by usage — if the manager does not support it, nobody below naturally will). The detailed figures on the slide: those who used it but did not continue were 58.8% of unique users, and daily active users lost 40% from the very start.

The remedy, and GENIE 2.0

The fixes were direct: on features, add multiple channels (desktop, mobile) and multimodal input and output (PDF, text, image, voice, HTML, video), add a prompt template library so people who cannot write instructions can copy one, and strengthen training — especially for managers, because if the manager does not support it nobody below will use it. GENIE 2.0 launched in September 2024 with new features added progressively, and paired with promotional activity in each unit, monthly usage showed 20% weighted growth. By the time of this conference, compared with the 2024 generative AI conference, monthly users had grown 200% and actual usage volume 600% (six-fold). Both growth figures are the speaker's own account, with no external audit found.

System architecture and compliance constraints

GENIE is not entirely built in-house but integrates the most suitable SaaS components from various cloud providers: the default model runs on Azure OpenAI, web retrieval uses GCP's Grounding with Google Search, and internal knowledge integrates the bank's existing question-answering bot, with GCP Agent Builder, AWS Bedrock and Microsoft Copilot Studio all evaluated as candidate modular components. Given the nature of banking, E.SUN puts correctness and precision of AI-generated content ahead of the spirit of governance: build guardrails to strengthen personal data and information security risk management, strengthen knowledge retrieval to reduce hallucination, and make sure answers reflect E.SUN's service quality and professional image. On cost control, every call records its originating unit and spend for ceiling tracking — the idea of FinOps.

This treatment — data cannot leave the bank, correctness before showmanship — echoes the Guidelines for the Use of Artificial Intelligence in the Financial Industry published by the Financial Supervisory Commission in June 2024. Those guidelines set out six core principles (establish governance and accountability, value fairness and human-centredness, protect privacy and customer rights, ensure system robustness and security, deliver transparency and explainability, promote sustainable development), and explicitly require that where a financial institution uses third-party generative AI, its own staff must still make an objective professional risk judgement on the output — the same logic as 黃仕鎮's account of E.SUN insisting on building GENIE itself and spending an extra year to get DLP right.

Purpose-built AI: the smart meeting minutes service

The first case is the in-house smart meeting minutes service. He addressed the inevitable question first: plenty of meeting minutes tools already exist, so why build one? The answer comes back to the nature of banking — confidential personal and business information cannot leave, so it has to be built in-house.

Technically, E.SUN does not simply hand the transcript to a large language model to summarise; it first runs a "clean-up" pass, stripping filler and repeated words to produce a cleaned transcript that makes the subsequent summarisation both more efficient and more accurate. Headline generation uses chunking: a one- to two-hour meeting is split into segments, each summarised with a segment headline extracted, and then all the segment headlines are pooled for the large language model to assess which are genuinely the important topics. E.SUN did once consider going back to letting users write their own headlines, but observed that from GPT-4 onward the model's headline judgement improved markedly, so the automated approach stayed. Action item extraction detects sentence patterns like "please have so-and-so do such-and-such" and lists them automatically; accuracy here is relatively lower — he noted that even the audio of this very talk would have several action items picked out of it by the system.

The quantified indicators on the slide are interesting: summary length as a proportion of cleaned transcript length falls between 1.1% and 32%; action item length as a proportion of cleaned transcript length is 0.6% to 33%; cleaned transcript length as a proportion of the original is 7% to 20% (which you can read as the filler ratio); and speaking speed is estimated as meeting duration divided by transcript length, about 0.17 characters per second. These figures then get used to analyse each colleague's meeting habits: whose meetings have the most key points, who has the most action items, who uses the most filler, who talks fastest — and even a word cloud letting a manager see at a glance what their people spend their days in meetings about.

Purpose-built AI: smart OCR

E.SUN's OCR work has always targeted fixed formats (cheques, ID cards) with in-house trained models, reaching recognition accuracy above 99.9% (speaker's own account, no external audit found). The other category is unconstrained formats — invoices, electricity bills, financial statements, each one laid out differently. E.SUN's new approach has a large language model read the image directly: tell the model which fields to extract and it reads the answers straight out, producing a template-free OCR. The slide notes this cut the smart OCR service's development cycle to one month.

Purpose-built AI: financial news aggregation

E.SUN's finance department staff have to read fifty websites a day covering the policy and market movements of eight currencies and their central banks (the US dollar and the Federal Reserve, the euro and the ECB, sterling and the Bank of England, the yen and the Bank of Japan, the renminbi and the People's Bank of China, the Australian dollar and the RBA, the New Zealand dollar and the RBNZ, the Canadian dollar and the Bank of Canada), and assembling one report a day by hand is extremely time-consuming. E.SUN's approach is to structure each article into a JSON summary first, then consolidate into a single JSON, write it up as a news piece, translate it into Taiwan-standard Traditional Chinese, and keep an index of the originals for checking back. The slide explains why this hierarchical, multi-layer prompt design: a single prompt has an input length limit and weaker control over output format and content, whereas a hierarchical design gives relatively stable quality. It is also paired with a self-questioning mechanism (having the model check its own classification) and similarity matching to find the cited sources, as ways of filtering noise and reducing hallucination.

Looking ahead

黃仕鎮 observes that internal AI applications are moving from assistive copilots toward autonomous agentic patterns, and the audience is shifting from bank staff toward customers. The vision on the slide is a dual track of a "customer AI agent" and a "staff AI agent": on the customer side, proactive asset allocation advice and proactively detecting deposit levels to prompt fund movements; on the staff side, deepening internal document retrieval, meeting minutes, and form and report generation. The next step is expanding GENIE's use cases and driving agent integration built on the MCP (Model Context Protocol) framework, aiming for a secure, reliable and usable financial AI ecosystem.

3. Figures and cases

4. Lines worth keeping

• "Banking is a licensed industry. Customers' data is our life, and we can't just throw our life outside the wall."
• "We can basically trust the headline it writes itself; it won't be worse than the one we'd write."
• "From an agentic point of view it's really componentisation — how do we turn a lot of AI into individual agentic AIs and then integrate them."

5. Tools and terms mentioned

• GENIE, GENIE 2.0: E.SUN Bank's in-house generative AI conversational platform
• DLP (data loss prevention): the masking of sensitive information before anything is sent to an external model
• RAG (retrieval-augmented generation): the core architecture for internal knowledge retrieval
• Azure OpenAI, GCP Grounding with Google Search, GCP Agent Builder, AWS Bedrock, Microsoft Copilot Studio: the cloud service components GENIE integrates
• MCP (Model Context Protocol): the framework planned for future agent integration
• SPOCR / GPOCR: the slide's shorthand for purpose-specific OCR and general-purpose OCR
• FinOps: the concept of cost control and tracking for AI services
• Pydantic structures: the output format specification mentioned in the financial news structured summarisation prompt

6. Wider observations

The real point of interest is not the later growth figures but his willingness to lay out an opening as ugly as "17.8%". Most enterprise case sessions tell you only the success story; by first making the shape of the failure clear and then taking the root causes apart, his is far more useful as a reference — especially the line that if the manager does not support it nobody dares use it. It sounds like a small thing, but it identifies that enterprise AI adoption usually gets stuck on governance incentives rather than on technology.

The trade-off of a bank preferring to build rather than buy an off-the-shelf integrated service is worth noting too: most startups reason that if SaaS will do, do not rebuild the wheel, whereas every one of E.SUN's purpose-built applications (meeting minutes, OCR, financial news) is built in-house, always for the same reason: customer data is our life and cannot leave. That echoes the developer conference's first session by 林鉦育 (also of E.SUN's Intelligent Banking Division, currently senior vice president engineer) on rolling out GitHub Copilot internally. The two sessions fell on different days and faced different directions (one internally on engineer productivity, one internally on a knowledge assistant for all staff), but both point at the same solid groundwork E.SUN has laid on generative AI over the past year or two. How the two of them actually divide the work, and their relationship, is unclear, and no speculation is offered here.

7. Sources