陳俊毅：91APP AI-Driven Development

1. In one sentence

91APP built an AI collaboration pipeline — the product owner does knowledge extraction to produce a PRD, engineers produce specs and code following arc42, and QA generates tests from the PRD — which raised one scrum team's delivery efficiency by thirty per cent without the number of system incidents getting any worse.

2. The session

The speaker's background

Levi 陳俊毅 has been at 91APP for over eleven years and is currently a senior manager in the product development division, mainly responsible for the SRE, platform and DBA teams. 91APP itself is one of this conference's sponsors. He opened by making the split clear: internally 91APP's AI work has two halves — using AI to develop products and improve process and efficiency, which is what this session covers, and building AI products to help retail clients run their sites more efficiently, which he flagged would be covered next by a speaker he called Andrew. Cross-checking the agenda, the next speaker up was indeed 91APP chief architect 吳剛志, so the nickname is a small detail the programme does not show.

How 91APP's teams grew

91APP's product development organisation splits into product teams and a central business group: the product teams are split by domain into multiple scrum teams, each with a product owner and a tech lead; the central business group builds tooling and improves process so the product teams move faster; and when requirement priorities between scrum teams collide, it escalates to "One Team" for a final decision, keeping the whole division pointed the same way. They spent ten years grinding this into shape, growing from about five people to about three hundred (speaker's own account, no external source). The scaling logic of the old model was simple: to raise output, add another scrum team — efficiency scaled out.

Why the old model stalled in the AI era

Historically they would take a project, bring in one scrum team, have the product owner and tech lead draw the flow on a whiteboard, and most projects ran fast and smooth that way. But when they set out to bring AI in last year, it did not work — because all that domain knowledge lives in senior product owners' and tech leads' heads and is very hard to scale out. That pushed them, from Q4 2024, to look for "a development method for the AI era", with a clear goal: not using AI for the sake of using AI, but scaling *up* a single scrum team's efficiency — raising speed or quality at the same time. Efficiency is the goal; AI is only the means.

A three-stage strategy

The strategy splits into blocks: in the first stage, let the product owner, QA and engineering functions each make a breakthrough at a single point, training everyone to solve problems within their own function using AI tools; then use AI and process to connect those points into an integrated PRD → SPEC → CODE → TEST pipeline; and ultimately build an AI-driven development flow that keeps flowing.

The product owner end: from knowledge extraction to a PRD

The old problem was that product owners and engineers each wrote their own things in Slides and Sheets, with knowledge neither accumulating effectively nor connecting up. Afterwards, product owners started accumulating their domain knowledge continuously on Google Drive; when a new project starts, they run knowledge extraction through NotebookLM to produce a "domain brief" — a knowledge snapshot tailored to that project. This does not only accelerate the project itself; it is also useful for onboarding, since a new joiner hitting an unfamiliar domain question can use the same flow to get up to speed on the product's context. Then, taking the domain brief plus a PRD template plus project-specific input, the product owner collaborates with the AI and adjusts step by step into a PRD (product requirement document); once the PRD is done it is broken into user stories and handed to engineering.

He stressed that the two most critical blocks in the PRD template are "what changes" and "acceptance criteria" — the AI has to know exactly what to change, how to change it and how it will be accepted. The full PRD template has eight parts: 1. project description (background, goals, main features, launch schedule, expected results); 2. feature research; 3. project scope; 4. functional requirements; 5. technical architecture; 6. team and schedule; 7. appendix; 8. change log. He also noted that the PRD is not only for the AI — the team uses the same document as their shared language when communicating with each other.

The engineering end: from an estimation experiment to the SPEC principles

Before rolling out training, they had engineers estimate the same exercise (write a console version of rock-paper-scissors-lizard-Spock, with 100% unit test coverage, a RESTful API and API spec, complete code comments, and lint cleaned to a perfect score). Before training, about seventy per cent of engineers thought it would take four to six hours; after training, over forty per cent could finish within two hours and over seventy per cent within four. They analysed two root causes for the efficiency gap: whether the developer knows the language or domain (his example: an engineer unfamiliar with Rust was assigned to write the exercise in Rust, could not tell whether the AI's code was right or wrong, could not debug it, and ended up spending *more* time); and differences in prompt engineering ability.

Out of that, engineering developed a set of "SPEC principles" (itself a pun on RD in Chinese, proposed by 91APP iOS team lead Justin): Specific, the question must be unambiguous; Precise, the language must not be vague; Explain, supply background and reasoning; Clarify, list the constraints and requirements. He gave a full worked example for an add-to-cart API: Specific states you want a POST /api/cart/add RESTful API; Precise states the input/output JSON structures, validation logic (productId and skuId must exist in the database, quantity cannot be below 1 or exceed stock) and business logic (same product and SKU increments the quantity, a new product adds an item); Explain states that this API serves the "add to cart" button on the storefront product page, that the cart service uses a distributed cache architecture and that the work must operate on their Redis-based cart service's data structures; Clarify states that promotions and coupon logic are out of scope, that login authentication is not needed (assume userId is already available), that a fixed error format must be returned when product data is bad, that it must follow 91APP's current C# and .NET Core coding style, and points at a specific existing file as a structural template.

arc42: borrowing an open-source architecture document template

Working through the SPEC principles, they found an off-the-shelf open-source document template that happened to fit: arc42 — whose official template has twelve chapters describing what documentation a large software project should carry. They did not adopt all of it, picking the six they considered most necessary: 1. introduction and goals (describing the system's goals, mapping each phase's features to functional and non-functional requirements); 2. context and scope (using the C4 model's system context diagram for external integration at system level and a container diagram for how components inside the system connect); 3. building block view (Swagger API spec, DB schema); 4. runtime view (sequence diagrams showing how the system behaves for each user story, for example creating, updating, disabling and revoking an API key or token); 5. design decisions; 6. glossary. He joked that some engineers initially worried writing all this documentation would take longer, but the documents themselves are produced in collaboration with the AI — "beating magic with magic".

Three working principles in the implementation stage

He summarised several principles for working with AI (collected by 91APP payments senior manager Ken): give instructions step by step with a plan, not expecting to get there in one shot but thinking the flow through, or planning the order with the AI, before issuing them one at a time; state explicitly where to change and where the output lands (for example, "add an AddToCart method in CartService.cs taking productId and quantity"); stop as soon as it goes off track, since redoing it or fixing it by hand is usually more efficient than forcing corrections; and aim for eighty out of a hundred rather than a hundred, because AI is there to accelerate you rather than to finish perfectly, so consciously schedule time for review and refactoring.

From the sprint's point of view, once an engineer receives a story they run generate code → unit test → code review → debug/fix, producing output in repeated collaboration with the AI, and only then reach git commit/push and the pull request (flow diagram collected by 91APP VP Alan).

The QA end: feeding the PRD and API spec straight into test cases

On the QA side they found that feeding the PRD and API spec to the AI, together with a clear description of the requirement, largely lets it produce test cases directly — because the PRD already spells out acceptance rules and test scenarios, and the Swagger spec already supplies the complete data fields, types and structures. The AI can then take those test cases further and produce Cypress test code. His conclusion was blunt: the more complete the PRD and spec, the more accurate the AI's testing; write a vague PRD and the AI can only guess.

Challenges still open

He was candid about two unresolved layers. In depth, the PRD is currently written mainly by the product owner alone, when some of its content should really be defined jointly with engineering and QA — how that collaboration should work is still under discussion. In breadth, only about three to five pilot teams are running smoothly, while the whole division has twenty-odd scrum teams, and they plan to spread this knowledge gradually through templates and sharing sessions. He added that it is not only the product organisation using AI at 91APP — operations use it for creative assets, HR uses it for interviews, and even managers are relearning how to lead a new kind of AI-literate talent.

3. Figures and cases

• End-to-end delivery efficiency (from receiving a requirement through development and testing to launch) up at least 30% for the teams that adopted it (speaker's own account, no external source).
• The number of system incidents was flat year on year: the efficiency gain did not come with a quality drop (speaker's own account, no external source).
• The product development organisation grew from about 5 people to about 300 over ten years (speaker's own account, no external source).
• Engineering estimation before and after training: beforehand 16% estimated within 2 hours, 41% four hours, 33% six hours, 8% eight hours; afterwards 41% finished within 2 hours, 25% within 4, 8% within 6, and 20% needed no extra time at all (internal training data per the speaker, no external source; the spoken "seventy per cent at four to six hours" and "forty per cent within two hours" are rough renderings of these figures).
• The official arc42 template has 12 chapters, of which 91APP adopted 6 (the 12 chapters are verifiable in the arc42.org chapter list; the proportion adopted is the speaker's account).
• About 3 to 5 pilot scrum teams are running smoothly, with roughly 20-odd scrum teams across the division still to reach (speaker's own account, no external source).

4. Lines worth keeping

• "Efficiency is the goal; AI is the means."
• "All this knowledge lives in people's heads, and it's hard to scale."
• "How you communicate effectively with AI — it's a pun on RD, really."
• "AI gets to eighty out of a hundred; it's still people and AI collaborating, with people reviewing what the AI produced and confirming it's correct before anything ships."
• "You'll produce all these documents in collaboration with the AI — beating magic with magic."
• "If it's gone off track, stop; don't waste time forcing corrections."
• "AI has become standard issue for a 91APPer."

5. Tools and terms mentioned

• **91APP, Inc.** (ticker 6741): a company providing omnichannel solutions for brands and retail, built on AI and insight analytics — the speaker's employer.
• arc42: an open-source software architecture documentation template; the official version has 12 chapters, of which 91APP adopted 6 into its internal process.
• **C4 model**: a layered visual method for describing software architecture; two layers are used here, the system context diagram and the container diagram.
• **NotebookLM**: Google's AI note-taking and knowledge extraction tool, which 91APP uses to distil domain knowledge on Google Drive into a domain brief.
• PRD (product requirement document): 91APP's requirement specification, generated internally from the domain brief and shared between the AI and the team.
• **Swagger / OpenAPI spec**: the specification format describing an API's data fields, types and structures, used as the basis for arc42's building block view and for QA test generation.
• **Redis**: the distributed cache database behind 91APP's cart service; the SPEC example requires development against the Redis-based cart service.
• **Cypress**: the end-to-end test framework the AI targets after generating test cases from the PRD and API spec.
• The SPEC principles (Specific / Precise / Explain / Clarify): 91APP engineering's own four-word mnemonic for prompting AI, proposed by 91APP iOS team lead Justin.

6. Wider observations

The point most worth keeping is the description of the pain point: knowledge does not accumulate, it only lives in senior product owners' and engineers' heads. That is not an AI-era problem — it is the old problem every organisation hits at a certain size. AI merely magnified the cost of never having written the knowledge down: senior people passing it on by word of mouth used to be enough to keep things moving, but to hand the work to an AI, knowledge that is not written down is genuinely unusable. The arc42 section is especially pragmatic — rather than copying a methodology wholesale, they worked out what kind of input the AI needs (structured, verifiable specs that map to where the code lands) and then went back and picked the six chapters of an existing template that were just enough, instead of imposing all twelve for the sake of completeness. The SPEC principles come from the same pragmatism: no universal theory of prompt engineering, just letting the engineering team hit the potholes themselves and then distilling four memorable words.

91APP was also a sponsor of this conference, and three of its people spoke across the two days (李昆謀, 吳剛志 and 陳俊毅 here); the other two sessions have their own notes, so they are not repeated here.

7. Sources