Research Agenda
From Using Software to Defining It: User-Defined Computing with AI
As AI agents become increasingly capable of generating interfaces, building applications, and interacting with digital environments, I've been thinking about a fundamental question: What does it mean for people to define their own computing experiences in the age of AI?
Traditionally, computing experiences have largely been defined by developers and service providers. Users interact with predefined interfaces, workflows, and functionalities. [1, 2, 3]
With advances in generative AI and agentic systems, this relationship is beginning to shift. People can increasingly participate in creating, modifying, and composing the computing environments they use. [4, 5, 6]
We conceptualize User-Defined Computing (UDC) as an approach to computing in which individuals exercise meaningful authority over the functionality, structure, and interaction of their computing experiences. Rather than merely consuming predefined applications or accepting system-driven personalization, users can specify, compose, evaluate, and revise computational capabilities according to their evolving goals and preferences. In AI-mediated environments, this authority need not require direct implementation; instead, users may delegate computational work to intelligent agents while retaining the ability to shape and renegotiate consequential design decisions.
For example, a shopper on Amazon could ask an AI agent to add a comparison panel and related YouTube reviews to a product page (what the environment can do), let the agent fill in specifications while they choose which products to compare (how it behaves), and hide sponsored listings that the platform would normally decide to show (who decides).
Three dimensions of
User-Defined Computing
Dimension 1
Functional Definition
What can the computing environment do?
Dimension 2
Interaction Definition
How should the computing environment behave?
Division of labor between agentic system and user
Dimension 3
Decision Authority
Who decides how the computing environment is defined?
Agentic system, user, or service provider
Agentic systemUserService provider
My research explores UDC through two interconnected directions. The first, human–AI co-creation, addresses functional and interaction definition: what people create with AI, and how the work is divided between them and the agentic system. The second, user-defined web experiences, addresses decision authority: who decides how a composed web experience is defined.
| Aspect | Direction 1Human–AI Co-Creation | Direction 2User-Defined Web Experiences |
|---|---|---|
| UDC dimensions |
|
|
| Shift | Using tools → creating tools | Fixed applications → composable environments |
| Key questions | Intent · Process · Evaluation | Representation and Architecture · Control and Agency |
| Methods |
|
|
| Projects |
|
|
| Contributions |
|
|
Direction 1
Human–AI Co-Creation
- UDC dimensions
- 1 · Functional Definition
- 2 · Interaction Definition
- Shift
- Using tools → creating tools
- Key questions
- Intent · Process · Evaluation
- Methods
- User Studies (Experts & Non-Experts)
- Participatory Research
- Projects
- Runtime Generative UI
- “Meaningful Knowledges”: a non-technical user community
- Contributions
- Empirical Insights
- Design Principles
- Community Knowledge
Direction 2
User-Defined Web Experiences
- UDC dimensions
- 3 · Decision Authority
- Shift
- Fixed applications → composable environments
- Key questions
- Representation and Architecture · Control and Agency
- Methods
- Research through Design
- Systems Building
- Technical Probes
- Projects
- Jacaranda
- Web Browsing Model–View–Controller (MVC)
- Contributions
- Composable Web Architecture
- New Interaction Substrates
1. Human–AI Co-Creation: From Using Tools to Creating Tools
AI-assisted design and development are lowering the barriers to creating personalized software. [7] However, generating an artifact is only part of the challenge.
I'm particularly interested in three questions:
- Intent: How do people communicate their goals, preferences, and constraints to AI systems? [8, 9]
- Process: How can users and AI agents coordinate to preserve design intentions throughout iterative creation? [9, 10]
- Evaluation: How do individuals develop and apply personal criteria when evaluating AI-generated artifacts? [11]
Through our ongoing research on Runtime Generative UI, we investigate how people with different levels of design expertise create and evaluate personalized interfaces with AI.
2. User-Defined Web Experiences: From Fixed Applications to Composable Environments
The web has traditionally been organized around applications and websites controlled by individual service providers.
But what if users could dynamically compose information, interfaces, and capabilities across services to create experiences that better reflect their needs? [12, 13, 14]
Through projects such as Jacaranda [15], I'm exploring two interconnected challenges:
- Representation and Architecture: How can web capabilities and information be represented so that users, not only platforms, can decide how they are composed? [13, 14]
- Control and Agency: How should users, agentic systems, and service providers share control over the composition of web experiences? [10, 16]
A Broader Research Vision
These two directions share a common theme: human agency in AI-mediated computing. [16, 17, 18]
As AI increasingly takes responsibility for generating and organizing computing experiences, an important challenge is ensuring that people retain meaningful authority over what gets created, how it evolves, and whether it meets their needs.
Perhaps the most interesting shift is that the boundary between using software and creating software is becoming increasingly blurred. [4, 7]
I'm excited to explore what this means for the future of Human–Computer Interaction—and how we might move toward computing environments that are not simply personalized for people, but meaningfully defined by them.
References
- [1]Allan MacLean, Kathleen Carter, Lennart Lövstrand, and Thomas Moran. 1990. User-Tailorable Systems: Pressing the Issues with Buttons. CHI ’90. DOI
- [2]Wendy E. Mackay. 1991. Triggers and Barriers to Customizing Software. CHI ’91. DOI
- [3]Amy J. Ko, Robin Abraham, Laura Beckwith, et al. 2011. The State of the Art in End-User Software Engineering. ACM Computing Surveys 43, 3. DOI
- [4]Gerhard Fischer, Elisa Giaccardi, Yunwen Ye, Alistair Sutcliffe, and Nikolay Mehandjiev. 2004. Meta-Design: A Manifesto for End-User Development. Communications of the ACM 47, 9. DOI
- [5]Priyan Vaithilingam, Elena L. Glassman, Jeevana Priya Inala, and Chenglong Wang. 2024. DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing. CHI ’24. DOI
- [6]Yining Cao, Peiling Jiang, and Haijun Xia. 2025. Generative and Malleable User Interfaces with Generative and Evolving Task-Driven Data Model. CHI ’25. DOI
- [7]Xinqi Zhang, Hariharan Subramonyam, Advait Sarkar, et al. 2026. Generative Design and Vibe Coding: Rethinking the Design-Development Divide for UI Prototyping. CHI ’26 Extended Abstracts. DOI
- [8]J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. CHI ’23. DOI
- [9]Hari Subramonyam, Roy Pea, Christopher Pondoc, Maneesh Agrawala, and Colleen Seifert. 2024. Bridging the Gulf of Envisioning: Cognitive Challenges in Prompt Based Interactions with LLMs. CHI ’24. DOI
- [10]Eric Horvitz. 1999. Principles of Mixed-Initiative User Interfaces. CHI ’99. DOI
- [11]Shreya Shankar, J.D. Zamfirescu-Pereira, Bjoern Hartmann, Aditya Parameswaran, and Ian Arawjo. 2024. Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences. UIST ’24. DOI
- [12]Michael Bolin, Matthew Webber, Philip Rha, Tom Wilson, and Robert C. Miller. 2005. Automation and Customization of Rendered Web Pages. UIST ’05. DOI
- [13]Clemens N. Klokmose, James R. Eagan, Siemen Baader, Wendy Mackay, and Michel Beaudouin-Lafon. 2015. Webstrates: Shareable Dynamic Media. UIST ’15. DOI
- [14]Geoffrey Litt, Daniel Jackson, Tyler Millis, and Jessica Quaye. 2020. End-User Software Customization by Direct Manipulation of Tabular Data. Onward! ’20. DOI
- [15]Xinqi Zhang, Mingyuan Zhong, Longjie Guo, Bryan Min, Yuanhao Zhang, and Kai Lukoff. 2026. Jacaranda: Grafting User-Editable Web Experiences Across Sites. UIST ’26 Adjunct. PDF
- [16]Jeffrey Heer. 2019. Agency plus Automation: Designing Artificial Intelligence into Interactive Systems. PNAS 116, 6. DOI
- [17]Dan Bennett, Oussama Metatla, Anne Roudaut, and Elisa D. Mekler. 2023. How does HCI Understand Human Agency and Autonomy? CHI ’23. DOI
- [18]Ben Shneiderman. 2020. Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy. International Journal of Human–Computer Interaction 36, 6. DOI