“Technology should serve people first.”

Research Agenda

From Using Software to Defining It: User-Defined Computing with AI

As AI agents become increasingly capable of generating interfaces, building applications, and interacting with digital environments, I've been thinking about a fundamental question: What does it mean for people to define their own computing experiences in the age of AI?

Traditionally, computing experiences have largely been defined by developers and service providers. Users interact with predefined interfaces, workflows, and functionalities. [1, 2, 3]

With advances in generative AI and agentic systems, this relationship is beginning to shift. People can increasingly participate in creating, modifying, and composing the computing environments they use. [4, 5, 6]

We conceptualize User-Defined Computing (UDC) as an approach to computing in which individuals exercise meaningful authority over the functionality, structure, and interaction of their computing experiences. Rather than merely consuming predefined applications or accepting system-driven personalization, users can specify, compose, evaluate, and revise computational capabilities according to their evolving goals and preferences. In AI-mediated environments, this authority need not require direct implementation; instead, users may delegate computational work to intelligent agents while retaining the ability to shape and renegotiate consequential design decisions.

For example, a shopper on Amazon could ask an AI agent to add a comparison panel and related YouTube reviews to a product page (what the environment can do), let the agent fill in specifications while they choose which products to compare (how it behaves), and hide sponsored listings that the platform would normally decide to show (who decides).

Three dimensions of

User-Defined Computing

  1. Dimension 1

    Functional Definition

    What can the computing environment do?

  2. Dimension 2

    Interaction Definition

    How should the computing environment behave?

    Division of labor between agentic system and user

  3. Dimension 3

    Decision Authority

    Who decides how the computing environment is defined?

    Agentic system, user, or service provider

    Agentic systemUserService provider

My research explores UDC through two interconnected directions. The first, human–AI co-creation, addresses functional and interaction definition: what people create with AI, and how the work is divided between them and the agentic system. The second, user-defined web experiences, addresses decision authority: who decides how a composed web experience is defined.

Direction 1

Human–AI Co-Creation

UDC dimensions
  • 1 · Functional Definition
  • 2 · Interaction Definition
Shift
Using tools → creating tools
Key questions
Intent · Process · Evaluation
Methods
  • User Studies (Experts & Non-Experts)
  • Participatory Research
Projects
  • Runtime Generative UI
  • “Meaningful Knowledges”: a non-technical user community
Contributions
  • Empirical Insights
  • Design Principles
  • Community Knowledge

Direction 2

User-Defined Web Experiences

UDC dimensions
  • 3 · Decision Authority
Shift
Fixed applications → composable environments
Key questions
Representation and Architecture · Control and Agency
Methods
  • Research through Design
  • Systems Building
  • Technical Probes
Projects
  • Jacaranda
  • Web Browsing Model–View–Controller (MVC)
Contributions
  • Composable Web Architecture
  • New Interaction Substrates

1. Human–AI Co-Creation: From Using Tools to Creating Tools

AI-assisted design and development are lowering the barriers to creating personalized software. [7] However, generating an artifact is only part of the challenge.

I'm particularly interested in three questions:

  • Intent: How do people communicate their goals, preferences, and constraints to AI systems? [8, 9]
  • Process: How can users and AI agents coordinate to preserve design intentions throughout iterative creation? [9, 10]
  • Evaluation: How do individuals develop and apply personal criteria when evaluating AI-generated artifacts? [11]

Through our ongoing research on Runtime Generative UI, we investigate how people with different levels of design expertise create and evaluate personalized interfaces with AI.

2. User-Defined Web Experiences: From Fixed Applications to Composable Environments

The web has traditionally been organized around applications and websites controlled by individual service providers.

But what if users could dynamically compose information, interfaces, and capabilities across services to create experiences that better reflect their needs? [12, 13, 14]

Through projects such as Jacaranda [15], I'm exploring two interconnected challenges:

  • Representation and Architecture: How can web capabilities and information be represented so that users, not only platforms, can decide how they are composed? [13, 14]
  • Control and Agency: How should users, agentic systems, and service providers share control over the composition of web experiences? [10, 16]

A Broader Research Vision

These two directions share a common theme: human agency in AI-mediated computing. [16, 17, 18]

As AI increasingly takes responsibility for generating and organizing computing experiences, an important challenge is ensuring that people retain meaningful authority over what gets created, how it evolves, and whether it meets their needs.

Perhaps the most interesting shift is that the boundary between using software and creating software is becoming increasingly blurred. [4, 7]

I'm excited to explore what this means for the future of Human–Computer Interaction—and how we might move toward computing environments that are not simply personalized for people, but meaningfully defined by them.

References

  1. [1]Allan MacLean, Kathleen Carter, Lennart Lövstrand, and Thomas Moran. 1990. User-Tailorable Systems: Pressing the Issues with Buttons. CHI ’90. DOI
  2. [2]Wendy E. Mackay. 1991. Triggers and Barriers to Customizing Software. CHI ’91. DOI
  3. [3]Amy J. Ko, Robin Abraham, Laura Beckwith, et al. 2011. The State of the Art in End-User Software Engineering. ACM Computing Surveys 43, 3. DOI
  4. [4]Gerhard Fischer, Elisa Giaccardi, Yunwen Ye, Alistair Sutcliffe, and Nikolay Mehandjiev. 2004. Meta-Design: A Manifesto for End-User Development. Communications of the ACM 47, 9. DOI
  5. [5]Priyan Vaithilingam, Elena L. Glassman, Jeevana Priya Inala, and Chenglong Wang. 2024. DynaVis: Dynamically Synthesized UI Widgets for Visualization Editing. CHI ’24. DOI
  6. [6]Yining Cao, Peiling Jiang, and Haijun Xia. 2025. Generative and Malleable User Interfaces with Generative and Evolving Task-Driven Data Model. CHI ’25. DOI
  7. [7]Xinqi Zhang, Hariharan Subramonyam, Advait Sarkar, et al. 2026. Generative Design and Vibe Coding: Rethinking the Design-Development Divide for UI Prototyping. CHI ’26 Extended Abstracts. DOI
  8. [8]J.D. Zamfirescu-Pereira, Richmond Y. Wong, Bjoern Hartmann, and Qian Yang. 2023. Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. CHI ’23. DOI
  9. [9]Hari Subramonyam, Roy Pea, Christopher Pondoc, Maneesh Agrawala, and Colleen Seifert. 2024. Bridging the Gulf of Envisioning: Cognitive Challenges in Prompt Based Interactions with LLMs. CHI ’24. DOI
  10. [10]Eric Horvitz. 1999. Principles of Mixed-Initiative User Interfaces. CHI ’99. DOI
  11. [11]Shreya Shankar, J.D. Zamfirescu-Pereira, Bjoern Hartmann, Aditya Parameswaran, and Ian Arawjo. 2024. Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences. UIST ’24. DOI
  12. [12]Michael Bolin, Matthew Webber, Philip Rha, Tom Wilson, and Robert C. Miller. 2005. Automation and Customization of Rendered Web Pages. UIST ’05. DOI
  13. [13]Clemens N. Klokmose, James R. Eagan, Siemen Baader, Wendy Mackay, and Michel Beaudouin-Lafon. 2015. Webstrates: Shareable Dynamic Media. UIST ’15. DOI
  14. [14]Geoffrey Litt, Daniel Jackson, Tyler Millis, and Jessica Quaye. 2020. End-User Software Customization by Direct Manipulation of Tabular Data. Onward! ’20. DOI
  15. [15]Xinqi Zhang, Mingyuan Zhong, Longjie Guo, Bryan Min, Yuanhao Zhang, and Kai Lukoff. 2026. Jacaranda: Grafting User-Editable Web Experiences Across Sites. UIST ’26 Adjunct. PDF
  16. [16]Jeffrey Heer. 2019. Agency plus Automation: Designing Artificial Intelligence into Interactive Systems. PNAS 116, 6. DOI
  17. [17]Dan Bennett, Oussama Metatla, Anne Roudaut, and Elisa D. Mekler. 2023. How does HCI Understand Human Agency and Autonomy? CHI ’23. DOI
  18. [18]Ben Shneiderman. 2020. Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy. International Journal of Human–Computer Interaction 36, 6. DOI

Last Updated: November 11, 2026