The Intersection of AI and UX: Best Practices and Future Directions

From coding assistants to creative design tools, AI has found its way into applications across Silicon Valley and beyond, fundamentally altering what we expect from the software we use. Products like Microsoft Copilot have leveraged this technology to streamline and enhance coding work and Adobe's suite has integrated AI to assist in creative tasks, offering unprecedented power to designers and artists.

However, the integration of AI in product design is not without its challenges. While some products have successfully harnessed its power to fundamentally change user experience, others have struggled, often reducing their generative AI features to a gimmick, a Clippy-style chat toy, rather than a substantive, useful tool.

I believe the mixed utilization of AI in product design is largely because it challenges the conventional standards that product designers have long adhered to. Traditionally, app design has been about clean, crisp navigation and well-defined components, guiding users efficiently towards the data and tools they need to achieve their tasks. Generative AI, however, introduces a more exploratory dimension for users. It has the capability to transform applications, not just the artsy ones, into playgrounds for creativity, where users are encouraged to experiment to achieve their tasks. This shift from linear, structured interactions to more open-ended, exploratory ones requires a rethinking of traditional UX principles.

With generative AI, users are presented with multiple potential outcomes and pathways in real-time, changing the dynamic from user-driven commands to AI-assisted discovery and co-creation. It's a shift from a user telling an app what to do, to the app itself opening a dialog with the user - suggesting and iterating best ways to achieve some task. All of this while remaining a completely optional feature and maintaining some sort of traditional experience in the background.

This new paradigm necessitates a different approach to UX design — one that prioritizes more flexibility, anticipates multiple avenues of user intent, and seamlessly incorporates AI-generated options into the user flow. It's about creating interfaces that can handle the spontaneity and variety of outputs that generative AI offers, making them intuitive and engaging without overwhelming the user or leading them down wrong pathways. In some ways, proper generative AI UX also means creating a digital friend or coworker — a persona that the user trusts and respects.

In this article, my aim is to pinpoint several examples of experiences that exemplify outstanding generative AI integration in product design, dissecting what they do well in an attempt to understand what makes them so successful. Alongside these analyses, I will also present a vision of the future, exploring the exciting directions this design evolution could lead us in the years to come.

What is in an AI-powered app anyways?

There is a distinct divide between 1) generative AI for creative applications, 2) generative AI for business-oriented applications, and 3) applications for building generative AI experiences themselves. I believe that distinction is crucial, particularly in understanding the complexity and implementation of their respective user experiences.

Creative generative AI applications often feature simplistic and intuitive user interfaces, such as a single text box that can generate elaborate text, visuals, or videos. The user inputs a prompt, and the AI responds with a creative output. This simplicity hides the sophisticated AI working behind the scenes, and from a user’s perspective the interaction is straightforward and direct — a single interaction is required to get amazing results. These creative applications also natively promote experimentation with “undo”/“redo” functionality and ways to mask the smell of AI, making them a great playground for generative AI.

On the other hand, integrating AI into business applications presents us with a considerably more complex challenge. Here, the AI isn’t just generating creative content based on a simple prompt; instead, it’s tasked with understanding and navigating a vast array of business processes, workflows, and user data points. It also needs a deep understanding of user nuance to be able to semantically interpret their queries to determine the best course of action. The integration of generative AI in business apps involves exposing potentially hundreds of pathways to an AI bot, essentially transforming an entire enterprise API surface into an interactive interface that users can query. This requires a deep intertwining of AI with the core business processes and an intuitive UI that can properly convey the complexity of the underlying tasks without overwhelming the user.

For instance, in a financial application, AI might be used to analyze and interpret complex financial data, automate customer service interactions, or streamline complicated supply chain decisions. Each of these functionalities requires a deep and intelligent integration of multiple types of AI that can interpret and interact with these business-specific data and workflows. This level of integration poses unique design challenges. The UX must be sophisticated enough to offer comprehensive functionality, yet intuitive enough for users to engage without requiring deep technical knowledge. It also needs to be cognizant of the fact that “undo”/”redo” are not native interactions in these environments, so potentially destructive actions must be first approved by the user. The challenge lies in creating an interface that provides powerful AI-driven insights and actions while still being accessible, user-friendly and non-destructive.

Finally, applications designed for building generative AI experiences are primarily tailored to AI developers. These platforms offer the essential tools and frameworks needed for the creation and refinement of AI models, prompts, agents, and various other components. They are characterized by their complex interfaces, which provide detailed controls for model training, testing, deployment, or workflow creation. The UX flavor in these platforms is distinctly technical, catering to developers who require comprehensive access to AI functionalities and parameters. Ironically, these tools are often less flashy, a series of forms or a workflow editor-type design, and their teams often have their own internal asks to develop AI-related features to improve their own UX.

Curious about these distinct areas, I decided to take a look at some of the platforms on the market to compare and contrast them from a UX perspective. I hope that by taking a look at them from a thousand foot view, we can potentially glean some insights about what works and what doesn’t in the current landscape of AI-powered tools.

Case studies

1) Text prediction & generation apps

• Subjects: (Copilot, Miro, Notion, and of course ChatGPT)
• Strengths:
  • Ease of use: These are entirely text-based systems which makes the barrier to entry extremely low.
  • Creative partner: Sometimes the generated result isn’t exactly what is expected, but its often not bad either. This creates a nice, back-and-forth feedback loop where you can iterate along with the AI to create something truly unique.
  • Facilitates user growth and proficiency: The real trick is getting these AIs to truly understand what we’re aiming for. Sometimes, it feels like they need a bit more context to get it right. Providing that context can be a skill in its own right, allowing the user to “master” the AI in the way they would any other tool.
• Weaknesses:
  • So much text!: Sometimes text is not the most appropriate way to convey ideas. Graphs, diagrams, illustrations, and images would greatly help to clarify complex ideas.
  • Overdelivers: I still find some joy in seeing ChatGPT send me a detailed, 8-paragraph essay, but that joy is quickly fading. For some tasks, a rapid fire conversation is more appropriate, but it can be difficult to get the LLM to respond to you in that manner.

The strengths of Copilot and ChatGPT lie in their ability to lower the barrier for creative writing/research, fostering user creativity and skill development. However, the user experience can be overwhelmed by the sheer volume and complexity of text generated, indicating a need for more intuitive and adaptive interaction models.

Going forward, these platforms might benefit from integrating visual elements like graphs, forms, images, cards, and more in order to complement text-based outputs. This evolution will ensure that these powerful tools not only continue to unlock new creative and productive possibilities but also align more closely with user preferences and contexts.

2) Image generation and art apps

• Subjects: Adobe Firefly, Midjourney, DALL-E
• Strengths:
  • I can’t draw to save my life: Frankly these tools are probably the only way that I could ever generate a decent, visually-appealing illustration. It “unlocks” a skill for a large percentage of the population that simply cannot create visual art.
  • Iteration: The process of iteration is built into these platforms in unique and powerful ways. You can pinpoint specific areas that need adjusting and tackle a design iteratively, plus the concept of undo/redo is built natively into the platform, allowing the user to “try on” different outputs from the AI during the iteration process
• Weaknesses:
  • Copyright considerations: Even more so than their text-based brethren, image generation apps have a terrible habit of spitting out blatantly copyrighted content. Adobe is doing a lot of work here in order to alleviate some of these concerns, but its still quite the challenge.

These apps represent a significant leap forward in democratizing artistic creation, allowing users who lack traditional drawing skills to produce visually stunning work. The iterative nature of these tools encourages experimentation and refinement, fostering an environment where creativity is limited only by imagination, not technical ability.

However, the rise of these image generation apps also brings forth complex ethical challenges, particularly around the use of copyrighted content. As these platforms evolve, there is a pressing need for more robust mechanisms to respect intellectual property while continuing to empower users with creative freedom. Addressing these ethical considerations will be key to ensuring that these powerful tools not only foster creativity but also adhere to the principles of responsible and respectful use of AI technology.

3) Apps which integrate generative AI for complex business processes

• Subjects: Salesforce Einstein, Atlassian Suite, Schneider Electric Resource Advisor, Tableau Pulse
• Strengths:
  • Business intelligence: These applications aim to tackle the challenging task of bringing huge software platforms into the AI age.
  • Multi-AI: Even before the rise of generative AI, products like Einstein were already strong AI features that allowed users to summarize email content and manage complexity of sales. Adding a generative AI context on top of all that just made their features that much stronger.
• Weaknesses:
  • Data privacy: Ensuring AI recommendations align with business objectives and maintaining data privacy are critical considerations. Apps in this space have a strong need to run models on-premise or via private models like Azure GPT in order to eliminate the possibility that public models are trained on confidential data.
  • Feature-by-feature: Instead of a holistic solution that covers the entire app, there is a tendency for products to “dip their feet” into AI at low-hanging fruit entry points. For example, (as of today and only from my personal research), Salesforce uses generative AI for tasks like email template curation, rather than a full, Copilot-like experience for text fields throughout their app. This greatly hinders the amount of potential impact that the technology could have on the product.

These apps attempt to chisel away at the task of integrating generative AI into core business processes. Platforms like Salesforce Einstein and Tableau Pulse show how AI can enhance productivity and decision-making in a business environment by synthesizing complex data and automating routine tasks. LLM elements take these functionalities a step further, allowing for more nuanced data interpretation and predictive analytics.

However, the integration of AI in such complex systems raises significant concerns around data privacy and security. As these platforms handle sensitive business information, ensuring that AI recommendations are aligned with business objectives and ethical guidelines is paramount. The challenge lies in developing AI solutions that are not only powerful and efficient but also secure and compliant with data protection regulations. This balance is critical for maintaining trust and reliability in AI-powered business tools, as they become increasingly integral to decision-making in various industries.

4) Apps which have incorporated generative AI in a fun and flavorful way

• Subjects: Spotify
• Strengths:
  • Fresh and fun!: I initially thought this feature was tacky, but after using it for a couple months now, it has grown on me and it’s the primary way I listen to music on Spotify now.
  • Simple implementation: Spotify was already an industry leader in music curation and recommendations — this feature simply provides a cute, virtual DJ assistant to provide you with small sets of songs.
  • Entering a new era of generated voice: I feel that this DJ represents a turning point where they’ve created an AI-generated voice that is actually pleasant to hear and passes the uncanny valley test. The tone is nice and conversational, the blurbs are short and to the point, and the voice doesn’t sound too much like Microsoft Sam.
• Weaknesses:
  • Not strong from an AI perspective: Implementation-wise there is not much new here. Spotify performs some data analytics to pick out song clusters based on user preference, and a voiced, ChatGPT-esque persona narrates the experience.

Spotify’s AI implementation showcases the potential of generative AI to not only serve functional purposes but also to add a layer of entertainment and engagement to the user experience. Spotify's success here is a testament to the ability of AI to transform a routine activity like music listening into a fresh experience.

However, from a technological standpoint, these implementations are not groundbreaking in terms of AI complexity. They leverage existing data analytics capabilities to curate content based on user preferences, complemented by a generative AI-driven conversational interface. The innovation lies not in the AI's technical prowess but in its application to create a unique, enjoyable user experience. For teams that already have a strong data analytics or AI posture, leveraging generative AI tech to present that information in a novel way can be a fun win for branding.

Julia’s principles of generative AI UX

With these studies out of the way, lets try and outline some key principles about what makes a good experience:

Have a deep understanding of the main interaction points in your app

Regardless of whether you are building a simple short-story generator using OpenAI APIs, or building a chatbot to help navigate through a complex maze of business processes, AI experiences need to understand and support many (if not all) of the potential avenues of interaction that a user might use in your app. To be an efficient tool, it needs to be able to interact with the underlying data structures — to be able to retrieve data via RAG, render items for the user, and execute actions for them. This is the key component which takes your AI implementation to the next level, transforming it from a Clippy clone to something completely powerful, new, and bespoke.

However, this comes with a responsibility to ensure these interactions are meaningful and contextually appropriate. For instance, in the Tableau Pulse example in the video above the user queries “What else should I know about air fryers?”, there is power in the fact that the system is able to understand and respond to the user’s query, access relevant data points around the topic, and provide possible actions like sharing/notification management. This is is a system which not only understands the context of the app itself, but also the unique context of the “air fryer” data stored within the users account.

I believe that this principle also means providing context-aware chatbot entry points throughout the app, creating a pipeline from the “traditional” UI experience into the generative AI one. Buttons can be provided throughout the app labeled “chat with this table” or “chat with this graph” in order to obtain more nuanced opinions about data in the “traditional” UI. This ensures that the AI feature is available whenever you need it, but doesn’t get in the way when you don’t.

Support personalization and facilitate user mastery

By adapting to individual users, understanding their preferences, and recognizing their goals, AI interfaces can offer a customized user experience. A prime example of this is ChatGPT's "Custom Instructions" feature, which personalizes the LLM’s interaction based on a user's brief biographical input. For example, one can provide the following input: “My name is Julia, I am a web developer. Please chat with me as a friend.” and the chat experience will refer to you by name and in a more conversational tone than it would use by default.

This not only makes responses more appropriate for the user, but it also builds user trust and I hypothesize that it would make them more likely to use the tool again. These types of personalization are going to be important for generative AI tooling across the board, allowing users to tailor their generated content to fit their needs and persona on an incredibly granular level. This is something that can either be done explicitly or determined over time via analysis of user behavior.

Another key aspect of this is the development of user mastery. Ultimately, an AI feature is a tool which a user will use to accomplish their tasks. Like any tool, there can be a certain learning curve in getting things to work the way that you want. I believe that this learning curve is ultimately a net positive for user retention because it can feel satisfying to develop mastery over your tools, but this satisfaction is also heavily affected by updates to the underlying prompting or the LLM itself. If an update causes the entire chat experience to be altered, it can affect user mastery and have an inverse effect on the users’ ability to use it efficiently, leading to a poor experience.

Meet the user where they are

The success of tools like GitHub Copilot underscores the importance of integrating AI into familiar user environments. By embedding generative AI functionalities in applications that they already use (like IDEs for developers), then both the barrier to entry and the learning curve are significantly reduced. The AI experience should feel like a natural extension of the existing tools rather than an intrusive add-on.

However, this can be one of the most challenging aspects for product development with generative AI. It’s no secret that generative AI features are being developed across the entire gamut of products in tech. If your users leave your app, they will likely be going to another app where there will be another bespoke AI. Thus, I believe that the act of supporting external avenues for adoption, while still maintaining the strongest AI features in your main product, will be the best way forward for most products.

Support user exploration and provide valuable feedback

Effective generative AI UX designs need to facilitate users in exploring and iterating multiple solutions to a single problem. This iterative process not only yields more accurate and useful responses but also cultivates user trust with the system. It creates an environment where users understand that if the first iteration doesn’t hit the mark, they have the tools and flexibility to refine and improve the outcome.

By prioritizing exploration and feedback in generative AI UX design, products can provide a more dynamic, user-friendly, and trustworthy experience. This approach acknowledges the AI’s role as a collaborator in problem-solving, rather than a sole decision-maker, fostering a more interactive and user-centric experience. Generative AI excels in offering multiple solutions and pathways for tasks, echoing the exploratory nature of human creativity. Creative tools already have a rich history of encouraging exploration and iteration, providing a model for how AI can be used, but translating that to less exploratory domains will be challenging.

Keep things fun and interesting

Leveraging AI to add elements of fun and whimsy can greatly enhance user engagement with your product. By adding a playful element to an already robust technology, Spotify has not only enhanced the user experience but also deepened the user's connection with the product. The key is to strike a balance between being fun and maintaining the authenticity that users have come to expect. This involves tailoring the AI's responses to align with the brand's values and the unique preferences of each user.

Even in a text-based context, a pleasant and conversational tone can go a long way in making users feel at home with the product. This is where the art of language in AI becomes pivotal. For instance, generative AI experiences that employ a friendly and engaging tone can transform mundane interactions into delightful experiences. This approachable tone helps in breaking down barriers between the user and the technology, making the interaction feel more like a chat with a friend rather than a transaction with a machine. By ensuring the AI's tone is consistently pleasant and user-friendly, products can foster a sense of comfort and familiarity that keeps users engaged and satisfied.

Save user time

The primary objective in integrating generative AI into product development is to enhance user efficiency. The effectiveness of AI is fundamentally measured by its ability to save time and streamline user workflows. However, achieving this efficiency is not without its challenges. If an AI system frequently provides inaccurate responses, misguides users, or underperforms, it risks quickly diminishing its value. For AI to truly resonate with users, it must not only speed up processes but also deliver precise and relevant information, thereby boosting overall productivity.

This concept might seem straightforward, yet quantifying the efficiency of AI can be very complex. A basic method is to incorporate feedback mechanisms within the AI interface, allowing users to rate their experience with a thumbs up or thumbs down. But this approach just barely touches the surface of gauging AI effectiveness. To thoroughly understand the quality of AI products, we need more nuanced metrics. This could include analyzing user engagement patterns, measuring the reduction in task completion time, or assessing the accuracy of AI responses over time. Advanced analytics that track user behavior before and after AI implementation can provide valuable insights into the tool's impact on efficiency and productivity.

Future developments should focus not only on refining AI's core capabilities but also on innovating ways to assess and enhance its utility. This includes evolving AI's learning algorithms to better understand user intent, expanding its contextual awareness, and integrating more adaptive feedback systems. By doing so, we can ensure that AI implementations in product development are not just innovative but also tangibly beneficial for the end-users for the long term.

Predictions for the future

Generative AI will continue to find its way into products at a rapid pace. I hypothesize that we will not only continue to see advancements and refinements in the principles above, but even more work done in areas like speech synthesis and personalization in order to further push the needle to make these AIs feel more like a close friend rather than just a tool.

I also believe that we will continue to see more generative AI features being developed within specific applications, rather than as part of cross-platform AI tools. This strategy not only helps companies keep their user base engaged within their own app but also allows them to create chat experiences uniquely tailored to their users' specific needs. There is a powerful consumer potential for cross-app AI SaaS platforms (similar to how Zapier is for API integrations), however I believe that specialized AI platforms embedded within individual products are likely to offer a more comprehensive range of functionalities, including advanced data analysis, user interactions, automations, and workflows, than generalist AI applications which would have to provide that same level of functionality across n-number of products.

On this note, I hypothesize that we will begin to see AI-to-AI integrations across SaaS platforms. In a similar fashion to B2B API integrations that we have today, we could have AIs that can query and prompt each other across business lines. When a user reaches out to a Twilio AI assistant, that assistant could in turn reach out to a Jira AI assistant for help in managing a ticket. When an admin chats with an AWS AI and learns that a potential security issue may have arisen, the AWS AI itself could reach out to a Slack AI to inform the necessary parties of the issue. This will provide immense power, but will increase the need for strict data governance and auditing for information that enters and leaves these assistants.

Finally, I hypothesize that as generative AI experiences become more commonplace, this will drastically shift what the “traditional” UI will look like. Imagine a generative AI-powered dashboard for example. Instead of a prescriptive dashboard, the user can provide a set of queries which load whenever the user visits the page. Instead of some hardcoded graphs or a list of recent orders, they see a dynamic UI powered by real time data generated from the prompt “I just got into work, show me what I need to see in order to be my best today.” and the most relevant data is presented to them.

To infinity…

In conclusion, the intersection of AI and UX design is a rapidly evolving landscape, one that promises to redefine how we interact with technology. Moving forward, the challenge for designers and developers will be to balance innovation with user-centricity, ensuring that AI tools are not only technologically advanced but also intuitive, engaging, and, most importantly, trust-building. By adhering to the principles outlined in this article, we can look forward to a future where AI not only augments our abilities but also enriches our experiences in ways we are just beginning to imagine. As we continue to push the boundaries of what's possible, it's clear that the fusion of AI and UX holds exciting potential for transforming our digital world.