Generative AI

We're in the middle of an AI Revolution,  and I think I can fairly compare it to the early   days of the internet or the beginning of the mobile phone industry. The world is changing fundamentally and irreversibly. And the way we work and live is tremendously impacted, whether we want it or not. Why now, when AI has been  around for so long? Well, it probably has to do with the impressive popularity and spectacular  adoption rates

of recently launched AI tools. What this adoption story tells us is that people are extremely curious, interested and open to this new world of tools and capabilities; interest that's most probably stemming from a mix of enthusiasm, fear, reluctance and excitement. And most of these tools are based on *generative AI*. So, what is generative AI? We know that machine learning is a  subset of AI, deep learning is a subset of machine learning,

and – you guessed it – generative AI is a subset of deep learning. To define it, putting it simply, generative AI is a type of artificial intelligence that creates new content based on what it has learned from existing content. Its main applications are in generating text, images or other media in response to prompts. How it works is that the model gets trained on data by applying machine learning techniques and is then able to create something entirely new, but with similar characteristics.

The main applications of generative AI are *language models* and *image models*. Language models learn about patterns in language through training data and then, given some text, they predict what comes next. Using text, these tools can generate new text, translations, summarizations, question answering, grammar correction, can generate images, videos, audio, speech

or decisions, tasks, play games. *Image models* produce new images using techniques such as *diffusion*, which works by destroying training data,  by adding noise, and then learning to recover the data by reversing this noising process. Then, given a prompt, they transform random noise into images. By inputting images in generative AI tools, you can get text, image captioning, image search, visual question answering; *images*, image completion, super  resolution, video, like animations.

The generative AI application landscape is already pretty vast, and we'll see more use cases and scenarios that these tools will address as technology and our understanding of it advance. Narrowing our scope to the design industry, some of the applications that help us most right now are *image generation* visual, mockups, icons, product illustrations, photography; *text generation* – UX copy, product descriptions, emails, documentation,

data augmentation. In cases where we have scarce research data, AI can help create synthetic data to supplement the real data. Not ideal – real data is the best, though. *Chatbots* for customer experience, and so on. The most prominent tools in generative AI are ChatGPT, Bard, Microsoft's Copilot, Midjourney, DALL-e, Stable Diffusion, to name a few. The landscape of tools is ever-changing with new tools emerging every day

and others being sunset after a short-lived life. The speed at which the market is transforming makes it hard to keep up with all the tool opportunities we can leverage in our work, but some of them are stable enough to be a safe bet for sticking around for a while; Bard, GPT, Copilot, probably here  to stay. And I'm excited to see how this prediction ages.

Neural networks are a type of artificial intelligence that can learn from data and perform various tasks, such as recognizing faces, translating languages, playing games and more. Neural networks are inspired by the structure and function of the human brain, which consists of billions of interconnected cells called neurons. They are made up of layers of artificial neurons that process and transmit information between each other. Neuronal networks can be trained using different algorithms

and architectures depending on the task and the data. Neural networks have many applications in various domains, such as image recognition, speech recognition, machine translation, natural language processing, medical diagnosis, financial forecasting, quality control, gaming and more. For example, Facebook uses a deep recognition system called Deep Face.

Google Translate uses a neural machine translation system with an encoded decoder architecture. Google DeepMind developed AlphaGo, a deep reinforcement learning system that defeated the world champion of Go. Neural networks are powerful tools for artificial intelligence. Because they can adapt to new data and situations, generalize from previous examples, and discover hidden patterns and features in the data. However, they face challenges and limitations

such as explainability, robustness, scalability and ethics. Some possible directions for the future of neural networks include neuroevolution, symbolic AI integration, and generative models. Neuroevolution is a form of artificial intelligence that uses evolutionary algorithms to generate artificial neural networks, parameters and rules. It is often applied in artificial life and general game playing.

Symbolic AI integration is a type of artificial intelligence that combines neural and symbolic AI architectures to address the weaknesses of each providing a robust AI capable of reasoning, learning, and cognitive modeling. These can be useful in autonomous cars and medical diagnosis systems. Generative models are a type of machine learning model that generates new data that is similar to a given dataset.

They can capture the underlying patterns or distributions from the data and provide realistic fake data that shares similar characteristics with the original data.

Before showing you an end to an AI-powered design process. That still happens under human guidance, which is the key point in this conversation. Let's go quickly over how AI can help us. So for ones that can decrease cognitive load, aid decision making by processing large volumes of data, can help us automate repetitive tasks such as formatting images or resizing text.

It can help us by providing more insights into human behavior and usage patterns. Also, with creating prototypes or mockups, all kinds of visual assets, it can spot usability issues and so on. AI can help us at every step of the design process. Covering our blind spots, augmenting our thinking, making us more efficient if we use it correctly. Here's how an end to end design process augmented by AI may look like. But the key to reading this schema are how I prefer to look at it:

there's always a person orchestrating this. The future in which AI could generate a cohesive, coherent, reliable and relevant design process end to end is a very distant future for now, and there's an impetuous need for someone governing over this process, applying critical thinking and showing intentionality at every stage of the design process. Also, because AI shows multiple limitations

on each of the steps shown in the schema. Nielsen Norman Group published an article unpacking the limits that currently surround the use of AI in research. To understand the context of their analysis. You need to know that there are currently two type of AI powered research tools we currently see on the market: insight generators and collaborators. AI Insight generators. These tools summarize user research sessions based only on the transcripts.

Since they don't accept any kind of additional information (context, past research, background information about the product and users and so on), they can be highly problematic in how they generate and present those summaries. While there are some workarounds like uploading background information as session notes to be added to the analysis, it's not the right framing for the source and it's not going to reflect correctly in the analysis and generation. Humans would be much better at this. The scoping and systems

thinking required to understand the interpretation landscape AI collaborators. These work similar to insight generators, but they're slightly better because they accept some contextual information provided by the researcher. For instance, the researcher might show to the AI some human generated interpretation to train it. The tool can then recommend tags for the thematic analysis of the data in addition to session transcripts, collaborators can also analyze researchers notes

and then create themes and insights based on input from multiple sources. But even though they appear to be a bit better, they're still significantly limited and pose a lot of problems, if not used with the right mindset and caution. The limitations they've identified and expand on in detail are: most AI tools can't process visual input, and the biggest problem with that is no human or AI tool can analyze usability testing sessions by the transcript alone.

Usability testing is a method that inherently relies on observing how the user is interacting with the product. Participants often think-aloud describing what they're doing and thinking. Their words do provide valuable information. However, you should never analyze usability tests based only on what participants say. Transcript-only analysis misses important context in user tests because participants don't verbalize all their actions don't describe every element in the product. Not always have a clear understanding or mental model of the product.

So for now, Nielsen Norman's group recommendation is do not trust AI tools that claim to be able to analyze usability testing sessions by transcripts alone. Future tools able to process video visuals will be much more useful for this method. Another problem is the limited understanding of the context. This remains a major problem. AI insight generators don't yet accept the study goals or research question insights or tags from previous

rounds of research, background information about a product or the user groups, contextual information about each participant, new user versus existing user, the list of tasks or interview questions. There is also a problem with the lack of citation and validation, which raises multiple concerns and problems. The tools aren't able to differentiate between the researchers notes and the actual session transcript. A major ethical concern here. We must always clearly separate our own interpretation or assumptions from what the participants said or did.

Another problem with the lack of citation is that it makes verifying accuracy very difficult. AI systems can sometimes produce information that sounds very plausible, but is actually incorrect. Unstable performance and usability issues are another problem. None of the tools they tested had solid usability or performance. They reported outages, errors and unstable performance in general. And then there's the problem of bias. According to Reva Schwartz and her colleagues, AI systems and applications can involve biases

at three levels: systematic, statistical and computational and human biases. AI must be trained on data which can introduce systematic such as historical and institutional and statistical biases, like a dataset sampling that is unrepresentative enough. When people are using a AI-powered results in decision making they can bring in human biases like anchoring bias. So bias can creep into research efforts on multiple levels,

and these tools don't yet have the mechanisms in place to prevent that. I wanted to discuss the limitations reported in the article in detail, because I believe we can easily extrapolate and expand them beyond just research tools. Most of these problems will be observable on other types of AI companions in the design process. Biases in image generation, limitations in being offered context, other kinds of input limitations, not accepting files or images,

output vagueness, generic results, and so on. So I think that this is a necessary frame to keep in mind when interacting and designing with the help of AI. Tools are not very reliable yet and accurate. So take everything they produce with a grain of salt and apply critical thinking at all times.