How to innovate: Product innovation by using AI

March 2, 2021 – Reading time: 12 minutes

In our interview with Eric von Hippel, trailblazer in lead user innovation and professor at the Massachusetts Institute of Technology (MIT), and Sandro Kaulartz, who is working in the area of social intelligence analytics for the global market research company Ipsos, we talk about a new approach to the division of labor in product innovation and innovation in general using artificial intelligence and user innovations.

Watch the summary of this video interview (12:35).

Volker Lippitz: Welcome. In this interview we will be talking about the topic of innovation, more specifically user innovation. With the advance of artificial intelligence, it becomes much easier to search for this innovation. Here with me are Professor Eric von Hippel from the MIT Sloan School of Management and Sandro Kaulartz from the International Marketing Company IPSOS. They have published an article on this topic with the title “Next-generation consumer innovation search: Identifying early-stage need-solution pairs on the web.” Thank you for joining this round to discuss your paper. My name is Volker Lippitz, I am the head of technical departments at the technology and innovation consultancy INVENSITY.

Eric, Sandro, can you shortly introduce yourselves?

Eric von Hippel: I’m a professor at MIT and research on the topic of innovation and where innovations come from. I’ve been doing it for thirty, forty years now. I have a large community of people who work with me on this topic.

Sandro Kaulartz: My name is Sandro Kaulartz and I’m working for IPSOS, a global market research company and I’ve been working in the area of social intelligence analytics. I am in charge of building analytical products, having the pleasure to work with people like Eric and academics to build new capabilities and methods, specifically for unstructured social data.

Eric von Hippel: I keep telling him to come over to MIT with me. I don’t know what’s taking you so long.

Volker Lippitz: My first question goes to you Eric. Within your article you point out that users innovate in all consumer good fields, examples are sports and kitesurfing. There’s also dwelling related innovation, medical related and so on. But is this limited to consumer goods only?

Eric von Hippel: No. Users, whether they are users of industrial goods or companies who are making new processes or consumers making new products with themselves, they all innovate to serve their own needs better. It’s simply that this particular paper focuses on a process for picking up consumer innovations in consumer generated content on the web. So, the phenomenon occurs in both in industry and amongst consumers. But the method we describe here is applied to consumers.

Volker Lippitz: Thank you. We will be talking a lot about need-solution pairs. Could you please shortly summarize that concept?

Eric von Hippel: Sure. Conventional problem solving assumes that you start with a need. In the U.S. we say in marketing research: find a need and fill it. So, you frame a problem and then you wonder if somebody already solved this problem or if you must solve it. When you look on the web, you are not restricted to a predefined need. You can search for a novel need that you’ve never thought of before, which is coupled to a user developed solution that proves its value.

So, when we get into the kind of web search that Sandro and I are talking about, what we are looking for is not people who have solved a fixed need. We are searching for people who have identified any need and then solved it. The result of this is that there are many innovations that companies never even thought of asking about that can be discovered and considered by companies for commercialization.

Volker Lippitz: So, basically this means you are looking for the unknown. Sandro: The first question that comes to my mind is: How can you systematically search for something if you cannot name it? Could you please summarize the approach you have depicted in your paper for us?

Sandro Kaulartz: I think the problem that Eric and I wanted to solve is that there is this wide field of user innovations. But it’s widely unknown to producers because there is no easy access to it. We see, and I think Covid-19 has taught us, that there is a tremendous number of users building their own solutions for problems that are essentially their real-world problems we see today. What we really wanted to do is using the web as a large innovation mine.

The challenge with it is that it´s widely unknown by producers. And here is where semantic AI algorithms enter to really discover relationships and from a very bottom-up approach discover these need-solution pairs that Eric was talking about. I think what we really wanted to do is give producers easy access to this landscape of user innovations they may not be aware of. And while that was challenging in the past, now with the available large data, the available AI methods and everybody’s fingertips, there is no reason not to look at it to design more user centric innovation processes.

Eric von Hippel: Volker, you asked how to search for the unknown. Sandro makes very clear that it’s known to the users. It’s just not known to the producers. So, you are searching for things about that users have said: I’ve come with it, you may not know about it and I find it very valuable. Those are the signals we’re looking for.

Volker Lippitz: Eric, you are very well-known for your lead user research concept. And in this context, a lead user is a single person that creates a new need-solution pair. When you look at classical marketing approaches, they usually try to identify the needs of a whole group. Would you say that the lead user approach is opposed to classical marketing research because of its focus on individual users rather than on the whole group?

Eric von Hippel: Marketing research, when it looks at the center of a market, is always behind. There is a leading edge where you discover people who have innovated for their own purposes. Our research shows that the pioneers are always users, essentially when function is novel.

Because producers cannot estimate the market, they have no idea at all how many people want a skateboard or a heart lung machine. But the doctor, who wants a heart lung machine, knows exactly what he wants and can justify building it.

You can´t really say that you are looking for single users. Sandro and I also looked for indications of social and commercial value. For example, we would not necessarily see the first person to develop a skateboard by the time we look and still – before producers are aware – there will be 10.000 skateboarders. They’re just all at the leading edge of the market rather than the center. So, they’re lead users, some of them are innovators, but it can be a community phenomenon as well as an individual one.

Volker Lippitz: How many would you say, out of the specific domain, are lead users?

Eric von Hippel: It’s a relative term. Lead users are simply defined as ahead of the market. Therefore, encountering needs that are sort of ahead of the commercial product area. But you don’t say it’s three percent or five percent, you simply say we can search the leading edge and discover important innovations there.

Volker Lippitz: What would then be the benefit for a company that would identify such lead users? Is it something like getting better products out of it, faster innovation processes or more successful products at the end?

Eric von Hippel: When you look at the center of your market, what you generally find is that users want some modification to what you’re already offering. They want an increased number of blueberries in the muffin. They want a reduction in the power requirements of a device.

When you look at the leading edge, what you discover are fundamentally new things that will become major. It’s not at all that conventional marketing research doesn’t work just fine to determine people’s relative wish for a reduced prize product versus an energy efficient one. It’s just that when you’re coming in with something absolutely new, the center of the market doesn’t know about it yet.

Volker Lippitz: So, you can have some kind of advantage over other companies if you are aware of that?

Eric von Hippel: Yes because companies are myopic. They look at the center of the market. They systematically throw out lead users as outliers because they want to analyze the center. It has always astonished me. When I said to look at a brand-new product, they said the user is very odd. It’s terrible, I so remember a little anecdote here:

It´s about a company where we did lead user studies. We came up with bikers who were doing long distant biking and had developed the equivalent of power bars to help them on their way. This was before there was anything about functional foods. We said that they can bike better, while the manufacturer said, that if we survey housewives from 29 to 40, they don’t seem to know anything about it or be interested. You know, ordinary market research methods focus on the center of the market, but they also tend to put blinders on the researchers.

Volker Lippitz: Yes.

Eric von Hippel: So, you said yes Volker, I’m glad you said yes.

Volker Lippitz: The users we are talking about have some kind of motivation to share their developments with the community, somehow self-rewarding. And they have fun to share it, informing other peers, learning from others. This is also a topic of your studies. Isn´t it also true that they´re hoping to gain the recognition and acknowledgement of the community?

Eric von Hippel: There are two sides to this. One of it is, that firms are set up still on the model of finding a need and developing the solution in house. So that even if you bring in solutions from outside, companies often do not welcome them. And yet, they are so important. Let me give you another example by means of producers of equipment for Type I diabetes.

We’re not at the leading edge here. People with Type I diabetes were dying because there was no artificial pancreas back then. One of the things you have to know is that very sophisticated people go home at night and become users. These are not DIY people with lesser skills than the R&D people in a company – in fact, they are the same people at night. If those people have a serious disease like diabetes, they’re not going to sit around waiting for a company to decide. It’s in their interest to develop an improvement.

There’s a wonderful group, specifically in the diabetes area, that adopted the model ‘we are not waiting’ because one of their members almost died. People came from everywhere to work on the next generation ‘artificial pancreas’. In a matter of weeks, they had completed what companies said would take five years. They put it on the web, thousands of users downloaded it and made copies for themselves and the manufacturers still resisted. It’s not like companies in general welcome this information. It’s a form of forcing them to move faster. They may or may not like it.

Volker Lippitz: So, there is usually no recognition aspect to it. The self-rewarding is in the center of the users.

Eric von Hippel: They are trying to save their own lives, doing something they enjoy or whatever it might be, but they are rewarding themselves, that’s right. And when the companies, by the way, adopt user developed innovations, they seldom acknowledge the source. They say to look at their marvelous XYZ. So, I think corrections have to be done there, too. My advice would be to search for free lead user innovations, improve them to make them producible and so on, but then acknowledge where you got it from.

Volker Lippitz: While we are talking about companies: Sandro, you are trying to commercialize this approach at your company IPSOS. I could imagine that there are some fields like kitesurfing or the example that we just heard where more ideas are shared, i.e. window cleaning. Have you ever noticed that effect?

Sandro Kaulartz: Yeah, right now we are piloting it with our clients to get more experience on their old belts. We certainly had cases where we came in with preconceptions where there was nothing going on in the area. But we all ended up pretty surprised how much there is. I think the richness is deeply underestimated, even for people who do that every day like me. So far, it’s quite amazing. Also, not just the volume, but also the broadness. And I think one key benefit of the methodology is that it’s very bottom-up based, it’s an explorative algorithm that would map out the entire landscape of user innovations.

There are certainly markets and domains of interest where the unmet needs prevail the solutions that consumers developed because of the reasons that Eric just mentioned. Not everybody brings the technical knowledge to it, but one can have an idea, describe it, can start something and use the web communities to work together on a problem. So far, I have been positively surprised, even in categories where I thought it wouldn’t end well.

Volker Lippitz: Very interesting. Also, I was very surprised when I read the paper and Eric said the research found out that 90% of the lead users are self-rewarding. For me, this huge share that we talked about is a very surprising finding. Eric, are there any other striking characteristics that most lead users have in common?

Eric von Hippel: Lead users, as we said, have their own needs as in the example of Type I diabetes. It can be an entertaining need like skateboarding, or it can be something vital to your health, like the artificial pancreas. First, they freely reveal because they’re collaborating openly. You couldn’t do it in a hidden context. Second, they don’t necessarily want to. One of the things you have to understand is that everybody – these are highly technically trained people typically – has a day job.

In my case, I am a tinkerer. You and I talked about the fact that back in the day, we had mechanical engineering training in our master’s degrees or whatever. So, I had a problem with my foot, strictly speaking the tendon. I went to the surgeon and he said to wear a thing that was was terrible and didn´t work at all. So, of course I built my own, showed it to the doctor and he said that it is marvelous. I told him that I want to give it to him, but he said to invest in starting a company because he can´t just receive it without buying it from somebody. I told him that I enjoy my current job and want him to take it as a gift, but that I won´t build it for him.

Volker Lippitz: Thank you. So, Sandro, the core of your paper describes the search approach leading to only unknown need-solution pairs and is paid the effort to spend. I don’t see any reason why producers of some consumer goods should not test this approach in your domain. So, to walk through the four-step process once like you describe in your paper, what effort would you expect in total?

Sandro Kaulartz: So far, it depends a little bit on the depth inside you want to deliver, it’s a questionable scope. But in the very last one it took us ten days with the team of four people from the moment that you have your data corpus collected from the web until you’re in a position to deliver the insights to your clients. It’s fairly fast.

There was a lot of work building the algorithm right and I’m completely ignoring the fact that we have been working with Eric for almost two years to get there. But now, as the algorithm is highly transferable across categories, there’s also less need for a data scientist to adjust it to specific categories. It can be incredibly fast. The moment you get it done once, the scalability is quite high.

Eric von Hippel: I must say, it’s been fantastic working with Sandro. This guy is amazing. When you’re really an expert, you can make it so that other people can use it who are less expert. That’s really what he and his colleagues engaged in.

Volker Lippitz: Are there any major effort drivers that you know of? That make it more complicated to apply the method.

Sandro Kaulartz: I think one critical step is properly defining the initial search field. So, where do you really want to find. There’s always the risk of going in too narrow, that you have a very precise definition of what you’re after, but then the data corpus is not sufficient enough to find anything. When you go in too broad, you pretty much inflate your data corpus and you know you will find a lot. But if it’s helping you to invent in a specific field is questionable / may be questionable.

To give you an example: If somebody would want us to look into foods, that’s too broad because we would look at 50 Mio. data points from last month. Just sifting and then cutting through the noise of this data corpus would be highly inefficient. So, focus and, obviously, data availability is a critical aspect. And then one of the things we learned on the way and we’re seeking for efficiency and so on. But I don’t want to give the impression that human expert and really bringing in a subject matter expertise in this process. It’s critical because it is really designed for human machine interaction.

Just to give you one example in this kitesurfing pilot that essentially is part of the paper. When you circle around the topic concept of innovation – anything that relates to fixing a problem, hacking something that existed before – you would discover things that first would feel pretty odd. For example, at some point, I saw the topic concept of Tea-bagging from kitesurfers. While initially you think, something needs to be removed or is a mistake, the algorithm for reasons can´t understand. But looking deeper into it with a subject matter expert, you will discover that Tea-bagging is a very common problem in kitesurfing. It’s when the wind is overpowering, you lose control of the kite and then the kite makes you dip in the water and lifts you up like a tea bag.

People use this term to start building new security features in kitesurfing equipment. Back until then, all innovations from companies had been like needing to create kites that are enabling faster surfing. But there was a mismatch between the security features and the speed of travel. People started to work on new security features. I’m mentioning it because it is a very classic example of ringing in an expert and he will tell you that Tea-bagging is something very interesting to look at. And I would have removed it because it felt like a mistake in the data corpus.

The last point I wanted to make is, bringing technology and some AI together with someone who knows the field very well, is fundamentally important to the success of applying this methodology.

Eric von Hippel: One thing we should point out is that Sandro and I bridge academia and his firm, but the agreement we have with his firm is that we publish methods fully and we are very happy when others come in and try to improve the field and so on. IPSOS is a head in many ways, they have secret sauces galore. Not to mention they have Sandro. But we invite participation. We want to build this field.

Volker Lippitz: Since we´re on the method itself: You have been describing in the paper that you use python libraries, Gensim and Spacey. How do these AI tools fit into research? Is there an input for the national language? What is the output?

Sandro Kaulartz: We actually use an internal built version of something called space model. What we have been giving in the paper because we do realize not everyone wants to work with us and reapply it by themselves. Spacey, i.e., is a great python library that can essentially do the same. We use an in-house model for the reasons that I just explained. We have created a process that allows human machine interaction where you collect the data corpus and let the algorithm search for the innovation concept in their data corpus. So, language based on vector models that relate to fixing problems, solving something, inventing something or hacking something.

What the algorithm essentially does: it turns words or expressions into vectors meaning the miracle values an then it looks at the neighborhoods. With doing that, you understand the meaning context of innovation. What are other expressions from consumers when talking about innovations? And then, after you have circled around this environment and you define your trails what innovation means in your particular case, you can start analyzing what the objects of innovations within are. Are these tricks or features? Are they equipment, kites or boards? You discover and that’s the beauty of it. It’s highly explorative in the way it works.

The output, as a first step, is cutting down to this innovation core in your large data corpus we are talking about. It’s also very important to collect a lot of historic data. I would not recommend looking only at the last three months. In the case of kitesurfing, we essentially went back to the hour zero when this board discipline started, which meant – if I remember correctly – up to thirty years of data that we collected.

You’re dealing with a high data mass. The algorithm we applied is a filtering mechanism. But then, as I said, you need a subject matter expert who brings in this knowledge to really discover if this is something that brings functionally novel features to the sport discipline or just somebody who wants to create an all-black kite and a more aesthetic innovation or reproduction of something that existed already.

This validation step is very important in the process. Otherwise, you’re just collecting words around innovation without really understanding the deeper meaning and why this individual created something to solve a particular problem, which then eventually leads you to need-solution pairs as we described earlier.

Volker Lippitz: So, that’s also one of the strengths of Artificial Intelligence, that you can train such models in advance and reuse them. Would you think it’s possible to have a specific word space model for a specific application for finding need-solution pairs that is pretrained and reusable?

Sandro Kaulartz: That’s what we did essentially. The time it took us was taking something that was part of our day to day work, but making it better and more specific for this need-solution pairs search. And that took us a while. But we are right now working on a second generation because there are ways to make that more efficient and with the latest achievements in natural language understanding and things that simply didn’t exist before, we started working on it. Because the landscape is changing so fast. We are right now exploring ways to make this process more efficient, which should ideally reduce the human time that is required. But also, as I said earlier, I think it is really important to have a subject matter expert as part of the process because I don’t think we will get there with meaningful outputs when we let it completely run by an algorithm.

Volker Lippitz: Thank you, that was very interesting. Any last sentences from your side?

Eric von Hippel: Well, in my case, I simply want to say that the lead user process or lead user concept has been around for several decades. But it was too hard to do. It was costly for companies; they would continuously come and say this is so hard. We know it’s out there, but it’s simply too hard. Sandro and I, as well as IPSOS and many academics think that people can finally start to interface with lead user innovations when we get this down to ten days.

The other thing I’d like to say is, when you do a search, remember how at the beginning of this conversation we talked about need-solution pairs. So, when you do a search, and let’s take kitesurfing as an example, you discover things that are a mix. You discover some things that are improvements. You might have been able to do this with standard marketing research, in other words a more reliable harness for somebody who is a kitesurfer. In the center of the market, people might well have complained about existing equipment, saying it’s just you need a better harness.

But what you also find is the breakthroughs. So, i.e. what we found there, was drone surfing. In drone surfing, you throw away the kite. All of a sudden, it’s a changed sport. Now you don’t need the wind. You can go up mountains. You can do all sorts of extraordinary things. Standard marketing researchers talking to standard kitesurfers in the middle of the market, would never discover these things that are going to totally change their field within the next ten years.

The method is now economical enviable and it is, I would say, remarkably good for breakthroughs. I mean Sandro with his standard clients found completely new things there where the R&D department said that they´ve never thought of that and that’s an old tired category. There are opportunities all over and we hope companies grasp them.

Volker Lippitz: Sandro?

Sandro Kaulartz: Nothing to add. I would just say the opportunity for us is really to help our clients to just bring the user right in front at the very fuzzy front end of early innovation where usually things are pretty internal. In marketing, the R&D departments, maybe some trend and future agencies will tell you about how their future in the category will look like.

We still see a lot of innovation failing in the markets. One reason for it is that they’re maybe not grounded deep enough on true consumer problems. Our hope for us is to learn constantly. Our vision is to embed that into a real time capability where every day you can learn about these problems that people are facing on the street and these are the solutions that correspond to these emerging needs. So, really enabling clients to become better at innovating and little less internal and listening more to how people live their lives and how products and brands are a part of it. I think that’s the truly interesting opportunity.

Volker Lippitz: Thank you very much for your time and your valuable input!

Watch the summary of this video interview (12:35).

Author

Link to the video summary

Here you can watch the summary of this interview on YouTube.

Resources