Adobe

Computer vision search 👁️

by John Canelis, Product Designer
Project Topics
  • marketplace
  • system design
  • interaction design
  • product design
  • prototyping
  • user research
  • web
  • search

Disclaimer: This post contains several images and videos of early concepts and prototypes. Some of the most interesting work from my time at Adobe was experimental and exploratory interaction design. My goal was to work with our technology team to find ways to identify objects within images and allow users to reposition them for their search results on Adobe Stock.

During my three years at Adobe I was a product designer on the Adobe Stock team. Adobe Stock is a marketplace for creative assets where people can buy and sell things like photos, videos, audio, and more. I designed features for browsing, searching, filtering, and saving content on the site.

One of my favorite projects during that time is what we referred to as advanced search. We created an experience that allowed users to select and move objects within an image to see similar content.

Several years later, this feature continues to live on the site.

Adobe Stock Search
Select an object within the image.

Adobe Stock Search
Reposition object

search-4
Select object within image

search-5
Reposition selected object

This feature was created to help people have more control over their search results. It was also an effort to demonstrate Adobe's advanced technology and strengthen its brand as a leader in understanding the needs of creatives.

I worked with data scientists, the head of product for Adobe Stock, dedicated user researchers, engineers, and other members of the Adobe Design team to make this possible.

Problem context

The business needs subscribers. Users need great content and control over their search results.

The key metric for Adobe with this product is subscribers (annual or monthly). Users of these creative marketplaces tend to only use one service. Because of this, it was important for us to have content and functionality that was clearly better than competing products.

Marketplace subscribers

One of the biggest obstacles for our business was to get people who had subscriptions with other services to switch to ours. This was especially important for larger creative teams (enterprise accounts). The cost of switching services for a company with many users can be high, so there needs to be enough perceived value in order to warrant making the switch.

User problems

Some of the key problems users expressed with using products like this is how much time it takes to find the right content and how little control they feel they have over their search results.

Emerging technology

During this time, Adobe was developing technology using concepts like computer vision. My role as a designer was to take that technology and use it for search features on Adobe Stock. I would work with the Adobe Search of data scientists to find ways our technology could deliver functionality unlike anything our competitors had.

Who is this for?

Freelance creatives, designers and business decision makers.

Our target user was anyone who needs content on a regular basis for their creative projects. More specifically, we thought about our users in three categories: full-time working designers on creative teams, freelancers, and business decision makers. An example of a “business decision maker” would be a director or manager of a team of creatives.That person ultimately makes the decision of what service their team or company will pay for.

Requirements

An open initiative to create compelling search features.

This began as an open-ended project.

Initially, there weren’t specific requirements other than a general initiative to use our search technology. The director of product on Adobe Stock set out a mission for us to use our emerging search tech to highlight Adobe’s strengths as a technology company and show that we understand creatives better than our competitors.

Deliver unique search features

We needed to differentiate Adobe Stock from our competitors. Whatever we create should be something that isn’t available with other products. Of course, it should solve a real customer need, but there was an extra focus here on innovation. This feature should enhance the Adobe brand and the value of Adobe Stock. More specifically, people should perceive these brands as leaders in creative technology.

Use Adobe’s design system

Adobe Stock was one of the first products within Adobe that was primarily on the web. The design would need to use our design system (Spectrum), which had been initially designed for desktop products. This project would use existing components but also push to innovate and inform the constant evolution of the Adobe design system to be more web friendly.

Team

Design, user research, engineering, and product management.

As the design lead for this project, I worked with a dedicated user researcher, a front-end engineer, and a dedicated product manager. Weekly design critiques were done within our team of about six designers focused on Adobe Stock.

Bi-weekly reviews were done with the larger product team, where I presented updates on the project to the director of product for Adobe Stock. My research partner gave me insight into our users’ needs and how they work. We also collaborated to determine what questions to ask and how to facilitate user testing sessions.

Data Science

A key part of this project was working with the Adobe Search team, which included data scientists and engineers. They had rough prototypes and ideas of what they felt was possible and potentially useful. My job was to act as a bridge between them and our users. I reviewed their concepts, asked if other functionality could be possible, and thought about how to tie everything together into a powerful, needed feature for people searching content on our site.

Below is an example of an early concept prototype created with the help of an engineer. It features the ability to position objects using our database of millions of images. It was important to validate that concepts worked well with the actual content we had available within the product.

Concept Search
An early concept using real data

Measuring success

More subscribers, more user control, and a stronger brand.

  • We should see an increase in subscriptions. To attribute an increase directly to this feature we can use surveys during sign ups to capture what features are influencing their decision.
  • Brand perception. Adobe Stock (and Adobe as a company) should be perceived as having stronger technology solutions than competitors. This was particularly important for the Adobe Stock product to be able to differentiate itself from sites like shutterstock.com.
  • We should see higher satisfaction rates from users in terms of their feelings of being in control over their search results.

Unique Challenges

An open-ended mission operating through a large organization.

Starting with technology

This project started with a company strategy to use our latest technology to solve a customer need. Starting from an open-ended, conceptual place is exciting but it was a challenge to find what technology we had available that was actually useful. Some concepts worked as an interesting demo but didn’t work well enough to launch. There were ideas that worked in very specific situations but provided strange results in others.

Progress within a large company

Adobe is a huge company. It’s the largest product team I’ve ever been on. Establishing consensus about how to move forward involves building multiple relationships, demoing concepts (many times), and building strong evidence to support the proposed solutions.

Adobe Search
An exanple search result on Adobe Stock

Outcomes

Launched and still live.

This feature launched in 2020. It was demoed at Adobe MAX, Adobe's annual creativity conference highlighting the latest updates to their products.

I’m happy to see that it continues to live on the site. Unfortunately, I don’t have data on its engagement rates or how often it is cited as a reason for people choosing to subscribe to Adobe Stock.

I would expect to see higher rates of conversion in subscriptions. I would expect this feature to appeal to people making decisions for creative teams. I would be curious to see how often this feature (or search features in general) is listed as a top reason for why users chose to subscribe to Adobe Stock.

I would expect to see a higher overall perception of quality for the Adobe Stock brand with regards to its search functionality compared to other products.

I would be curious to see rates of engagement. But because this feature is visible after users click "View similar" – a feature that already had one of the highest rates of engagement, I would expect that users naturally discover this experience. Users tend to start with an image they like and then seek similar items.

Other Projects

Sticky search, filters, design system, and software plugins.

adobe-global-search
Sticky search bar.

adobe-stock-structure
Search, filter, and save with links to landing pages and your Adobe account.

adobe-stock-navigation-pattern
Navigation and search patterns throughout Adobe web products.

adobe-stock-design-system
Adobe Stock design system in Sketch.

adobe-stock-framer
Plugins for other applications.