Designing search for the Start screen

Given the ton of interest in the design of the new Start screen we wanted to dive deeper into the topic of search. There's a clear focus on efficiency and overall professional productivity in the comments. For professional scenarios, every keystroke matters. One new aspect of the Windows 8 platform is the ability for Metro style apps to deliver a customized search "contract." For this post we'll focus on the built-in search capabilities for files, settings, and apps, which update the Windows 7 search features. You can learn more from our //build/ session on search, which provides a detailed look at the topic of this post. With that lens, Brian Uphoff, a program manager on our Search, View, and Command user experience team, authored this post.
--Steven​

In our previous related posts (Evolving the Start menu, Designing the Start screen, and Reflecting on your comments on the Start screen) we discussed the evolution of the Start menu and the reasoning behind the design. We also discussed how organizational mechanisms and search are powerful tools that make it easier to find and launch apps. As you install more and more apps, these tools become increasingly important. For the past several releases, searching from the Start menu has been established as the quickest way to find and launch apps, particularly for keyboard users.
When planning Windows 8, we wanted to make sure the efficiency and dexterity of the Windows 7 Start menu search was carried forward into the new Start screen. Before we dive into the details of the new experience, let’s take a quick look at the evolution of search from the Start menu, and how people are using it today.
Evolution of searching from Start

The search box in the Start menu as we know it today first made its appearance in Windows Vista. It became easy for users to search for programs or apps, settings, and files on the desktop and in personal folders like Documents, Pictures, Music, and Videos. The search experience aggregated different types of results in one view with programs and settings combined in a single group. The results of a query displayed a small set of items in heuristically sized groups. You needed to click “See all results” to see the rest in Windows Explorer, which aggregated everything into one ungrouped and unsorted view.

Figure 1: Start menu search in Windows Vista
In Windows 7, we expanded results to include detailed Control Panel tasks in addition to the main Control Panel pages. We also separated out Control Panel items from programs into a unique group that allowed you to more easily focus on the type of result you were looking for.
The overall experience aggregated different types of items and had a fixed limit on the number of results that could appear. This was because the result set was limited to the size of the Start menu. Clicking a group header took you to Windows Explorer for programs and files or to Control Panel for settings. Each experience had a type-specific view, though the search results order diverged from what was shown in the Start menu. Showing an aggregated view in the Start menu required compromising on performance in addition to space because we would search all programs, Control Panel items, and files, even if you were looking for only one of these data types.
Figure 2: Start menu search in Windows 7
When we look at the usage data of how people are using the Start menu to search in Windows 7, it’s clear that searching to launch programs is the most frequent and important activity users engage in with Start search.
Our telemetry data shows that 67% of all searches in Windows 7 are used to find and launch programs. Searching for files accounts for 22% of all Windows 7 Start menu searches, and searching for Control Panel items about 9%. Searching for email messages via Start Menu is very rare (less than 0.05%). The remaining 2% are searches executing the “Run” functionality.
Figure 3: Windows 7 Start menu search usage data
Searching from Start in Windows 8

Searching via the Start menu has continued to evolve with each release. The Windows 8 Start search experience builds on top of search features available in Windows 7 and provides a unique view for each of the three system groups - Apps, Settings and Files. These search result views are a natural progression from the Windows 7 groups and are easily accessible from anywhere in the operating system via the Search charm or keyboard shortcuts. Separating the search results into views means we can tailor the experience for each data type. For example, the File search view provides you with filters and search suggestions while typing to quickly complete your query.
In Windows 8, we expect people will be acquiring and installing more apps than ever before. Had we continued using the Windows 7 Start menu search interface to search for a Control Panel item, you would always see app or program results before Control Panel results, displacing many Control Panel items from being the first match. This and other constraints on the existing design required us to develop a new approach—this is especially true as we consider the increasing use of larger monitors or higher DPI screens where longer menus become even more difficult to use and navigate. In Windows 7, the total number of results that could be shown in the Start menu was limited. Depending on the number of groups with matching results, an average of 3-4 results were shown per group. Very rarely did all results for a group show up, and the organization of the results was pretty unpredictable.
With Windows 8, on the other hand, we’re following an app-first model, where each app developer understands their data and users best, and knows the best way to present the information to them. Using the same model for search, we believe that always having a quick and consistent way to get directly to settings or file search results gives you precision and control over the type of results you’re looking for. In Windows 8, each view is tailored for the type of content you’re searching for, and shows all the results, instead of limiting them due to screen real-estate.
One change a few of you will notice is that file search results no longer include email messages and contacts. The inclusion of email search never got the generalized support from mail clients that we had hoped for, though at least one mail client did support it (one reason why email searches are rare in the Start menu <0.05% of total searches). With the app-first approach in Windows 8, Metro style email apps will use the search contract to provide a rich set of filtered search results in a view customized for email. In comparison, email clients and other apps in Windows 7 have no control over how their search results are presented.
We paid special attention to ensuring the number of keystrokes required to find and launch apps, settings, or files is at parity with or better than in Windows 7. We’ve introduce a set of keyboard shortcuts to help users quickly and efficiently get to settings search results (WIN key + W) or file search results (WIN key + F), thus reducing the total number of keystrokes needed to find and launch settings or files. We’ll cover how we maintained and increased keyboard efficiency across these views in more detail later in the post.
Searching apps

App search results show the full set of apps (both their “friendly” names and executable names) for which the search term matches the name. As the number of installed apps increases, it becomes difficult to browse through a large list to find an infrequently used app. Search helps quickly filter and reduce a large list of apps down in just a few key strokes. We wanted to make sure we preserved the same keyboard usage patterns as Windows 7. You don’t have to first click on the Search charm to begin searching – simply start typing in the Start screen and you’ll see your list of apps filter down to the one you are looking for.
Figure 4: Full-screen app search results
Also note that the Most Frequently Used (MFU)-based ranking of app search results from Windows 7 is preserved in Windows 8. For example, if you type “paint” in the developer preview you get 2 apps back as search results – PaintPlay and Paint. If you predominantly just use Paint, it will be ranked higher than PaintPlay as you use it more often. So, launching Paint (or other apps you frequently use) becomes more efficient the more you use app search.
Some of you have pointed out that many users won’t discover that they can simply type to start searching in the Start screen. Search is closely associated with typing— the most common pattern to search in the Start menu is to bring up the Start menu by using the Windows key or by clicking the Start button and typing. That exact and efficient behavior is preserved in Windows 8 as we have observed and found that pattern is what users care about most. Our experience in user tests, and even when people at //build/ tried the Develop Preview for the first time, shows that people tend to serendipitously discover this feature early in using Windows 8, and so we're confident it will not be a hindrance to usability. Nevertheless the Search charm is highly visible, and selecting it shows the Search box.
The Windows 7 Start menu also included “Run” functionality for commanding and navigating Windows. This has been carried over to Windows 8 as well—tasks like running scripts and .exes in the user’s PATH are still possible and supported in App search. Search continues to support launching folders in Windows Explorer by typing in full paths. For example, typing “C:\” in Start search results in the set of folders in the C: drive appearing below the search box. Pressing the Down Arrow key moves selection through the list and autocompletes the folder name in the search box, allowing users to continue typing to further refine the path. You can do the same with UNC (\\foo\example) paths as well. And of course WIN key + R will switch to the desktop and bring up the classic Run dialog, just as you would expect.
Figure 5: Typing a path into Start search
Searching settings

The settings search experience brings together all settings and Control Panel items across the system in one view. Settings search results are matched not only to the name of the Control Panel applet or task, but also to the various keywords that may describe it. We have also heard your frustration that shutdown is not available as a search result, and we will address this along with improvements to the Start user interface for shutdown (as a reminder, you can also just use the power button or close the lid).
Figure 6: Full-screen settings search results
Searching files

The number of files on PCs keeps increasing over time as users continue to acquire and create more documents, music, photos, and videos. Our goal, while redesigning the file search experience, was to make it seamless and complete so you can achieve your task of quickly finding a file without having to transition to Windows Explorer.
In File search, you’ll also see search suggestions as you type to help you quickly and efficiently complete the search. The indexer provides these search suggestions based on the content and properties of files it knows about. Search suggestions are a very powerful concept made popular and used extensively on the web—they help you to pinpoint relevant search terms with just a few keystrokes. In Windows 8 we built search suggestions into the file search experience and also made this feature available in the platform for all Metro style apps to use. Note, this feature also accounts for typos or spelling errors, and suggests the auto-corrected search term as you type. Using the arrow keys to choose suggestions autocompletes the term in the box. This makes it easy to add more terms to the query and quickly narrow down the set of results to find the one you want.
You can also still search using AQS (Advanced Query Syntax) from Windows 7. AQS allows for greater precision and control when constructing the query to get targeted results. Here are some sample searches and their advanced query syntax:



Query

AQS Syntax


Find all files authored by Brian or David

author: (Brian OR David)


Find all photos with an F-stop of 2.8 where no flash was fired

f-stop:2.8 flashmode:no flash


Find all files where the file name contains a word starting with Metro and the file size is greater than 1MB

filename:$<Metro* size:>1mb


Figure 7: Search suggestions based on the content and properties of files
Figure 8: Full-screen file search showing results
Separating searches for apps, settings, and files into their own views allows room for each of them to evolve and breathe— this way they can each provide their own ideal display format—unlike the single list of results in previous versions, which required conformity to achieve aggregation in the limited space. For example, the file search view also provides filters to easily refine the results based on the type of file you’re searching for. Filtering by type is a powerful way to efficiently reduce the results set, irrespective of where the file is saved.
More relevant and contextual information for each file is also now displayed to make the search experience complete. This helps differentiate between similar results and also makes it clear to you why a given result was returned, by highlighting the property that matched the search term—something not possible in the Start menu before. For example, when searching for the term “performance,” the results now highlight where “performance” matched. In one result, it matched on the Title property, and is clearly indicated that way in the result. The results also show file type and size to help further disambiguate between results.
Figure 9 File search results highlight the property that matches the search term
Using a mouse, hovering over a result reveals a rich tooltip with some additional details. For example, for the video result shown below, the rich tooltip shows the duration of the video, frame height, frame width, date modified, and the full path to the file. Using touch, pressing and holding on an item reveals the tooltip.
Figure 10: Rich tooltip reveals additional details in file search
Designing Start search for dexterity

Designing for efficiency and dexterity is a core goal of the Start search feature team. As such, using the keyboard to launch apps, settings, and files from search is a very important part of the Start search experience. We also put a lot of thought into preserving existing keyboard patterns, which both average and advanced users have come to rely on, and have built muscle memory around.
Our telemetry data shows that many users leverage the Start menu as a means of commanding Windows. They use specific key-combinations to efficiently launch apps. For example, pressing the WIN key, typing “calc”, and pressing ENTER launches Calculator. Many advanced users know that typing “cmd” and then CTRL+SHIFT+ENTER opens an elevated command, and that typing “notepad c:\mynotes” creates or opens a .txt file. If you watch the keyboard demonstration from the //build/ keynote, you will see many of these used.
These keyboard patterns continue to work in Windows 8 just as well as in previous versions. Pressing the WIN key takes you to the Start screen. Simply start typing in the Start screen and the Search pane automatically opens with the search term in the search box, and the view filtered to show apps that match the term.
The fastest way to search settings and files from anywhere in the system is to use a set of keyboard shortcuts introduced to increase efficiency. These Windows 8 shortcuts reduce the number of keystrokes needed to launch a setting or file to a number equal to (or less than in many cases) what was required in Windows 7. Alternatively, you can also use the Search pane, which indicates the number of results matching the search in each view, to switch between apps, settings, and file searches.



and type

Apps search


+W

Settings search


+F

Files search

Figure 11: Windows 8 Start search keyboard shortcuts
Based in part on the feedback here, we are working on a change that we hope to have available in our beta release, which will take you directly to app search results when you select the Search charm in the desktop.
The efficiency of using the keyboard doesn’t stop at just typing to start a search. Sometimes the app, setting, or file that you want to launch is not the first result shown. You can use the arrow keys to quickly move down to the desired app in the results list, and then press ENTER to launch it. The white box that shows keyboard focus tells you which app will be launched on ENTER. This enables you to efficiently launch any app, setting, or file matching a search. In Windows 7, you could only launch one of the top 3-4 results with this kind of efficiency.
When we looked at some of the common Control Panel items people were searching for (for example, searching for power options using the term “power”), it quickly became clear that because we favored app results first in our Windows 7 design, “Power options” was the fourth result, below all the power shell app results. If you installed Office and frequently used PowerPoint, you saw PowerPoint along with Power shell (32 bit, 64 bit, and the Help file) ahead of ”Power options.” Extending a similar design in Windows 8 would have meant that the position of the “Power options” result would continue to fluctuate as you installed more apps on your system. This forces you to scan through increasingly large result-sets every time you search for a particular app or setting or file.
In Windows 8, we also provide the count of results per system view, so you immediately know how many apps or settings or files match the search term. Switching search views is also designed so you can easily switch views without taking your hands off of the keyboard. In the example shown below, to switch to settings search, press the Down Arrow key and the focus shifts to settings in the search list. Press ENTER and you will see settings search results. As mentioned before, you can continue to use the arrow keys to choose the desired item and press ENTER to launch it. Pressing TAB allows you to quickly switch between the search results list and the Search pane.
Figure 12: No results for apps – but use the arrow keys to switch to settings view, which shows 17 results
Figure 13: Use the arrow keys to choose a settings search result
To add to our earlier point on preserving search efficiency, here are some comparisons of the number of keystrokes for launching frequently used apps via search. In Windows 7, you would press the WIN key, start typing in your search term, and then press ENTER to launch the program. We count all the keystrokes end to end. In Windows 8, you can apply the same pattern for searching for apps (WIN key, type in the search term, press ENTER to launch). Launching Word, Calculator, Paint, or Media Player by pressing the WIN key and typing "word", "calc", "calculator", "paint", "player", or "media" in the search box takes precisely the same number of keystrokes in Windows 8 as it does in Windows 7.
To launch settings in Windows 7, you would press the WIN key, type in the query, arrow down to the result you wanted, and press ENTER to launch it. In Windows 8, you can use WIN key + W to launch settings search, type in a query, and press ENTER to launch. Typing WIN key + W and typing "uninstall", "device manager", or "defender" gives you the same results with precisely the same number of keystrokes in Windows 8 as in Windows 7. In some cases, it takes even fewer keystrokes than in Windows 7 (for example, pressing WIN key + W, typing “power” and then pressing ENTER to launch power options).
This dexterity-focused design isn’t all we have done to make search more efficient. We have also made key performance investments across the system. In current testing of Windows 8, our search performance improvements have cut app search time in half for desktops and laptops. The improvements are even larger on netbooks.
Figure 14: Performance comparison showing % decrease in execution time of app search
Designed for touch, too

We discussed the details of designing search for dexterity while using the keyboard, but this design works equally well for touch. To begin searching in Start, simply swipe the edge and tap on the Search charm. This opens the full list of installed apps. You can use the touch keyboard to search for a program to launch, but you can also use semantic zoom to zoom out, and then tap on the section that contains your app. Start search is lightweight, fast, fluid, and quickly gets out of the way as you pan through your list of apps, settings, or files.
The Search pane makes it easy to continue searching for the same term in other system views or Metro style apps with just one tap. Touch-friendly search suggestions minimize typing on a touch screen, and the search contract provides a framework for search suggestions that developers can use for their own Metro style apps as well. In addition, we designed touch-friendly filters with result counts in the file search view to help users quickly refine the search results set.
The new Start search experience makes it easier than ever to search for content in your PC or in apps from anywhere in the system. It’s been designed to work seamlessly and efficiently across the range of devices that Windows will run on, and across different input mechanisms such as the mouse, keyboard, and touch. Start search brings apps, settings, and files together with other Metro style apps that implement the Search contract, creating a unified and consistent search experience. We will talk more about the search experience and the search contract in a future post. In the meantime, you can get more information from our talk at //build/ on the Search contract.
Here’s a video showing how easy and efficient it is to quickly launch apps, settings, and files from anywhere in the system using the keyboard:
Your browser doesn't support HTML5 video.
Download this video to view it in your favorite media player:
High quality MP4 | Lower quality MP4
We look forward to your continued feedback as you try out the Windows 8 Start search experience!
Thanks,
Brian

Continue reading...