Ubuntu Smart Scopes project announced and explained

At Ubuntu Developer Summit R, the developers affirmed that, in Raring Ringtail, 100 scopes will be implemented by default (number started in Raring and continued in upcoming Ubuntu releases).

Weeks ago, the developers created Ubuntu Scopes, an interesting launchpad project, designed with gathering, developing and optimizing scopes, later to be landed in Raring Ringtail as default.

The main purpose of the mentioned development effort is the creation of a powerful Dash Home, meaning, pressing the Super button and typing a term in the search area, will populate the Dash with matched results, as generated by numerous and numerous scopes (data sources).

As previously stated, the Dash Home is targeted to become a global search place, incorporating results from both local data (recently opened images, previously-edited text documents, commonly used applications, etc) and online sources (Ubuntu One Music Store, etc), presenting to the user a filled-with-results experience.

It seems that, judging by an official wiki webpage, the Dash Home is to be structured in Ubuntu 13.04 as incorporating:

  • default scopes (as generated by Unity Applications lens, Unity Music lens, Unity Files lens, etc)
  • certain installed scopes (probably third-party scopes, similar to Ubuntu 12.10's third-party Clementine scope, Gmusicbrowser scope, etc)
  • remote scopes located on an online server

With the above-mentioned scopes (both local and remote), the user, after typing a word in the search area, will be able to enjoy a rich-spectrum of data results, including the results from the 100 scopes, "the goal of the 100 Scopes project and of Ubuntu in general is to provide lots and lots of scopes, and there should be lots and lots more in the future for every data source that anyone might want".

Typing a word in the Dash Home search area will be performed as: the Dash Home Scope communicates with the Smart Scopes Service (the server), the Default Scopes (Ubuntu One Music Store, etc) and other installed-by-user scopes (like for example Reddit); from this bundle of scopes, the Home Scope will generate matched results and matched categories (and populate the Dash Home view).

Typing (for example) mysearchterm in the Dash Home search area will:

  • send informations to the server (containing session ID, search term, state of the filters, user's platform, user's geographic info and user's locale)
  • immediately search the Default Scopes
  • then gather an ordered-by-importance list of recommendations and a list of remote Scopes (categorized by importance and relevancy)
  • finally, use the ranking information returned from the Smart Scopes Service and visually populate the Dash Home with results from Default Scopes and Remote Scopes
  • "from the list of ranked scopes returned by the service, figure out which ones are not covered by the default scopes or remote scopes, to activate & search them (i.e 100 scopes)"

Due to the massive number of new scopes, the Smart Scopes Service is to be refined and optimized in time based on feedback and data collections, "The Smart Scopes service collects metrics from the Dash about which scope results correlate to which queries, and uses those metrics to predict which scopes will provide the most useful result for a given query. The smart scopes service is in charge of selecting scopes from the client and server which should be queried to produce the results for the current dash search".

"To maximize our relevancy metric, we want to have for each query the list of scopes ranked by how likely it is that the user will click on them. This is easy to build for the most popular query terms. The most naive method is to present the users with random ranking and then see which scopes get clicked. This is called 'exploration'".

Worth mentioning
As usual, the users will have the ability to switch all online content OFF via System Settings-->Privacy