Introducing the WinUI 2.6 Pager Controls

In this article we walk through a UWP app that demonstrates two new pagination controls that ship with the upcoming WinUI 2.6. The Windows UI 2.x Library is a project that’s factoring out the XAML UWP controls from the operating system into a NuGet package. New XAML controls and styles as well as backward compatible updates on existing native controls are rolled out via WinUI 2. This happens at a much higher pace than with the Windows 10 SDKs. WinUI 2 is allowing UWP developers to build and deploy apps that behave better on target platforms with different Windows 10 versions.

The next iteration of WinUI 2 will bring two new pagination controls: a traditional Pager control and a visual PipsPager control. The need for such a family of XAML pagination controls was raised in 2018, and today they are ready to use … well almost. WinUI v2.6. is currently in still prerelease. Make sure to check ‘include prerelease’ when looking for the NuGet package:

WinUI_2_PreRelease

Pagination controls make the most sense when they’re connected to large DataGrids. For the sake of simplicity our sample app just uses a FlipView with some images.

Pager Control

From the specifications we learn that the last official pagination control in XAML was the DataPager in … Silverlight. It’s good to see that this gap will finally be closed with a new XAML Pager control. Here’s the new Pager control in its simplest form – with its default display mode, and NumberOfPages and SelectedPageIndex bound to its associated FlipView:

<FlipView x:Name="ImageRepeater"
          ItemsSource="{x:Bind Pictures}" />
<!-- Default Display Mode -->
<winui:PagerControl NumberOfPages="{x:Bind Pictures.Count}"
                    SelectedPageIndex="{x:Bind ImageRepeater.SelectedIndex, Mode=TwoWay}" />

Here’s how it looks like – no surprises here:

Pager

By changing the DisplayMode to NumberBox you can trade the DropDown page selector for an editable NumberBox, more convenient when there are a lot of pages:

NumberBox

The third DisplayMode is named ButtonPanel and displays page numbers as a hyperlink, and an ellipsis if there are too many. Here’s the XAML:

<winui:PagerControl NumberOfPages="{x:Bind Pictures.Count}"
                    SelectedPageIndex="{x:Bind ImageRepeater.SelectedIndex, Mode=TwoWay}"
                    FirstButtonVisibility="Hidden"
                    LastButtonVisibility="Hidden"
                    NextButtonVisibility="Hidden"
                    PreviousButtonVisibility="Hidden"
                    DisplayMode="ButtonPanel" />

Here’s how this looks like:

ButtonPanelMode

In this configuration the Pager always displays a page number hyperlink to the previous, next, first, and last pages. It would be nice to hide the obsolete default arrow buttons for this. The API supports this, but unfortunately there’s this bug that prevents hide these buttons. The fix is not yet released.

The synchronization between the pagination control and the host of the pages can be configured via binding to (or programmatically updating) properties -like we did on this page- or by implementing event handlers on the button clicks – what we will do in the PipsPager sample.

WinUI is developed in the open, you find the (technical) documentation and C++ source code for the Pager control right here. The new Pager looks and feels totally as expected.

FlipView companion

The specifications for a glyph-based pagination control were issued long before the first implementation of the PipsPager. The description reminded us of a similar control that we wrote (many) years ago in the Windows 8 age: the FlipViewIndicator, a companion pagination control for … the FlipView. The control -basically a templated ListBox- ended up in Tim Heuer’s Callisto framework. For old times sake we dug up the source code and pasted it in the sample app. Here’s how the result looks like:

Indicator

We’re glad to see it still works nicely. Here’s how to use it in an app:

<FlipView x:Name="ImageRepeater"
            ItemsSource="{x:Bind Pictures}" />
<controls:FlipViewIndicator FlipView="{Binding ElementName=ImageRepeater}" /> />

For more details on its implementation, check this antique blog post.

PipsPager

Most web based image carousels use a pagination control made of dots, like this:

FlipCarousel

Pips are small but easily countable items, such as the dots on dominoes and dice, and that’s were PipsPager got its name from. It’s a row (or column) of dots -optionally extended with arrow buttons- that allows you to paginate. You find the full specs here.

Here’s its default look and feel in our sample app:

PipsPager

In an attempt to stretch the new control, we implemented a circular carousel: when the user reaches the end, the first item will reappear on the next navigation. To achieve this, we needed to override (or rather bypass) some of the PipsPager’s default behaviors. By default, the PipsPager

  • disables its ‘Previous’ button when the pip that corresponds to the first page is displayed,
  • does not react on clicking on the pip that corresponds to the displayed page, and
  • disables its ‘Next’ button when the pip that corresponds to the last page is displayed.

Instead, we built a carousel that

  • always keeps the pip for the selected page in the middle,
  • uses all other pips to navigate (slower with the more central pips, faster with the peripheral ones), and
  • implements circular (endless) navigation.

Here’s how it looks like:

PipsPagerCarousel

PipsPager supports this scenario: it allows setting NumberOfPages to a negative number to indicate an undefined or unknown number of pages, and it can hide its ‘Previous’ and ‘Next’ buttons. For the interaction, we’re using SelectedIndexChanged:

Here’s the XAML for our carousel PipsPager:

<FlipView x:Name="ImageRepeater"
            ItemsSource="{x:Bind Pictures}"
            SelectionChanged="ImageRepeater_SelectionChanged" />
<winui:PipsPager x:Name="Pager"
                    NumberOfPages="-1"
                    NextButtonVisibility="Collapsed"
                    PreviousButtonVisibility="Collapsed"
                    SelectedIndexChanged="Pager_SelectedIndexChanged"
                    MaxVisiblePips="5" />

To make sure that the pip for the first page never appears (it would not respond to clicking hence block the circular navigation) we decided to make the control not rotate between pages 1 and n (the number of images) but instead to make it rotate between n and 2n. So the PipsPager always believes he’s somewhere in the middle of the page collection.

Here’s the initialization:

Pager.SelectedPageIndex = Pictures.Count;

And here are the event handlers that implement change selection in the pager but also in its host. FlipView has its built-in navigation support, so it can change the selected item by itself, in which case we need to update the pager:

private void ImageRepeater_SelectionChanged(
	object sender, 
	SelectionChangedEventArgs e)
{
    // Keeps the selection in the middle of the pager.
    Pager.SelectedPageIndex = ImageRepeater.SelectedIndex;
}

private void Pager_SelectedIndexChanged(
	WinUI.PipsPager sender, 
	WinUI.PipsPagerSelectedIndexChangedEventArgs args)
{
    // Good that this doesn't create an infinite loop.
    Pager.SelectedPageIndex = Pager.SelectedPageIndex % Pictures.Count + Pictures.Count;
    ImageRepeater.SelectedIndex = Pager.SelectedPageIndex % Pictures.Count;
}

Just like the regular Pager this new PipsPager looks and feels as expected. There’s still some room to fix bugs and improve these controls: the official release of WinUI 2.6 is scheduled for next month. We’re happy with these two new pagination controls. It’s great to see the UWP ecosystem extended with good-working and good-looking essential controls.

Our sample app lives here on GitHub.

Enjoy!

Introducing the WinUI InfoBar control

According to its roadmap WinUI 3.0 is currently focusing on Win32 desktop applications and .NET 5. In mean time WinUI 2.x keeps on evolving, bringing XAML features and controls for UWP. One of these new controls is the InfoBar control which was released as a component of WinUI 2.5.

In its original proposal we read that Microsoft was looking for a uniform way to display “long-lived app-wide status messages for scenarios such as updates available or errors that may occur which prevent the user from experiencing an app or its features working as intended“.

Here’s an example of such an ‘update available’ message in Office:

OfficeStatusBar

Here’s another example of the type of message that InfoBar is designed to display: the ‘we are recording this session’ message in Teams. This is clearly more than just a notification, hence the control needs to be explicitly closed by the user:

RecordingStartedTeams

Unlike similar controls (notifications, teaching tip) the InfoBar is not a member of the Flyout family, so you should adapt your layout for it.

We made a little UWP sample app to play with this newcomer – and stretch it a little bit. Here’s how this app looks like, it demonstrates some typical usages of InfoBar:

Classics

The InfoBar API

Here’s an overview of InfoBar’s public interface:

InfoBar

The InfoBar controls displays a Message with a Title and an Icon (with background color and icon adapting to the Severity) and an optional ActionButton. You can take a look at its documentation here and its source code right here – it’s spread over a remarkably huge number of files.

Here’s an example of a XAML declaration in its simplest form – a warning without action button that displays a validation message of a text box:

<!-- Validation Panel -->
<winui:InfoBar Title="Not amused"
                Message="I think we need a recount ..."
                Severity="Warning"
                IsOpen="True" />

Here are some samples of typical declarations of an informational wide horizontal InfoBar at the top, with an action button (regular button or hyperlink):

<!-- Office 365 Status Bar -->
<winui:InfoBar Title="UPDATES ARE AVAILABLE"
                Message="Do you want to update this app right now? It will close and reopen."
                Severity="Informational"
                IsOpen="True">
    <winui:InfoBar.ActionButton>
        <Button Content="Update" />
    </winui:InfoBar.ActionButton>
</winui:InfoBar>

<!-- Visual Studio Status Bar -->
<winui:InfoBar Title="Performance report"
                Message="There's this weird extension slowing down the startup."
                Severity="Informational"
                IsOpen="True">
    <winui:InfoBar.ActionButton>
        <HyperlinkButton Content="Tell me more about this" />
    </winui:InfoBar.ActionButton>
</winui:InfoBar>

Here’s the Live Visual Tree for all these controls. Notice that all InfoBar instances live inside the XAML page. This means you need to preserve space for them:

LiveVisualTree

We also created an InfoBar with a WinUI DropDownButton as its ActionButton and IsClosable to false to force the user to select an option in the dropdown if she wants to close the message. Here’s the XAML:

<!-- One that Edge and Outlook should have -->
<winui:InfoBar x:Name="PreferencesInfoBar"
                Title="Preferences updated"
                Message="We changed your user preferences, you like it?"
                Severity="Informational"
                IsOpen="True"
                IsClosable="False">
    <winui:InfoBar.ActionButton>
        <winui:DropDownButton>
            <TextBlock FontFamily="Segoe MDL2 Assets"
                        FontSize="14"
                        Text="" />
            <winui:DropDownButton.Flyout>
                <MenuFlyout Placement="BottomEdgeAlignedLeft">
                    <MenuFlyoutItem Icon="Emoji2"
                                    Text="Keep"
                                    Click="MenuFlyoutItem_Click" />
                    <MenuFlyoutItem Text="Revert"
                                    Click="MenuFlyoutItem_Click">
                        <MenuFlyoutItem.Icon>
                            <FontIcon Glyph="" />
                        </MenuFlyoutItem.Icon>
                    </MenuFlyoutItem>
                    <MenuFlyoutItem Icon="Emoji"
                                    Text="Revert and never touch my preferences again"
                                    Click="MenuFlyoutItem_Click" />
                </MenuFlyout>
            </winui:DropDownButton.Flyout>
        </winui:DropDownButton>
    </winui:InfoBar.ActionButton>
</winui:InfoBar>

All menu items in the sample are hooked to the same handler that closes the InfoBar by setting the IsOpen property:

private void MenuFlyoutItem_Click(object sender, RoutedEventArgs e)
{
    PreferencesInfoBar.IsOpen = false;
}

Here’s how it looks like in the sample app. Notice how the Live Visual Tree indicates that the menu flyout is hovering over the other controls – you don’t need to preserve space for these:

LiveVisualTree2

Here’s yet another typical example, but this time we created it programmatically. It’s an error message at the bottom of the page, much like the old WinForms StatusStrip:

var infoBar = new WinUI.InfoBar
{
    Title = "OOPS",
    Message = "Division by zero. Dude, I told you so ...",
    Severity = WinUI.InfoBarSeverity.Error,
    IsOpen = true,
    VerticalAlignment = VerticalAlignment.Bottom,
    Margin = new Thickness(0, 0, 0, 80)
};

RootGrid.Children.Add(infoBar);

The last of the ‘classical’ examples is an informational message at the right side of the page. We’re pretty close to a notification scenario here – except that the control still requires manual closing:

<!-- Toast (with ignored customizations) -->
<winui:InfoBar Title="WOOHOO"
                Message="Data has been saved properly."
                Severity="Success"
                IsOpen="True"
                CornerRadius="8"
                VerticalAlignment="Center"
                HorizontalAlignment="Right"
                Height="120">
    <winui:InfoBar.Content>
        <TextBlock Text="Keep up the good work ..."
                    Foreground="DimGray"
                    Padding="0 0 0 12" />
    </winui:InfoBar.Content>
</winui:InfoBar>

In the previous InfoBar instance, we tried to apply some customizations such as a specific Height and CornerRadius. The current control template just ignores these. That’s largely understandable, since the purpose of this new control is to bring a common look-and-feel for this type of scenario and steer away of custom solutions.

Customizing the InfoBar

On one hand it’s good to finally have a standard control for error and warning messages with a default look (size, color, icon) and a default behavior (requires explicit closing). We strongly believe that we should try to stick to this standard. On the other hand our customers very often require corporate UI and UX patterns that may deviate from the defaults.

Adapting the Template

It’s almost never a good idea to entirely re-template a control, especially since WinUI targets support across all Windows 10 versions. Hence its control templates may contain multiple IsApiContractPresent conditions so you’ll have to test your retemplated control for each and every version – and then you’re still not future-proof. On top of that you also have to consider accessibility when customizing controls.

There’s nothing wrong with lightweight styling (i.e. overriding some system brushes to sync the color scheme to your corporate style). WinUI controls facilitate this by making maximum use of XAML Theme Resources – you can check the ones that are used by the InfoBar right here. In the sample app we went a little bit further. We couldn’t resist to create a template for the InfoBar’s originally proposed look and feel:

Specials

we started with right-clicking on an instance in the Visual Studio designer, to create a copy of the current template:

CreateTemplate

You find the latest version of this template right here. It wouldn’t add value to post the source code of our template here, so we won’t. We’re happy with the outcome of this little exercise.

Adding Auto-Closing Behavior

The documentation is pretty clear on on where InfoBar should be used and where not. The control should e.g. NOT be used for just confirming an action, and in several GitHub issues –such as this one– it is stated that the control is NOT meant to be auto-closed. These are of course just recommendations, and we developers and our customers have been known to bypass recommendations sometimes, no?

In a lot of apps user interface consistency is –rightly- more important than following recommendations. Here’s an example of such consistency in Visual Studio. When you’re checking in code into a source control system (Azure DevOps, Git, whatever), the VS Team Explorer window displays an warning InfoBar as long as the action is busy, and an error InfoBar when something went wrong. After a successful check-in, the same UI is used for displaying the result:

GitHubVisualStudio

Most of us will agree that this behavior makes sense, but this last message is far from  “long-lived app-wide status messages for scenarios such as updates available or errors that may occur which prevent the user from experiencing an app or its features working as intended“.

It’s good to reuse the InfoBar default look here, but we believe that this last message deserves a different behavior: it should auto-close after a few seconds. That’s just what we did in the sample app. We created an InfoBar subclass with an extra public AutoCloseInterval and some private methods to implement the behavior (feel free to add a fancy Community Toolkit fade animation yourself). Here’s the class diagram for the control:

 AutoClosingInfoBar

Since the API of InfoBar is similar to the TeachingTip control, we were able to reuse the code for our own AutoClosingTeachingTip. The child class registers a notification function to listen for changes in the IsOpen property:

this.RegisterPropertyChangedCallback(IsOpenProperty, IsOpenChanged);

A timer is started:

private void Open()
{
    _timer = new DispatcherTimer();
    _timer.Tick += Timer_Tick;
    _timer.Interval = TimeSpan.FromMilliseconds(AutoCloseInterval);
    _timer.Start();
}

After the interval, the control is closed:

private void Timer_Tick(object sender, object e)
{
    this.IsOpen = false;
}

Allow us to repeat that with InfoBar it’s good to finally have a standard control for error and warning messages with a default look and a default behavior. We welcome this new member of the WinUI control family.

Our sample app lives here on GitHub.

Enjoy!

A lap around the Microsoft MVVM Toolkit

In this article we’ll walk through a UWP sample app to experiment with the features of the new Microsoft.Toolkit.Mvvm package that is part of the Microsoft Community Toolkit.

MVVM

MVVM is a software architectural pattern introduced by Microsoft in 2005 to support the development of XAML apps, originally in WPF and later on in Silverlight. MVVM stays relevant to the more recent XAML environments such as UWP, Xamarin (MAUI), the Uno platform, and WinUI.

In a nutshell MVVM allows you to separate UI logic from business logic by dividing your code into

  • Models that represent data or information – think ‘domain objects’ or ‘entities’,
  • Views that represent the structure and layout that the user interacts with – think ‘pages’ and ‘controls’, and
  • ViewModels that express Model data and business logic through methods, properties and commands that are accessible to the Views,  preferably via declarative data-binding.

MVVMCore

MVVM requires the strong data-binding capabilities that we see in XAML through its {Binding} and {x:Bind} markup extensions. The pattern was also implemented in other technology stacks such as AngularJS/Angular.io, Ext JS and Vue.js.

Microsoft started recommending MVVM for XAML 15 years ago and they still do. Oddly enough, they only provided ‘bindable controls’ and never came up with official helper classes (like base classes that implement INotifyPropertyChanged or ICommand) or an MVVM development framework (with event aggregation, messaging, or viewmodel location). The only exception is the Composite Application Library, which later grew into Prism. Regarding MVVM, Microsoft developers always had to rely on (Open Source) third parties, of which the most popular (in alphabetical order) are/were Caliburn, MVVM Light, and Prism.

We had great experiences with each of these frameworks. However -except for Prism- none of them are still actively maintained. We assume that the ecosystem for XAML apps is evolving too rapidly for the maintainers of larger Open Source libraries. Indeed none of the existing MVVM frameworks was ever designed with .NET Core, Uno, MAUI or WinUI 3 in mind…

Maybe it’s time for a new wind to blow.

Enter Windows MVVM Toolkit

The Microsoft.Toolkit.Mvvm package is a modern, fast, and modular MVVM library that is part of the Windows Community Toolkit (WCT). The package targets .NET Standard 2.* so it can be used on

  • any .NET app platform: UWP, WinForms, WPF, Xamarin, Uno, WinUI and more, and on
  • any .NET runtime: .NET Native, .NET Core, .NET Framework, or Mono.

Its API surface is the same in all environments, making it perfect for building shared libraries.

The object model and terminology are definitely inspired by MVVM Light and the development team is aiming to provide a smooth migration path from MVVM Light to the new toolkit. Apart from functionality WCT Toolkit also pays high attention to performance. Many benchmarks like this one (on the Messenger class) are continuously run during the development:

perf2

The package comes with documentation and a nice sample app. WCT MVVM is currently in ‘preview 4’ so everything is still work in progress.

A UWP Sample App

A few months ago this GitHub issue revealed the initial feature set of Microsoft MVVM Toolkit, preliminary documentation, and even an operational preview package. We decided to create a little UWP app to test that package. Here’s how the home page looks like – standard clean WinUI 2.5:

HomePage

We’ll use this app throughout this article to explore the features of the new MVVM Toolkit in town.

Core classes

The Building Blocks page of the sample app illustrates the core classes for data binding and commanding:

BuildingblocksPage

ObservableObject

ObservableObject provides a base class for types to implement INotifyPropertyChanged and INotifyPropertyChanging – typically Models and ViewModels. You may have encountered similar helpers in other frameworks under the name ObservableBase, BindableBase, or NotifyPropertyChanged. Typically such a class comes with an easy way to raise the PropertyChanged event in property Setters, and that’s exactly what SetProperty() does.

Here’s what it looks like in one of our Model classes:

public class SuperHero : ObservableObject
{
    private string _name;

    public string Name
    {
        get => _name;
        set => SetProperty(ref _name, value);
    }

    // ...
}

Here’s another example from the sample app, this time for a ViewModel – the one that is bound to the BuildingBlocksPage View:

public class BuildingBlocksPageViewModel : ObservableObject
{
    private SuperHero _superHero;

    public SuperHero SuperHero
    {
        get => _superHero;
        set => SetProperty(ref _superHero, value);
    }

    // ...

}

Here’s how the ViewModel is declaratively bound to the View as its DataContext:

<Page.DataContext>
    <viewModels:BuildingBlocksPageViewModel x:Name="ViewModel" />
</Page.DataContext>

When we change a property on the SuperHero, the UI will update:

ViewModel.SuperHero.Nemesis = _nemeses[rnd.Next(0, 5)];

When the property that you want to monitor is a Task<T>, then you can use SetPropertyAndNotifyOnCompletion() instead of SetProperty(). It wil set the property but with the help of an internal TaskNotifier<T> it will delay the notification to when the task was completed:

private TaskNotifier<string> _saveTheUniverseTask;

public Task<string> SaveTheUniverseTask
{
    get => _saveTheUniverseTask;
    private set
    {
        SetPropertyAndNotifyOnCompletion(ref _saveTheUniverseTask, value);
    }
}

There’s even an overload that allows you to register a callback for when the task finishes. Here’s the ViewModel code triggered by the save button on the sample page. It assigns a new value to the task, and then starts it:

public async Task SaveTheUniverse()
{
    SaveTheUniverseTask = new Task<string>(
        () =>
        {
            Task.Delay(2000).Wait();

            if (rnd.Next(2) > 0)
            {
                return $"We're doomed, I lost my {SuperHero.Tool}.";
            }

            return "We're saved ... this time.";
        }
    );

    _saveTheUniverseTask.Start();
}

The task was made observable, so you can monitor its status and result. A simple binding suffices to follow up on the Status [note: The sample app has some extra code to reveal more statuses than ‘initial’ and ‘completed’.]:

<Run Text="{x:Bind ViewModel.SaveTheUniverseTask.Status, Mode=OneWay}" />

We needed to add some extra code to bind to the Result of the task, since UWP XAML does not allow you to bind to a generic. Here’s a ‘wrapper’ property for the result:

public string SaveTheUniverseTaskResult => 
	_saveTheUniverseTask.Status == TaskStatus.RanToCompletion 
	? _saveTheUniverseTask.Result 
	: "(hold your breath)";

And again, the declarative binding:

<Run Text="{x:Bind ViewModel.SaveTheUniverseTaskResult, Mode=OneWay}" />

RelayCommand

The RelayCommand class – a.k.a. DelegateCommand in some frameworks- and its generic and asynchronous siblings provide a default implementation of the ICommand interface. It’s used to declaratively bind events in UI controls to methods in the viewmodel. In the sample app the BuildingBlocksPageViewModel has the following method to toggle its data provider:

private void SwitchDataProvider()
{
    if (_dataProvider is RedDataProvider)
    {
        DataProvider = new BlueDataProvider();
    }
    else
    {
        DataProvider = new RedDataProvider();
    }

    SuperHero = _dataProvider.SuperHero();
}

To trigger this method from a button, we declare a property of the ICommand type:

public ICommand SwitchDataProviderCommand { get; }

Then we let a RelayCommand refer to it:

SwitchDataProviderCommand = new RelayCommand(SwitchDataProvider);

In the view we can now bind the button’s Command property to the ViewModel’s property of type ICommand:

<Button Command="{x:Bind ViewModel.SwitchDataProviderCommand}" />

AsyncRelayCommand extends the ICommand behavior with support for asynchronous operations. Here’s an example of an asynchronous data provider switch:

private async Task SwitchDataProviderAsync()
{
    await Task.Delay(1000);

    SwitchDataProvider();
}

The corresponding property is defined as IAsyncRelayCommand:

public IAsyncRelayCommand SwitchDataProviderAsyncCommand { get; }

And assigned to an AsyncRelayCommand instance:

SwitchDataProviderAsyncCommand = new AsyncRelayCommand(SwitchDataProviderAsync);

IAsyncRelayCommand inherits from ICommand, so you can bind it to a Button’s Command:

<Button Command="{x:Bind ViewModel.SwitchDataProviderAsyncCommand}" />

The ExecutionTask property gives you access to the underlying Task, so you can monitor it. There’s also a convenient IsRunning property that indicates when the task is … running. You can use this to display a busy indicator, such as a ProgressRing. Here’s how such binding looks like:

<winui:ProgressRing
   IsActive="{x:Bind 
	ViewModel.SwitchDataProviderAsyncCommand.IsRunning, 
	FallbackValue=False, 
	Mode=OneWay}" />

ObservableValidator

ObservableValidator is an ObservableObject that hosts a default implementation of INotifyDataErrorInfo. This enables Models and ViewModels to implement validation rules and expose validation results. INotifyDataErrorInfo used to be an important interface some years ago, when all out-of-the-box controls in e.g. Windows Forms, ASP.NET Forms, and WPF came with error templates that implemented the UI part of this interface. It looks like the built-in class comments in the validation annotation attributes are still living in that era – not really reassuring for today’s .NET Client developers:

RegexValidator

In the modern XAML stacks you need to do all of the control templating manually or rely on a third-party control library. But here’s the good news: rumor has it that WinUI 3 will bring back error templates for XAML controls, so we welcome this WCT MVVM Toolkit’s ObservableValidator very much.

The ObservableValidator class comes with

  • members that implement the INotifyDataErrorInfo interface (HasErrors, GetErrors(), ErrorsChanged, …), 
  • overloads of SetProperty() that allow you to specify whether or not validation should run on the assignment of a new property value,
  • a TrySetProperty() that does not assign the value if it induces validation errors, and 
  • support for all kinds of validation attributes from the System.Component.DataAnnotations namespace.

Here’s the StudyGroup class from our sample app. Its properties are decorated with validation rules on requirement, length, numerical range, and regular expression pattern:

public class StudyGroup : ObservableValidator
{
    [Required(ErrorMessage = "Topic is Required")]
    [MinLength(2, ErrorMessage = "Topic should be longer than one character")]
    public string Topic
    {
        get => _topic;
        set => SetProperty(ref _topic, value, true);
    }

    [Range(2009, 2015, ErrorMessage = "Class should be from 2009 to 2015")]
    public int Class
    {
        get => _class;
        set => SetProperty(ref _class, value, true);
    }

    [RegularExpression(@".*[pP]aintball.*", ErrorMessage = "Hobbies should contain 'Paintball'.")]
    public string Hobbies
    {
        get => _hobbies;
        set => SetProperty(ref _hobbies, value, true);
    }

    // ...
}

Many more validation attributes exist. Some allow you to compare the new value of a property to the old one, or compare the value of two properties within the instance. It’s also very easy to roll your own custom validation attribute.

Using the INotifyDataErrorInfo members a View can react appropriately to the the validation state of its ViewModels. In our sample app we decided to let StudyGroup expose an Errors string with the concatenated list of validation errors. A symbol icon with a tooltip displays the messages:

<SymbolIcon Symbol="ReportHacked"
            Foreground="Red"
            Visibility="{x:Bind ViewModel.StudyGroup.HasErrors, Mode=OneWay}"
            HorizontalAlignment="Right"
            Margin="0 4">
    <ToolTipService.ToolTip>
        <TextBlock Text="{x:Bind ViewModel.StudyGroup.Errors, Mode=OneWay}"
                    Foreground="Red" />
    </ToolTipService.ToolTip>
</SymbolIcon>

Here’s how it looks like in action:

Validation

Messaging

In an MVVM application, ViewModels as well as other components may need to communicate with each other in a loosely coupled way. Most MVVM libraries have a Messenger infrastructure for this purpose.

MVVMMessenger

Microsoft MVVM Toolkit’s messaging API allows independent modules to exchange information via messages through a publish/subscribe mechanism. Our sample app hosts two scenario’s. Both scenario’s have different View/ViewModel pairs (a.k.a. “Modules”) on the same page. They are unaware of each other, but need to exchange information about the current ‘theme’.

Here’s the first messaging scenario:

  • the ViewModel of the Shell hosts the current ‘Theme’ (just a color),
  • when the sample page opens, all modules request the current theme and adapt their UI,
  • a ‘Theme Module’ allows to update this current theme through a ToggleSwitch,
  • the other modules update their UI immediately, and
  • the ‘Theme’ is not a global variable (and neither is the ShellViewModel).

Here’s how the page looks like:

MessengerPage

Messages

In this scenario we use two of the message base classes that come with MVVM Toolkit. Here’s the definition of the message that is sent by the theme module when the theme changed, a ValueChangedMessage<T>:

public class ThemeChangedMessage : ValueChangedMessage<Theme>
{
    public ThemeChangedMessage(Theme value) : base(value)
    {
    }
}

Here’s the definition of the message that all modules use to get the current theme, a RequestMessage<T>:

public class ThemeRequestMessage : RequestMessage<Theme>
{
}

ObservableRecipient and Messenger

All ViewModels in this scenario inherit from ObservableRecipient  (which was called ViewModelBase in Preview 1) to get access to an IMessenger instance via the Messenger property.

Here’s the ShellViewModel from our sample app. It switches to the default theme, and starts listening for the ThemeRequestMessage (which it answers with the theme) and the ThemeChangedMessage (which updates the theme). In a production app, the theme would probably be persisted in the Settings.

Here’s the definition of the ShellViewModel:

public class ShellViewModel : ObservableRecipient
{
    private Theme _theme = Theme.Default;

    public ShellViewModel()
    {
        Messenger.Register<ThemeRequestMessage>(this, m =>
        {
            m.Reply(_theme); 
        });

        Messenger.Register<ThemeChangedMessage>(this, m =>
        {
            _theme = m.Value;
        });
    }
}

Here’s the call that the theme-aware modules make to fetch the current theme when they are displayed:

_theme = Messenger.Send<ThemeRequestMessage>();

Just like the ShellViewModel they also subscribe to the ThemeChangedMessage to update:

Messenger.Register<ThemeChangedMessage>(this, m =>
   {
      // ...
   }

Sending a message is as easy as calling Send() on the Messenger:

Messenger.Send(new ThemeChangedMessage(_theme));

Under the hood, the messaging infrastructure is using classic .NET Events and Delegates. It’s important that ViewModels unsubscribe their callbacks when they stop listening for messages. The IMessenger interface comes with several Unregister() members, of which this one is the most radical:

Messenger.UnregisterAll(this);

Generally a ViewModel should only send messages and respond to messages when it is in an operational state – i.e. when it is bound to a loaded View. This operational state can be defined with the IsActive property. When the property becomes true, the OnActivated() method is called, so that’s the appropriate place to register your callbacks:

protected override void OnActivated()
{
    base.OnActivated();

    Messenger.Register<ThemeChangedMessage>(this, m =>
    {
        // ...
    });
}

When IsActive becomes false, OnDeactivated() is called. Its default implementation unregisters all callbacks, so in most cases you need to do … nothing.

Here’s how a View in the sample app marks out the operational state of its ViewModel:

private void ColorModule_Loaded(object sender, RoutedEventArgs e)
{
    _colorModuleViewModel.IsActive = true;
}

private void ColorModule_Unloaded(object sender, RoutedEventArgs e)
{
    _colorModuleViewModel.IsActive = false;
}

The default Messenger in the current version of WCT MVVM uses WeakReferences under the hood, so dangling ViewModels with non-unregistered callbacks will eventually be cleaned up by the Garbage Collector. If you’re sure that your app nicely unregisters all messaging handlers, then you may switch to the original (faster!) Messenger that relies on strong references. Here’s how to do this:

public class ShellViewModel : ObservableRecipient
{
    public ShellViewModel() : base(StrongReferenceMessenger.Default)
    {}

    // ...

}

Message Tokens

Some apps may host groups of modules that listen to different variations of messages. Message tokens can be very useful to reduce the communication overhead. Here’s a screenshot of our sample app – the page is based of a real-life scenario and shows four modules:

  • one that broadcasts all pillow-and-blanket war casualties,
  • one that is only interested in pillow victims,
  • one that is only interested in blanket victims, and
  • one that is interested in all casualty messages.

MessengerWithTokenPage

The message itself is empty – it’s good to see that no specific base class is needed:

public class CasualtyMessage // No specific base class needed.
{
}

When a casualty is reported, the broadcast module sends the message, and uses the victim’s party (pillow or blanket) as token:

Messenger.Default.Send<CasualtyMessage, string>(new CasualtyMessage(), "pillow");
// OR
Messenger.Default.Send<CasualtyMessage, string>(new CasualtyMessage(), "blanket");

The next to modules register a callback for CasualtyMessage, but only for their own token:

Messenger.Register<CasualtyMessage, string>(this, "blanket", m => { OnCasualtyMessageReceived(); });

The fourth module registers twice, once for each token. It’s impossible to receive tokenized messages without tokenized registration. So in out sample app, the following will not receive a single message:

// Does not see the messages with a token.
Messenger.Register<CasualtyMessage>(this, m => { OnCasualtyMessageReceived(); });

In our sample app –and many other- Enumations would make an ideal candidate for tokens. After all: the use of hardcoded strings is risky business. Unfortunately token classes are required to implement IEquatable<T> as you see in the definition:

void Register<TMessage, TToken>(object recipient, TToken token, Action<TMessage> action)
            where TMessage : class
            where TToken : IEquatable<TToken>;

This constraint rules out the use of Enumerations. So we created the Party class that looks and feels like an Enum, but implements the interface:

public sealed class Party : IEquatable<Party>
{
    public static readonly Party Pillow = new Party(1, nameof(Pillow));
    public static readonly Party Blanket = new Party(2, nameof(Blanket));

    public string Name { get; private set; }

    public int Id { get; private set; }

    private Party(int id, string name)
    {
        Id = id;
        Name = name;
    }

    public bool Equals(Party other)
    {
        return Id == other.Id;
    }
}

The class can be used as token, instead of String, and we ended up with a much better maintainable code:

Messenger.Register<CasualtyMessage, Party>(this, Party.Blanket, m => { OnCasualtyMessageReceived(); });
Messenger.Default.Send<CasualtyMessage, Party>(new CasualtyMessage(), Party.Pillow);

Dependency Injection

In an MVVM architecture your code base is divided into Views, ViewModels, and Models. Any other reusable code should be grouped into Services. There should be a way to locate and call these services from any component.

MVVMServices

ServiceLocator and DependencyInjection are two (related) common patterns to implement this. These patterns are independent from MVVM, but having a service container available in an MVVM app is useful/required in many cases, like

  • when you want to cache expensive ViewModels instead of recreating them every time a View is instantiated, or
  • when you want to access the messenging infrastructure from a non-ObservableRecipient (like a View).

WCT MVVM comes with an Ioc helper class around the IServiceProvider interface. It supports creating and using a service container, but does not come itself with another service provider. This allows you to stick to your favorite dependency injection framework: Autofac, Ninject, Castle Windsor, Microsoft.Extensions.DependencyInjection or any other. Our sample app uses that last one – very popular in ASP.NET Core:

DependencyInjectionNuGet

Here’s a screenshot of the test page, it demonstrates fetching services from the container, and constructor injection:

InversionOfControlPage

When the app starts up, it registers its services in the container (the Messenger instance, an expensive ViewModel, a service for Logging, and one for Dialogs) and builds the provider:

Ioc.Default.ConfigureServices
        (new ServiceCollection()
            .AddSingleton<IMessenger>(WeakReferenceMessenger.Default)
            .AddSingleton<ILoggingService, DebugLoggingService>()
            .AddSingleton<ColorModuleViewModel>()
            .AddSingleton<ModalView>()
            .BuildServiceProvider()
        );

At runtime, the app can now pull services out of the container with a call to one of the GetService() family members. Here’s how the expensive ViewModel is fetched by a View:

var viewModel = Ioc.Default.GetService<ColorModuleViewModel>();

And here’s a View getting a reference to the Messenger to send a message:

Ioc.Default.GetService<IMessenger>()
	.Send<CasualtyMessage, Party>(new CasualtyMessage(), Party.Pillow);

For the sake of completeness (and unrelated to MVVM) the sample app also demonstrates constructor injection. Here’s how a ViewModel fetches the Logging and the Dialog services when it’s instantiated:

public ColorModuleViewModel(ILoggingService loggingService, ModalView modalView)
{
    // ...
}

Wait, there’s more!

The new WCT MVVM library is familiar, fast, and flexible. But there’s more coming your way than just the NuGet package. In the near future you may also expect:

In mean time our sample UWP app lives here on Github.

Enjoy!

A WinUI 2 Reference App

In this article we’ll walk through a UWP app that heavily relies on WinUI 2. WinUI 2.x (currently version 2.4) is the production-ready predecessor of the ambitious WinUI 3 platform. Version 3.x aims to become -in a not too distant future- the native runtime UI library for all apps that run on Windows – think Win32, UWP, Xamarin, Uno, and React Native.

To prepare ourselves for this WinUI 3.x we created a small but –hopefully- representative UWP app using WinUI 2.4. It fetches and displays live financial data (stock prices and news) from IEX Cloud. Here’s how the app looks like:

PortfolioPage

The app

It is our intention to upgrade this app with every new version of one of the underlying dependencies (specifically WinUI). This will give us an idea on how hard or easy it is to upgrade existing UWP apps to the new platform. In the mean time this reference app may also help you getting started with WinUI in UWP.

The app displays stock market data that is returned from services by IEXCloud, a company that publishes (and we quote from their home page) “institutional grade data, including fundamentals, ownership, international equities, mutual funds, options, real-time data, and alternative data – all in one API“. The data is exposed for free (if you stay below 50.000 core calls per month) but you need to register an account. The app’s Settings page allows you to enter the access keys that come with that account.

To become familiar with the financials API, we started to play around with IEXSharp – an Open Source C# IEXCloud client. We rapidly decided to continue to use the client: it’s complete, easy to work with, and regularly updated.

Here’s the full list of the app’s dependencies, with WinUI v2.5.0 already lurking as a preview:

Dependencies

Let’s dive into the source code.

The Shell Page

The root page of the app hosts the navigation infrastructure and shared UI assets such as the background image. Due to our long history as Prism framework user, we keep calling this type of page the Shell.

Look and feel

The main control in the Shell page is a NavigationView. We’ll use ‘winui’ as namespace alias for the WinUI-specific controls:

xmlns:winui="using:Microsoft.UI.Xaml.Controls"

Here’s the XAML declaration of a NavigationView with its menu on the left, a page header and no back button:

<winui:NavigationView 
	x:Name="NavigationView"
	Header="My stocks"
	IsBackButtonVisible="Collapsed">
	<winui:NavigationView.MenuItems>
		<!-- menu items -->
	</winui:NavigationView.MenuItems>
</winui:NavigationView>

The menu is a list of NavigationViewItem instances. You can nest these to obtain hierarchical navigation. Here’s the XAML definition of the History menu item, demonstrating not only a hierarchical construct but also three types of icons to use (symbol, SVG path, and multicolor bitmap):

<winui:NavigationViewItem Content="History">
    <winui:NavigationViewItem.Icon>
        <FontIcon Glyph="" />
    </winui:NavigationViewItem.Icon>
    <winui:NavigationViewItem.MenuItems>
        <winui:NavigationViewItem Content="AAPL">
            <winui:NavigationViewItem.Icon>
                <PathIcon Data="M32.295, etcetera ..." />
            </winui:NavigationViewItem.Icon>
        </winui:NavigationViewItem>
        <winui:NavigationViewItem Content="MSFT">
            <winui:NavigationViewItem.Icon>
                <BitmapIcon UriSource="/Assets/microsoft.png"
                            ShowAsMonochrome="False" />
            </winui:NavigationViewItem.Icon>
        </winui:NavigationViewItem>
    </winui:NavigationViewItem.MenuItems>
</winui:NavigationViewItem>

Here’s how the Shell page looks like. The Fluent reveal highlight effect is by default enabled in the control, as well as the acrylic background (we did override the BackgroundSource however):

Shell

The emptiness on the right of the menu is a Frame control to host the different user pages.

Navigation

Navigation is very straightforward: each NavigationViewItem keeps the class name of the target content page in its Tag property:

<winui:NavigationViewItem 
	Content="Watchlist"
        Tag="XamlBrewer.UWP.IEXCloud.Sample.Views.WatchListPage" />

The menu conveniently responds to ItemInvoked (click) as well as SelectionChanged (click on a unselected item). Here’s how we Navigate the Frame to the selected content page, and update the Header with the text of the (root node of the) selected item:

private void NavigationView_SelectionChanged(
	WinUI.NavigationView sender, 
	WinUI.NavigationViewSelectionChangedEventArgs args)
{
    if (args.IsSettingsSelected)
    {
        ContentFrame.Navigate(typeof(SettingsPage));
        NavigationView.Header = "Settings";
        return;
    }

    var item = args.SelectedItemContainer as WinUI.NavigationViewItem;

    if (item.Tag != null)
    {
        ContentFrame.Navigate(Type.GetType(item.Tag.ToString()), item.Content);
        NavigationView.Header = sender.SelectedItemsPath().First().Content;
    }
}

The NavigationView makes a distinction between the Settings menu item and the others, in case you would want to open a settings dialog or ye olde Windows 8 SettingsPane. The current guidelines for app settings however specify that “the app settings window should open full-screen and fill the whole window”. Here’s the corresponding from the (by the way excellent) docs:

appsettings-layout-navpane-desktop

It looks like we’re supposed to treat the Settings page not different from the other pages. If that’s the case for all devices, then API members such as IsSettingsSelected are actually obsolete.

NavigationView 2.4 supports a hierarchical menu but is missing a SelectedItems property – representing the path from the root menu to the selected leaf menu. This would be extremely helpful in some common scenario’s, e.g. when

  • you want to display a bread crumb, or
  • your navigation logic requires information from different levels.

In our sample app, the root menu defines the target class for navigation while the leaf menu provides the stock symbol to show. We created a SelectedItemsPath extension method that goes recursively through the selected menu items and returns the full path as a list:

public static List<WinUI.NavigationViewItem> SelectedItemsPath(
	this WinUI.NavigationView navigationView)
{
    var result = new List<WinUI.NavigationViewItem>();
    GetSelectedItems(navigationView.MenuItems, ref result);

    return result;
}

private static void GetSelectedItems(
	IList<object> items, 
	ref List<WinUI.NavigationViewItem> result)
{
    foreach (WinUI.NavigationViewItem item in items)
    {
        if (item.IsSelected)
        {
            result.Insert(0, item);
        }
        else
        {
            if (item.MenuItems?.Count > 0)
            {
                var count = result.Count;
                GetSelectedItems(item.MenuItems, ref result);
                if (result.Count > count)
                {
                    result.Insert(0, item);
                }
            }
        }
    }
}

Teaching tip

All of the content pages visualize stock related data from the IEXCloud services. This requires a (free) account. The Shell page hosts a TeachingTip control to inform you of this:

<winui:TeachingTip 
	x:Name="SettingsTip"
	Title="IEX Cloud Account Required"
	CloseButtonClick="SettingsTip_CloseButtonClick"
	IsOpen="False">
   <winui:TeachingTip.Content>
      <!-- ... -->
   </winui:TeachingTip.Content>
</winui:TeachingTip>

Here’s how it looks like:

TeachingTip

The TeachingTip is only opened when the required tokens are not found in the settings, and its arrow points to the center of the Settings menu item:

private void Shell_Loaded(
	object sender, 
	RoutedEventArgs e)
{
    var settings = new Settings();
    if (String.IsNullOrEmpty(settings.PublishableKey) && 
	String.IsNullOrEmpty(settings.PublishableSandBoxKey))
    {
        SettingsTip.Target = NavigationView.SettingsItem as FrameworkElement;
        SettingsTip.PreferredPlacement = WinUI.TeachingTipPlacementMode.TopLeft;
        SettingsTip.IsOpen = true;
    }
}

When the teaching tip is closed, the app autonavigates to the Settings page.

The Settings page

In the Settings page the user can enter (and test) the different tokens that are required to call the IEXCloud services, and also choose between ‘production’ mode (limited number of calls, but real data) and ‘sandbox’ mode (unlimited calls, but fake data). During development and testing it definitely made sense to activate ‘sandbox’ mode – some of the diagrams eat a lot of data.

Using Observable Settings

A while ago we came across this very elegant ObservableSettings base class for bindable settings in UWP. Here are the settings for our WinUI reference app – two pairs of token keys, and an indicator for sandbox mode:

public class Settings : ObservableSettings
{
    public static Settings Default { get; } = new Settings();

    public Settings()
        : base(ApplicationData.Current.LocalSettings)
    {}

    public string PublishableKey
    {
        get { return Get<string>(); }
        set { Set(value); }
    }

    public string SecretKey
    {
        get { return Get<string>(); }
        set { Set(value); }
    }

    // Similar code for sandbox keys.
   // ...

    [DefaultSettingValue(Value = true)]
    public bool UseSandBox
    {
        get { return Get<bool>(); }
        set { Set(value); }
    }
}

All you need to do is create a Settings instance …

private Settings Settings => new Settings();

… and bind its properties it to the corresponding UI elements:

<TextBox Header="Publishable key"
            Text="{x:Bind Settings.PublishableKey, Mode=TwoWay}" />
<PasswordBox Header="Secret key"
                Password="{x:Bind Settings.SecretKey, Mode=TwoWay}" />

Changes are automatically saved to local settings. How convenient!

Here’s how the Settings page looks like:

Settingspage

Applying ThemeShadow

The panels in the Settings page have shadow around them. ThemeShadow is defined as a shared resource and then applied to each panel:

<Grid Background="Transparent">
    <Grid.Resources>
        <ThemeShadow x:Name="SharedShadow" />
    </Grid.Resources>
    <Grid x:Name="ShadowCatcher"
            Margin="-8" />
    <VariableSizedWrapGrid x:Name="SettingsGrid"
                            Orientation="Horizontal">
        <StackPanel x:Name="ContentGrid"
                    Shadow="{StaticResource SharedShadow}">
            <!-- Production Content -->
        </StackPanel>
        <!-- Gets the animation -->
        <Grid>
            <!-- Casts the shadow -->
            <StackPanel x:Name="SandboxContentGrid"
                        Shadow="{StaticResource SharedShadow}">
                <!-- Sandbox Content -->
            </StackPanel>
        </Grid>
    </VariableSizedWrapGrid>
</Grid>

When the app starts, the panels are lifted by a Translation on the z-axis:

public SettingsPage()
{
    this.InitializeComponent();
    ContentGrid.Translation += new Vector3(0, 0, 6);
    // ...
}

The control underneath the WrapGrid is added to the shadow’s Receivers:

private void SettingsPage_Loaded(object sender, RoutedEventArgs e)
{
    SharedShadow.Receivers.Add(ShadowCatcher);
}

Applying Implicit Transformations

Since smooth transition is not (yet) the default, we let the different panels in the Settings page fluently respond to changes in window size via implicit animations:

SettingsAnimation

We had to add an extra Grid in the panel template to make this happen: adding ThemeShadow and implicit animations to the same control did not work …

Let’s dive into the real content pages now.

The Watchlist page

The Watchlist page hosts a list of high level properties of some stocks. We admit that the current Telerik Rad Datagrid would have been an ideal container for this collection, but we wanted to get some more experience with the WinUI ItemsRepeater. Here’s the XAML structure of the Watchlist page:

<winui:ItemsRepeater x:Name="Quotes">
    <winui:ItemsRepeater.Layout>
        <winui:StackLayout 
	Orientation="Vertical"
	Spacing="20" />
    </winui:ItemsRepeater.Layout>
    <winui:ItemsRepeater.ItemTemplate>
        <DataTemplate x:DataType="response:Quote">
            <!-- template content -->
        </DataTemplate>
    </winui:ItemsRepeater.ItemTemplate>
</winui:ItemsRepeater>

Here’s what the page looks like:

WatchlistPage

There’s nothing more to report on this page, except that –after all those years- we learned that you can format positive, negative, and zero values with a single expression:

<TextBlock Text="{x:Bind sys:String.Format('{0:+0.00%;-0.00%;0%}', changePercent)}" />

And again, we’re using an items repeater to mimic a data grid here. As soon as this app will make the move to WinUI 3, this page will become an ideal host for testing the upcoming Telerik WinUI 3 Data Grid.

The News page

The News page displays the latest news for a stock symbol in an ItemsRepeater. The symbol is taken from the Content of the hierarchical menu item and passed via the navigation logic:

protected override void OnNavigatedTo(NavigationEventArgs e)
{
    base.OnNavigatedTo(e);

    _symbol = e.Parameter.ToString();
}

Here’s how the page looks like:NewsPage

We observed that the RelativePanel is the best host for data templates that contain images with different source sizes.

The History page

The main beef of the History page is a Telerik RadCartesianChart that displays stock price history with

  • the closing price as a LineSeries,
  • a graphical representation of the OHLC (open-high-low-close) data as a CandleStickSeries, and
  • a customized ChartTrackBall that reveals the OHLC data when hovering over the chart with the mouse.

Here’s how the History page looks like:

HistoryPage

The focus of the reference app –and this article- is not on the charts but on WinUI 2, so let’s take a look on the TabView on top of the page. It starts with TabItems for the two predefined stock symbols from the menu (AAPL and MSPF). It also allows to add (removable) custom tabs, and it comes with a TabStripFooter with a (not-yet-implemented) Save button.

Here’s its XAML definition:

<winui:TabView 
	x:Name="SymbolsTab"
	SelectionChanged="SymbolsTab_SelectionChanged"
	AddTabButtonClick="SymbolsTab_AddTabButtonClick"
	TabCloseRequested="SymbolsTab_TabCloseRequested">
    <winui:TabViewItem Header="AAPL"
                        IsClosable="False" />
    <winui:TabViewItem Header="MSFT"
                        IsClosable="False" />
    <winui:TabView.TabStripFooter>
        <!-- Save Button -->
    </winui:TabView.TabStripFooter>
</winui:TabView>

When we navigate to the History page, we pick up the stock symbol – just like in the News page- and then use it to select the appropriate Tab item:

protected override void OnNavigatedTo(NavigationEventArgs e)
{
    base.OnNavigatedTo(e);

    _symbol = e.Parameter.ToString();

    try
    {
        // Synchronize menu to tab.
        SymbolsTab.SelectedItem = SymbolsTab
                                    .TabItems
                                    .Where(ti => (ti as WinUI.TabViewItem).Header.ToString() == _symbol)
                                    .First();
    }
    catch (Exception)
    {
        // Synchronization failed.
        SymbolsTab.SelectedIndex = 0;
    }
}

When the user selects another Tab –that’s a SelectionChanged– we fetch its symbol and refresh the page’s data:

private async void SymbolsTab_SelectionChanged(
	object sender, 
	SelectionChangedEventArgs e)
{
    if (!e.AddedItems.Any())
    {
        return;
    }

    _symbol = (e.AddedItems.First() as WinUI.TabViewItem).Header.ToString();

    // Here's where we should try to sync from tab to menu without starting a loop.
    // ...

    ProgressRing.IsActive = true;

    using (var iexCloudClient = IEXCloudService.GetClient())
    {
        try
        {
            var response = await iexCloudClient.StockPrices.HistoricalPriceAsync(_symbol);
            if (response.ErrorMessage != null)
            {
                HistoricPrices.ItemsSource = null;
                CandleSticks.ItemsSource = null;
            }
            else
            {
                HistoricPrices.ItemsSource = response.Data;
                CandleSticks.ItemsSource = response.Data;
            }
        }
        catch (Exception ex)
        {
            HistoricPrices.ItemsSource = null;
            CandleSticks.ItemsSource = null;
        }
    }

    ProgressRing.IsActive = false;
}

When the ‘plus’ button is hit, we open a small ContentDialog to request a new stock symbol:

HistoryPageAdd

When the user confirms the new stock symbol, we add a new TabViewItem and select it:

if (await dialog.ShowAsync() == ContentDialogResult.Primary)
{
    var newTab = new WinUI.TabViewItem();
    newTab.Header = textBox.Text;
    sender.TabItems.Add(newTab);
    sender.SelectedIndex = sender.TabItems.Count - 1;
}

Here’s another straightforward piece of code. When the user closes one of the custom tabs, we simply remove it:

private void SymbolsTab_TabCloseRequested(
	WinUI.TabView sender, 
	WinUI.TabViewTabCloseRequestedEventArgs args)
{
    sender.TabItems.Remove(args.Item);
}

The Portfolio page

All the controls and techniques that were used in the previous pages were just a warming-up for the Portfolio page. It displays a GridView with (fictional) stock positions: original value, current value, turnover, and a history sparkline (technicaly another Telerik RadChart, this time showing a SplineSeries). Here’s how the page looks like:

PortfolioPage

We’ll focus on the menu on top of the page. It allows to specify the time period for the sparklines. It is another NavigationView instance, this time demonstrating a NavigationViewItemHeader and a different PaneDisplayMode:

<winui:NavigationView 
	x:Name="TopNavigationView"
	PaneDisplayMode="Top"
	IsSettingsVisible="False"
	IsBackButtonVisible="Collapsed"
	SelectionChanged="TopNavigationView_SelectionChanged">
    <winui:NavigationView.MenuItems>
        <winui:NavigationViewItemHeader Content="  Chart period: " />
        <winui:NavigationViewItem Content="1 Month"
                                    Tag="OneMonth" />
        <winui:NavigationViewItem Content="3 Months"
                                    Tag="ThreeMonths"
                                    IsSelected="True" />
        <winui:NavigationViewItem Content="1 Year"
                                    Tag="OneYear" />
    </winui:NavigationView.MenuItems>
</winui:NavigationView>

A navigation view menu comes with its own styling – including a default acrylic background. We needed to override one of its many resources to make that background transparent to blend in the page:

<SolidColorBrush x:Key="NavigationViewTopPaneBackground"
                    Color="Transparent" />

Selecting a menu refreshes the query and the page, just like the tab in the History page.

For the sake of completeness, here’s how the implicit animations look like (this should be the default) on this page:

PortfolioAnimation

The Source

Our WinUI 2 reference app lives here on GitHub. You may expect regular updates to it.

Enoy!

Using LiteDB as a local NoSQL database in UWP

In this article we’ll show how to use a LiteDB database instance to hold local data in an UWP app. LiteDB is a NoSQL database with an API that is inspired by MongoDB. It focuses on storing ‘documents’: loosely typed nested key-value or key-array pairs similar to JSON objects.

Whenever you need to manipulate, query, and persist a collection of dynamic complex objects in your app, a NoSQL database is probably the better choice. A relational database such a SQLite would impose a less convenient, fixed, strongly-typed multi-table schema on your data.

We built a sample UWP app that locally stores some NetFlix series data. The app’s data layer covers

  • creating the local database,
  • running basic CRUD-queries,
  • querying metadata,
  • running advanced queries,
  • handling foreign-key relationships, and
  • dealing with images.

Here’s how that sample app looks like:

FileStorage

Configuring the app

In order to use LiteDB, your app needs a dll that has its source code in a very active GitHub repo and is distributed via NuGet:

LiteDB_NuGet

Our sample UWP app also references Microsoft.UI.XAML a.k.a. WinUI. This is to ensure that we’re using the latest set of XAML controls and styles – it’s where the rounded corners come from.

When you open the app’s bin folder after adding the LiteDB NuGet package, you’ll notice that its dll takes less than half a megabyte – not bad for a whole database engine:

PackageFolder

Creating the database instance

When creating an instance of a LiteDatabase you need to provide a connection string with at least the path to the file where it needs to be stored. For an UWP app, a file in ApplicationData.LocalFolder is an obvious choice:

private static LiteDatabase MyDatabase
{
    get
    {
        var databaseName = "MySeries";
        var filePath = Path.Combine(
            Windows.Storage.ApplicationData.Current.LocalFolder.Path, 
            databaseName);

        return new LiteDatabase(filePath);
    }
}

Here’s how the local data folder looks like after this code has run:

AppDataFolder

Creating Document Collections

The ‘complex dynamic objects’ that the sample app will store, represent NetFlix series with actors, and with seasons that have episodes. We defined these in POCO classes. LiteDB translates these to BsonDocuments (binary JSON) using its object mapping strategy. This strategy contains data type mapping and primary key generation/determination:

Here’s how the sample app’s entities class diagram looks like:

Entities

We’ll store these entities in two collections: one for the Series and one for the Actors. Observe that in a relational database the same schema would

  • require at least five tables – one for each entity and one to hold the n-to-n relationship between Series and Actor, and
  • leave no room for new or unexpected attributes.

Let’s focus on storing documents that represent Series. Here’s the code that

  • (re)creates a LiteCollection to hold the Series instances using DropCollection() and GetCollection<T>(),
  • places an index on it with EnsureIndex(), and
  • populates it with sample data with some Insert() statements,
  • in an Awaitable method:
public static Task Reset()
{
    return Task.Run(() =>
    {
        using (var db = MyDatabase)
        {
            // Remove collection.
            db.DropCollection("series");

            // Get a collection (create it if it doesn't exist)
            var seriesCollection = db.GetCollection("series");

            // Index on Name.
            seriesCollection.EnsureIndex(x => x.Name);

            // Populate.
            foreach (var series in Series.SampleData)
            {
                seriesCollection.Insert(series);
            }

            // ...
        }
    });
}

The database now knows its schema and is ready for some action.

Basic Queries

A call to FindAll() against a document collection returns all instances in it, sorted by their primary key – in our sample app this is the numeric Id field:

public static List SelectAll()
{
    using (var db = MyDatabase)
    {
        var col = db.GetCollection("series");
        return col.FindAll().ToList();
    }
}

A call to Query() returns an ILiteQueryable<T> which exposes LINQ capabilities. Here’s a query that returns all the Series in the database that have a season in a specific year, sorted by Name:

public static List SelectFromYear(int year)
{
    using (var db = MyDatabase)
    {
        var col = db.GetCollection("series");
        return col
            .Query()
            .Where(x => x.Seasons.Select(s => s.Year).Any(y => y == year))
            .OrderBy(x => x.Name)
            .ToList();
    }
}

The call to fetch a single document from a collection is conveniently called FindOne(). It takes a Lambda expression for predicate:

var ac = seriesCollection.FindOne(s => s.Name == "Altered Carbon");

The rest of the API for CRUD queries is also straightforward. Here are some samples of Insert(), Update() and Delete() statements. After each call, we pass back the last Actor in the collection, serialized into a JSON-string. Here’s the code:

public static IEnumerable Crud()
{
    using (var db = MyDatabase)
    {
        var actors = db.GetCollection("actors");
        var ws = new Actor { Name = "Will Smiff" };
        actors.Insert(ws);
        yield return BsonMapper.Global.Serialize(actors.FindAll().Last()).ToString();
        ws.Name = "Will Smith";
        actors.Update(ws);
        yield return BsonMapper.Global.Serialize(actors.FindAll().Last()).ToString();
        actors.Delete(ws.ActorId);
        yield return BsonMapper.Global.Serialize(actors.FindAll().Last()).ToString();
    }
}

Here’s how it looks like in the sample app:

Crud

Querying metadata

Just like in a relational database, you can use the API that queries user content to also fetch system metadata. The database itself is exposed as a collection named ‘$database’. Here’s how to enumerate all its properties – things like name, size, and configuration:

public static IEnumerable SelectDatabaseProperties()
{
    using (var db = MyDatabase)
    {
        var col = db.GetCollection("$database");
        var doc = col.FindAll().ToList().First();
        foreach (var item in doc.Keys)
        {
            yield return $"{item}: {doc[item]}";
        }
    }
}

Here’s how to get the list of all user collections in the database, using the expression-based syntax that we’ll cover shortly:

public static IEnumerable SelectUserCollections()
{
    using (var db = MyDatabase)
    {
        var col = db.GetCollection("$cols");
        //var cols = col.Find(BsonExpression.Create("$.type = 'user'")).ToList();
        var cols = col.Find("$.type = 'user'").ToList();
        var name = "name";
        foreach (var item in cols)
        {
            yield return $"{item[name]}";
        }
    }
}

Here’s the sample app displaying the results:

Metadata

Advanced Queries

Next to its standard LINQ API LiteDB also supports a SQL syntax. Inside the statements you can use expressions that are based on JsonPath – a lightweight (but still powerful) JSON version of XPath.

Here are some sample queries and their corresponding expressions:

The titles of all episodes in a 2020 season $.Seasons[@.Year = 2020].Episodes[*].Title
All season finales (last episode of each season) $.Seasons[*].Episodes[-1]
All episodes with the text ‘fight’ in its description $.Seasons[*].Episodes[@.Description LIKE ‘%fight%’]

Here’s how to write and execute these queries from C#. You Execute() a command and then Read() through its result set, pretty much like in the good old ADO.NET days:

public static IEnumerable Select2020Seasons()
{
    using (var db = MyDatabase)
    {
        // This year's episodes.
        var reader = db.Execute(
            "SELECT $.Name, $.Seasons[@.Year = 2020].Episodes[*].Title AS Episodes 
             FROM series");

        while (reader.Read())
        {
            yield return reader.Current.ToString();
        }
    }
}

public static IEnumerable SelectSeasonFinales()
{
    using (var db = MyDatabase)
    {
        // Season finales per season.
        var reader = db.Execute("
            SELECT $.Name, $.Seasons[*].Episodes[-1] AS SeasonFinales 
            FROM series");

        while (reader.Read())
        {
            yield return reader.Current.ToString();
        }
    }
}

public static IEnumerable SelectFightEpisodes()
{
    using (var db = MyDatabase)
    {
        // Episodes about fighting.
        var reader = db.Execute("
            SELECT $.Name, $.Seasons[*].Episodes[@.Description LIKE '%fight%'] AS Fights 
            FROM series");

        while (reader.Read())
        {
            yield return reader.Current.ToString();
        }
    }
}

Here’s how the results look like in the sample app:

AdvancedQueries

Handling Foreign Key Relationships

Similar to foreign keys in a relational database, DBRef allows you to create references between collections – mainly to avoid duplication. In our sample entity model, each series has a cast of actors. We didn’t want to embed the actor’s details in the series document. While an actor may appear in multiple series, it makes sense to store properties like name and birthday and Academy Award nominations only once.

In the series entity we decorated the property holding the list of actors with a BsonRef attribute:

public class Series
{
    // ...

    [BsonRef("actors")]
    public Actor[] Cast { get; set; }

    // ...
}

The Cast now refers to Actor instances instead of embedding these. The Series instance will store only the identity of each Actor. The identity is the primary key property that you define by using naming conventions or applying the BsonId attribute (more details here):

public class Actor
{
    [BsonId]
    public int ActorId { get; set; }

    // ...
}

When adding actors to the cast of a series, make sure that the Actor documents are first added to the database – they need to have their identity. Then you can insert/update/upsert the Series:

var f1 = seriesCollection.FindOne(s => s.Name == "Formula1: Drive to survive");

var drivers = new List
{
    new Actor { Name = "Alex Albon" },
    new Actor { Name = "Carlos Sainz" },
    new Actor { Name = "Charles Leclerc" }
};
actorsCollection.Upsert(drivers);
f1.Cast = drivers.ToArray();
seriesCollection.Upsert(f1);

That same order applies to data modifications inside a transaction (yes, LiteDB supports transactions):

var rm = seriesCollection.FindOne(s => s.Name == "Rick and Morty");

var cartoons = new List
{
    new Actor { Name = "Rick Sanchez" },
    new Actor { Name = "Morty Smith" },
    new Actor { Name = "Mr. Meeseeks" }
};
rm.Cast = cartoons.ToArray();
db.BeginTrans();
actorsCollection.Upsert(cartoons); // Actors added.
seriesCollection.Upsert(rm);
// actorsCollection.Upsert(cartoons); // Actors not added - even inside a transaction the order is important.
db.Commit();

To join the Series with their Actors, just do the same as in Entity Framework and Include() the related entity. Here’s the LINQ query:

var col = db.GetCollection("series");
return col
    .Query()
    .Include(s => s.Cast)
    .OrderBy(x => x.Name)
    .ToList();

Here’s how the result looks like in the sample app:

Joins

Using FileStorage

LiteDB limits the maximum size for an individual document to 1 MB. For regular data types that should suffice. When you’re dealing with images and streams, you should store these in so-called FileStorage. When you start using the FileStorage feature, LiteDB creates two collections to store the metadata (‘_files’) and the content (‘_chunks’) of the files.

Here’s how to upload a local file (a poster image for a Series) together with its metadata –also a BsonDocument- into FileStorage:

public static string SaveFile(string fileId, string filePath, string series)
{
    using (var db = MyDatabase)
    {
        var fs = db.FileStorage;
        var fileInfo = fs.Upload(fileId, filePath);
        fs.SetMetadata(fileInfo.Id, new BsonDocument { ["series"] = series });

        return $"Imported {fileInfo.Filename} ({fileInfo.Length} bytes) as {fileInfo.Id}.";
    }
}

Here’s the call from the app to store the file:

fileInfo = DataLayer.SaveFile(
        @"$/series/rickandmorty.jpg", 
        path, 
        "Rick and Morty");

Observe that we apply a folder structure to the uploaded file(s), as if they’re saved in a ‘series’ folder.

Here’s how to use the resulting file Id to fetch the contents back:

public static Stream SelectFile(string fileId)
{
    using (var db = MyDatabase)
    {
        var stream = new MemoryStream();
        var fs = db.FileStorage;
        var fileInfo = fs.Download(fileId, stream);
        return stream;
    }
}

Here’s how to fetch all files from a virtual folder:

public static IEnumerable QueryFolder(string folder)
{
    using (var db = MyDatabase)
    {
        var fs = db.FileStorage;
        var infos = fs.Find("_id LIKE @0", folder + "%");
        if (infos != null)
        {
            foreach (var info in infos)
            {
                yield return info.Id;
            }
        }
    }
}

And the call from the client:

var files = DataLayer.QueryFolder(@"$/series/");

Here’s how to fetch files by their meta data:

public static Stream FindFileByMetadata(string series)
{
    using (var db = MyDatabase)
    {
        var stream = new MemoryStream();
        var fs = db.FileStorage;
        var fileInfo = fs.Find(x => x.Metadata["series"] == "Altered Carbon").FirstOrDefault();
        fs.Download(fileInfo.Id, stream);
        return stream;
    }
}

For the sake of completeness, here’s how to remove a file from FileStorage:

public static bool DeleteFile(string fileId)
{
    using (var db = MyDatabase)
    {
        var fs = db.FileStorage;
        return fs.Delete(fileId);
    }
}

Here’s the result from these queries in the sample app:FileStorage

The Verdict

Whenever you need to manipulate, query, and locally persist collections of dynamic complex objects in a .NET based app, it’s worth considering LiteDB. It’s lightweight, it‘s easy to use, and it comes with rich capabilities. It’s also free of charge.

The Source

The sample app lives here on GitHub.

Enjoy and stay healthy!

Getting started with ML.NET in Jupyter Notebooks

For many years .NET developers have been building classic console, desktop or web applications through a stop-modify-recompile-restart cycle. In machine learning and other data-centric tasks this cycle creates so much overhead an delay that it simply doesn’t make sense. Data scientists involved in data analysis, data preparation, and model training prefer a fully interactive environment that allows mixing content, live source code, and program output in a single (web) page that gives immediate feedback when the data or the code changes.

The Jupyter ecosystem provides such an interactive environment, and there’s good news for .NET developers. The list of language kernels supported by Jupyter -including Python, R, Julia, Matlab and many others- has been extended with .NET Core.

The Jupyter Notebook app enables us today to run on-premise interactive machine learning scenarios with ML.NET using C# or F# in a web browser, without bringing software or hardware costs. In this article we try to get you started with developing and running C# machine learning scenarios in Jupyter notebooks. We’re going to assume that you already know the basics of ML.NET.

Installation

Download

There are many ways to install the environment, but you’ll always need these two ingredients:

  • the Jupiter Notebook (a Python program), and
  • dotnet interactive (formerly known as dotnet try), an version of the .NET Core runtime that allows you to run C# and F# as a scripting language.

Here’s probably the easiest way get operational – although leaner procedures may exist:

  1. Install the latest Anaconda:
    Anaconda
  2. Install the latest .NET Core SDK:
    NetSdk

Configure

On the command line, type the following to install dotnet interactive as a global tool:

dotnet tool install -g dotnet-try

Open the Anaconda Prompt and run the following command to register the .NET kernel in Jupyter:

dotnet try jupyter install

Validate

To verify the installation:

  • open the Anaconda3 menu from the start menu,
  • start the Jupyter Notebook app, and
  • press the ‘new’ button in the top right corner to create a new Notebook.

If .NET C# and .NET F# appear in the list of kernels, then you’re ready to go:

NetKernels

You find the most recent installation guide right here.

First steps

Fire it up

Starting a Notebook spawns a language kernel process with two clients

  • a browser with the IDE to edit and run notebooks, and
  • a terminal window displaying the logs of the associated kernel.

Here’ what to expect on your screen:

NotebookAndServer

Hello World

The browser IDE allows you to maintain a list of so-called cells. Each of these that can host documentation (markup or markdown) or source code (in the language of the kernel you selected).

Jupyter Notebooks code cells accept regular C#. Each cell can be individually ran, while the kernel keeps track of the state.

It’s safe to assume that the canonical “Hello World” would look like this:
HelloWorld1

The C# kernel hosts extra functions to control the output of cells, such as the ‘display()’ function:
HelloWorld2

Display() is more than just syntactic sugar for Console.WriteLine(). It can also render HTML, SVG, and Charts. There’s more info on this function right here.

Jupyter Notebook runs a Read-Eval-Print Loop (a.k.a. REPL). Instructions are executed, and expressions are evaluated against the current state that is maintained in and by the kernel. So instead of an instruction to print a string, you can simply type “Hello World” in a cell (without a semicolon). It will we treated as an expression:
HelloWorld3

Loading NuGet packages

Code cells in Jupyter Notebooks can host instructions, expressions, class definitions and functions, but you can also load dll’s from NuGet packages. The #r instruction loads a NuGet package into the kernel:

NuGet

 

#r and using statements do not have to be written at the top of the document – you can just add them whenever you need them in the notebook.

Doing diagrams

One of the NuGet packages that you definitely want to use is XPlot, a data visualization framework written in F#. It allows you create a huge number of chart types delegating the rendering to well-known open source graphing libraries such as Plotly and Google Charts.

Here’s how easy it is to define and render a chart in a C# Jupyter Notebook:

XPlotSample

You’ll find many more examples here.

Running a Canonical Machine Learning Scenario

Over the last couple of months we have been busy creating an elaborated set of machine learning scenario’s in a UWP sample app. In the last couple of weeks we managed to migrate most of these to the Jupyter Notebook, and added some new ones.

Let’s run through one of the new samples. It predicts the quality of white wine based on 11 physicochemical characteristics. The problem is solved as a regression – in our UWP sample app we solved it as a binary and a multiclass classification.

The sample runs a representative scenario of

  • reading the raw data,
  • preparing the data,
  • training the model,
  • calculating and visualizing the quality metrics,
  • calculating and visualizing the feature contributions, and finally
  • predicting.

In this article we won’t dive into the ML.NET details. Here’s how the full scenario looks like in a Jupyter Notebook. :

Regression1

(some helpers were omitted here)Regression2

Regression3

Regression4

Regression5

Regression6

Here’s how the full web page with source and results rendering looks like.

The Jupyter Notebook provides a much more productive environment to create such a scenario than classic .NET application development does. Let’s dive into some of the reasons and take a deeper look into some Jupyter features that make the ML.NET-Jupyter combo attractive to data scientists.

Let’s focus on Data Analysis

Interactive Diagrams

Data analysis requires many types of diagrams, and Jupyter notebooks makes it easy to define and modify these.

Here’s how to import the XPlot NuGet package and render a simple interactive boxplot chart. The rendered diagram highlights the details of the element under the mouse:

InteractiveDiagram

We created a more elaborated example of boxplot analysis right here. Here’s how the resulting diagram looks like:

ComplexBoxPlot

As part of Principal Component Analysis data scientists may want to draw a heat map with the correlation between different features. Here’s how easy it is to create such a chart:

HeatMap

We created a more elaborated example on the well-know Titanic data set right here. This is the resulting diagram:

TitanicHeatMap

These diagrams and many more come out-of-the-box with XPlot.

Object Formatters

Just as developers, data scientists spend most of their time debugging. They continuously need detailed feedback on the work in progress. We already encountered the new display() function that prints the value of an expression. Jupyter notebooks allow you to override the HTML that is printed for a specific class by registering an ObjectFormatter – something that sits between decorating a class with the DebuggerDisplay attribute and writing a custom debugger visualizer.

Let’s write a small –but very useful- example. Here’s how an instance of a ML.NET ConfusionMatrix is displayed by default:
ConfusionMatrix1

That’s pretty confusing, right? [Yes: pun intended!] Let’s fix this.

The few ObjectFormatter examples that we already encountered, were spawning IHtmlContent instances (from ASP.NET Core) that were created and styled through the so-called PocketView API – for which there is no documentation yet. Here’s how to fetch the list of HTML tags that you can create with it, and a small sample on how to apply a style to them:

var pocketViewTagMethods = typeof(PocketViewTags)
    .GetProperties()
    .Select(m => m.Name);
display(pocketViewTagMethods);

var pocketView = table[style: "width: 100%"](tr(td[style:"border: 1px solid black"]("Hello!")));
display(pocketView);

Here’s the result:

PocketView

Here’s how to register a formatter that nicely displays a table with the look-and-feel that a data analyst expects for a (binary!) confusion matrix:

Formatter.Register((df, writer) =>
{
    var rows = new List();

    var cells = new List();
    var n = df.Counts[0][0] + df.Counts[0][1] + df.Counts[1][0] + df.Counts[1][1];
    cells.Add(td[rowspan: 2, colspan: 2, style: "text-align: center; background-color: transparent"]("n = " + n));
    cells.Add(td[colspan: 2, style: "border: 1px solid black; text-align: center; padding: 24px; background-color: lightsteelblue"](b("Predicted")));
    rows.Add(tr[style: "background-color: transparent"](cells));

    cells = new List();
    cells.Add(td[style:"border: 1px solid black; padding: 24px; background-color: #E3EAF3"](b("True")));
    cells.Add(td[style:"border: 1px solid black; padding: 24px; background-color: #E3EAF3"](b("False")));
    rows.Add(tr[style: "background-color: transparent"](cells));

    cells = new List();
    cells.Add(td[rowspan: 2, style:"border: 1px solid black; text-align: center; padding: 24px;  background-color: lightsteelblue"](b("Actual")));
    cells.Add(td[style:"border: 1px solid black; text-align: center; padding: 24px; background-color: #E3EAF3"](b("True")));    
    cells.Add(td[style:"border: 1px solid black; padding: 24px"](df.Counts[0][0]));
    cells.Add(td[style:"border: 1px solid black; padding: 24px"](df.Counts[0][1]));
    rows.Add(tr[style: "background-color: transparent"](cells));

    cells = new List();
    cells.Add(td[style:"border: 1px solid black; text-align: center; padding: 24px; background-color: #E3EAF3"](b("False")));
    cells.Add(td[style:"border: 1px solid black; padding: 24px"](df.Counts[1][0]));
    cells.Add(td[style:"border: 1px solid black; padding: 24px"](df.Counts[1][1]));
    rows.Add(tr(cells));

    var t = table(
        tbody(
            rows));

    writer.Write(t);
}, "text/html");

Here’s how a confusion matrix now looks like – much more intuitive:
ConfusionMatrix2

DataFrame

It’s not always easy to prepare the data for a machine learning pipeline or a diagram, and this is where the new DataFrame API comes in. DataFrame allows you to manipulate tabular in-memory data in a spreadsheet way: you can select, add, and/or filter rows and columns, apply formulas and so on. Here’s how to pull in the NuGet package, and add a custom formatter for the base class. DataFrame is currently only in version 0.2 so you may expect some changes. You may also expect the object formatters to be embedded in future releases:

 

DataFrame1

The DataFrame class knows how to read input data from a CSV:

DataFrame2

In our binary classification sample we use some of the DataFrame methods to replace the “Quality” column holding the taster’s evaluation score (a number from 1 to 10) by a “Label” column with the Boolean indicating whether the wine is good or not (i.e. the score was 6 or higher). Here’s how amazingly easy this is:

var labelCol = trainingData["Quality"].ElementwiseGreaterThanOrEqual(6);
labelCol.SetName("Label");
trainingData.Columns.Add(labelCol);
trainingData.Columns.Remove(trainingData["Quality"]);

Here’s the result (compare the last column with the one in the previous screenshot):

DataFrame3

Since there’s no documentation available yet, we had to dig into the C# source code.

DataFrame is a very promising new .NET API for tabular data manipulation, useful in machine learning and other scenarios.

Sharing is Caring

The source code of our Jupyter notebooks lives here on GitHub (if you like it then you should have put a Star on it). The easiest way to explore the code and the rendered results is via nbViewer.

We also invite you to take a look at these other ML.NET-Jupyter samples from Microsoft and from fellow MVP Alexander Slotte.

Enjoy!

Introducing WinUI ItemsRepeater and Friends

In this article we’ll build a fluent NetFlix-inspired single-page UWP app on top of iTunes Movie Trailers data. The app uses some of the newer WinUI controls such as ItemsRepeater, TeachingTip, CommandBarFlyout, as well as classic XAML elements like MediaPlayerElement, acrylic brushes and animation. 

The app allows you to scroll horizontally and vertically through a list of movies per genre, select a movie, and play its trailer. The app is fully functional in touch, mouse, and keyboard mode. Here’s how it looks like – we decided to call it XamlFlix (the runners-up names were .NET Flix and WinUI tunes):

WinUISample

In a previous blog post we described WinUI as the future of XAML development.  For more details, take a look at this great session on “Windows App Development Roadmap: Making Sense of WinUI, UWP, Win32, .NET” from the Ignite event. For a summary of this session, check Paul Thurrot’s article from which we borrowed the following illustration:

win10-app-platform

Getting the data

Our sample app starts with getting the most recent iTunes Movie Trailers content. It is exposed as a public XML document that looks like this:

iTunesXml

We defined a class to represent a movie, with its title and the whereabouts of its poster image and QuickTime trailer:

public class Movie
{
    public string Title { get; set; }

    public string PosterUrl { get; set; }

    public string TrailerUrl { get; set; }
}

For convenience, movies are grouped by genre. Here’s the corresponding class:

public class Genre
{
    public string Name { get; set; }

    public List<Movie> Movies { get; set; }
}

Using HttpClient.GetStringAsyc() we fetch the XML from the internet. Then we Parse() it into an XDocument. First we get the genre names through a fancy XPATH expression using XPathSelectElements():

using (var client = new HttpClient())
{
    xml = await client.GetStringAsync("http://trailers.apple.com/trailers/home/xml/current.xml");
}

var movies = XDocument.Parse(xml);

var genreNames = movies.XPathSelectElements("//genre/name")
                .Select(m => m.Value)
                .OrderBy(m => m)
                .Distinct()
                .ToList();

Then we use some more of this XPATH and LINQ magic (Eat my shorts, JSON!) to query the movies per genre –a movie may appear in more than one genre- and immediately populate all the element collections:

foreach (var genreName in genreNames)
{
    _genres.Add(new Genre()
    {
        Name = genreName,
        Movies = movies.XPathSelectElements("//genre[name='" + genreName + "']")
            .Ancestors("movieinfo")
            .Select(m => new Movie()
            {
                Title = m.XPathSelectElement("info/title").Value,
                PosterUrl = m.XPathSelectElement("poster/xlarge").Value,
                TrailerUrl = m.XPathSelectElement("preview/large").Value
            })
            //.OrderBy(m => m.Title)
            .ToList()
    });
}

Here’s the DataTemplate that represents a Movie in the app. It’s just an image that we made clickable and focusable by placing it inside a button:

        <DataTemplate x:Key="MovieTemplate"
                      x:DataType="local:Movie">
            <Button Click="Movie_Click"
                    DataContext="{x:Bind}"
                    BorderThickness="0"
                    Padding="0"
                    CornerRadius="0">
                <Image Source="{x:Bind PosterUrl}"
                       PointerEntered="Element_PointerEntered"
                       PointerExited="Element_PointerExited"
                       Height="360"
                       Stretch="UniformToFill"
                       HorizontalAlignment="Stretch"
                       VerticalAlignment="Stretch"
                       ToolTipService.ToolTip="{x:Bind Title}">
                </Image>
            </Button>
        </DataTemplate>

Adding some WinUI components

Let’s bring in some WinUI components to create the visual foundation of the app. These components live in the Microsoft.UI.Xaml NuGet package:

WinUINuGet

After adding the NuGet package, don’t forget to import its styles and other WinUI resources. App.xaml is the best place to do this:

<Application.Resources>
    <ResourceDictionary>
        <ResourceDictionary.MergedDictionaries>
            <XamlControlsResources xmlns="using:Microsoft.UI.Xaml.Controls" />
            <!-- Other merged dictionaries here -->
        </ResourceDictionary.MergedDictionaries>
        <!-- Other app resources here -->
        <x:Double x:Key="ContentDialogMaxWidth">800</x:Double>
    </ResourceDictionary>
</Application.Resources>

Here comes ItemsRepeater

XamlFlix is built around collections: it displays a collection of genres that each display a collection of movies. ItemsRepeater is an ideal host for this: it’s a WinUI element that is designed to be used inside custom controls that display collections. It does not come with a default UI and it provides no policy around focus, selection, or user interaction. It supports virtualization so it allows you to deal with very large collections out of the box.

The main beef of the app’s UI are ItemsRepeater controls: there’s one that repeats the genres vertically, and genre comes a horizontal repeater on the movies.

We started with defining two  reusable StackLayout resources: 

<controls:StackLayout x:Key="HorizontalStackLayout"
                        Orientation="Horizontal" />
<controls:StackLayout x:Key="VerticalStackLayout"
                        Orientation="Vertical"
                        Spacing="0" />

An ItemsRepeater is a data-driven panel that does not come with its own scrolling infrastructure, so you may need to wrap it in a ScrollViewer. Here’s the declaration of the repeater for the Genre instances:

<ScrollViewer VerticalScrollBarVisibility="Auto"
                VerticalScrollMode="Auto"
                HorizontalScrollMode="Disabled"
                Grid.Row="2">
    <controls:ItemsRepeater x:Name="GenreRepeater"
                            ItemTemplate="{StaticResource GenreTemplate}"
                            Layout="{StaticResource VerticalStackLayout}"
                            HorizontalAlignment="Stretch"
                            VerticalAlignment="Stretch" />
</ScrollViewer>

Each genre has its name displayed on top of a horizontal list of Movie instances – again implemented as an ItemsRepeater inside a ScrollViewer:

<DataTemplate x:Key="GenreTemplate"
                x:DataType="local:Genre">
    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition Height="auto" />
            <RowDefinition Height="*" />
        </Grid.RowDefinitions>
        <Image VerticalAlignment="Stretch"
                HorizontalAlignment="Left"
                Margin="6 0 0 0"
                Width="52"
                Source="/Assets/FilmStrip.png"
                Stretch="Fill"
                Grid.RowSpan="2" />
        <TextBlock Foreground="Silver"
                    FontSize="36"
                    Margin="66 0 0 0"
                    Text="{x:Bind Name}" />
        <ScrollViewer HorizontalScrollBarVisibility="Visible"
                        HorizontalScrollMode="Enabled"
                        VerticalScrollMode="Disabled"
                        Margin="66 0 0 0"
                        Grid.Row="1">
            <controls:ItemsRepeater ItemsSource="{x:Bind Movies}"
                                    ItemTemplate="{StaticResource MovieTemplate}"
                                    Layout="{StaticResource HorizontalStackLayout}" />
        </ScrollViewer>
    </Grid>
</DataTemplate>

After the content was loaded, we set its items source programmatically:

GenreRepeater.ItemsSource = Genres

The page then looks like this:

ItemsRepeater

Let’s move our focus to the behavior now.

On a touch screen the page reacts appropriately to horizontal and vertical panning. Mouse scrolling is a bit problematic: the movie repeaters take almost all the screen area and hence they get the mouse input and only trigger horizontal scrolling. So we decided to place a film strip on the left. It creates an area that facilitates vertical scrolling through the genres.

Hello TeachingTip

The mouse behavior is not very intuitive for first-time users, so we added a TeachingTip –a WinUI Flyout with an arrow- to draw the user’s attention to the film strip on the left.

Here’s its XAML declaration:

<controls:TeachingTip x:Name="ScrollTeachingTip"
                        Target="{x:Bind FakeTarget}"
                        Title="I looks like you are trying to scroll."
                        Subtitle="With the mouse on this filmstrip you'll scroll vertically."
                        PreferredPlacement="Right"
                        IsOpen="False">
    <controls:TeachingTip.IconSource>
        <controls:SymbolIconSource Symbol="Sort" />
    </controls:TeachingTip.IconSource>
</controls:TeachingTip>

The TeachingTip needs to point to the film strip as its a Target. While visually the strip looks like a whole, it’s actually composed of multiple images: it’s repeated per genre. That makes it hard to use as target for the teaching tip – at least declaratively.

We provided an alternative Target by placing a UI-less control center left of the page:

<ContentControl x:Name="FakeTarget"
                VerticalAlignment="Center"
                HorizontalAlignment="Left"
                Width="40"
                Grid.Row="2" />

Here’s the result, the teaching tip points right to the middle of the film strip:

TeachingTip

We couldn’t resist creating a version with an image of Clippy as HeroContent:

TeachingTipClippy

It doesn’t make sense to pop-up the tip while the movie images are still loading, so we delayed its appearance:

await Task.Delay(2000);
ScrollTeachingTip.IsOpen = true;

We covered both touch and mouse input for navigation. Let’s now take a look at keyboard input.

To our great surprise we observed that the page already behaved properly on keyboard input. Here’s how navigation with the arrow keys looks like – not bad for a lightweight control that does not support selection. The visual border around the movie is the focus rectangle of the button in the Movie data template:

ArrowNavigation

To highlight the movie under the mouse cursor, our first attempt was a ToolTip with the title. It’s hardly visible and it feels like HTML. Here’s an example (hint: it’s on Spice in disguise):

Tooltip

We went for a more fluent experience. When digging through the awesome XAML Controls Gallery we found this nice sample that animates the size of a button on hover (don’t try it out here, it’s just a screenshot):

XamlControlsGalleryAnimation

We decided to subtly grow and shrink the image under the mouse cursor on hovering. It only required a straight copy/paste from the gallery sample code. The code uses the Compositor to hook a SpringVector3NaturalMotionAnimation to the scale of the image. We increase it by 2% on PointerEntered and then shrink back on PointerExited:

private Compositor _compositor = Window.Current.Compositor;
private SpringVector3NaturalMotionAnimation _springAnimation;

private void CreateOrUpdateSpringAnimation(float finalValue)
{
    if (_springAnimation == null)
    {
        _springAnimation = _compositor.CreateSpringVector3Animation();
        _springAnimation.Target = "Scale";
    }

    _springAnimation.FinalValue = new Vector3(finalValue);
}

private void Element_PointerEntered(object sender, PointerRoutedEventArgs e)
{
    // Scale up a little.
    CreateOrUpdateSpringAnimation(1.02f);

    (sender as UIElement).StartAnimation(_springAnimation);
}

private void Element_PointerExited(object sender, PointerRoutedEventArgs e)
{
    // Scale back down.
    CreateOrUpdateSpringAnimation(1.0f);

    (sender as UIElement).StartAnimation(_springAnimation);
}

Here’s how the result looks like – the real animation is a lot smoother that the animated gif suggests:

ScaleAnimation

That looks decent, no? Let’s add some more functionality now.

Introducing CommandBarFlyout

The CommandBarFlyout is another WinUI control that lives up to its name: it is literally a CommandBar in a Flyout. It groups AppBarButton instances in primary and secondary commands that are applicable to a specific UI element.

XamlFlix uses a CommandBarFlyout to display the possible actions for the selected movie: play, buy, rate, …. For the sake of simplicity we hooked them all to the same event handler: whatever menu you select, you always get to play the movie trailer.

Here’s the declaration of the movie menu:

<controls:CommandBarFlyout x:Name="MovieCommands"
                            Placement="Right">
    <AppBarButton Label="Play"
                    Icon="Play"
                    ToolTipService.ToolTip="Play"
                    Click="Element_Click" />
    <AppBarButton Label="Info"
                    Icon="List"
                    ToolTipService.ToolTip="Info"
                    Click="Element_Click" />
    <AppBarButton Label="Download"
                    Icon="Download"
                    ToolTipService.ToolTip="Download"
                    Click="Element_Click" />
    <controls:CommandBarFlyout.SecondaryCommands>
        <AppBarButton Label="Buy"
                        Click="Element_Click" />
        <AppBarButton Label="Rate"
                        Click="Element_Click" />
    </controls:CommandBarFlyout.SecondaryCommands>
</controls:CommandBarFlyout>

The control comes with several FlyoutShowOptions to configure position, placement, and behavior. Here we define it to appear on top of the image, in an expanded state and grabbing the focus. The ShowAt() method is called when a button in a movie data template is clicked. It opens the menu for the targeted UI element:

private void Movie_Click(object sender, RoutedEventArgs e)
{
    FlyoutShowOptions options = new FlyoutShowOptions();
    options.ShowMode = FlyoutShowMode.Standard;
    options.Placement = FlyoutPlacementMode.Top;

    MovieCommands.ShowAt(sender as FrameworkElement, options);
}

This is how the menu looks like in XamlFlix (on top of Top Gun):

CommandBarFlyout

Here’s MediaPlayerElement

For playing the movie trailer there are not too much options. We went for the classic UWP MediaPlayerElement and placed it in a ContentDialog:

<ContentDialog x:Name="MediaPlayerDialog"
                Closing="MediaPlayerDialog_Closing"
                CloseButtonText="Close">
    <StackPanel>
        <TextBlock x:Name="TitleText"
                    Margin="0 0 0 20" />
        <MediaPlayerElement x:Name="Player"
                            AreTransportControlsEnabled="True"
                            MinWidth="600"
                            AutoPlay="True" />
    </StackPanel>
</ContentDialog>

The dialog needs more space than the maximum of 548 pixels that a popup normally gets in UWP, so you have to override ContentDialogMaxWidth in app.xaml:

<x:Double x:Key="ContentDialogMaxWidth">800</x:Double>

When an element of the command bar flyout is clicked, we open the dialog and create a MediaSource from the trailer’s URL. We need to Hide() the command bar flyout, since it lives in the same layer as the dialog:

private async void Element_Click(object sender, RoutedEventArgs e)
{
    // It stays on top of the dialog.
    MovieCommands.Hide();

    var movie = (sender as FrameworkElement)?.DataContext as Movie;
    var source = MediaSource.CreateFromUri(new Uri(movie.TrailerUrl));

    TitleText.Text = movie.Title;
    Player.Source = source;
    await MediaPlayerDialog.ShowAsync();
}

private void MediaPlayerDialog_Closing(ContentDialog sender, ContentDialogClosingEventArgs args)
{
    // Prevent the player to continue playing.
    Player.Source = null;
}

Here’s the resulting UI:

MediaPlayer

The styles of the current WinUI v2.2 do not apply to ContentDialog yet, that’s why the dialog itself and its buttons have no rounded corners. Don’t worry: this will change in WinUI v2.3.

Let’s sprinkle some Acrylic

These days, fluent apps need a touch of acrylic material. The XamlFlix UI area is almost entirely covered with movie poster images, so we decided to use an AcrylicBrush for the whole background:

<Page.Background>
    <AcrylicBrush BackgroundSource="HostBackdrop"
                    TintColor="{ThemeResource SystemColorBackgroundColor}"
                    TintOpacity="0.9"
                    FallbackColor="{ThemeResource ApplicationPageBackgroundThemeBrush}" />
</Page.Background>

We blended that background into the title bar:

private void ExtendAcrylicIntoTitleBar()
{
    CoreApplication.GetCurrentView().TitleBar.ExtendViewIntoTitleBar = true;
    ApplicationViewTitleBar titleBar = ApplicationView.GetForCurrentView().TitleBar;
    titleBar.ButtonBackgroundColor = Colors.Transparent;
    titleBar.ButtonInactiveBackgroundColor = Colors.Transparent;
}

In this setting Windows only draws the system buttons, and makes you responsible for displaying the app title. Here’s an appropriate (and reusable) window title declaration:

<TextBlock xmlns:appmodel="using:Windows.ApplicationModel"
            Text="{x:Bind appmodel:Package.Current.DisplayName}"
            Style="{StaticResource CaptionTextBlockStyle}"
            IsHitTestVisible="False"
            Margin="12 8 0 0" />

The Code

The XamlFlix sample app lives here on GitHub. For more WinUI samples also check the XAML Controls Gallery and the Windows Community Toolkit Sample App in the Store (sources are also on GitHub).

Building explainable Machine Learning models with ML.NET in UWP

In this article we’ll describe how to design and build explainable machine learning models with ML.NET. We used a UWP app to host these models, and OxyPlot to create the diagrams to visualize the importance of features for a model and/or a prediction.

We will discuss the following three tasks that you can execute with the ML.NET Explainability API:

  • calculate the feature weights for a linear regression model,
  • calculate the feature contributions for a prediction, and
  • calculate Permutation Feature Importance (PFI) for a model.

The two corresponding pages it the UWP sample app look like this:

PredictionFeatureContribution

PfiCalculation

In many Machine Learning scenarios it is not only important to have a model that is accurate enough, but also to have one that is interpretable. In Health Care or Finance, models should be transparent enough to explain why they made a particular prediction. Conclusions like “You are 75% healthy” or “Your loan application was not approved” may require a better excuse than “because the computer says so”. Model explainability –knowing the importance of its features- is not only useful in justifying predictions, but also in refining the model itself – through feature selection. Investigating explainability allows you to remove features that are not significant for a model, so that you probably end up with shorter training times and less resource intensive prediction making.

The code in this article is built around the 11-features white wine quality dataset that we already covered several times in this article series. We solved it as a binary classification problem and as a multiclass classification problem for AutoML. This time we finally approach it the correct way: as a regression problem where the outcome is a (continuous) numerical value – the score– within a range. We’ll use an Stochastic Dual Coordinate Ascent regression trainer as the core of the model pipeline. Let’s first build and train that model.

Here’s the class to store the input data:

public class FeatureContributionData
{
    [LoadColumn(0)]
    public float FixedAcidity;

    [LoadColumn(1)]
    public float VolatileAcidity;

    [LoadColumn(2)]
    public float CitricAcid;

    // More features ...

    [LoadColumn(9)]
    public float Sulphates;

    [LoadColumn(10)]
    public float Alcohol;

    [LoadColumn(11), ColumnName("Label")]
    public float Label;
}

As in any ML.NET scenario we need to instantiate an MLContext:

public MLContext MLContext { get; } = new MLContext(seed: null);

Here’s the code to build and train the model. We’ll refine it later in several ways to enhance its explainability:

private IEnumerable<FeatureContributionData> _trainData;
private IDataView _transformedData;
private ITransformer _transformationModel;
private RegressionPredictionTransformer<LinearRegressionModelParameters> _regressionModel;

public List<float> BuildAndTrain(string trainingDataPath)
{
    IEstimator<ITransformer> pipeline =
        MLContext.Transforms.ReplaceMissingValues(
            outputColumnName: "FixedAcidity",
            replacementMode: MissingValueReplacingEstimator.ReplacementMode.Mean)
        .Append(MLContext.Transforms.Concatenate("Features",
            new[]
            {
                "FixedAcidity",
                "VolatileAcidity",
                "CitricAcid",
                "ResidualSugar",
                "Chlorides",
                "FreeSulfurDioxide",
                "TotalSulfurDioxide",
                "Density",
                "Ph",
                "Sulphates",
                "Alcohol"}))
        .Append(MLContext.Transforms.NormalizeMeanVariance("Features"));

    var trainData = MLContext.Data.LoadFromTextFile<FeatureContributionData>(
            path: trainingDataPath,
            separatorChar: ';',
            hasHeader: true);

    // Keep the data avalailable.
    _trainData = MLContext.Data.CreateEnumerable<FeatureContributionData>(trainData, true);

    // Cache the data view in memory. For an iterative algorithm such as SDCA this makes a huge difference.
    trainData = MLContext.Data.Cache(trainData);

    _transformationModel = pipeline.Fit(trainData);

    // Prepare the data for the algorithm.
    _transformedData = _transformationModel.Transform(trainData);

    // Choose a regression algorithm.
    var algorithm = MLContext.Regression.Trainers.Sdca();

    // Train the model and score it on the transformed data.
    _regressionModel = algorithm.Fit(_transformedData);

    // ...
}

Which features are hot, and which are not

Feature Weights in linear models

Machine Learning has several techniques for calculating how important features are in explaining/justifying the prediction. When your main algorithm is a linear classifier (e.g. linear regression) then it’s relatively easy to calculate feature contributions. The prediction is the linear combination of the features values, weighted by the model coefficients. So at model level there’s already a notion of feature contribution. In ML.NET these Weights are found in the LinearModelParameters class. This is the base class for all linear model parameter classes, like LinearRegressionModelParameters (used by SDCA) and OlsModelParameters (used by the Ordinary Least Squares trainer).

For any linear model, you can fetch the overall feature weights like this:

// Return the weights.
return _regressionModel.Model.Weights.ToList();

Here’s how the weights for the wine quality model look like in a diagram:

ModelFeatureContribution

The alcohol feature seems to dominate this particular model. Its weight is positive: a higher alcohol percentage results in a higher appreciation score. We may have just scientifically proven that alcohol is important in wine quality perception. In other news there are also at least three characteristics that could be ignored in this model – and maybe up to seven.

Feature Contribution Calculation

A second component in ML.NET’s Explainability API is a calculator that computes the list of feature contributions for a specific prediction: the FeatureContributionCalculator. Just like the overall feature weights from the previous section, the calculated feature contributions can be positive or negative. The calculator works for a large number of regression, binary classification, and ranking algorithms. The documentation page on the FeatureContributionCalculatingEstimator class contains a list of all compatible trainers. It includes

  • all linear models – because they inherently come with feature weights,
  • all Generalized Additive Models (GAM) – because their squiggly, wiggly shape functions are created by combining linear models, and
  • all tree based models – because they can calculate feature importance based on the values in the decision paths in the tree(s).

Thanks to ML.NET’s modular approach it’s easy to plug a feature contribution calculator into a model pipeline, even if the model is already trained. Here’s how we did this in the sample app.

First we extended the prediction data structure (which originally had only the Score) with an array to hold the contribution values for each feature:

public class FeatureContributionPrediction : FeatureContributionData
{
    public float Score { get; set; }

    public float[] FeatureContributions { get; set; }
}

Using the CalculateFeatureContribution() method we created and estimator, and trained it on just one input sample to become a transformer. This adds the calculation to the pipeline and the feature contributions to the output schema. The transformer was then appended to the trained model.

Here’s how that looks like in C#:

private PredictionEngine<FeatureContributionData, FeatureContributionPrediction> _predictionEngine;

public void CreatePredictionModel()
{
    // Get one row of sample data.
    var regressionData = _regressionModel.Transform(MLContext.Data.TakeRows(_transformedData, 1));

    // Define a feature contribution calculator for all the features.
    // 'Train' it on the sample row.
    var featureContributionCalculator = MLContext.Transforms
        .CalculateFeatureContribution(_regressionModel, normalize: false)
        .Fit(regressionData);

    // Create the full transformer chain.
    var scoringPipeline = _transformationModel
        .Append(_regressionModel)
        .Append(featureContributionCalculator);

    // Create the prediction engine.
    _predictionEngine = MLContext.Model.CreatePredictionEngine<FeatureContributionData, FeatureContributionPrediction>(scoringPipeline);
}

For testing the model, we didn’t dare to bother you with an input form for 11 features. Instead we added a button that randomly fetches one of the almost 4000 training samples and calculates the score and feature contributions for that sample.

Here’s the code behind this button:

public FeatureContributionPrediction GetRandomPrediction()
{
    return _predictionEngine.Predict(_trainData.ElementAt(new Random().Next(3918)));
}

Here’s how the results look like in a plot. This is an example of a linear model, so we can compare the overall model weights with the feature contributions for the specific prediction. We did not normalize the results in the feature contribution calculator configuration (the second parameter in the call) to keep these in the same range as the model weights:

PredictionFeatureContribution

Feature Contribution Calculation also works for models that don’t come with overall weights. Here’s how the results look like for a LightGBM regression trainer – one of the decision tree based algorithms:

LightGbmFeatureContribution

Check the comments in the sample app source code for its details. Also in comments, is the code for using an Ordinary Least Squares linear regression trainer. This 18th century algorithm is one is even more biased towards alcohol than our initial SDCA trainer.

For yet another example, check this official sample that adds explainability to a model covering the classic Taxi Fare Prediction scenario.

Permutation Feature Importance

The last calculator in the ML.NET Explainability API is the most computationally expensive one. It calculates the Permutation Feature Importance (PFI). Here’s how PFI calculation works:

  1. A baseline model is trained and its main quality metrics (accuracy, R squared, …) are recorded.
  2. The values of one feature are shuffled or partly replaced by random values – to undermine the relationship between the feature and the score.
  3. The modified data set is passed to the model to get new predictions and new values for the quality metrics. The result is expected to be worse than the baseline. If your model got better on random data then there was definitely something wrong with it.
  4. The feature importance is calculated as the degradation of a selected quality metric versus the one in the baseline.
  5. Steps 2, 3, and 4 are repeated for each feature so that the respective degradations can be compared: the more degradation for a feature, the more the model depends on that feature.

In ML.NET you fire up this process with a call to the PermutationFeatureImportance() method. You need to provide the model, the baseline data set, and the number of permutations – i.e. the number of feature values to replace:

// ... (same as the previous sample)

// Prepare the data for the algorithm.
var transformedData = transformationModel.Transform(trainData);

// Choose a regression algorithm.
var algorithm = MLContext.Regression.Trainers.Sdca();

// Train the model and score it on the transformed data.
var regressionModel = algorithm.Fit(transformedData);

// Calculate the PFI metrics.
var permutationMetrics = MLContext.Regression.PermutationFeatureImportance(
    regressionModel, 
    transformedData, 
    permutationCount: 50);

The call returns an array of quality metric statistics for the model type. For a regression model it’s an array of RegressionMetricStatistics instances – each holding summary statistics over multiple observations of RegressionMetrics. In the sample app we decided R Squared to be the most important quality metric. So the decrease in this value determines feature importance.

We defined a data structure to hold the result:

public class FeatureImportance
{
    public string Name { get; set; }
    public double R2Decrease { get; set; }
}

The list of feature importances is created from the result of the call and visualized in a diagram. Since we decided that R Squared is the main quality metric for our model, we used RSquared.Mean (the mean decrease in R Squared during the calculation) as the target value for feature importance:

for (int i = 0; i < permutationMetrics.Length; i++)
{
    result[i].R2Decrease = permutationMetrics[i].RSquared.Mean;
}

Here’s how the plot looks like:

PfiCalculation

 

We used the same regression algorithm –SDCA- as in the previous samples, so the addiction to alcohol (the feature!) should not come as a surprise anymore.

Here’s a detailed view on one of the metric statistics when debugging:

PrivateMetricsStatistics

That is an awful lot of information. Unfortunately the current API keeps most of the details private – well, you can always use Reflection. Access to more details would allow us to create more insightful diagrams like this one (source). It does not only show the mean, but also min and max values during the calculation:

importance-bike-1

For another example of explaining model predictions using Permutation Feature Importance, check this official ML.NET sample that covers house pricing.

The Source

In this article we described three components of the ML.NET Explainability API that may help you to create machine learning models that have the ability clarify their predictions. The UWP sample app hosts many more ML.NET scenario’s. It lives here on GitHub.

Enjoy!

A Lap around the WinUI TeachingTip Control

In this article we will run through a couple of scenarios using the UWP TeachingTip control. This relatively new control is an animated flyout that according to the documentation “draws the user’s attention on new or important updates and features, remind a user of nonessential options that would improve the experience, or teach a user how a task should be completed”.

The high quality of this official documentation made us decide to skip a high-level introduction and to immediately expose the TeachingTip control to some more challenging ‘enterprise-ish’ scenarios. Here are the things you can expect us to cover in this article:

  • programmatically creating a TeachingTip,
  • precision targeting a XAML control,
  • state management,
  • auto-hiding a TeachingTip on time-out and navigation, and
  • building an inherited control.

We also added a sample that expresses our concerns on the light dismiss behavior, and identified an interesting use case for the TeachingTip as ‘Form Field Wizard’. Since all the official samples use the light theme, we decided to go for dark – with a custom highlighted border.

As usual we built a small sample app, this is how it looks like:

HomePage

The TeachingTip control is shipped as a part of the Windows UI Library (a.k.a. WinUI) which is mostly known for its down-level compatibility versions of the official native UWP controls. The Windows UI Library also hosts brand new controls that aren’t shipped as part of the default Windows platform. Here are some of the controls that were shipped with WinUI 2.1 in April 2019:

The latest WinUI 2.2 release from September 2019 introduced a very promising TabView control – and slightly rounded corners for all controls. In the beginning of 2020 also an Edge Chromium based WebView is expected. 

Getting started with the Windows UI Library

The WinUI toolkit is available as NuGet packages that can be added to any existing or new project. Make sure not to forget to add the Windows UI Theme Resources to your App.xaml resources – as explained in the Getting Started guide. Here’s how this is done in the sample app:

<Application.Resources>
    <ResourceDictionary>
        <ResourceDictionary.MergedDictionaries>
            <!-- Win UI Controls -->
            <XamlControlsResources xmlns="using:Microsoft.UI.Xaml.Controls" />
            <!-- Other Dictionaries and Styles -->
            <!-- ... -->
        </ResourceDictionary.MergedDictionaries>
    </ResourceDictionary>
</Application.Resources>

WinUI exposes an extended API and a set of new controls. Currently these controls live in the Microsoft.UI.Xaml namespace to distinct them from the classic (future legacy?) UWP controls in Windows.UI.Xaml. IntelliSense will reveal some of the some doubles:

ControlsList

According to the roadmap WinUI’s intention is to decouple the entire XAML stack from the rest of UWP, to ship it as a NuGet package. UWP developers will observe that existing UWP XAML APIs (currently shipping as part of the OS) will fade out to no longer receive new feature updates – only security updates and critical fixes. Other UWP features such as application and security model, media pipeline, shell integrations, and broad device support will continue to evolve. WPF, Windows Forms and MFC developers will observe that WinUI is going to absorb (or render obsolete?) XAML Islands, a feature that is currently available through the Windows Community Toolkit.

Here’s an illustration of the current and target architecture:

roadmap_winui2

roadmap_winui3

The TeachingTip Control

A teaching tip is a control that provides contextual information, often used for informing, reminding, and teaching users about important and new features. Visually the TeachingTip is a Flyout with a Tail (an arrow pointing to its Target) and smooth opening and closing animations (love these!). The Teaching Tip is truly an Open Source control: it has been on GitHub from its proposal and specifications to its C++ source code. The TeachingTip documentation will get you started on the fly, and the API is simple. It’s safe to infer from the state of the related GitHub Issues that the control is new but stable.

It’s time to make our hands dirty.

Programmatically instantiating a TeachingTip

Most of our sample pages come with a Home page that describes the sample, and a Main page with all the action. One of the TeachingTip’s missions is to get the user started or to point out new features, so we decided to immediately open one on the Home page when the app starts. Here’s the C# code to define a TeachingTip programmatically, and hook it into the visual tree:

_mainMenuTeachingTip = new TeachingTip
{
    Target = glyph,
    Title = "Welcome",
    Content = "The Main page is where all the action is.",
    HeroContent = new Image
    {
        Source = new BitmapImage(new Uri("ms-appx:///Assets/MindFlayer.jpg"))
    },
    PreferredPlacement = TeachingTipPlacementMode.BottomRight,
    IsOpen = true,
    BorderThickness = new Thickness(.5),
    BorderBrush = new SolidColorBrush(Colors.DarkRed)
};
_mainMenuTeachingTip.Closed += MainMenuTeachingTip_Closed;

contentGrid.Children.Add(_mainMenuTeachingTip);

Precision Targeting

A TeachingTip is a Flyout, with an optional arrow that points to its Target. The positioning sample on the Main page displays what the impact is of PreferredPlacement for targeted and non-targeted teaching tips. Here’s a screenshot of it:

Positioning

Keep in mind that the TeachingTip will find its own place when there is not enough room for it. Here’s what happens when we specify LeftBottom when there’s no space there:

PositioningOverride

If you aim for visual perfection, you have to identify the exact Target that you want your TeachingTip to point its arrow to. In our sample app we wanted to target a menu item – a templated list item. Targeting the list item itself produced a rather fuzzy result, because of the default position calculation and the fact that TeachingTip never touches its Target. So we decided to target the icon inside the menu item instead of the menu item itself.

In most samples the Target is declared in XAML referencing a static control on the page. In most real life scenarios you will probably programmatically dig through a complex dynamic XAML structure, relying on things like the VisualTreeHelper, the GetChild() and FindName() methods, and looking up a XAML element that corresponds to an item in a list, with ContainerFromIndex().

Here’s how the sample app looks up the menu icon to be set as Target:

// Find the Main menu item and the Content grid.
var shell = (Window.Current.Content as Frame)?.Content as FrameworkElement;
var contentGrid = shell?.FindName("ContentGrid") as Grid;
var menu = shell?.FindName("Menu") as ListView;
var mainPageMenu = menu?.ContainerFromIndex(1) as ListViewItem;

// Find the Icon.
var itemsPresenter = ((VisualTreeHelper.GetChild(mainPageMenu, 0)) as FrameworkElement);
var stackPanel = ((VisualTreeHelper.GetChild(itemsPresenter, 0)) as FrameworkElement);
var glyph = stackPanel?.FindName("Glyph") as FrameworkElement;

State management

We assume that you liked the animated Teachingtip when opening the app for the first time, and perhaps the second time too. Just be aware that automatically opening TeachingTips becomes boring and annoying very rapidly to the end user. It really makes sense to let your remember that it showed a tip a couple of times, and then hide it forever.

To enhance this type of annoyment we made a second TeachingTip that opens when the first one raises its Closed event:

ReplayTip

We remember whether or not the TeachingTip has been displayed in an entry in the LocalSettings of the current ApplicationData:

public static void DisplayReplayButtonTip()
{
    var localSettings = Windows.Storage.ApplicationData.Current.LocalSettings;

    if (localSettings.Values["replayButtonTeachingTipDisplayed"] != null &&
        localSettings.Values["replayButtonTeachingTipDisplayed"].ToString() == "True")
    {
        return;
    }

    // ...

    _replayButtonTeachingTip = new TeachingTip
    {
        Target = replayButton,
        // ...
    };

    localSettings.Values["replayButtonTeachingTipDisplayed"] = "True";

    (homePage.Content as Grid).Children.Add(_replayButtonTeachingTip);
}

Here’s the code to remove the state again, e.g. in case of a reset-to-factory-settings situation:

private void ResetButton_Click(object sender, Windows.UI.Xaml.RoutedEventArgs e)
{
    var containerSettings = (ApplicationDataContainerSettings)ApplicationData.Current.LocalSettings.Values;
    var keys = containerSettings.Keys;
    foreach (var key in keys)
    {
        ApplicationData.Current.LocalSettings.Values.Remove(key);
    }

    ResetButtonTeachingTip.IsOpen = true;
}

A Word about Light-Dismiss

The IsLightDismissEnabled property makes an open teaching tip dismiss when the user scrolls or interacts with other elements of the application. In this mode, the PopupRoot –the top layer that hosts Flyouts, Dialogs, and TeachingTips- invisibly covers the whole page and swallows all UI events. Except for the system buttons in the top right corner, none of the app’s buttons will respond to a click or tap – including the Back button, the Hamburger menu and all Navigation items in our sample app.

We’re not a big fan of this mode. If you want to see for yourself, there’s one light-dismiss enabled TeachingTip in the sample app:

LightDismiss

Auto-Hiding a TeachingTip Control

It makes sense for a TeachingTip to hide itself when it’s no longer relevant, like when its Target is not displayed anymore (e.g. on Navigation) or when we can assume that the user had enough time to admire it. Let’s write some code to support such scenarios.

Hiding on Navigation

By default an open TeachingTip remains open when the user navigates to another page. When the code behind that TeachingTip (e.g. a Close event handler) refers to its Target or other items on the original page, your app will crash:

PopupRoot

On the left hand side in the above screenshot you see the Visual Studio Live Visual Tree. It reveals how the PopupRoot holding the TeachingTip is completely separated from the RootScrollViewer holding its Frame, Page, and Target. It’s up to you to synchronize these.

It’s easy for a Page to close a TeachingTip in its OnNavigatingFrom event:

protected override void OnNavigatingFrom(NavigatingCancelEventArgs e)
{
    PositioningTip.IsOpen = false;
    base.OnNavigatingFrom(e);
}

A more reusable solution is to enumerate all open Popup elements (Dialogs, Flyouts, ToolTips, TeachingTips) with GetOpenPopups() and then close each of these:

var openPopups = VisualTreeHelper.GetOpenPopups(Window.Current);
foreach (var popup in openPopups)
{
    popup.IsOpen = false;
}

Here’s an alternative for the fans of one-liners:

VisualTreeHelper.GetOpenPopups(Window.Current).ToList().ForEach(p => p.IsOpen = false);

Hiding on Timeout

If you want to close a TeachingTip after a certain time interval then you need to start a DispatcherTimer when you open it:

var timer = new DispatcherTimer();
timer.Tick += MainMenuTimer_Tick;
timer.Interval = TimeSpan.FromSeconds(5);
timer.Start();

The code that you write in its Tick() event executes on the UI Thread, so it can visually impact controls. You can set the IsOpen property of the TeachingTip to false there, and do more cleanup if you wish. Don’t forget to stop the timer:

private static void MainMenuTimer_Tick(object sender, object e)
{
    (sender as DispatcherTimer).Stop();

    if (_mainMenuTeachingTip == null)
    {
        return;
    }

    // Close and cleanup the TeachingTip
    _mainMenuTeachingTip.IsOpen = false;
}

Building an Inherited Control

Some time ago most Microsoft (and Open Source) teams that build UWP controls stopped making these controls sealed. Since then it is relatively easy to create inherited controls – such as a TeachingTip that auto-closes itself when its AutoCloseInterval elapses.

The sample app hosts such a control, here’s how an instance is declared in one of the XAML pages:

<xbcontrols:AutoCloseTeachingTip x:Name="ResetButtonTeachingTip"
                                    AutoCloseInterval="3000"
                                    Title="Reset"
                                    Content="Your app was reset to factory settings."
                                    PreferredPlacement="Right">
    <controls:TeachingTip.IconSource>
        <controls:SymbolIconSource Symbol="Repair" />
    </controls:TeachingTip.IconSource>
</xbcontrols:AutoCloseTeachingTip>

Here are the steps to create the control:

  1. Create a class that inherits from TeachingTip (no shit, Sherlock),
  2. add a property to hold the timeout value -it does not need to be a dependency property– with a decent default value,
  3. call RegisterPropertyChangedCallback() to define a listener for changes in the IsOpen dependency property,
  4. implement the DispatcherTimer based auto-close algorithm, and
  5. make sure to unregister all event handlers to avoid memory leaks.

Here’s the entire class definition:

/// <summary>
/// A teaching tip that closes itself after an interval.
/// </summary>
public class AutoCloseTeachingTip : Microsoft.UI.Xaml.Controls.TeachingTip
{
    private DispatcherTimer _timer;
    private long _token;

    public AutoCloseTeachingTip() : base()
    {
        this.Loaded += AutoCloseTeachingTip_Loaded;
        this.Unloaded += AutoCloseTeachingTip_Unloaded;
    }

    /// <summary>
    /// Gets or sets the auto-close interval, in milliseconds.
    /// </summary>
    public int AutoCloseInterval { get; set; } = 5000;

    private void AutoCloseTeachingTip_Loaded(object sender, RoutedEventArgs e)
    {
        _token = this.RegisterPropertyChangedCallback(IsOpenProperty, IsOpenChanged);
        if (IsOpen)
        {
            Open();
        }
    }

    private void AutoCloseTeachingTip_Unloaded(object sender, RoutedEventArgs e)
    {
        this.UnregisterPropertyChangedCallback(IsOpenProperty, _token);
    }

    private void IsOpenChanged(DependencyObject o, DependencyProperty p)
    {
        var that = o as AutoCloseTeachingTip;
        if (that == null)
        {
            return;
        }

        if (p != IsOpenProperty)
        {
            return;
        }

        if (that.IsOpen)
        {
            that.Open();
        }
        else
        {
            that.Close();
        }
    }

    private void Open()
    {
        _timer = new DispatcherTimer();
        _timer.Tick += Timer_Tick;
        _timer.Interval = TimeSpan.FromMilliseconds(AutoCloseInterval);
        _timer.Start();
    }

    private void Close()
    {
        if (_timer == null)
        {
            return;
        }

        _timer.Stop();
        _timer.Tick -= Timer_Tick;
    }

    private void Timer_Tick(object sender, object e)
    {
        this.IsOpen = false;
    }
}

For the sake of completeness, here’s how an instance of the control is created programmatically:

var _closeTeachingTip = new AutoCloseTeachingTip
{
    Title = "Then what are you still doing there?",
    Content = "We are all waiting for you in the factory.",
    HeroContent = new Image
    {
        Source = new BitmapImage(new Uri("ms-appx:///Assets/BrimbornSteelworks.png"))
    },
    PreferredPlacement = TeachingTipPlacementMode.Right,
    IsOpen = true
};

Using a TeachingTip as Form Field Wizard

Out of the box the TeachingTip supports an optional button: the Action Button. When one is defined, the cross icon in the upper left corner is replaced by a genuine Close Button at the bottom. The corresponding action can be defined in a traditional ActionButtonClick handler or in a more MVVM way through an ActionButtonCommand property. The action button does not close the TeachingTip.

Here’s how a TeachingTip with two buttons is declared in the sample app:

<controls:TeachingTip x:Name="ButtonsTip"
                        Target="{x:Bind ButtonsButton}"
                        Title="Were you already Flayed?"
                        CloseButtonContent="Yes"
                        CloseButtonCommand="{x:Bind CloseCommand}"
                        ActionButtonContent="Not sure"
                        ActionButtonCommand="{x:Bind ActionCommand}" />

And this is how it looks like – the action button was clicked, so the TeachingTip remained open when we opened the untargeted one on the left:

ActionButton

TeachingTip inherits from ContentControl, so it supports rich content. Together with its titles, its buttons, and its tail, this makes it an ideal host for a local wizard that can help the end user filling out a specific input field on a form – as a small dialog box that provides options or a calculation tool. This functionality is typically visually represented by decorating the form field with a tiny button showing an ellipsis or a magic wand or a calculator icon.

Our UWP app contains an example of using the TeachingTip as a form field wizard. The ellipsis button next to the TextBox opens a teaching tip presenting some options. The standard Action and Close buttons allow the user to select one of the options, or ignore the suggestion:

WizardTip

This is a scenario in which the TeachingTip could really shine, and we’re definitely planning to use it like this in some of our apps.

TeachingTip is a useful new control in the UWP ecosystem, with a simple API and smooth animations.

The Code

The sample app lives here of GitHub.

Enjoy!

Machine Learning with ML.NET in UWP: Automated Learning

In this article we take the ML.NET automated machine learning API for a spin to demonstrate how it can be used in a C# UWP app for discovering, training, and fine-tuning the most appropriate prediction model for a specific machine learning use case.

The Automated Machine Learning feature of ML.NET was announced at Build 2019 as a framework that automatically iterates over a set of algorithms and hyperparameters to select and create a prediction model. You only have to provide

  • the problem type (binary classification, multiclass classification, or regression),
  • the quality metric to optimize (accuracy, log loss, area under the curve, …), and
  • a dataset with training data.

The ML.NET automated machine learning functionality is exposed as

This article focuses on the automated ML API, we’ll refer to it with its nickname ‘AutoML’. In our UWP sample app we tried to implement a more or less realistic scenario for this feature. Here’s the corresponding XAML page from that app. It shows the results of a so-called experiment:

AutoML

Stating the problem

In the sample app we reused the white wine dataset from our binary classification sample. Its raw data contains the values for 11 physicochemical  characteristics -the ‘features’- of white wines together with an appreciation score from 0 to 10 – the ‘label’:

Input

We’ll rely on AutoML to build us a model that uses these physicochemical features to predict the score label. We’ll treat it as a multiclass classification problem where each distinct score value is considered a category.

Creating the DataView

AutoML requires you to provide an IDataView instance with the training data, and optionally one with test data. If the latter is not provided, it will split the training data itself. For the training data, a TextLoader on a .csv file would do the job: by default AutoML will use all non-label fields as feature, and create a pipeline with the necessary components to fill out missing values and transform everything to numeric fields. In a real world scenario you would want to programmatically perform some of these tasks yourself – overruling the defaults. That’s what we did in the sample app.

We used the LoadFromTextFile<T>() method to read the data into a new pipeline, so we needed a data structure to describe the incoming data with LoadColumn and ColumnName attributes:

public class AutomationData
{
    [LoadColumn(0), ColumnName("OriginalFixedAcidity")]
    public float FixedAcidity;

    [LoadColumn(1)]
    public float VolatileAcidity;

    [LoadColumn(2)]
    public float CitricAcid;

    [LoadColumn(3)]
    public float ResidualSugar;

    [LoadColumn(4)]
    public float Chlorides;

    [LoadColumn(5)]
    public float FreeSulfurDioxide;

    [LoadColumn(6)]
    public float TotalSulfurDioxide;

    [LoadColumn(7)]
    public float Density;

    [LoadColumn(8)]
    public float Ph;

    [LoadColumn(9)]
    public float Sulphates;

    [LoadColumn(10)]
    public float Alcohol;

    [LoadColumn(11)]
    public float Label;
}

We added a ReplaceMissingValues transformation on the FixedAcidity field to keep control over the ReplacementMode and the column names, and then removed the original column with a DropColumns transformation.

Here’s the pipeline that we used in the sample app to manipulate the raw data:

// Pipeline
IEstimator<ITransformer> pipeline =
    MLContext.Transforms.ReplaceMissingValues(
        outputColumnName: "FixedAcidity",
        inputColumnName: "OriginalFixedAcidity",
        replacementMode: MissingValueReplacingEstimator.ReplacementMode.Mean)
    .Append(MLContext.Transforms.DropColumns("OriginalFixedAcidity"));
                
    // No need to add this, it will be done automatically.
    //.Append(MLContext.Transforms.Concatenate("Features",
    //    new[]
    //    {
    //        "FixedAcidity",
    //        "VolatileAcidity",
    //        "CitricAcid",
    //        "ResidualSugar",
    //        "Chlorides",
    //        "FreeSulfurDioxide",
    //        "TotalSulfurDioxide",
    //        "Density",
    //        "Ph",
    //        "Sulphates",
    //        "Alcohol"}));

A model is created from this pipeline using the Fit() method, and the Transform() call creates the IDataView that provides the training data to the experiment:

// Training data
var trainingData = MLContext.Data.LoadFromTextFile<AutomationData>(
        path: trainingDataPath,
        separatorChar: ';',
        hasHeader: true);
ITransformer model = pipeline.Fit(trainingData);
_trainingDataView = model.Transform(trainingData);
_trainingDataView = MLContext.Data.Cache(_trainingDataView);

// Check the content on a breakpoint:
var sneakPeek = _trainingDataView.Preview();

Here’s the result of the Preview() call that allows to peek at the contents of the data view:

DataViewPreview

Keep in mind that AutoML only sees this resulting data view and has no knowledge of the pipeline that created it. It will for example struggle with data views that have duplicate column names – quite common in ML.NET pipelines.

Round 1: Algorithm Selection

Defining the experiment

In the first round of our scenario, we’ll run an AutoML experiment to find one or two candidate algorithms that we would like to explore further. Every experiment category (binary classification, multiclass classification, and regression) comes with its own ExperimentSettings class where you specify things like

  • a maximum duration for the whole experiment (AutoML will complete the test that’s running at the deadline),
  • the metric to optimize for (metrics depend on the category), and
  • the algorithms to use (by default all algorithms of the category are included in the experiment).

The experiment is then instantiated with a call to one of the Create() methods in the AutoCatalog. In the sample app we decided to optimize on Logarithmic Loss: it gives a more nuanced view into the performance then accuracy, since it punishes uncertainty. We also decided to ignore the two FastTree algorithms that are not yet 100% UWP compliant. Here’s the experiment definition:

var settings = new MulticlassExperimentSettings
{
    MaxExperimentTimeInSeconds = 18,
    OptimizingMetric = MulticlassClassificationMetric.LogLoss,
    CacheDirectory = null
};

// These two trainers yield no metrics in UWP:
settings.Trainers.Remove(MulticlassClassificationTrainer.FastTreeOva);
settings.Trainers.Remove(MulticlassClassificationTrainer.FastForestOva);

_experiment = MLContext.Auto().CreateMulticlassClassificationExperiment(settings);

Running the experiment

To execute the experiment … just call Execute() on the experiment, providing the data view and an optional progress handler to receive the trainer name and quality metrics after each individual test. The winning model is returned in the BestRun property of the experiment’s result:

var result = _experiment.Execute(
    trainData: _trainingDataView,
    labelColumnName: "Label",
    progressHandler: this);

return result.BestRun.TrainerName;

The progress handler must implement the IProgress interface which declares a Report() method that is called each time an individual test in the experiment finishes. In the sample app we let the MVVM Model implement this interface, and pass the algorithm name and the quality metrics to the MVVM ViewModel via an event. Eventually the diagram in the MVVM View -the XAML page- will be updated.

Here’s the code in the Model:

internal class AutomationModel : 
    IProgress<RunDetail<MulticlassClassificationMetrics>>
{
    // ...

    public event EventHandler<ProgressEventArgs> Progressed;

    // ...

    public void Report(RunDetail<MulticlassClassificationMetrics> value)
    {
        Progressed?.Invoke(this, new ProgressEventArgs
        {
            Model = new AutomationExperiment
            {
                Trainer = value.TrainerName,
                LogLoss = value.ValidationMetrics?.LogLoss,
                LogLossReduction = value.ValidationMetrics?.LogLossReduction,
                MicroAccuracy = value.ValidationMetrics?.MicroAccuracy,
                MacroAccuracy = value.ValidationMetrics?.MacroAccuracy
            }
        });
    }
}

The next screenshot shows the result of the algorithm selection phase in the sample app. The proposed model is not super good, but that’s mainly our own fault – the wine scoring problem is more a regression than a multiclass classification. If you consider that these models are unaware of the score order (they don’t realize that 8 is better than 7 is better than 6, etcetera – so they also not realize that a score of 4 may be appropriate if you hesitate between a 3 and a 5), then you will realize that they’re actually pretty accurate.

Here’s a graphical overview of the different models in the experiment. Notice the correlation –positive or negative- between the various quality metrics:

Round1

Some of the individual tests return really bad models. Here’s an example of an instance with a negative value for the Log Loss Reduction quality metric:

WorseThanRandom

This model performs worse than just randomly selecting a score. Make sure to run experiments long enough to eliminate such candidates.

Round 2: Parameter Sweeping

When using AutoML, we propose to first run a set of high level experiments to discover the algorithms that best suit your specific machine learning problem and, and then run a second set of experiments with a limited number of algorithms –just one or two- to fine-tune their appropriate hyperparameters. Data scientists call this parameter sweeping. For developers the source code for both sets is almost identical. In round 1 we start with all algorithms and Remove() some, and in round 2 we first Clear() the ICollection of trainers first and then Add() the few that we want to evaluate.

Here’s the full parameter sweeping code in the sample app:

var settings = new MulticlassExperimentSettings
{
    MaxExperimentTimeInSeconds = 180,
    OptimizingMetric = MulticlassClassificationMetric.LogLoss,
    CacheDirectory = null
};

settings.Trainers.Clear();
settings.Trainers.Add(MulticlassClassificationTrainer.LightGbm);

var experiment = MLContext.Auto().CreateMulticlassClassificationExperiment(settings);

var result = experiment.Execute(
    trainData: _trainingDataView,
    labelColumnName: "Label",
    progressHandler: this);

var model = result.BestRun.Model as TransformerChain<ITransformer>;

Here’s the result of parameter sweeping on the LightGbmMulti algorithm, the winner of the first round in the sample app. If you compare the diagram to the Round 1 values, you’ll observe a general improvement of the quality metrics. The orange Log Loss curve consistently shows lower values:

Round2

Not all parameter swiping experiments are equally useful. Here’s the result that compares different parameters for the LbfgsMaximumEntropy algorithm in the sample. All tests return pretty much the same (bad) model. This experiment just confirms that this is not the right algorithm for the scenario:

Round2Bis

Inspecting the Winner

After running some experiments you probably want to dive into the details of the winning model. Unfortunately this is the place where the API currently falls short: most if not all of the hyperparameters of the generated models are stored in private properties. Your options to drill down to the details are

  • open the model that you saved (it’s just a .zip file with flat texts in it), or
  • rely on Reflection.

We decided to rely on the Reflection features in the Visual Studio debugger. In all multiclass classification experiments that we did, the prediction model was the last transformer in the first step of the generated pipeline. So in the sample app we assigned this to a variable to facilitate inspection via a breakpoint.

Here are the OneVersusAll parameters of the winning model. They’re the bias, the weights, splits and leaf values of the underlying RegressionTreeEnsemble for each possible score:

HyperParameters1

That sounds like a pretty complex structure, so let’s shed some light on it. For starters, LightGbmMulti is a so-called One-Versus-All (OVA) algorithm. OVA is a technique to solve a multiclass classification problem by a group of binary classifiers.

The following diagram illustrates using three binary classifiers to recognize squares, triangles or crosses:

one-vs-all

When the model is asked to create a prediction, it delegates the question to all three classifiers and then deducts the result. If the answers are

  • I think it’s a square,
  • I think it’s not a triangle, and
  • I think it’s not a cross,

then you can be pretty sure that it’s a square, no?

The winning model in the sample app detected 7 values for the label, so it created 7 binary classifiers. This means that not all the scores from 0 to 10 were given. [This observation made us realize that we should have treated this problem as a regression instead of as a classification.] Each of these 7 binary classifiers is a a LightGBM trainer – a gradient boosting framework that uses tree-based learning algorithms. Gradient boosting is –just like OVA- a technique that solves a problem using multiple classifiers. LightGBM builds a strong learner by combining an ensemble of weak learners. These weak learners are typically decision trees. Apparently each of the 7 classifiers in the sample app scenario hosts an ensemble of 100 trees, each with a different weight, bias, and a set of leafs and split values for each branch.

The following screenshot shows a simpler set of hyperparameters. It’s the result of a parameter sweeping round for the LbfgsMaximumEntropy algorithm, also known as multinomial logistic regression. This one is also a One-Versus-All trainer, so there are again 7 submodels. This time the models are simpler. The algorithm created a regression function for each of the score values. The parameters are the weight of each feature in that function:

HyperParameters2

At this point in time the API’s main target is to support the Command Line Tool and the Model Builder, and that’s probably why the model’s details are declared private. All of them already appear in the output of the CLI however, so we assume that full programmatic access to the models (and the source code to generate them!) is just a matter of time.

Wow!

Here’s the overview of a canonical machine learning use case. The phases that are covered by AutoML are colored:

MachineLearningModel_2_3_4

To complete the scenario, all you need to do is

  • make your raw data available in an IDataView-friendly way: a .csv file or an IEnumerable (e.g. from a database query), and
  • consume the generated model in your apps, that’s just three lines of code: load the .zip file, create a prediction engine, call the prediction engine.

AutoML will do the rest. Impressive, no? There’s no excuse for not starting to embed machine learning in your .NET apps…

The Source

The sample app lives here on GitHub.

Enjoy!