matejknopp.com

I have been interested in desktop applications ever since I first saw Turbo Vision. Those text mode resizable windows in DOS felt like magic to me. It sparked an interest in user interface frameworks that's still going strong, more than 20-something years later.

In the last decade or so the spotlight has been largely shifted to web and mobile, which does not make me particularly happy. So it feels like it's time to crawl out of the shadows, leave my comfort zone and try to help bringing some of the spotlight it back where it belongs. To the desktop! :)

The road to Flutter

The last desktop application I worked on was (still is) Airflow. It's a mixture of Qt and a fair chunk of platform specific code. I'm quite contented with the end result, if I do say so myself, but the developer experience and overall productivity leaves a lot to be desired.

About two years ago, I needed an Airflow companion app for iOS and Android. After several prototypes, decision was made and I went with Flutter. I do like to think that I have my share of UI development experience, after all, I have worked with nearly a dozen GUI frameworks on various platforms, so there's not much that can surprise me at this point right? Wrong. The biggest surprise of them all was just how good working with Flutter felt. Never, not once in my life, have building user interface made this much sense.

Wouldn't it be amazing if I could build desktop applications this way? Well, it would, but reality is a harsh mistress and at that point desktop embedders were still in their infancy. So back to Qt it is. But I coudn't get the idea out of my head.

Fast forward a year or two, a lot has changed. There's still work to do, but desktop embedders have matured quite a bit and Flutter on desktop is starting to be a viable option.

Flutter desktop embedders

Now you might be asking: Matt, doesn’t Flutter already have desktop embedders? So what is all this fuss about?

Why yes, it does indeed. And NativeShell builds right on top of them. You can imagine the Flutter desktop embedder as being a platform view component (think GtkWidget, NSView or, dare I say, HWND). It handles mouse and keyboard input, painting, but it doesn’t try to manage windows, or Flutter engines / isolates. Or do things like platform menus and drag & drop. To make things more complicated, Flutter embedders have completely different API on each platform. So if you want to create an engine or register platform channel handler for some low level code, you need to do this separately for each platform.

This is where NativeShell steps in

NativeShell starts right where Flutter desktop embedders end. It provides a consistent, platform agnostic API to existing Flutter embedders. It manages engines and windows. It provides drag & drop support, access to platform menus, and other functionality that is out of scope of Flutter embedders. And it exposes all of this through easy to use Dart API.

NativeShell is written in Rust. Rust is great because it lets you write efficient low level platform specific code if you need to, but it also let you use NativeShell without having to know any Rust. Simply executing cargo run is all it takes to get things going. Cargo is Rust package manager (like pub is for Dart), it takes care of downloading and building all dependencies.

Getting Started

$ flutter config --enable-windows-desktop
$ flutter config --enable-macos-desktop
$ flutter config --enable-linux-desktop

$ flutter channel master
$ flutter upgrade

After this, you should be good to go 🚀:

$ git clone https://github.com/nativeshell/examples.git
$ cd examples
$ cargo run

NativeShell transparently integrates Flutter build process with cargo. If Rust and Dart gods are smiling at you, this is what you should now see:

Platform Channels

If you need to call native code from a Flutter app, the two options are platform channels or FFI. For general use platform channels are preffered, since they are easier to use and properly bounce the messages between platform and UI threads.

This is what registering a platform channel handler looks like with NativeShell (also the only Rust code here, I promise :)

fn register_example_channel(context: Rc<Context>) {
    context
        .message_manager
        .borrow_mut()
        .register_method_handler("example_channel", |call, reply, engine| {
            match call.method.as_str() {
                "echo" => {
                    reply.send_ok(call.args);
                }
                _ => {}
            }
        });
}

To do this directly using the existing platform embedders API, you would need to write this code separately for each platform using platform specific API. And then make sure your handler gets registered every time you create new engine (and possibly unregistered when you shut it down).

With NativeShell you only need to register your handler once, and it can be called from any engine. Messages can be transparently serialized and deserialized to Rust structures with Serde (using Flutter's StandardMethodCodec format).

Window Management

Presumably you'd want your desktop application to have multiple windows? NativeShell has you covered. Resize windows to content or set minimal window size so that Flutter layout doesn't underflow? It can do that too. It also makes sure to only show windows once the contents is ready, elimiating ugly flicker.

Currently each Window runs as a separate isolate. NativeShell provides API for creating window, setting and quering geometry, updating style and window title. It also provides API for easy communication between windows.

Videos can be sized to content, or resizable with minimum size to fit intrinsic content size.

This would be a minimal demonstrations of how multiple windows are created and managed in Dart:

void main() async {
  runApp(MinimalApp());
}

class MinimalApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    // Widgets above WindowWidget will be same in all windows. The actual
    // window content will be determined by WindowState
    return WindowWidget(
      onCreateState: (initData) {
        WindowState? context;
        context ??= OtherWindowState.fromInitData(initData);
        // possibly no init data, this is main window
        context ??= MainWindowState();
        return context;
      },
    );
  }
}

class MainWindowState extends WindowState {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: WindowLayoutProbe(
        child: TextButton(
          onPressed: () async {
            // This will create new isolate for a window. Whatever is given to
            // Window.create will be provided by WindowWidget in new isolate
            final window = await Window.create(OtherWindowState.toInitData());
            // you can use the window object to communicate with newly created
            // window or register handlers for window events
            window.closeEvent.addListener(() {
              print('Window closed');
            });
          },
          child: Text('Open Another Window'),
        ),
      ),
    );
  }
}

class OtherWindowState extends WindowState {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: WindowLayoutProbe(child: Text('This is Another Window!')),
    );
  }

  // This can be anything that fromInitData recognizes
  static dynamic toInitData() => {
        'class': 'OtherWindow',
      };

  static OtherWindowState? fromInitData(dynamic initData) {
    if (initData is Map && initData['class'] == 'OtherWindow') {
      return OtherWindowState();
    }
    return null;
  }
}

Drag & Drop

It's hard to imagine any self respecting desktop UI framework that wouldn't support Drag & Drop. NativeShell supports dragging and dropping file paths, URLs, custom dart data (serializable by StandardMethodCodec) and can even be extended to handle custom platform specific formats.

It should be quite easy to use, and I'm happy with how it turned out, even though it did involve writing some downright scary looking code.

Popup Menu

It often surprises me how many frameworks and applications have this wrong. It wasn't until very recently that Firefox started using native popup menu on macOS. No matter how polished your app is, if you get the menus wrong, it won't feel right.

NativeShell lets you easily create and show context menus. The menu API is deceptively simple, given how powerful the menu system is. Menus are reactive. You can ask menu to be rebuilt while visible and NativeShell will compute the delta and just update menu items that have actually changed.

  int _counter = 0;

  void _showContextMenu(TapDownDetails e) async {
    final menu = Menu(_buildContextMenu);

    // Menu can be updated while visible
    final timer = Timer.periodic(Duration(milliseconds: 500), (timer) {
      ++_counter;
      // This will call the _buildContextMenu() function, diff the old
      // and new menu items and only update those platform menu items that
      // actually changed
      menu.update();
    });

    await Window.of(context).showPopupMenu(menu, e.globalPosition);

    timer.cancel();
  }

  List<MenuItem> _buildContextMenu() => [
        MenuItem(title: 'Context menu Item', action: () {}),
        MenuItem(title: 'Menu Update Counter $_counter', action: null),
        MenuItem.separator(),
        MenuItem.children(title: 'Submenu', children: [
          MenuItem(title: 'Submenu Item 1', action: () {}),
          MenuItem(title: 'Submenu Item 2', action: () {}),
        ]),
      ];

MenuBar

The MenuBar widget is possibly my favorite feature in NativeShell. On macOS, it renders as empty widget and instead puts the menu in the system menu bar (on top of screen). On Windows and Linux, it renders the top level menu items using Flutter widgets and then uses native menus for the rest. That means the menu bar can be anywhere in your widget hierarchy, it's not limited to the top of the window and it doesn't rely on GDI or Gtk to paint iself.

It supports mouse tracking and keyboard navigation, just like regular system menubar would, but without any of the limitations.

Where things are right now

NativeShell is under heavy develoment. Things will likely break. More documentation and examples are severely needed. But I think it's in a shape where it could be useful for some people.

All three supported platforms (macOS, Windows, Linux) have full feature parity.

If you made it all the way here, you can continue to nativeshell.dev.

Feedback is appreciated!