matejknopp.com

> arcane arts of desktop software development

Anatomy of a NativeShell Project

June 9, 2021

One of the first questions I got after releasing NativeShell was about adding it to an existing Flutter project. So it seems like a good idea to write a little bit about NativeShell projects.

Let's start with some cool unicode block diagrams :)

This is what a regular Futter desktop project looks like:

                            ┊
Your Project                ┊   Flutter
                            ┊
Platform specific code      ┊
to create and manage        ┊
windows, engines            ┊
                            ┊
Generated code added        ┊
by Flutter Tool             ┊
                            ┊
╭───────────────────────╮   ┊   ╭──────────────────╮
│ macOS specific code   │   ┊   │ macOS Flutter    │
│ (Objc/Swift + Cocoa)  │───────│ Desktop Embedder │────┐
╰───────────────────────╯   ┊   ╰──────────────────╯    │
╭───────────────────────╮   ┊   ╭──────────────────╮    │    ╭─────────╮
│ Windows specific code │   ┊   │ Windows Flutter  │    │    │ Flutter │
│ (C++, Win32/UWP)      │───────│ Desktop Embedder │────┼────│ Engine  │
╰───────────────────────╯   ┊   ╰──────────────────╯    │    ╰─────────╯
╭───────────────────────╮   ┊   ╭──────────────────╮    │
│ Linux specific code   │   ┊   │ Linux Flutter    │    │
│ (C, Gtk)              │───────│ Desktop Embedder │────┘
╰───────────────────────╯   ┊   ╰──────────────────╯
                            ┊

For comparison, this is a project built with NativeShell:

                   ┊                        ┊
Your Project       ┊  NativeShell           ┊  Flutter
                   ┊                        ┊
                   ┊  Window Management     ┊   ╭──────────────────╮
                   ┊  Engine Management     ┊   │ macOS Flutter    │
                   ┊  DnD, Menus, etc.   ┌──────│ Desktop Embedder │────┐
                   ┊                     │  ┊   ╰──────────────────╯    │
╭───────────────╮  ┊                     │  ┊   ╭──────────────────╮    │    ╭─────────╮
│ Platform      │  ┊  ╭──────────────╮   │  ┊   │ Windows Flutter  │    │    │ Flutter │
│ agnostic Code │─────│ NativeShell  │───┼──────│ Desktop Embedder │────┼────│ Engine  │
│ (Rust)        │  ┊  ╰──────────────╯   │  ┊   ╰──────────────────╯    │    ╰─────────╯
╰───────────────╯  ┊                     │  ┊   ╭──────────────────╮    │
                   ┊                     │  ┊   │ Linux Flutter    │    │
                   ┊                     └──────│ Desktop Embedder │────┘
                   ┊                        ┊   ╰──────────────────╯
                   ┊                        ┊

NativeShell provides all the necessary platform abstractions for things that you might not want to deal with directly, but at the same time it will let you write low level platform specific code when you need to.

This is what a minimal NativeShell project contains:

├ .cargo
│ └ config.toml   # Settings for Rust compiler to set RPATH on macOS and Linux.*
├ lib             # Dart code
│ └ main.dart     # Dart entry-point. The main() method that will be called for every window.
├ src             # Rust code
│ └ main.rs       # Rust entry-point.
├ Cargo.toml      # Cargo crate configuration (similar to what pubspec.yaml is for Dart)
├ build.rs        # Rust build script. This is the secret sauce - it integrates Flutter# build with cargo (Rust build system). More details on this below.
├ pubspec.yaml    # Dart package configuration.

* Hopefully no longer necessary once cargo:rustc-link-arg=FLAG becomes stable

If you compare this to a regular Flutter desktop project you will notice that this structure is much simpler. The regular Flutter project is a somewhat messy combination of source files, IDE projects (i.e. XCode), CMake files, generated files and ephemeral libraries (added to your tree by Flutter tool to get around the fact that C++ has no standard package manager or stable ABI).

I personally find all this a little bit wonky. For mobile it may have sense to have separate projects for Android and iOS, the platforms are very different and platform IDEs may be required to launch on device and debug native code, but I think for desktop we can do better. So you'll need none of those files with NativeShell. There are no generated files or platform specific projects in the source tree. All these details are handled transparently by build.rs.

So what does build.rs do?

Cargo (the Rust build system) has one really nice feature - it can run arbitrary Rust code during build. NativeShell takes full advantage of this when building your project: It makes sure that Dart code is compiled to kernel snapshots for debug builds, resources from Dart packages are assembled, AOT native libraries are built for release builds, and all required artifacts (i.e. the desktop embedder library) are copied from Flutter cache to your build folder. It can now also build, link and register native Flutter plugins if your project is using any.

All this is done transparently and everything is contained within the build folder.

This is what a simple build.rs file looks like:

// Import the nativeshell_build crate which contains the logic required for
// Flutter project build
use nativeshell_build::{
AppBundleOptions, BuildResult, Flutter, FlutterOptions, MacOSBundle
};

fn build_flutter() -> BuildResult<()> {
// Static method that takes care of Flutter build process; In FlutterOption
// you can specify the main dart file if it is something other than
// lib/main.dart, or choose local engine name and path if you built the flutter
// engine yourself.
Flutter::build(FlutterOptions {
..Default::default()
})?;

// macOS specific: This will create a "fake" app bundle structure that links
// to real executable. This is necessary because running unbundled macOS
// apps has unexpected side-effects.
if cfg!(target_os = "macos") {
let options = AppBundleOptions {
bundle_name: "AppTemplate.app".into(),
bundle_display_name: "App Template".into(),
icon_file: "icons/AppIcon.icns".into(),
..Default::default()
};
// Build the macOS bundle and return path to bundle resources
let resources = MacOSBundle::build(options)?;

// Make subdirectory in bundle resources and copy app icons
resources.mkdir("icons")?;
resources.link("resources/mac_icon.icns", "icons/AppIcon.icns")?;
}

Ok(())
}

fn main() {
// Call the build_flutter method and if it fails, print error and abort build.
if let Err(error) = build_flutter() {
println!("\n** Build failed with error **\n\n{}", error);
panic!();
}
}

The Rust code might seem weird at bit first glance (it is quite easy to get used to), but hopefully with comments it shouldn't be too difficult to figure out what's going on.

Which finally brings us back to...

Adding NativeShell to existing project

If you have an existing Flutter project, fingers crossed adding NativeShell to it shoud be fairly straightforward. Check out the NativeShell App Template and copy the following items to your existing project:

The resources folder only contains macOS app icon. You can skip it, just make sure you remove the icon symlink step from build.rs.

Make sure to also merge contents of .gitignore as cargo uses the target folder for build output, which you don't want to check in your version control.

After this is done, install the nativeshell dart package:

$ flutter pub add nativeshell

The last remaining step is adding WindowWidget and WindowLayoutProbe widgets to your widget hierarchy. These should be high enough to cover the entire contents of your app, otherwise window sizing might not work properly. You can see a simple example of using the WindowWidget here.

Introducing NativeShell for Flutter

June 3, 2021

I have been interested in desktop applications ever since I first saw Turbo Vision. Those text mode resizable windows in DOS felt like magic to me. It sparked an interest in user interface frameworks that's still going strong, more than 20-something years later.

In the last decade or so the spotlight has been largely shifted to web and mobile, which does not make me particularly happy. So it feels like it's time to crawl out of the shadows, leave my comfort zone and try to help bringing some of the spotlight it back where it belongs. To the desktop! :)

The road to Flutter

The last desktop application I worked on was (still is) Airflow. It's a mixture of Qt and a fair chunk of platform specific code. I'm quite contented with the end result, if I do say so myself, but the developer experience and overall productivity leaves a lot to be desired.

About two years ago, I needed an Airflow companion app for iOS and Android. After several prototypes, decision was made and I went with Flutter. I do like to think that I have my share of UI development experience, after all, I have worked with nearly a dozen GUI frameworks on various platforms, so there's not much that can surprise me at this point right? Wrong. The biggest surprise of them all was just how good working with Flutter felt. Never, not once in my life, have building user interface made this much sense.

Wouldn't it be amazing if I could build desktop applications this way? Well, it would, but reality is a harsh mistress and at that point desktop embedders were still in their infancy. So back to Qt it is. But I coudn't get the idea out of my head.

Fast forward a year or two, a lot has changed. There's still work to do, but desktop embedders have matured quite a bit and Flutter on desktop is starting to be a viable option.

Flutter desktop embedders

Now you might be asking: Matt, doesn’t Flutter already have desktop embedders? So what is all this fuss about?

Why yes, it does indeed. And NativeShell builds right on top of them. You can imagine the Flutter desktop embedder as being a platform view component (think GtkWidget, NSView or, dare I say, HWND). It handles mouse and keyboard input, painting, but it doesn’t try to manage windows, or Flutter engines / isolates. Or do things like platform menus and drag & drop. To make things more complicated, Flutter embedders have completely different API on each platform. So if you want to create an engine or register platform channel handler for some low level code, you need to do this separately for each platform.

This is where NativeShell steps in

NativeShell starts right where Flutter desktop embedders end. It provides a consistent, platform agnostic API to existing Flutter embedders. It manages engines and windows. It provides drag & drop support, access to platform menus, and other functionality that is out of scope of Flutter embedders. And it exposes all of this through easy to use Dart API.

NativeShell is written in Rust. Rust is great because it lets you write efficient low level platform specific code if you need to, but it also let you use NativeShell without having to know any Rust. Simply executing cargo run is all it takes to get things going. Cargo is Rust package manager (like pub is for Dart), it takes care of downloading and building all dependencies.

Getting Started

  1. Install Rust
  2. Install Flutter
  3. Enable desktop support in Flutter (choose the one for your platform)
$ flutter config --enable-windows-desktop
$ flutter config --enable-macos-desktop
$ flutter config --enable-linux-desktop
  1. Switch to Flutter Master (for the time being)
$ flutter channel master
$ flutter upgrade

After this, you should be good to go 🚀:

$ git clone https://github.com/nativeshell/examples.git
$ cd examples
$ cargo run

NativeShell transparently integrates Flutter build process with cargo. If Rust and Dart gods are smiling at you, this is what you should now see:

Platform Channels

If you need to call native code from a Flutter app, the two options are platform channels or FFI. For general use platform channels are preffered, since they are easier to use and properly bounce the messages between platform and UI threads.

This is what registering a platform channel handler looks like with NativeShell (also the only Rust code here, I promise :)

fn register_example_channel(context: Rc<Context>) {
context
.message_manager
.borrow_mut()
.register_method_handler("example_channel", |call, reply, engine| {
match call.method.as_str() {
"echo" => {
reply.send_ok(call.args);
}
_ => {}
}
});
}

To do this directly using the existing platform embedders API, you would need to write this code separately for each platform using platform specific API. And then make sure your handler gets registered every time you create new engine (and possibly unregistered when you shut it down).

With NativeShell you only need to register your handler once, and it can be called from any engine. Messages can be transparently serialized and deserialized to Rust structures with Serde (using Flutter's StandardMethodCodec format).

Window Management

Presumably you'd want your desktop application to have multiple windows? NativeShell has you covered. Resize windows to content or set minimal window size so that Flutter layout doesn't underflow? It can do that too. It also makes sure to only show windows once the contents is ready, elimiating ugly flicker.

Currently each Window runs as a separate isolate. NativeShell provides API for creating window, setting and quering geometry, updating style and window title. It also provides API for easy communication between windows.

Videos can be sized to content, or resizable with minimum size to fit intrinsic content size.

This would be a minimal demonstrations of how multiple windows are created and managed in Dart:

void main() async {
runApp(MinimalApp());
}

class MinimalApp extends StatelessWidget {

Widget build(BuildContext context) {
// Widgets above WindowWidget will be same in all windows. The actual
// window content will be determined by WindowState
return WindowWidget(
onCreateState: (initData) {
WindowState? context;
context ??= OtherWindowState.fromInitData(initData);
// possibly no init data, this is main window
context ??= MainWindowState();
return context;
},
);
}
}

class MainWindowState extends WindowState {

Widget build(BuildContext context) {
return MaterialApp(
home: WindowLayoutProbe(
child: TextButton(
onPressed: () async {
// This will create new isolate for a window. Whatever is given to
// Window.create will be provided by WindowWidget in new isolate
final window = await Window.create(OtherWindowState.toInitData());
// you can use the window object to communicate with newly created
// window or register handlers for window events
window.closeEvent.addListener(() {
print('Window closed');
});
},
child: Text('Open Another Window'),
),
),
);
}
}

class OtherWindowState extends WindowState {

Widget build(BuildContext context) {
return MaterialApp(
home: WindowLayoutProbe(child: Text('This is Another Window!')),
);
}

// This can be anything that fromInitData recognizes
static dynamic toInitData() => {
'class': 'OtherWindow',
};

static OtherWindowState? fromInitData(dynamic initData) {
if (initData is Map && initData['class'] == 'OtherWindow') {
return OtherWindowState();
}
return null;
}
}

Drag & Drop

It's hard to imagine any self respecting desktop UI framework that wouldn't support Drag & Drop. NativeShell supports dragging and dropping file paths, URLs, custom dart data (serializable by StandardMethodCodec) and can even be extended to handle custom platform specific formats.

It should be quite easy to use, and I'm happy with how it turned out, even though it did involve writing some downright scary looking code.

Popup Menu

It often surprises me how many frameworks and applications have this wrong. It wasn't until very recently that Firefox started using native popup menu on macOS. No matter how polished your app is, if you get the menus wrong, it won't feel right.

NativeShell lets you easily create and show context menus. The menu API is deceptively simple, given how powerful the menu system is. Menus are reactive. You can ask menu to be rebuilt while visible and NativeShell will compute the delta and just update menu items that have actually changed.

  int _counter = 0;

void _showContextMenu(TapDownDetails e) async {
final menu = Menu(_buildContextMenu);

// Menu can be updated while visible
final timer = Timer.periodic(Duration(milliseconds: 500), (timer) {
++_counter;
// This will call the _buildContextMenu() function, diff the old
// and new menu items and only update those platform menu items that
// actually changed
menu.update();
});

await Window.of(context).showPopupMenu(menu, e.globalPosition);

timer.cancel();
}

List<MenuItem> _buildContextMenu() => [
MenuItem(title: 'Context menu Item', action: () {}),
MenuItem(title: 'Menu Update Counter $_counter', action: null),
MenuItem.separator(),
MenuItem.children(title: 'Submenu', children: [
MenuItem(title: 'Submenu Item 1', action: () {}),
MenuItem(title: 'Submenu Item 2', action: () {}),
]),
];

MenuBar

The MenuBar widget is possibly my favorite feature in NativeShell. On macOS, it renders as empty widget and instead puts the menu in the system menu bar (on top of screen). On Windows and Linux, it renders the top level menu items using Flutter widgets and then uses native menus for the rest. That means the menu bar can be anywhere in your widget hierarchy, it's not limited to the top of the window and it doesn't rely on GDI or Gtk to paint iself.

It supports mouse tracking and keyboard navigation, just like regular system menubar would, but without any of the limitations.

Where things are right now

NativeShell is under heavy develoment. Things will likely break. More documentation and examples are severely needed. But I think it's in a shape where it could be useful for some people.

All three supported platforms (macOS, Windows, Linux) have full feature parity.

If you made it all the way here, you can continue to nativeshell.dev.

Feedback is appreciated!

RSS feed icon RSS