By Zhang Han, nicknamed Menliu.
When Flutter was first designed by Google, the web ecosystem was not considered. The reason was simple: both technologies had different design concepts, and forcible integration would likely deprive them of each of their respective advantages. However, many teams in the industry are now attempting to do just this, which shows that a demand indeed does exist. Today, Zhang Han, a wireless development expert at Alibaba, will show you how to connect Flutter with the web ecosystem.
In short, my initial thought is that the two should not be connected. But, before we can make informed conclusions, it's important that we consider all angles, and first we need to think about why we would want to connect them in the first place.
One important question to consider before anything else is why we may want to integrate Flutter into the web ecosystem in the first place.
For a bit of context, Flutter is currently one of the most popular cross-platform technologies capable of running code on Android, iOS, PCs, IoT-enabled devices, and browsers. Because of this, it is often regarded as a next-generation cross-platform technology. Compared with Weex and React Native, which can solve consistency problems between multiple platforms and have similar native rendering performance, Flutter's upper layer is more lightweight than the encapsulation layer of JavaScript (JS), and the overall performance of Flutter is also slightly better.
However, the first question that most people who are learning about Flutter is: Why does Flutter use Dart? Getting into a brand new language means you have to learn a lot more. But isn't JS good enough? If you're not into JS, what about TypeScript (TS)? In fact, Flutter abandoned not only the JS language but also HTML and Cascading Style Sheets (CSS), and designed an improved and decoupled widget system. In short, Flutter abandoned the entire web ecosystem. Instead, it is committed to creating a new ecosystem, but this ecosystem cannot reuse the code and solutions of the web ecosystem. In particular, all previous cross-platform solutions, such as Hybrid, React Native, and Weex, are compatible with the web ecosystem. This makes Flutter an outlier and deters most frontend developers from using it.
The following table shows an analysis of the costs to frontend developers who want to use Flutter:
As the development mode of Flutter is similar to that of the frontend framework, which we can more or less think of as a copy of React, the framework learning cost is not high, and the Dart learning cost is only slightly higher. However, we also need to learn how to assemble UIs with widgets. Although many layout widgets are similar to CSS-based programs in design, they provide much less flexibility. To use widgets in real projects, you have to transform the entire toolchain and implement development from the Native First perspective. The development process for Flutter is similar to that of native applications, but quite different from the development of frontend pages. The highest cost lies in the ecosystem cost. It is difficult to reuse code and technical solutions accumulated in the frontend ecosystem, which is the most significant disadvantage of Flutter. Accordingly, the ecosystem becomes Flutter's weakest link.
Regardless of whether Flutter abandoned the web ecosystem to pursue advanced technologies or business profit, the practical problem is that the overwhelming majority of UI developers are frontend developers, and the most abundant ecosystem is exactly the web ecosystem. In my opinion, web technology is also the most efficient way to develop UIs. If we can use web technology stacks to develop web applications at the upper layer and use Flutter to implement cross-platform rendering at a lower layer, this would improve development efficiency, performance, and cross-platform consistency. This would also allow us to reuse a large amount of code and solutions accumulated in web technology stacks.
But, perhaps these reasons are not sufficient alone. However, we will assume you are convinced and wish to connect Flutter to the web ecosystem. Later, we will discuss whether this is a good idea.
Integrating Flutter and the web ecosystem can be done in two ways:
First, let's analyze the Flutter architecture and see where we can start.
Flutter can be divided into two parts: the framework and the engine. The engine is located at a lower layer and is relatively stable, so it is better not to alter it. The framework implemented by using Dart, on the other hand, needs to be modified. To integrate the web ecosystem, we need to introduce the JS engine, whether the Dart VM should be retained is pending discussion. As shown in the preceding figure, the Material and Cupertino UI libraries at the top are not required, because the frontend has its own replacements. Here, Widgets is the major concern. Will you develop UIs in HTML or CSS, or keep widgets but use JS as the development language? Well, different situations call for different solutions.
In fact, there are many integration solutions, and the industry has tried many approaches. Here, I will summarize three methods:
dart2js.The TS transformation approach completely abandons the Dart VM and uses TS to re-implement the Flutter framework written in Dart.
Then, why is TS rather than JS used in this case? This is not because TS is currently a trendy language and is backward compatible with JS. Rather, this solution starts by asking, "can Dart for Flutter be replaced with JS?" The easiest ways to do this is to translate Dart into TS, or directly compile code into JS by using dart2js. However, with JS, the compiled code contains a lot of library encapsulations such as dart:ui, the generated package is huge, and it is difficult to customize APIs to be exported. Therefore, it is better to rewrite the code with TS because the toolchain is more familiar and customizations are allowed.
Theoretically, most Flutter functions will still be supported after the translation, and the solution can reuse various npm packages and be dynamic. However, we will lose the AOT capability. Therefore, the execution performance of JS will likely be inferior to Dart. In addition, the layout operations for all nodes run in JS, and only basic graphic capabilities are required for the underlying layer. This resembles developing a UI framework based on the Canvas API. In this solution, the performance is not necessarily as high as that of existing frontend frameworks.
The biggest problem, however, is how to maintain consistency with the official Flutter version. Assuming we translate the code from the 1.13 version, will an update is needed when the 1.15 version is available? This process involves little technical skill, but does require continuous investment, which is quite annoying.
In addition, it is also necessary to consider whether the upper layer develops UIs by using widgets or the HTML + CSS combination that is familiar to the frontend team. If you continue to use widgets, most of the frontend components still cannot be used, and the UIs still have to be redeveloped. Actually, if you determine to redevelop the UIs, the cost is the same, so you might as well redevelop them in Dart. Simply using the official versions of Flutter removes the need to translate the Dart code every time Flutter is updated. Therefore, since we have chosen to connect to the frontend ecosystem, we must connect to CSS. Otherwise, this solution is not worth our time. However, the connection between CSS and widgets is a tedious process and involves completeness problems.
Now that code translation is not an elegant solution, so let's keep Dart and connect JS or CSS to the widgets.
Well, this method does work as it uses Flutter as the underlying rendering engine, whereas the upper layer maintains the programming style of the frontend framework, with only the rendering part linked to Flutter. In fact, many existing frontend frameworks abstract underlying rendering capabilities and can connect to different rendering engines. For example, Vue and Rax support both browsers and Weex. Likewise, they can support Flutter additionally.
This method provides good compatibility with the frontend framework, but the process is too long. The business code calls the frontend framework interface for rendering, and the rendering instruction is issued after some operations are run. Then, this rendering instruction is passed to the Flutter framework in a communication manner. This communication involves a cross-language conversion from JS to C++ and then to Dart. After the rendering instruction is received, it must be converted into the corresponding widget tree, and the CSS-to-widget conversion is still very complicated. Moreover, the widget itself is stateful and is responsively updated. When it is updated, a new widget is generated and the diff algorithm is used for the new widget. If the UI is updated at the frontend, the diff algorithm is used once for vdom in JS in the frontend framework and then is used for the widget after the rendering instruction is passed into Flutter.
If you want to bypass Widgets and directly connect to the rendering layer shown in the figure, you can bypass the widget diff operation but have to modify the rendering process of the Flutter framework. Now that you need to modify the Flutter framework, why not simply use the TS transformation approach and avoid the need for JS-to-Dart communication as well? As a result, this returns us to the first solution.
In short, the advantages of this solution are that it is easy to implement and that the frontend development experience is retained to the greatest extent possible. Comparatively, the disadvantages are the long rendering process, high communication costs, responsive logic conflicts, and also incomplete CSS-to-widget conversion.
To get rid of the Dart VM, you need to re-implement the framework developed with Dart in another language, such as JS, TS, or C++. The most thoroughgoing method is to use C++ to re-implement the Flutter framework, integrate the JS engine, and expose C++ APIs to the JS environment through binding. You can still develop upper-layer applications in JS.
By developing the framework layer in C++, you can enjoy better performance and more supported languages. The Flutter framework was originally built on a Dart VM and relies on it to run. Therefore, the Flutter framework is highly dependent on Dart. After re-implementation with C++, the JS engine is built on top of the C++ framework, so the framework itself does not depend on the JS engine. Therefore, it can interface with other languages, such as Java and Kotlin, or connect back to the Dart VM for continued support for Dart.
This solution can enhance performance and maintain consistency with Flutter, but the transformation and maintenance costs are very high. Development in C++ is undoubtedly not as efficient as Dart. It is also difficult to keep up with Flutter's rapid iteration. If you don't keep up or the implementation is inconsistent, differentiation is the likely result. The CSS-to-widget conversion is another problem that has to be faced.
If we drew a schematic of the preceding three solutions, it would look something like the following figure:
In the preceding figure, the solid lines indicate cross-language communication, and excessively frequent communication can affect the performance of the overall framework. The dotted lines indicate other possible connections.
Starting at the bottom, the Flutter engine does not need to be modified. This layer is the key to cross-platform development. The framework is available in three languages, which are JS/TS, Dart, and C++. C++ provides the best performance, whereas Dart features the lowest cost. Moving up, you need to deal with HTML or CSS and widget problems. Here, you can directly connect to a frontend framework, or you can directly implement it at the C++ layer. Otherwise, there are too many binding APIs to be exposed, and frequent communication will occur.
So far, this function has been officially implemented. You can compile apps developed in Dart into web apps and run them on the browser. The official documentation mainly introduces the usage and APIs. Here, we will briefly go through the specific internal implementation solution.
According to the Flutter architecture diagram, it is the upper-layer framework that must be transformed from web to Flutter, whereas the underlying engine needs to be transformed from Flutter to web.
The framework's core dependency on the engine is dart:ui, which is the library implemented in the engine. The library abstracts the APIs for rendering the UI layer, while the underlying layer integrates the Skia implementation and exposes Dart APIs to upper layers. As you can see, the integration method is relatively simple:
dart2js to compile the framework into the JS format.dart:ui (specifically dart:web_ui) based on browser APIs.Compiling Dart into JS is no problem. The performance may be slightly affected, but the functions can be completely retained. The key is the implementation of dart:web_ui. In the native engine, dart:ui relies on SkCanvas exposed by Skia for rendering. This is a set of underlying graphic APIs, which only define underlying capabilities such as drawing lines, polygons, and textures. It is still difficult to implement this set of APIs through browser APIs. As shown in the preceding figure, the web-version engine is implemented based on DOM and Canvas. The underlying layer defines two graphic APIs, DomCanvas and BitmapCanvas. The transmitted layer tree will be rendered into the element tree of the browser. However, the node only contains styles such as position, transform, and opacity, which are only a small subset of CSS. Rather, some more complex renderings are directly implemented in 2D.
I have compiled a rather complicated demo and tried it. The performance is not ideal, the scrolling is not smooth, and sometimes the images flash. The generated JS code has a size of 1.1 MB (without gzip after minify), and the node level is relatively deep. I estimate that, developing the page in the frontend would not produce a code size greater than 300 KB, and the number of nodes could be cut in half at the very least.
If you look at the issues in the Flutter repository and filter out platform-web data, you will see a large number of issues, such as text editing failures, the cursor cannot be found, ListView is not scrollable on iOS, unexpected checkbox or button behaviors, scrolling lag and image flashing when you scroll images on Android, invalid fonts, videos cannot be played on some models, selected text cannot be copied, and debug failures. With Flutter for Web, we have fallen into a trap. It reminds me of the nightmare that frontend developers had to deal with to ensure compatibility with different browsers.
The main reason for these performance and compatibility issues is that browsers do not expose enough underlying capabilities and process gestures and user input very differently from what Flutter does.
The Flutter engine requires underlying graphic APIs and system capabilities. Although similar graphic APIs are provided, if all of them are implemented with Canvas, it is difficult to deal with issues such as accessibility, text selection, gestures, and forms. This will also lead to many compatibility issues. Therefore, the true solution combines Canvas with DOM, but the encapsulation level is too deep and the rendering process is too long. It is as if, in the Flutter framework, after a series of difficult operations, the nodes are generated, the layout is calculated, and the drawing attributes are also processed. All that remains to do is render a canvas with these resources. Then, the process is handed over to the browser, where elements are generated again and the layout is calculated again in the rendering process. Only then do we go to the underlying graphic library for drawing.
Another example is scrolling on a long page. Only CSS (overflow:scroll) code in the browser can make elements scrollable. Gesture listening, page scrolling, and scrolling animation are implemented natively by the browser. There is no need to interact with JS, or even to perform layout and paint again but composite. As shown in the preceding figure, Animation and Gesture are implemented by Dart in Flutter and are compiled with JS. The browser does not know whether this element is scrollable but continuously dispatches the touchmove event. JS calculates the node offset according to the event attribute, computes the animation, and then applies transform or a new position to the node. Then, the browser goes through the complete rendering process again.
Performance and compatibility issues still remain. In the short term, we can solve the individual issues. For long-term optimization, there are two official approaches:
1. Use the CSS Painting API for rendering.
a. This is a new standard that has been proposed. It holds that JS should be used to implement some rendering functions and custom CSS properties.
b. This solution has not been implemented yet, so we must wait until browsers can support CSS Houdini stably.
2. Use Skia in the WebAssembly version for rendering. https://skia.org/user/modules/canvaskit
a. This approach gives full play to the performance advantages of Wasm and keeps Skia functions consistent. However, Wasm does not necessarily provide performance advantages in a browser environment. Here, we will not go into this issue.
b. This function has been partially implemented. For more information about how to enable it, see the configuration at: https://github.com/flutter/flutter/issues/41062#issuecomment-533952994.
Both solutions attempt to make more use of the browser's underlying capabilities. Here, the idea is that only when the browser exposes more underlying capabilities can they better implement the Flutter web engine. However, this will take a long time, and we cannot participate in the process. To use Flutter for Web at the current stage, we still have to maintain the existing architecture and work together to solve the issues, giving priority to ensuring functionality. At this stage, performance optimization is of secondary importance.
If we are permitted to imagine, how can we better integrate the Flutter and web ecosystems from an architectural perspective?
Looking back at the architecture diagram at the beginning of the article, the framework (Dart) is on top, and the engine (C++) is under it. Divided by the foundation layer, the interaction between both components uses geometric figure information. If this architecture is retained, we can move the division between both components upward to between the Widgets and Rendering layers, as shown in the following figure. Theoretically, this change would be imperceptible to Flutter developers because the upper-layer development language and widget APIs remain unchanged.
If this is the case, the interaction between the framework and the engine would no longer involve geometric figures but node information. Widget combination, setState responsive updating, and widget diff operations would still be done in Dart, while the layout, rendering, cropping, and animation of expanded RenderObjects would all be done in C++. This solution would offer better performance and improved integration with the engine.
Alternatively, we could keep the original design of the engine and encapsulate the sunk logic into a Renderer layer, as shown in the following figure:
These three layers each have a clear purpose:
This solution not only provides performance advantages, but also removes the rendering engine's dependency on Dart and supports multiple languages and development modes. If you integrate a Dart VM, you can use Dart to write code. If you integrate a JS engine or a JVM, you can respectively use JS or Java for the same purpose. Whichever is the case, the underlying rendering capability remains the same regardless of the used language, and the layout algorithms and animation and gesture processing behaviors are also consistent among all cases.
With this architecture, it would be easier to connect to the web ecosystem. Dart and widgets are not wanted at the frontend. I hope they can be replaced by JS and CSS, but I also want an underlying rendering engine that is consistent across platforms. Therefore, I can start the integration from the Renderer layer, bypassing what I do not want and keeping what I do.
This would also make it easier to implement Flutter for Web. When integrating at the engine layer, we always suffer from the insufficient underlying capabilities exposed by the browser. This situation is improved when we integrate at the Renderer layer. We can encapsulate a set of rendering APIs based on JS, CSS, DOM, and Canvas capabilities for widget calls, simplifying the rendering process. However, we still must address the compatibility issues between widgets and DOM or CSS.
As shown in our technical analysis, to connect the Flutter ecosystem to the web ecosystem, you need to invest a lot. Therefore, you must be sure you really want to do this.
First, Google's official positioning of Flutter is ambiguous. When they first designed Flutter, they did not consider the web ecosystem, or even deliberately avoided it. Instead, the team advocated a more native development method. The reason why I said that both ecosystems should not be connected at the beginning of the article is simple: the design concepts of the two technologies are different and they are not moving in the same direction. Their ecosystems and technical solutions are not connected, so forcible integration is likely to lose the advantages of both ecosystems. However, many teams in the industry are trying to do just this. This shows there is a real need for a solution. However, if Google resists this trend, it will be difficult to find a satisfactory approach. Nevertheless, Flutter for Web is now officially supported, which marks a step closer to the web ecosystem. It is also possible that Google will make it easier to integrate with web in the future.
Another difficulty facing our attempts to integrate these ecosystems is cross-platform technology itself. Browsers have developed for 20 or 30 years and are already very powerful cross-platform products. In fact, they are almost synonymous with the web. However, they are also bloated and carry a great deal of historical problems. For example, they do not provide satisfactory performance or user experience and have poor compatibility with native apps, especially on mobile and Information of Things (IoT) platforms. Although hardware performance is continuously improving, this is not true for software. The performance and experience provided by browsers are always worse than those of native pages. This gap provides an opportunity for new businesses and scenarios. If we look at the business scenarios built in recent years, we will see that many of them, such as artificial intelligence (AI), augmented reality (AR), video streaming, and live streaming, only become popular after taking advantage of the new "native" capabilities provided by Android and iOS. Is there a business model out there waiting to be born from new web APIs?
If you think you have the answer, please share your views in the comments.
Are you eager to know the latest tech trends in Alibaba Cloud? Hear it from our top experts in our newly launched series, Tech Show!
Alibaba Clouder - September 21, 2020
淘系技术 - December 9, 2020
Alibaba Clouder - May 17, 2021
Alibaba Clouder - December 22, 2020
XianYu Tech - September 11, 2020
Alibaba Clouder - February 8, 2021
Mobile Testing
Provides comprehensive quality assurance for the release of your apps.
Learn More
Simple Log Service
An all-in-one service for log-type data
Learn More
mPaaS
Help enterprises build high-quality, stable mobile apps
Learn More
Log Management for AIOps Solution
Log into an artificial intelligence for IT operations (AIOps) environment with an intelligent, all-in-one, and out-of-the-box log management solution
Learn More