By Jingshu, from Xianyu Technology Team
The images Xianyu uses are based on a proprietary external texture:
The texture data in the Flutter application layer is not cached, thus requiring Bitmap data to be rendered into texture data and delivered to the Flutter application layer each time. Since the loaded Native images and image library of Flutter are cached separately in memory, plenty of memory is consumed. The Flutter image caches refer to stored local resource images. However, most of the images in Flutter pages are external texture images downloaded online. Thus, the cached resource utilization is very low.
Put aside the technical implementation, what is an ideal solution to solve the preceding three problems?
An ideal solution would see one memory cache exist in the app that could cache the textures and image data loaded by the Flutter Image Widget.
ImageCache is officially attached and can't be removed. Image Widget is also used in the Xianyu app. Now, the ideal solution would see cache texture data in ImageCache, from which textures are acquired.
The following figure shows the current Flutter image loading logic and the process of image caching.
The chart above shows the Flutter image loading using ImageCache.putIfAbsent
to retrieve the cache. If it fails, the system uses the passed-in loader to establish the corresponding ImageStreamCompleter
for the image loading.
If ImageCache.putIfAbsent
works, putIfAbsent
returns ImageStreamCompleter
directly, which contains imageInfo
. Then, Image Widget renders the ui.Image
of imageInfo
.
For ImageCache, putIfAbsent
is the only way to retrieve caches for externals.
In the beginning, it is assumed to establish the corresponding key, loader, and ImageStreamCompleter
according to the parameters of ImageCache.putIfAbsent
, and then use putIfAbsent
to obtain the cache.
The attempt failed. As shown in the following figure, when the image is successfully downloaded and decoded, the listener method is called back. With the listener method, images will be stored in the cache queue of ImageCache.
The listener callback has two parameters with ui.Image
stored in ImageInfo
.
It's impossible to establish ui.Image
at the application layer because it is set to the application layer after the underlying layer of the Flutter engine completes image decoding. Besides, the application layer cannot actively set values. As a result, the imageSize value cannot be computed in the listener or be stored in the cache.
Since the cache queue of ImageCache is private, putIfAbsent
is the only way to store data in the queue. Therefore, it's feasible to start with the source code of ImageCache. ImageCache can be customized by modifying the code, and its functions can be extended.
The code of Flutter ImageCache cannot be modified directly, so the source code of ImageCache is copied. Then, ImageCache of PaintingBinding
is replaced with a customized ImageCache.
As is shown, the createImageCache
method can be observed in Flutter PaintingBinding
. The method can be rewritten to get a customized ImageCache by inheriting WidgetsFlutterBinding
. It is also feasible to set various cache sizes for ImageCache.
A new texture caching method is defined to avoid modifying the ImageCache code as much as possible, which aligns the logic of putIfAbsent
. The core code logic is listed below:
This method is mainly implemented by referring to the logic of putIfAbsent
. You need several key extensions to cache textures into ImageCache:
TextureCacheKey
is the only key to identify textures based on their width, height, and URL.TextureImageStreamCompleter
belongs to texture management, which inherits ImageStreamCompleter
and internally stores callbacks of texture data and successful downloads. When the cache is hit, the hit cache is returned to the application layer, and texture ID is obtained and rendered by Texture Widget.TextureImageStreamCompleter
and executes the logic of texture loading. At the same time, a listener callback is established and registered in TextureImageStreamCompleter
.Note: Images are normally Dart objects and will be automatically recycled by Dart VM. However, the real data of a texture object is located in the shared memory of the Flutter engine. Therefore, it is necessary to manage the release of the texture manually. Based on the reference counting of textures, textures will be truly released; only when no widget holds the texture and the reference counting is 0.
Similarly, when the upper-layer Texture Widget is disposing, the interface provided by ImageCache will also be called to see if the current texture is cached or being used. Only when no texture is cached or being used, will the texture be released.
Take the search results page as the test page. Many large pictures and various duplicate small label pictures are displayed on the page.
The test procedure is listed below:
During this period, the physical memory is sampled in second and lasts for 100s total. The following data are obtained.
The blue curve represents the memory usage before optimization, and the orange curve represents the memory usage after optimization. In the beginning, the memory usage is generally the same. The decrease in memory usage during browsing is caused after GC recycles the app memory. Generally, the total memory usage after optimization is less than before because glitches caused by GC are reduced.
The solution above implements the goal of one memory cache in one app. It also stores the textures and Flutter images, saves memory space, and improves memory usage. However, it still intrudes into the ImageCache source code. Thus, additional work is necessary to upgrade the Flutter engine and maintain the code.
Since Flutter uses putIfAbsent
to load native images with the original size, one image may take up several MB in the app. As a result, the function of large image monitoring is added to putIfAbsent
. When the size of the loaded image exceeds 2 MB, the data, including the URL, usage information, and size of the image, will be reported. By doing so, several cases of improper image use are found. For example, original images are loaded using Image.network
, or Image.asset loads a large image from local resources.
How Does Xianyu Improve the Technical Experience Based on Flutter?
56 posts | 4 followers
FollowXianYu Tech - August 6, 2020
XianYu Tech - September 4, 2020
XianYu Tech - August 10, 2021
XianYu Tech - May 11, 2021
Alibaba Clouder - February 8, 2021
XianYu Tech - September 3, 2020
56 posts | 4 followers
FollowExplore Web Hosting solutions that can power your personal website or empower your online business.
Learn MorePlan and optimize your storage budget with flexible storage services
Learn MoreA low-code development platform to make work easier
Learn MoreExplore how our Web Hosting solutions help small and medium sized companies power their websites and online businesses.
Learn MoreMore Posts by XianYu Tech