That's my blog… Life and Linux

GNOME Fractional (and multi-monitor) Scaling Hackfest, the report

This wasn't a joke!As previously announced, few days ago I attended the GNOME Fractional Scaling Hackfest that me and Red Hat‘s Jonas Ådahl organized at the Canonical office in Taipei 101.
Although the location was chosen mostly because it was the one closest to Jonas and near enough to my temporary place, it turned out to be the best we could use, since the huge amount of hardware that was available there, including some 4k monitors and HiDPI laptops.
Being there also allowed another local Caonical employee (Shih-Yuan Lee) to join our efforts!

As this being said I’ve to thank my employer, for allowing me to do this and for sponsoring the event in order to help making GNOME a better desktop for Ubuntu (and not only).

Going deeper into the event (for which we tracked the various more technical items in a WIP journal), it has been a very though week, hard working till late while trying to look for the various edge cases and discovering bugs that the new “logically sized” framebuffer and actors were causing.

In fact, as I’ve already quickly explained, the whole idea is to paint all the screen actors at the maximum scale value across the displays they intersect and then using scaled framebuffers when painting, so that we can redefine the screen coordinates in logical pixels, more than using pixel units. However, since we want to be able to use any sized element scaled at (potentially any) fractional value, we might incur in problems when eventually we go back to the pixel level, where everything is integer-indexed.

We started by defining the work items for the week and setting up some other HiDPI laptops (Dell XPS 15 and XPS 13 mostly) we got from the office with jhbuild, then as you can see we defined some list of things to care about:

  • Supporting multiple scaling values: allowing to scale up and down (< 1.0) the interface, not only to well-known value, but providing a wider range of floats we support
  • Non-perfect-scaling: covering the cases in which the actor (or the whole monitor) when scaled up/down to a fractional level has not anymore a pixel-friendly size, and thus there are input and outputs issues to handle due to rounding.
  • GNOME Shell UI: the shell StWidget‘s need to be drawn at proper resource scaling value, so that when they’re painted they won’t look blurred.
  • Toolkit supports: there are some Gtk issues when scaling more than 2x, while Qt has support for Fractional scaling.
  • Wayland protocol improvements: related to the point above we might define a way to tell toolkits the actual fractional scaling value, so that they could be scaled at the real value, instead of asking them to scale up to the upper integer scaling level. Also when it comes to games and video players, they should not be scaled up/down at all.
  • X11 clients: supporting XWayland clients

Read the rest of this entry »

GNOME Hackfest for Fractional Scaling

As Ubuntu users know, Unity desktop shell had HiDPI displays support since 14.04, however while the unity shell has been able to scale at any fractional value (despite we limited the setting to only 8 values per integer), GTK3 and GNOME Shell just supported integer scaling values.

Although I see the technical reason for that (pixels can’t be divided!), I think that our approach still worked quite well by using proper round functions, as it’s really hard to notice the visual flaws that this introduced at such high resolutions.

At the same time, to get proper scaling for GTK apps, we used the stratagem of mixing the UI scaling with the text scaling factor, so that the multiplication between the two values will match the user-requested scaling level.
This worked pretty well in single-monitor instances, while in multi-monitor environments the idea was to use xrandr scaling to get matching results, but this has not been done, mostly due to an X11 bug which didn’t allow to go further…

Anyway, this is the past and present, but let’s talk about the future… Ubuntu will use GNOME Shell (in Wayland, when possible), and so far there’s no support for multi-monitor scaling or fractional scaling, but things are changing!

Jonas Ådahl is leading this efforts and he started with supporting a new configuration API for monitors, that is a prerequisite for pursuing the GNOME Fractional Scaling initiative.
Basically, the main implementation idea is to make GTK and various toolkits to scale at an higher value, and then using mutter to scale actors down at composition level.
While it might be a little more resource intense, it’s also true that this will work nicely (especially in multi-monitor environments with different scaling values) and that there’s no other option, given that GTK scaling system can’t be changed at this point (too many things now assume it’s an integer).

Ubuntu cares about having a proper HiDPI support in next releases, so the Desktop Team decided to join the upstream initiative, and since I’m currently around asia, we’ve organized a GNOME Hackfest, that will be hosted by Canonical in its Taipei office in Taipei 101 in order to continue the work Jonas is doing and plan how to proceed with future work items.