Treviño's Blog

The fractional scaling era for GNOME shell has finally arrived!

The news spread out quite quickly, once last Friday Jonas pressed the button and that triggered the last-second merge for the relevant proposals we prepared for Mutter and GNOME Shell in order to get this available for GNOME 3.32.

As someone might recall, we started this work some years ago (ouch!) and lead to an Hackfest in Taipei, but in between other work to do and priorities which caused this to be delayed a bit. While the first iteration was ready for some time now. But at every review we improved things fixing bugs (like missing scaled widgets) and optimizing some code paths, so hopefully this time helped in serving better quality :).

We’ve still quite a lot of work to do (see these issues for mutter and shell) and some fixes that we have in queue already, but the main task is there. So starting from now the shell will paint all its elements properly and in good visual quality at any fractional scaled value, and independently for every monitor.

Monitors with different scaling values, and a window drawn in between the two

As you might have noticed in the screenshot above, the X11 apps are still not really scaled in quality, while it’s not possible for them all (like xterm there), we need to work for a solution that will cover the legacy applications which does support scaling, and at the same time those which doesn’t want to be scaled at all (games!).

As per what said above, this feature is still considered experimental and then you need to enable it via:

gsettings set org.gnome.mutter experimental-features "['scale-monitor-framebuffer']"

Doing this will allow you to set more a wider set of scaling values under Control Center, display panel.

For what concerns Extensions, most of them should work with no change, but not those using directly the St.TextureCache, as we changed the methods signature by adding a resource_scale parameter.
We discussed weather adding another method instead (as gir doesn’t support default values), but since 3.32 will need anyway a rewrite of most extensions anyways, and since it’s better to have them to behave properly with resource scale since the beginning (instead of blurred contents), we decided not to do it.
So, please sync with this change (and sorry :)).

Putting my Ubuntu hat now, this won’t change much things for default ubuntu experience, since it’s still using X11 (although I’ve something in the works for this too), but people who want to take advantage of this can easily just login using the Ubuntu on Wayland session, enable the experimental setting and profit.

As final world, thanks to people who helped having this in by reviewing and testing the code.

As previously announced, few days ago I attended the GNOME Fractional Scaling Hackfest that me and Red Hat‘s Jonas Ådahl organized at the Canonical office in Taipei 101.
Although the location was chosen mostly because it was the one closest to Jonas and near enough to my temporary place, it turned out to be the best we could use, since the huge amount of hardware that was available there, including some 4k monitors and HiDPI laptops.
Being there also allowed another local Caonical employee (Shih-Yuan Lee) to join our efforts!

As this being said I’ve to thank my employer, for allowing me to do this and for sponsoring the event in order to help making GNOME a better desktop for Ubuntu (and not only).

Going deeper into the event (for which we tracked the various more technical items in a WIP journal), it has been a very though week, hard working till late while trying to look for the various edge cases and discovering bugs that the new “logically sized” framebuffer and actors were causing.

In fact, as I’ve already quickly explained, the whole idea is to paint all the screen actors at the maximum scale value across the displays they intersect and then using scaled framebuffers when painting, so that we can redefine the screen coordinates in logical pixels, more than using pixel units. However, since we want to be able to use any sized element scaled at (potentially any) fractional value, we might incur in problems when eventually we go back to the pixel level, where everything is integer-indexed.

We started by defining the work items for the week and setting up some other HiDPI laptops (Dell XPS 15 and XPS 13 mostly) we got from the office with jhbuild, then as you can see we defined some list of things to care about:

• Supporting multiple scaling values: allowing to scale up and down (< 1.0) the interface, not only to well-known value, but providing a wider range of floats we support
• Non-perfect-scaling: covering the cases in which the actor (or the whole monitor) when scaled up/down to a fractional level has not anymore a pixel-friendly size, and thus there are input and outputs issues to handle due to rounding.
• GNOME Shell UI: the shell StWidget‘s need to be drawn at proper resource scaling value, so that when they’re painted they won’t look blurred.
• Toolkit supports: there are some Gtk issues when scaling more than 2x, while Qt has support for Fractional scaling.
• Wayland protocol improvements: related to the point above we might define a way to tell toolkits the actual fractional scaling value, so that they could be scaled at the real value, instead of asking them to scale up to the upper integer scaling level. Also when it comes to games and video players, they should not be scaled up/down at all.
• X11 clients: supporting XWayland clients

Read the rest of this entry »

As Ubuntu users know, Unity desktop shell had HiDPI displays support since 14.04, however while the unity shell has been able to scale at any fractional value (despite we limited the setting to only 8 values per integer), GTK3 and GNOME Shell just supported integer scaling values.

Although I see the technical reason for that (pixels can’t be divided!), I think that our approach still worked quite well by using proper round functions, as it’s really hard to notice the visual flaws that this introduced at such high resolutions.

At the same time, to get proper scaling for GTK apps, we used the stratagem of mixing the UI scaling with the text scaling factor, so that the multiplication between the two values will match the user-requested scaling level.
This worked pretty well in single-monitor instances, while in multi-monitor environments the idea was to use xrandr scaling to get matching results, but this has not been done, mostly due to an X11 bug which didn’t allow to go further…

Anyway, this is the past and present, but let’s talk about the future… Ubuntu will use GNOME Shell (in Wayland, when possible), and so far there’s no support for multi-monitor scaling or fractional scaling, but things are changing!

Jonas Ådahl is leading this efforts and he started with supporting a new configuration API for monitors, that is a prerequisite for pursuing the GNOME Fractional Scaling initiative.
Basically, the main implementation idea is to make GTK and various toolkits to scale at an higher value, and then using mutter to scale actors down at composition level.
While it might be a little more resource intense, it’s also true that this will work nicely (especially in multi-monitor environments with different scaling values) and that there’s no other option, given that GTK scaling system can’t be changed at this point (too many things now assume it’s an integer).

Ubuntu cares about having a proper HiDPI support in next releases, so the Desktop Team decided to join the upstream initiative, and since I’m currently around asia, we’ve organized a GNOME Hackfest, that will be hosted by Canonical in its Taipei office in Taipei 101 in order to continue the work Jonas is doing and plan how to proceed with future work items.