Being wrong

I think it’s important to note this in public: Sometimes, I’m wrong.

I don’t know everything. Sometimes, my opinions are based on things I’ve been told second-hand, or on things I don’t fully understand yet. Sometimes, the facts that I believe to be truthful and accurate turn out to be outdated, or incorrect. Sometimes, I do or say things that I believe to be the best possible action or statement to make at the time, and it isn’t.

The Web, and the Internet at large (which includes non-Web ecosystems like email, IRC, and so on), is full of too many people that never want to admit they are wrong. The Internet is a fairly permanent medium of record; if you are wrong, it will be forever very obvious that you were wrong. I believe this contributes to people going out of their way to always be “right”, even when they know they aren’t. Otherwise they will look foolish in public forever.

Well, sometimes I’m wrong. That wrongness has been recorded before. In public. Forever. It will almost assuredly happen again, too. And when it does, I’ll note it, and possibly write a correction article, or a response email, or ping an affected party on IRC, and note that I was wrong.

The Internet needs more people to admit to being less than perfect, and to note their assumptions when writing. I hope that leads to more intellectual honesty, of which I find the Internet to be increasingly devoid.

The problem with “patches welcome” culture

I’m going to tell you a secret.

Most computer users cannot write computer code. (Shock!) This is not anything new, and I would dare say this is not even necessarily a problem that needs to be corrected. In a similar vein, a great deal of first-world citizens use cars daily, but I doubt many drivers would be able to fully rebuild an engine, or even describe the difference between EFI and carburetors.

This is the fundamental flaw behind the “patches welcome” culture, and why some libre open source projects have less-than-ideal user experiences, and some even have communities that most would describe as “elitist”.

While drafting this article, a few people told me that they agreed with the substance but did not like that I was using “patches welcome” to describe the culture. While it is correct that most libre software projects should be welcoming of patches, that is not what this article is about. When these people say “patches welcome”, it is a deflection; they don’t want to put forth the effort to properly maintain their software.

Let’s consider an example of this. In a welcoming environment that fosters participation and communication, a request for a feature from a user typically goes somewhat like this:

User: I would really like to be able to select an entire sentence using a key combination so that I can make the sentence bold or underline without dragging.
Developer: Okay. We’ll add that to the list of features that have been requested. Thank you for telling us!

Sometimes these features take time to add; maybe some will never see the light of day. Nevertheless, the user still informed the developer of the software what they needed, which allows the developers to make better choices about how they approach building the software, and what features to prioritise.

Now, in a project with “patches welcome” culture, the users are ignored or even chastised. I’ve actually seen discussions very similar to the following take place:

User: I would really like to be able to select an entire sentence using a key combination so that I can make the sentence bold or underline without dragging.
Developer: patches welcome
User: I don’t know how to write code.
Developer: Then you shouldn’t ask us for help.

This behaviour undermines what libre software is supposed to stand for. It gives the user reason to go back to using proprietary software, where they can call up Microsoft or Apple and tell them what they want or ask for help when they need it. Even if the proprietary software corporations never add their suggestion, they still feel more connected and respected than they do by this example libre project. This behaviour gives the user no reason to use the software that respects them and their freedoms. Free Software is meaningless if it has no users to use it.

Some may argue that people should be empowered to learn to program, and there definitely is a case to be made for that. However, you really need to consider all the reasons people *would not* want to learn to program:

  • They would rather spend that time with family, friends, or their hobbies.
  • They have learning or mental disabilities that make algorithmic reasoning, logic, or concentration required to program difficult.
  • They have physical disabilities that make programming difficult.
  • They simply aren’t interested.

That last item is especially important. Do you want someone who has no interest in programming – no interest in security, or correctness, or doing things the Right Way (or even the way you want them done) – to commit code to your repository? Are patches really welcome, or are you just deflecting the requests of your users so you don’t have to maintain the software you’ve written?

All people deserve Free Software. Nobody deserves to be denigrated, shamed, or ignored because of their inability or lack of desire to program a computer. My personal suggestion to those who do not want to accept feature requests unless patches are attached is to not publicly release your software. If you do, add a notice stating that you do not wish to be contacted about your software by users, so that they may make an informed decision about the software that they use.

Further thoughts on Wayland

Previously on The Cat Fox Life, I wrote an accidentally now-infamous article on what I felt were lies/misconceptions that Wayland users were spreading about the Wayland system.

One of the Wayland community’s more prolific developers wrote a rebuttal to my article, then directly responded to me on HN. I hadn’t yet gotten around to properly responding to this in earnest, because I had a lot of personal IRL drama to deal with in February. I’m feeling a little under the weather today, with a minor head cold and sore throat, but Drew has reached out to me personally and the time to write this is now.

Before I begin, I feel like the original introduction to my previous article may have been worded slightly incorrectly. This was never an indictment of the developers of Wayland, but rather the rabid fanboys who spread BS on Reddit, forums, IRC, and such. There are rabid fanboys of systems I actually like, too; musl, s6, KDE, and Firefox come to mind. I don’t like those fanboys either, and I could probably write articles about lies they come up with, too. (Welcome to the Internet, I suppose.)

Drew wrote: LD_PRELOLAD hacks don't work if the compositor launches the programs - some simple .bashrc trick won't work, and getting the LD_PRELOAD into .bashrc requires being unsandboxed, which itself opens up a wealth of side channel attacks. None of this is a mark against Wayland - it's only one part of a secure system, and the other parts are mandatory.

That’s perfectly correct, and I noted in my previous article: “I’ve been told that mentioning something that uses LD_PRELOAD is cheating and that you could own any application, not just Wayland. That is true! But this is being sold as “impossible to keylog”. It isn’t.”. This isn’t a mark against Wayland, but again, a mark against dumb people saying things they shouldn’t be and generating buzz. There have been multiple people on IRC and Reddit that have said that Wayland is immune to keylogging without clarifying that it’s the protocol, not the implementation. The developers of Wayland know that you need to secure the whole system, but these fanboys don’t.

Drew wrote: many implementations are widely supported by older hardware. wlroots, which is the dominant Wayland ecosystem with over 75% of all compositors using it for their rendering, requires only GLESv2, which is the most broadly supported OpenGL standard.

I said in my article: “Wayland compositors universally require OpenGL profiles that older hardware, less expensive hardware, libre hardware, and most embedded chipsets do not provide.” GLESv2 is still not going to work on framebuffers, libre FPGAs, or embedded chipsets. At least, not without LLVMPipe, which would use a lot of CPU time and power on the hardware where it’d be relevant. That said, Drew wrote in his response: “writing an fbdev backend is totally possible and I’d merge it in wlroots if someone put in the time.” This gives me hope that perhaps it may be possible to run Wayland compositors on framebuffers some day. Additionally, I was too harsh when I said that Wayland is “designed” to “require” blobs. Of course it isn’t, and I shouldn’t have said that.

I would also like to note that although wlroots is used by “over 75% of all compositors”, there are four main compositors that I would suggest people think of when they think of Wayland: KWin (KDE Plasma), Mutter (GNOME, replacing Metacity), Sway, and Enlightenment. As far as I know, wlroots is only used by Sway out of these four. So while it has high number of market share, I’m not sure it has the same high number of installed user base, which coloured my initial perception on compositor GPU requirements. I am not sure what Mutter requires out of GPUs, but KWin’s Wayland compositor requires more than base GLESv2, and far more than KWin’s X11 compositor, which supports XRender (software) and OpenGL 2.0. Similarly, in my experience Enlightenment runs much better on older hardware using X11 rather than Wayland. It is good to know, though, that not all Wayland developers feel that they should be overutilising GPUs so much. I am glad that if wlroots had to require OpenGL, GLESv2 is the standard they chose.

Drew wrote: For network transparency, it wouldn't be difficult but no one who cares has stepped up to do the work. Multiple clipboards are now supported. Many of the pieces of remote desktop are in place, and again someone who cares needs to step up to implement the rest and tie it together. We are ready and waiting to support anyone who wants to step up to complete this work.

This was actually very nice to hear! Wayland developers seem to want network transparency, but just don’t have the interested parties right now to make it reality. It’s good to know that ideas are not rejected in Wayland like the Wayland proponents in /r/linux said they would be because “that’s legacy X11 garbage”. (I suppose this comes down, again, to fanboys yelling and screeching.)

Drew wrote: A bug in xorg-server will similarly bring down your X session. Driver bugs affect both. Bugs are addressed when they're found, what more do you want from us? Regardless, restart protocols are in the research phase and your comments about the security holes implicated are blatant speculation.

The difference being I rarely hit driver bugs or Xorg server bugs, but regularly hit compositor bugs. (KWin has crashed 27 times in the past year. Xorg, once, due to – you guessed it – a radeon.ko bug.)

Yes, the security holes are blatant speculation and were phrased in a mean and combative way. It was wrong and I publicly apologise, not only to Drew and other Wayland developers, but to any that may have felt personally attacked. Of course I didn’t mean it to be a personal attack, but it came off that way due to being written poorly. I regret that, and hope to learn from this experience.

As a conclusion, I still personally don’t see myself running Wayland any time soon. However, I would be happy and willing to merge Wayland packages in to Adélie Linux if someone is willing to maintain them.

Booting a custom kernel/OS on the Raspberry Pi 3 B (64-bit)

On Wednesday afternoon, we were having a discussion of various 64-bit ARM SoCs on the Adélie Linux chat. I mentioned that the only one I had was a PINE64, which did not support graphical display on the version of the Linux kernel we ship. Michael, one of our contributors, suggested I should buy a Raspberry Pi 3 for testing graphical software (such as KDE and Firefox). I was surprised to learn that Target carries the RPi3 and they had a few in stock locally. Off I went, stopping in for a 128 GB µSD card as well.

Then began the fun: I had a RPi 2B some years ago, but it was lost in the move. I only have experience booting these things using NOOBS, and I wanted to use NOOBS anyway because that would allow me an easy way to have multiple system images and choose which one to boot. This will let me perform 32-bit ARM testing on the RPi3 later on, when we support 32-bit ARM a bit better. Therefore the first order of business was to figure out how NOOBS’ boot menu works, so that I could write a custom configuration with the partition layout how I desired:

  • Shared /home, 40 GB XFS
  • 64-bit /boot volume, 1 GB FAT32
  • 64-bit root (/) volume, 39 GB ext4
  • 32-bit /boot volume, 1 GB FAT32
  • 32-bit root (/) volume, 39 GB ext4

Since NOOBS erases the SD card on first boot, I booted NOOBS once on the Pi before continuing. Then I went to work in fdisk on my Talos:


Disk /dev/sdb: 119.3 GiB, 128043712512 bytes, 250085376 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xb41dea51

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sdb1            8192    137215    129024    63M  e W95 FAT16 (LBA)
/dev/sdb2          137216 250085375 249948160 119.2G  5 Extended
/dev/sdb5          139264    204797     65534    32M 83 Linux
/dev/sdb6          206848  81995775  81788928    39G 83 Linux
/dev/sdb7        81997824  84092927   2095104  1023M  c W95 FAT32 (LBA)
/dev/sdb8        84094976 167981055  83886080    40G 83 Linux
/dev/sdb9       167983104 170078207   2095104  1023M  c W95 FAT32 (LBA)
/dev/sdb10      170080256 250085375  80005120  38.2G 83 Linux

Through trial and error, with a fairly good reference and reading the source code of NOOBS, I was able to divine the layout of my desired installed_os.json file. Once NOOBS is booted for the first time, it makes a small “settings” partition; this was /dev/mmcblk0p5 on the Pi and /dev/sdb5 in the SD card reader of my workstation.


[
        {
                "description": "A friendly, easy-to-use desktop environment that uses little resources",
                "folder": "/mnt/os/Adelie64",
                "icon": "/settings/os/adelie.png",
                "name": "Adélie Linux (64-bit)",
                "partitions" : [ "/dev/mmcblk0p7", "/dev/mmcblk0p8" ],
                "release_date": "2019-02-21"
        },
        {
                "description": "A friendly, easy-to-use desktop environment that uses little resources",
                "folder": "/mnt/os/Adelie32",
                "icon": "/settings/os/adelie.png",
                "name": "Adélie Linux (32-bit)",
                "partitions" : [ "/dev/mmcblk0p9", "/dev/mmcblk0p10" ],
                "release_date": "2019-02-21"
        }
]

I placed this file in the root of the settings partition (/dev/sdb5 in this example) and downloaded the header image from our Web site to /os/adelie.png on the same. Now the structure looked like:


gwyn /mnt/PiStuff # tree
.
├── cache
│   ├── data7
[ lots of irrelevant files removed ]
├── installed_os.json
├── lost+found
├── noobs.conf
├── os
│   └── adelie.png
└── wpa_supplicant.conf

I then unpacked the Adélie Linux aarch64 root FS image to the root partition (/dev/sdb8 in this example). I downloaded the needed firmware files for /boot from the official Raspberry Pi Foundation GitHub repository, and put them in the boot partition (/dev/sdb7 in this example). Then came a full day of trial and error with the kernel.

Kernel config

Our Adélie easy-kernel package boots great on the PINE64 and a few other SoCs, but it needed small adjustments to be comfortable booting on the Raspberry Pi 3 B. In addition to the configuration knobs noted by the Raspberry Pi Foundation, we added CONFIG_FB_SSD1307=m and CONFIG_FB_UDL=m for eventual support of the Pi’s touchscreen interface. (I have not yet tested that and do not know if this is sufficient or not.)

Next was spending four hours trying to understand why the system booted properly but had no display on-screen. dmesg contained:


[   11.423042] vc4-drm soc:gpu: failed to bind 3f902000.hdmi (ops vc4_hdmi_ops [vc4]): -517
[   11.423147] vc4-drm soc:gpu: master bind failed: -517

As it turns out, this was because I was using DTB files from a different kernel build. It’s incredibly important to use the DTB files that ship with the kernel you are testing with! After installing the correct DTB files to the SD card and booting again, a different message was output:


[   14.347564] vc4_hdmi 3f902000.hdmi: vc4-hdmi-hifi  3f902000.hdmi mapping ok
[   14.348456] vc4-drm soc:gpu: bound 3f902000.hdmi (ops vc4_hdmi_ops [vc4])
[   14.348689] vc4-drm soc:gpu: bound 3f806000.vec (ops vc4_vec_ops [vc4])
[   14.349175] vc4-drm soc:gpu: bound 3f400000.hvs (ops vc4_hvs_ops [vc4])
[   14.349459] vc4-drm soc:gpu: bound 3f206000.pixelvalve (ops vc4_crtc_ops [vc4])
[   14.349687] vc4-drm soc:gpu: bound 3f207000.pixelvalve (ops vc4_crtc_ops [vc4])
[   14.349903] vc4-drm soc:gpu: bound 3f807000.pixelvalve (ops vc4_crtc_ops [vc4])
[   14.349968] vc4-drm soc:gpu: failed to allocate buffer with size 16777216
[   14.349995] vc4-drm soc:gpu: failed to allocate buffer with size 16777216
[   14.350080] [drm:vc4_bo_create [vc4]] *ERROR* Failed to allocate from CMA:
[   14.350092] vc4_v3d 3fc00000.v3d: Failed to allocate memory for tile binning: -12. You may need to enable CMA or give it more memory.
[   14.350195] vc4-drm soc:gpu: failed to bind 3fc00000.v3d (ops vc4_v3d_ops [vc4]): -12
[   14.386347] vc4-drm soc:gpu: master bind failed: -12
[   14.386385] vc4-drm: probe of soc:gpu failed with error -12

This was correct; the solution was adding cma=256M@256M to the kernel command line. The final command line that I use is: root=/dev/mmcblk0p8 rootwait ro cma=256M@256M.

I rebuilt Mesa with VC4 support; our 32-bit ARM package had it already, but we did not enable it on 64-bit ARM. Then I installed KDE and X.Org: blackjack ~ # apk add kde x11 mesa-dri-vc4. Behold!

Adélie Linux running KDE Plasma 5 on a Raspberry Pi 3 in 64-bit mode

Please consider supporting our continued adventures bringing quality Libre Software to more platforms! You can find ways to help out on the Adélie Linux Contribution page, or directly on Patreon, PayPal, or Ko-fi. I hope this article was able to help you learn something new about your Pi.