Category: Technology

The musl preprocessor debate

Today, I would like to discuss a project that I care very deeply about: the musl libc. One of the most controversial and long-standing debates in the musl community is that musl does not define a preprocessor macro.

What’s in a macro?

Simply put, preprocessor macros allow C code to build parts of itself conditionally. For example, the GNU libc defines the “__GLIBC__” macro. If your code needs to do something specific to function properly on systems using that library, it can conditionally build that code using “#ifdef __GLIBC__”.

The authors of musl have said that they will not add a preprocessor macro identifying the platform as musl because:

It’s a bug to assume a certain implementation has particular properties rather than testing.

Rich Felker, “Re: #define __MUSL__ in features.h”, 2013-03-29

I agree with this sentiment in theory, and in an idealised world this would hold up. However, I’d like to discuss why I think this may need to be reconsidered moving forward.

Sometimes you can’t test

One major reason this is an issue is that sometimes it is not possible to do what the authors consider the “correct” form of testing, which is compile-testing.

This practice requires you to build a small test program, determine whether it built properly, determine its runtime characteristics, and then use the results of that test to influence how your actual software is built. This is an alternative to using the conditional code with preprocessor macros.

However, there are many reasons you may not be able to successfully perform such testing. Cross compilation is a large gap here. In fact, many years ago when I was starting the Adélie project, this caused failures in the base image I was building.

The Bash shell could not perform any compile-time or run-time checks because it was being cross-compiled from a GNU libc system to a musl libc system. This caused it to use “fallback” code that worked improperly. If musl had defined a __MUSL__ macro, Bash would not have needed to assume it was running on a pre-POSIX system.

Similarly, the mailing list thread that made me feel strongly enough to write this article involves a header-only library. These types of libraries are meant to be “drop-in” and function without any changes to a developer’s build system. If header-only libraries start requiring you to use build-time tests, you lose the main reason to use them in the first place.

The author of this thread correctly points out that FreeBSD versions their API with a preprocessor macro. Any software that requires a certain API can simply ensure that __FreeBSD_version is defined as greater-or-equal than the versions that introduced that API.

The main reason that the musl project is fearful of this approach, at least to my observation, is that features or APIs (or indeed, bug fixes) can be backported to prior versions. I feel very strongly that this is not the responsibility of the libc.

If a distribution backports a feature, API, or patch to an older version of a library, it is that distribution’s responsibility to ensure that the software they build against it continues to function. When I backported an API from Qt 5.10 to 5.9 to ensure KDE continued building for Adélie, it was my responsibility as maintainer of those packages to keep them building properly. It certainly does not mean Qt should stop defining a preprocessor macro to determine the version being built against.

Additionally, some APIs are privileged. Determining whether these APIs work correctly using run-time testing can prevent CI/CD from working properly because the CI user does not have permission to use them.

A versioned macro like FreeBSD’s makes sense

I feel that the best way forward for musl is to define a macro like FreeBSD’s. It monotonically increases as APIs or features are added.

I agree that simple bug fixes, and even behavioural changes, probably should not be tracked with this macro. However, this would make it significantly easier to use new APIs as they are introduced.

It also makes builds more efficient. The cost of compile-time tests racks up quickly. On my POWER9 Talos workstation, typical ./configure runs take longer than the builds themselves. This is because fork+exec is still a slow path on POWER. It is similar on ARM, MIPS, and many other RISC architectures.

Macros like these don’t fully eliminate the need for ./configure, but they lessen the workload. Compile-time tests make sense for behaviour detection, but they do not make sense for API detection.

Daily-driving a Mac, one year later

It has been about a year since I published Really leaving the Linux desktop behind. This marks the first year I’ve used Mac OS as my primary computing environment since 2014. Now, I want to summarise my thoughts and feelings on using the Apple ecosystem as my primary platform – good and bad.

The Amazing

Universal Clipboard

Universal Clipboard has dramatically simplified my blog workflow. Typically, all of my articles are drafted and composed on my iPad Pro, as WordPress offers a great native app. This allows me to avoid using a browser. As I write this article, I am copying the links out of Safari on my Mac. They immediately show up in the pasteboard of the iPad.

KDE Connect does offer a Clipboard plugin, but it only supports plain text at this time. Apple’s implementation allows you to copy rich text, photos, files, and more.

Something else I would really like to note is that these features will work on High Sierra and later. It is transparent to the user no matter what version of the OS they are running. This somewhat alleviates the issue of newer OS versions having newer device requirements.

Safari Tab Groups

Safari 15 introduced the concept of Tab Groups, which is something I have been missing a lot since Firefox killed off extensions and replaced them with a severely limited alternative. Tab Groups simply allow you to categorise groups of tabs into cohesive sets. You can almost consider it a “focus window” where a logical set of tabs live.

The best part is that Safari’s Tab Groups sync between iCloud devices, which means I can use, add to, and manipulate tab groups on my tablet and phone as well. The replacement extension I used for Firefox had most of the features I enjoyed, but it doesn’t sync between devices using Firefox Sync, which meant I could really only browse the Web in this powerful way on my main desktop (then, my Talos II).

Having tab groups that sync between devices has allowed me to bring order to my previously chaotic Web browsing habits, allowing me to focus better and waste less time being distracted.

AirPlay

I love having the ability to AirPlay my screen directly to a TV. As far as I am aware, it is not possible on Linux to share your entire display to a smart TV without complicated command-line invocations that change regularly.

I used this to show my grandmother family photos while she was recovering from a health challenge. We use this monthly for budget planning in our household – just share Excel to the TV and we can see and discuss where the money is going this month.

There is no reason that this couldn’t be implemented on Linux, but off the top of my head I can think of a few challenges: the TV may have a different DPI than the computer screen (which has always been challenging for Linux windowing systems and toolkits), compressing the video using a libre codec while providing good picture quality and low bandwidth usage, and the general sorry state of wireless networking in Linux (which is due to the chipmakers, I know).

Apple Maps

Having a Maps app on my computer that I can use to view place details, satellite imagery, landmark information, and plan routes is a very powerful tool. I use this regularly to find places to shop local, and to plan weekend excursions to parks and attractions.

The closest thing I found on Linux was Marble. While I did enjoy the fact that Marble integrated so well with OSM, the views were always slightly grainy and off. Zoom and pan needed work and I could never understand the code well enough to contribute a fix.

You can still boot Linux on them

The Asahi Linux project has done an amazing job on building a boot loader for the M1 that should allow a whole host of alternative systems working. This includes not just Linux but also the BSDs, and perhaps even illumos when they bring up ARM64 support.

I remember when the M1 came out, everyone thought the firmware would be locked down and prevent non-Mac OS systems from running at all. It turns out that not only did this not happen, but you can actually sign your own kernels and have Trusted Boot using your own compiled Linux. This may end up making the M1 more libre-friendly than x86 systems.

Misc

  • Apps like Things really demonstrate the power of the Mac platform and what is on offer. You could probably make something as nice and integrated as Things on Linux, but for someone as busy as me, it is nice to use what is already there.
  • I feel much more in control of notifications on the Mac platform than I did on Linux with libnotify and Plasma. Notifications can be handled per-app, not just per-notification in the app itself. “Focus modes” (DnD) sync with my other devices like my phone and tablet. I can set repeating schedules (or one-offs) with profiles that allow some apps through but not others.
  • Older devices really are still supported. Even if you can’t boot Big Sur or Monterey on them, which is a big list if you are willing to play with a patching system, most of the niceties I’ve written about work back to High Sierra.

The drawbacks

The only real drawback that I’ve found in this year is that since the Mac isn’t a fully libre open-source system, I can’t fix the few bugs that I’ve run into.

I have not felt “trapped” or “helpless” or at all like I am living in a walled garden. Terminal is still there, unsigned apps can still be run with a simple context-click, and AppleScript (and now Shortcuts) is available to automate workflows.

I still believe that libre software ideals are correct and the goal of having a libre operating environment is a good one. However, I also believe that it was perhaps naive of me to believe that such a thing can truly exist in the way I hoped it could. The people who develop libre operating environments have different priorities.

And when you are spending your days using technology instead of making technology, the libre software ideals genuinely do become more of a theoretical than something in your face. This can be good, or bad, depending on your viewpoint.

At the end of the day, my goal in life is to make a difference, and also have a bit of fun. I want a system that is out of my way and lets me focus on that. For me, in 2022, that system is a Mac.

A final word on cost

Far too many people are priced out of the Apple ecosystem. I understand that part of the high cost of Apple products are to subsidise the R&D of making all these things work so well. However, they also have pretty high profit margins beyond their R&D expenditures.

I wish that Apple would lower their price, even a little, so that this amazing technology that works so well could be in the hands of more people.

Everyone on Earth deserves technology that is easy to use and lets them have a fun, happy life. That was my goal when I started the Adélie Linux project, and I only wish that more open source projects would do the same. Until then, I will continue doing my part to make the world a little bit better from the keyboard of a Mac.

Using MailWrap on macOS Monterey

Apple’s Mail Plug-In system is quite amazing, and has led to some innovative and brave developments as an MUA (Mail User Agent). These brave developments include: wrapping long lines like a decent MUA, quoting messages in replies correctly like a decent MUA, and an option to Wrap Text like a decent MUA.

I actually forgot to install MailWrap on my M1 when I built it up, and haven’t noticed mostly because I haven’t been posting to mailing lists lately. However, I feel deep personal shame for posting on lkml without remembering to install it first. Look at those long lines!

So I set out to install MailWrap. It certainly is a lot more difficult than it used to be.

#1: Allowing the installer to access ~/Library/Mail

The first time I tried to install MailWrap, I received the unhelpful message that access to ~/Library/Mail/Bundles was denied. This is because I had to grant Terminal the Full Disk Access permission.

You can do this in System Preferences under Security & Privacy. You’ll be helpfully reminded that you have to restart Terminal. Hope you don’t have six active SSH connections open, like I did!

#2: Using the correct UUID

Now we need to add the correct UUID to the Info.plist file. Open ~/Library/Mail/Bundles/MailWrap.mailbundle/Contents/Info.plist in your favourite plain-text editor. Scroll to where you’ll find “Supported10.16PluginCompatibilityUUIDs” and then add the following lines under the “</array>” line:

        <key>Supported12.2PluginCompatibilityUUIDs</key>
        <array>
                <string>6FF8B077-81FA-45A4-BD57-17CDE79F13A5</string>
                <string>25288CEF-7D9B-49A8-BE6B-E41DA6277CF3</string>
        </array>

Note that this says “12.2”; when 12.3 comes out, we will need to change this again.

#3: Sign and allow the bundle to run

Gatekeeper will try to keep you safe from untrusted code, which is generally a good thing. We can sign our bundle now:

$ cd ~/Library/Mail/Bundles
$ codesign -f -s - MailWrap.mailbundle

And now that it is signed properly, we can tell Gatekeeper to trust the signature:

$ sudo spctl --add --label “MailExtensions” MailWrap.mailbundle
$ sudo spctl --enable --label “MailExtensions”

Troubleshooting

Incompatible Plug-ins Disabled. Mail has disabled the following plug-ins: MailWrap.mailbundle Contact the makers of these plug-ins for versions that are compatible with Mail 15.0.
Incompatible Plug-ins Disabled

If you receive this Incompatible Plug-ins Disabled message, then something has gone wrong with your UUIDs. You’ll need to try again and make sure that you’ve pasted those lines in the correct spot.

“MailWrap.mailbundle” is damaged and can’t be opened. You should move it to the Bin. Mail created this file on an unknown date.
“MailWrap.mailbundle” is damaged and can’t be opened.

I received this message when I edited the Info.plist file after running codesign. It means the CodeSignature doesn’t match the contents. You need to re-run the codesign command every time you change any file in the bundle to keep the signature updated.

“Mail” needs to be updated. This app will not work with future versions of macOS and needs to be updated to improve compatibility. Contact the developer for more information.
“Mail” needs to be updated.

This message is because MailWrap uses Python 2.7. Hopefully I will have some time to update it to Python 3 before the eventual removal of Python 2.7 from macOS. I’ve had success doing this before, so hopefully it goes well.

In conclusion

Now my emails are nice and wrapped and I’m not breaking a bunch of email clients in faraway lands. And all was quiet in the world. (Except not: the kernel is still broken, and Ukraine is still being invaded.)

I hope this post was useful to you. Happy hacking!

2 TB USB drive on a PowerBook G3 Pismo

I have a 2 TB USB SSD for my photo library, and I wondered: would it work on my PowerBook G3 Pismo with Mac OS 9? Let’s find out!

Here’s a quick, fun anecdote from the Retro Lab. I bought a Sandisk Extreme 2 TB USB NVMe drive on Black Friday. (Actually, I bought it the Wednesday before Thanksgiving.) My intention is to use it for storing my entire photo library.

I primarily intend to have it connected to my M1 MacBook Pro, but it comes with a USB-C to USB-A adaptor, and states it is compatible with “any computer with a USB port”. I decided to put that statement to the test with my trusty Pismo.

This computer was the top of the line for the year 2000, including a 500 MHz CPU and Mac OS 9. I tried to do some searches online to see the maximum volume size that Mac OS 9 can support. Most of my searches simply showed “more than 200 GB”. Okay, then!

I booted the Pismo and connected the drive to the rear USB port. Lo and behold, there really was no step 2: it showed up immediately in the Finder.

My new 2 TB NVMe SSD working perfectly on a Pismo running Mac OS 9.

It makes me happy that if I ever feel the desire to fire up Kai’s Power Goo again, I can do so with any photo in my library. Have fun, everyone!