What’s the deal with Cisco devices in `file` output, anyway?

If you work on PowerPC systems of some kind – or maybe you work on car MCUs that use the NEC V800 CPU – you may have run across some strange output when you run the file command on any binary:

/usr/bin/file: ELF 32-bit MSB pie executable, PowerPC or cisco 4500, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-powerpc.so.1, stripped

Of course it’s a PowerPC binary, but why the mention of “cisco 4500” (or Cisco 7500s for 64-bit PowerPC binaries, or Cisco 12000s for NEC V800s)? The reason behind this is a fascinating insight into the world of proprietary computing architectures and the somewhat inventive way Cisco tried to lock down some of their older systems.

A brief primer on ELF

ELF, which stands for Extensible Linking Format or Executable and Linkable Format and is not a Will Ferrell character, is a file format for executable files and shared libraries (among others).

In layman’s terms, ELF specifies things like what processor the executable runs on, the ABI that it uses, the endianness and word size (32-bit or 64-bit, for example) that it uses, and so on.

One of the fields in an ELF file is the e_machine field, which specifies the type of machine the file is designed to run on. 0x02 is SPARC, 0x03 is the Intel x86, 0x14 is 32-bit PowerPC, 0x15 is 64-bit PowerPC, and so on.

This is the identifier that allows your OS to tell you “Exec format error” (or similar) when you run an executable for a CPU other than the one you are currently using. As a side note, it is also this field that allows qemu-user binfmt to work, if you are curious.

Cisco’s use of e_machine

The boot loader for Cisco IOS machines, also known as ROMMON, will refuse to load firmware for a different router model than the system. For example, on a Cisco 2911, you may see:

loadprog: error - Invalid image for platform
e_machine = 30, cpu_type = 194

ROMMON uses e_machine as a sort of “model number”. The Cisco 4500 uses cpu_type 20 or 0x14, which happens to also be the ELF e_machine for PowerPC.

The “magic” library that the file command uses to determine the machine type of ELF binaries only knows a few models of Cisco. I haven’t been able to determine their criteria for inclusion, or why some are present and some aren’t.

References

The ROMMON error was gleaned from an OpenWrt forum post; I don’t have hardware to show this error myself.

More information about how older Cisco devices use ELF can be found on the LinuxMIPS wiki.

This question was originally asked by some curious people on the #talos-workstation IRC channel on Libera.Chat. I knew the basics of Cisco’s ELF-scapades, but they were the ones who inspired me to make this write-up and learn a bit more.

Expanding the Retro Lab, and Putting It to Work

Over the past month, I have been blessed with being in the right place at the right time to acquire a significant amount of really cool computers (and other technology) for the Retro Lab.

Between the collection I already had and these new “hauls”, I now have a lot of computers. I was, ahem, encouraged to stop using the closets in my flat to store them and finally obtained a storage locker for the computers I’m not using. It’s close to home, so I can swap between what I want to work on virtually at will.

Now I am thinking about ways to track all of the machines I have. One idea I’ve had is to use FileMaker Pro for the Power Macintosh to track the Macs, and FoxPro to track the PCs. One of my best friends, Horst, suggested I could even use ODBC to potentially connect the two.

This led me to all sorts of ideas regarding ways to safely and securely run some server services on older systems and software. One of my acquisitions was a Tyan 440LX-based server board with dual Pentium II processors. I’m thinking this would be a fun computer to use for NT. I have a legitimate boxed copy of BackOffice Server 2.5 that would be perfect for it, even!

Connecting this system to the Internet, though, would present a challenge if I want to have any modicum of security – so I’ve thought it out. And this is my plan for an eventual “Retro Cloud”.

Being a cybersecurity professional, my first thought was to completely isolate it on the network. I can set up a VLAN on my primary router, and connect that VLAN to a dedicated secondary router. That secondary router would have total isolation from my present network, so the “Retro Cloud” would have its own subnet and no way to touch any other system. This makes it safer to have an outbound connection. I’ll be able to explore Gopherspace, download updates via FTP, and all that good stuff.

Next, I’m thinking that it would make a lot of sense to have updated, secure software to proxy inbound connections. Apache and Postfix can hand sanitised requests to IIS and Exchange without exposing their old, potentially vulnerable protocol handlers directly to the Internet.

And finally, as long as everything on the NT system is public knowledge anyway – don’t (re)use any important passwords on it, don’t have private data stored on it – the risk is minimal even if an attacker were able to gain access despite these protections.

I’m still in the planning stages with this project, so I would love to hear further comments. Has anyone else set up a retro server build and had success securing it? Are there other cool projects that I may not have even thought of yet? Share your comments with me below!

The musl preprocessor debate

Today, I would like to discuss a project that I care very deeply about: the musl libc. One of the most controversial and long-standing debates in the musl community is that musl does not define a preprocessor macro.

What’s in a macro?

Simply put, preprocessor macros allow C code to build parts of itself conditionally. For example, the GNU libc defines the “__GLIBC__” macro. If your code needs to do something specific to function properly on systems using that library, it can conditionally build that code using “#ifdef __GLIBC__”.

The authors of musl have said that they will not add a preprocessor macro identifying the platform as musl because:

It’s a bug to assume a certain implementation has particular properties rather than testing.

Rich Felker, “Re: #define __MUSL__ in features.h”, 2013-03-29

I agree with this sentiment in theory, and in an idealised world this would hold up. However, I’d like to discuss why I think this may need to be reconsidered moving forward.

Sometimes you can’t test

One major reason this is an issue is that sometimes it is not possible to do what the authors consider the “correct” form of testing, which is compile-testing.

This practice requires you to build a small test program, determine whether it built properly, determine its runtime characteristics, and then use the results of that test to influence how your actual software is built. This is an alternative to using the conditional code with preprocessor macros.

However, there are many reasons you may not be able to successfully perform such testing. Cross compilation is a large gap here. In fact, many years ago when I was starting the Adélie project, this caused failures in the base image I was building.

The Bash shell could not perform any compile-time or run-time checks because it was being cross-compiled from a GNU libc system to a musl libc system. This caused it to use “fallback” code that worked improperly. If musl had defined a __MUSL__ macro, Bash would not have needed to assume it was running on a pre-POSIX system.

Similarly, the mailing list thread that made me feel strongly enough to write this article involves a header-only library. These types of libraries are meant to be “drop-in” and function without any changes to a developer’s build system. If header-only libraries start requiring you to use build-time tests, you lose the main reason to use them in the first place.

The author of this thread correctly points out that FreeBSD versions their API with a preprocessor macro. Any software that requires a certain API can simply ensure that __FreeBSD_version is defined as greater-or-equal than the versions that introduced that API.

The main reason that the musl project is fearful of this approach, at least to my observation, is that features or APIs (or indeed, bug fixes) can be backported to prior versions. I feel very strongly that this is not the responsibility of the libc.

If a distribution backports a feature, API, or patch to an older version of a library, it is that distribution’s responsibility to ensure that the software they build against it continues to function. When I backported an API from Qt 5.10 to 5.9 to ensure KDE continued building for Adélie, it was my responsibility as maintainer of those packages to keep them building properly. It certainly does not mean Qt should stop defining a preprocessor macro to determine the version being built against.

Additionally, some APIs are privileged. Determining whether these APIs work correctly using run-time testing can prevent CI/CD from working properly because the CI user does not have permission to use them.

A versioned macro like FreeBSD’s makes sense

I feel that the best way forward for musl is to define a macro like FreeBSD’s. It monotonically increases as APIs or features are added.

I agree that simple bug fixes, and even behavioural changes, probably should not be tracked with this macro. However, this would make it significantly easier to use new APIs as they are introduced.

It also makes builds more efficient. The cost of compile-time tests racks up quickly. On my POWER9 Talos workstation, typical ./configure runs take longer than the builds themselves. This is because fork+exec is still a slow path on POWER. It is similar on ARM, MIPS, and many other RISC architectures.

Macros like these don’t fully eliminate the need for ./configure, but they lessen the workload. Compile-time tests make sense for behaviour detection, but they do not make sense for API detection.

2 TB USB drive on a PowerBook G3 Pismo

I have a 2 TB USB SSD for my photo library, and I wondered: would it work on my PowerBook G3 Pismo with Mac OS 9? Let’s find out!

Here’s a quick, fun anecdote from the Retro Lab. I bought a Sandisk Extreme 2 TB USB NVMe drive on Black Friday. (Actually, I bought it the Wednesday before Thanksgiving.) My intention is to use it for storing my entire photo library.

I primarily intend to have it connected to my M1 MacBook Pro, but it comes with a USB-C to USB-A adaptor, and states it is compatible with “any computer with a USB port”. I decided to put that statement to the test with my trusty Pismo.

This computer was the top of the line for the year 2000, including a 500 MHz CPU and Mac OS 9. I tried to do some searches online to see the maximum volume size that Mac OS 9 can support. Most of my searches simply showed “more than 200 GB”. Okay, then!

I booted the Pismo and connected the drive to the rear USB port. Lo and behold, there really was no step 2: it showed up immediately in the Finder.

My new 2 TB NVMe SSD working perfectly on a Pismo running Mac OS 9.

It makes me happy that if I ever feel the desire to fire up Kai’s Power Goo again, I can do so with any photo in my library. Have fun, everyone!