Tag: Linux

Blogging in general, and a new project

It’s been a long time since I wrote here. In the past few months, I have moved across the country, and helped four other people do the same. It is exhausting and tiring but so rewarding to improve not only my own life but the lives of others by sharing in new experiences.

Enough of that, though. I am starting up a new Linux distribution, titled Adélie Linux, aimed at being very fast, very small, and fully POSIX® compliant. It’s almost meeting those three goals! Going forward, I think I will be starting a new blog specifically about my adventures with Adélie, which will probably take up a considerable amount of my writing time. This blog will stay around, though, not only for memories past but for non-Adélie related things in my life. I am still interested in Python, writing emulators, music, and other general geekiness; I just now have a new project that is taking up a large amount of my free time.

Going IPv6 native without IPv4

Now that I have finally moved in to my new apartment (which requires a long blog of its own), I have new routing equipment and a new network infrastructure. The native IPv6 on Cox Communications seems to be a bit better than the native IPv6 offered by Comcast Business; namely, Cox seems to be peered more widely and therefore ping times are much lower. Of course, this could be specific to the market I’m in – eastern Oklahoma – so YMMV.

However, because DHCP is a terrible protocol, it is constantly flaking, leaving me with IPv6-only access to the Internet. That is, no access to IPv4 whatsoever. Surprisingly, it’s nearly usable. However, I am highly disappointed in a few surprises I’ve found that do not work over IPv6:

  • EVERY SINGLE CODE HOSTING SERVICE ON THE INTERNET. This really, really, really, really upsets me. Luckily, I don’t have to care any more, because I run my own now.
  • DuckDuckGo. I am incredulous that a modern search engine is not accessible over IPv6.
  • eBay and PayPal. This isn’t really surprising, I suppose, since eBay were running Windows NT 4 as recently as 2006… they always have been a decade off of the current technologies.
  • Any news Web site I tried: Bloomberg, BBC, New York Times, Washington Post.
  • The entire StackExchange family of properties, five YEARS after being asked for even a trial of IPv6 access. This is entirely unacceptable. I expect news organisations and e-commerce conglomerates to be woefully behind the times, but a company designed from the ground up for computer scientists by computer scientists? I can’t believe this is real.
  • Weather.gov. The US government actually has an IPv6 project with real time online completion progress, even available itself via IPv6.  However, while NOAA’s flashy Web 3.0 marketing pages are available over IPv6, the important research, life-saving data, and forecast information made by the National Weather Service are entirely IPv4-only. I understand that internally, their infrastructure is not entirely ready for IPv6, but they should be able to run the main radar and warning information over IPv6 at least. Americans need not feel singled out, though; the UK’s Met Office is also unavailable over IPv6.

At least Wikipedia and the Google properties are usable, so I have music, videos, and a reference library.

Web browsers, music players, workarounds, and PulseAudio

As security researchers have discovered yet another horrible security bug in Chrome, and Google yet again decides to put off fixing it, I decided to finally give up Chrome entirely. I had dwindled down my usage of it from primary browser in 2009; to secondary browser for Flash and videos in 2013; and finally using it solely for streaming Google Play Music and Spotify, along with the occasional site compatibility test for my work, in 2015. Firefox’s inspector tools and Firebug are good enough, and I have a Mac running Safari if I need a WebKit test, so I decided it was no longer important to test on Chrome. That left the issue of music streaming.

Can’t handle it, can’t handle it

Google Play Music, however, has a fatal flaw. It is a mess of terrible “one page” JavaScript. After only a few hours of music streaming, it had already leaked 150 MB(!) worth of orphan DOM nodes, and 282 MB(!!!!) worth of uncollectable JavaScript objects. This basically means it created buttons, links, and so on, and didn’t properly remove them when it was done, so that memory is leaked out and I would have to restart Firefox to get that memory back. Restarting Firefox multiple times a day is not an option for me.

What’s worse is that one of Firefox’s best and most unknown features was also making my life worse. Every 10 seconds, it scans its memory to see if any of it can be reclaimed, to make sure that it does not use too much memory. Since Google Play Music’s interface had leaked so much memory, the scan was taking about 2 seconds – during which the browser became completely unresponsive. That means that for about 20% of the entire time it was open, it was unresponsive (frozen, locked, etc), all because Google has no idea how to write JavaScript.

My mother (bless her soul, she’s openly embraced Debian) suggested I try Rekonq, but it could not even load Google Play Music’s user interface. I also tried Opera Classic (pre-Blink), and it too could not load Google Play Music. At this point I am very upset at Google; why did you write such a cluster#*$@ of terrible code instead of writing a simple multi-page player like YouTube? YouTube does not suffer from any of these issues, and is a Google product!* Anyway, my next goal was to see what I could do for streaming music that did not require a Web browser.

* I am aware that YouTube has a single-page mode, but I found a way to disable it except while using playlists. It works great and does not leak half a gigabyte of memory.

Done, done, and I’m on to the next one

It turns out that Google Play Music has no official API and no non-browser clients. Even Spotify has unofficial ones that are of questionable quality and legality, but Google has done a very good job of making their API so hard to use that nobody bothers to even try with them. (Future project idea: Make one anyway.)

Then I realised their Android app is pretty reliable and certainly better than having my browser locked for 20% of the time it’s open. However, I still need to be able to hear other things on my computer (if someone links me to a video or presentation, for instance), and I don’t want to have to keep flipping back and forth between my phone and desktop.

My work-provided desktop did not come with a sound card (even though we use sound a lot internally…), so I am using a USB Griffin iMic as my sound “card”. It works fantastically in Linux/ALSA, but one thing I could not figure out was how to make it play line in as a monitor (i.e. playthrough, listening to line in/mic with headphones/out, whatever you like to call it). Thankfully, I found a very helpful blog post about this very issue, and a solution involving PulseAudio: pactl load-module module-loopback was all it took to listen to crystal-clear, low-latency, glorious Nexus audio on my desktop!

Final thoughts

  • While it certainly is great that PulseAudio offers the same great passthrough functionality that OS X had since Jaguar (and lost in Mavericks), they really need to document PulseAudio modules better.
  • Google needs to rethink making their music player in one page JavaScript. A native app would be amazing and make me a much happier catfox.
  • It just feels like… if Google hadn’t royally screwed up Chrome, and they hadn’t royally screwed up their music player, then hours of my life would have been saved because then I would not have had to learn how to monitor line in within Linux. It was interesting learning all this, but I still have this feeling that it should be entirely unnecessary, and like this is a very unclean workaround for what amounts to “Google is terrible at writing code”.

Oh well, at least Android 6.0 is good. (For now.)

 

The Joys of Unix Programming: MAP_ANON(YMOUS)

I was trying to do a little late-night hacking last night on SuperGameHerm, the Game Boy emulator my friends and I are writing, and I hit an error in the memory mapper. Specifically, certain OSes that used to be named after cats don’t like calling mmap on /dev/zero (neither does Android). I thought it was odd that it was falling back to that code in the first place, though, because Apple’s Mac OS X — I mean, a certain cat themed OS — has always supported MAP_ANON, and I confirmed that by going to man mmap on a Mac.

What was going on? I dug deeper and saw MAP_ANON was guarded in sys/mman.h, so CMake wasn’t finding it and it was instead compiling our fallback code. And so I started digging up other issues related to big endian machines and realised that I had only tested OS X and FreeBSD on big endian, and never tested OS X or FreeBSD on little endian. So was my big mistake.

This is a comprehensive guide to How to Make MAP_ANON(YMOUS) Visible, for every OS I could find information on:

Mac OS X

On 10.3 and below, this is easy; it’s always there! It is not guarded by any #ifdef.

On 10.5 and above, it is slighly harder; you must define _DARWIN_C_SOURCE to cause MAP_ANON to be visible in a public scope.

On 10.4 only, it’s much harder! It is only protected by #ifndef _POSIX_C_SOURCE, so to use MAP_ANON against the 10.4 SDK, you must completely undefine _POSIX_C_SOURCE. You don’t have any other choice.

I suppose that means my overall advice then is to use the 10.5 SDK no matter what, if you have a Leopard computer handy, because it can target as low as 10.0. Otherwise, use the Panther SDK included with Tiger’s Xcode Tools. Don’t ever use Tiger’s SDK if you want MAP_ANON.

FreeBSD

Before 5.0, it’s always visible, just as in OS X 10.3. There are no preprocessor options to show or hide MAP_ANON.

On 5.0 or above, the only way to cause MAP_ANON to be visible is to define __BSD_VISIBLE somewhere. Undefining _POSIX_C_SOURCE won’t save you here.

Other BSDs (NetBSD, OpenBSD, DragonFly BSD)

It’s never guarded. MAP_ANON is always available.

Solaris

I could only get my hands on OpenSolaris, but considering the header having a copyright date of 1989 (by AT&T), I can’t imagine it’s any different on Real Solaris (or Oracle Solaris). There are no guards here, either; that’s to be expected since they invented the damn thing.

Linux

glibc: I don’t understand /usr/include/bits in the slightest. It seems to be always available no matter what options I toss to clang, but it is guarded by.. __USE_MISC? I presume this is some sort of feature macro buried deep in glibc that I don’t care about or understand.

musl: It’s always available, at least on 1.1.11 which is what I have on my test box.

Android: After searching through their spaghetti of includes to get to the actual file that defines constants, it appears they are all completely unguarded, though that isn’t surprising since it is Linux and embedded.

In conclusion

Perhaps it’s best to avoid anonymous mmap(2) in applications that you want to actually be portable.