I posted recently about my experiences shifting to systemd and how I found it, for all three of my Arch machines, to be a relatively simple and straightforward process which, in the intervening weeks, has borne out the views of the developers and other members of the community who have been using it for a longer period—it works, and works well. At the start of this week, Stéphane Gaudreault announced on Arch-dev-public the intention to replace the original initscripts with systemd.

The reaction by some to this announcement was, to me, quite disappointing. My view of open source communities is that by participating you are entering into a social contract. That contract is that, as you are reliant on the goodwill of the wider community it is incumbent on you to ensure that your overall contribution—in whatever form—brings more to the community than it takes away. At the very least, you can strive to ensure that it is a value neutral exchange.

Sadly, for some of the trolls on the MLs, that is patently not the case. I want to take a closer look at just one of those threads, for two reasons. First, it is a textbook case of a particularly pernicious troll. And second, in conjunction with some other factors, it had an especially damaging effect on the wider community.

The thread in question is on Arch-general, the largest of the mailing lists. It has the suitably alarmist title Think twice before moving to systemd. The troll, who has had only minimal involvement with the Arch community up until this unfortunate moment starts by framing his “concern”¹:

I just became aware that Arch Linux plans to switch to systemd, and this worries me for several reasons.

Felipe is worried. Ergo, we should be worried. Why should we be worried? Because, despite the fact that systemd has been discussed and used by Archers since the middle of 2010, it has just come to his attention and now, with the formidable powers of his intellect, he has detected a disturbance in the force.

I tried systemd a while ago in a brand new machine with Arch Linux and the boot was *much slower*. After some exchanges with Lennart Poettering and other people in Google+[1], it became clear I was on my own. Eventually I found the culprit: Fedora uses CONFIG_HZ_1000, and Arch Linux uses CONFIG_HZ_300. It became clear to me that systemd was not ready for prime time, it wasn't thoroughly tested in a lot of machines, and if you have problems Lennart Poettering will blame you (PulseAudio sounds familiar?).

The salient phrases here are “it became clear I was on my own” and the even more telling, “it became clear to me that systemd was not ready for prime time.” Once I had managed to wipe away the tears of gratitude that were coursing down my cheeks, I realized that Arch finally had finally found what it had been missing for all these years: a saviour. We were going to be OK after all.

Felipe has “found the culprit.” Despite the best efforts of the Arch developers, upstream, the other distros that have been using it for considerably longer, and the experience of the Arch users reported on our boards, no-one had the heroic ability to see through to the heart of this conundrum and expose systemd for the fraud that it is.

But our gratitude should be tempered with some shame for we, the Arch community, have let Felipe down. We haven’t lived up to the standards he expects of us. As he gently admonishes us: “I was expecting more from the Arch Linux community.” Yes, Felipe, and you were right to. We let you down; can you forgive us?

The thread goes on to generate more than 50 replies. All of them wasted. His trolling is not interested in facts, in reason or logic; his method of argumentation is not rational, and he is not interested in working toward an understanding of the truth; he just wants to be right. At whatever cost to the wider community. These posts, and trolls in other fora like this one,² are really about gratifying Felipe’s own voracious appetite for self-aggrandisement. In reality, of course, it is a stunning illustration of the Dunning-Kruger effect. It would be pathetic if it were a victimless pursuit; but it is not.

Further down in the thread we see the inevitable frustration of the developer who leads this work for Arch, Tom Gunderson, finally boil over. This precipitates another thread where Tom announces that he will no longer participate in this list. This is from the developer that has patiently and constructively engaged with Arch users on the list around this issue for the past several months. On his own time.

That is the impact of trolls like Felipe. That is a direct effect of his selfishness on the wider community. What has he brought to the community? Dissent, distraction and posturing. What has he taken away? The ability for all of us to engage with one of the developers of the software we all use. His presence on the list, and that of a few others, has been nothing short of corrosive.

And as for the smoking gun, the config setting that no-one was smart enough to discover? Denis A. Altoé Falqueto went to the trouble to build another kernel and run a test and, contrary to Felipe who provided no such evidence, posted the results. You can guess the outcome.

Notes

1. See the concern troll.
2. Read the whole thread to fully understand how obstinance can trump reason… See also his own blog where he expresses incredulity that his comments on another blog have been deleted.

Creative Commons image of the troll on Flickr by Comrade Foot.

I posted a couple of weeks ago about the impact of some of the current changes to Arch Linux. There seems to be quite a bit of it about, because in conjunction with the move from /lib to /usr/lib—and motivated by the same overarching shifts in the other distributions—the other major change making its way through the Arch community is the move to systemd. This change, however, has been more controversial because it marks a shift away from Arch’s central configuration file, /etc/rc.conf; a move that has disconcerted a number of people.

I’m not going to get into whether or not this move is consistent with The Arch Way, because that has been thrashed over extensively on the mailing lists, but also—and more importantly—because Arch is a rolling release, which means change is inevitable. What is more, after watching all the back and forth on the ML and on the boards, I am convinced that the developer overseeing this shift both knows what he is doing and is acting in the best interests of Arch.

This view has been validated by my experience of shifting to systemd on my machines.

When the new initscripts package was pushed to [testing] in the middle of last month, I stripped out all of the redundant options in /etc/rc.conf and took the ten minutes or so to read about and then create the new files. I then spent some time reading up on systemd, principally Lennart Poettering’s series of posts, the Arch wiki entry and the many and various manual pages.

Over the last couple of days, I have migrated all of my machines over to systemd. I found it to be a very straightforward process. I went for a pure systemd approach, for reasons I will return to. Essentially, this is how I tackled it.

First, for each machine, I established what daemons were running. I did this by using this shell script and cross checking entries in my /etc/rc.conf. Then it was just a matter of installing systemd:¹

1

sudo pacman -S systemd systemd-sysvcompat systemd-arch-units

The only outstanding file to be created was /etc/timezone, the others—all documented in man rc.conf—I had created earlier in the month. Then, with the list of the services I would need running on each box, I enabled them. I think it is fair to say that the only thing that is clunky about systemd is the command syntax for managing units. Repeatedly typing systemctl enable foo.service is as tedious as it is error prone, so I hacked up a wrapper script:

1
2
3

sysd enable acpid
sysd enable lvm
etc…

I also activated some additional gettys, as I like to log into a tmux session in TTY2 when I don’t need or want to start X:

1

ln -sf /usr/lib/systemd/system/getty@.service /etc/systemd/system/getty.target.wants/getty@tty2.service

And I misguidedly wrote a couple of custom service files that turned out to be completely unnecessary; and then I was done.

On all three boxes I rebooted and that was it. The only issue I experienced was the final shutdown after removing initscripts and before booting up with systemd. After issuing shutdown -r now the process hung with a message about a missing inittab file. In retrospect, I should have added init=/bin/systemd to my kernel parameters and not unistalled instiscripts until I had booted with systemd. This has now been added to the systemd wiki page.

Overall, though: no real drama, just a blur of text² and then the login prompt. While I haven’t seen a dramatic decrease in boot time, booting does feel a bit snappier and two of my three machines shutdown markedly more quickly. Consider me sold.

That brings me back to the decision to opt for a pure systemd approach, rather than hanging on to a completely eviscerated /etc/rc.conf. The Arch community has been told that the intention is to support the rc.conf format forever; I can’t help but think this is a mistake. In the same way that partial upgrades are unsupported, I think that systemd is Arch’s inevitable future and the community should be encouraged to accept that, Syu and move on…

Having said that, I appreciate the consideration shown by the developers in wanting to assure people that their choice would continue to be catered for; and perhaps, over time, people still using initscripts will be regarded with the same sense of wry amusement as those who still show up from time to time with hal in their DAEMONS line.

Update 4/8/12

Tom Gunderson, the developer who looks after this part of Arch, has clarified the comment about the long term support for initscripts. As comments are hosted on a third-party paltform, I thought that it should be included in the body of the post.

Just a clarification: regarding the (probably very unwise) promise of "supporting rc.conf forever", the intended meaning was that initscripts will support the syntax, not that initscripts themselves will always stay around (nor that systemd will gain support for the syntax in case initscripts is dropped).

At the moment systemd does support rc.conf (when installed in conjunction with initscripts), but this is meant as a transitional measure only, and will likely go away eventually (though probably not soon).

Notes

1. Note this approach will remove sysvinit and initscripts and mv /etc/rc.conf{,.pacsave}.
2. Make sure you remove quiet from your kernel parameters for your first systemd boot so you can see that everything starts correctly…

Creative commons image on Flickr by Tricky

In my last post, on Mercurial Queues, I outlined how you might use MQ to manage a dwm patchset. Mercurial makes this process ridiculously easy, and this functionality is more than enough to recommend it. However, where I think it really gets interesting is when you share that same patchset across multiple machines and are easily able to customize which patches in the queue are applied, based upon the particular machine you are building dwm on.

For example, in my case I use dwm on all three of my machines: my desktop at home, my laptop at work and my EeePC. Given the nature of each of these three machines, though, I want a slightly different stack of patches applied. These differences might extend only to particular rules applied in config.def.h based upon the number of tags I use, or the different applications that I run at home as opposed to at work.

In any event, what I want to be able to do is maintain a single patchset in a mercurial repository, clone it to all my machines and then, using MQ, filter which specific patches are applied. This is all possible using a killer feature of MQ called guards. Guards allow you to conditionally apply patches in a queue. Essentially, in a queue, you “tag” a patch with a guard and—depending on whether or not the guard is positive (to be applied), or negative (to be skipped)—when you hg qpush -a MQ takes care of applying the correct patches in the queue.

So, in the case of my work laptop, I begin by cloning dwm and intializing a patch queue:

1
2
3

cd Build/
hg clone http://hg.suckless.org/dwm
hg qinit -c

I setup my .hgrc within the patches directory so that it points at the correct mercurial repository:

1
2
3
4
5
6

 # Mercurial config file
[ui]
username = jasonwryan <jasonwryan@gmail.com>
ssh = ssh -i ~/.ssh/bb -C
[paths]
default = ssh://hg@bitbucket.com/jasonwryan/dwm-patchset

Then I pull down my dwm patchset from the mercurial repository with hg pull:

1
2
3
4
5
6
7

pulling from ssh://hg@bitbucket.com/jasonwryan/dwm-patchset
requesting all changes
adding changesets
adding manifests
adding file changes
added 6 changesets with 16 changes to 10 files
(run 'hg update' to get a working copy)

After I update my local repo,¹ I can see the full patchset with hg qseries:

1
2
3
4
5
6
7

setup.makefile
base.config.customizations
statuscolours
cycle
push
bstack
centurion.config

Then, as I decribed in my previous post, I make the changes to config.def.h that are specific to this machine:

1
2
3
4

hg qnew veles.config
 # “hack, hack, hack…”
hg qrefresh
hg qcommit -m 'Adaptions for Veles'

My patchset now has an additional patch in it:

1
2
3
4
5
6
7
8
9
10

┌─[Veles ~/Build/dwm]
└─╼ hg qseries
setup.makefile
base.config.customizations
statuscolours
cycle
push
bstack
centurion.config
veles.config

This is where guards come in to play. If I wasn’t interested in sharing the patchset, it would just be a matter of deleting the patch for Centurion. However, what I want to do is direct MQ to ignore that specific patch when applying the queue. I do that by creating a negative guard and then activating it:

1
2
3

hg qguard centurion.config -- -block
hg qselect block
number of unguarded, unapplied patches has changed from 8 to 7

That’s it! Adding a verbose flag to hg qseries² will print out the patchset and the accompanying guard status:

1
2
3
4
5
6
7
8
9
10

┌─[Veles ~/Build/dwm]
└─╼ hg qseries -v
0 U setup.makefile
1 U base.config.customizations
2 U statuscolours
3 U cycle
4 U push
5 U bstack
6 G centurion.config
7 U veles.config

The real test, of course, is pushing the patchset onto the fresh dwm code:

1
2
3
4
5
6
7
8
9
10
11

┌─[Veles ~/Build/dwm]
└─╼ hg qpush -a
applying setup.makefile
applying base.config.customizations
applying statuscolours
applying cycle
applying push
applying bstack
skipping centurion.config - guarded by '-block'
applying veles.config
now at: veles.config

How cool is that? One central patchset, applied conditionally depending upon the machine you are using at the time.

Notes

1. Any guards applied on other machines will be present in the newly pulled series file: these conflicts will need to be manually merged…
2. The options for all of the hg commands can be read with hg -v help $command. For those related to queues, just prepend a q to $command.

Creative Commons image on Flickr by Thorsten Becker