ben.eficium is the blog of Ben Slack, citizen and principal consultant at Systems Xpert.
All posts are Copyright © Ben Slack on the date of publishing.

14 March 2016

FreeBSD cross-compiling with gcc and poudriere

I've been experimenting with Raspberry Pi using FreeBSD for a while now.

To build the ports I'm using qemu and poudriere on a dedicated build machine, as a) compiling on the rpi takes so long, and b) I have a few of them doing various tasks, so having a dedicated package server makes sense.

I've had to use the HEAD version of FreeBSD (currently 11.0-CURRENT) to get it working with the SD cards I bought to host the rpi OS.

My problem is getting a handful of ports to build with clang - the default C compiler for FreeBSD from version 10. The compile errors are happening with poudriere on the build machine and on the rpis themselves.

IMHO the move from gcc to clang was premature, but as a non-contributor to the FreeBSD project, I'm not complaining.

The offending ports that won't (or don't) build at time of writing are:


These are dependencies for a lot of other ports, so not being able to build them is a showstopper. No doubt there are plenty of other ports that won't build for the armv6 architecture with clang.

Via roundabout diagnosis, I discovered that all these ports do build with gcc but not with clang when compiling directly on one of my rpis. Hence the need to build using gcc.

So, the question then becomes - how do you build using poudriere and gcc? This is not trivial - as attested by the "Using gcc under poudriere" thread on FreeBSD Forums.

The solution suggested here by Mega56 is not ideal. Editing the poudriere port scripts with changes that shouldn't be in the master is never a good idea. For a start you'd have to make the edits after each upgrade, which would be a real pain.

The better solution I came up with was to install gcc in the poudriere model or root jail, a copy of which is made to a "reference" jail each time a poudriere "bulk" build is started. This is far easier than any of the other suggested solutions in the thread. Some of the suggestions, such as simply creating a new poudriere jail - won't work at all.

Then just add the USE_GCC?= yes variable into the Makefile for each port that requires gcc compiling. This has to be done every time you upgrade the ports, but if required you could easily write a script with sed to automate the task. For the two ports I am using this is not really worthwhile.

Getting the gcc compiler installed in the model jail is a complex process, but remember it only has to be completed once. It's set and forget.

The Details
This explanation assumes the following:
  • You've set up a qemu and poudriere cross-compile environment for the armv6 architecture. I used the instructions provided at Doug's Domain to get this set up. Big thanks to Doug Vetter - who is awesome.
  • You have read the man pages and basic doco for the qemu and poudriere ports.
  • My poudriere home is located at /home/poudriere, my jail is called "110armv6" (as I'm using FreeBSD 11.0-CURRENT) and I am using the "default" ports tree. Adjust the instructions for your own environment - am sure you can work out the detail.
Build the gcc packages
Rather than building the lang/gcc port inside the master jail we can use a standard poudriere build to make the packages. We can then install them in the master jail using FreeBSDs pkg manager. If there are other ports you'd find useful in the master jail, then add them also. I think Perl and Python are good candidates.

If you haven't already done so, you should probably get the latest ports tree for poudriere by issuing the following:

# poudriere ports -u

I use a port list file /usr/local/etc/poudriere.d/pkg.list for poudriere to compute bulk builds. I strongly recommend you do similar and ensure the line lang/gcc and any other ports you want in the master jail are listed in your equivalent file.

Issue the standard poudriere "bulk" command and gcc will build (under clang of course).

# cd /usr/local/etc/poudriere.d
# poudriere bulk -vv -j 110armv6 -f ./pkg.list

I always use the verbose debugging options -vv.

Hopefully your gcc port will build without issue. I have never experienced a problem with this step.

Set up the QEMU environment
The great thing about qemu is that once the environment is set up you can simply chroot to the poudriere master jail and all the emulation is handled under the hood.

However, you first have to mount a few directories and the device tree into the master file system in order to access a few dependencies. To do this I used the instructions "Using qemu-user to chroot and bootstrap other architectures on #FreeBSD" on The Ignorant Hack's blog. Big thanks to the Hack.

First, mount the device tree to the /dev folder in the jail's file system.

# mkdir /home/poudriere/jails/110armv6/dev
# mount -t devfs devfs /home/poudriere/jails/110armv6/dev/

Then we mount the packages folder so we can install from the poudriere built gcc packages we built earlier.

# mkdir /home/poudriere/jails/110armv6/packages
# mount -t nullfs /home/poudriere/data/packages/110armv6-default/All \

N.B. if you do a ports update and rebuild between these instructions, the symbolic link to the All packages directory will change, so you'll have to unmount and re-mount the packages directory in the master jail.

Install the package management tool in the master jail
We'll need to install the pkg package manager into the jail. Unfortunately, the jail does not have networking as is, so we'll have to get the package manually from the FreeBSD package site.

Get the latest package for the armv6 architecture. I've put it into the jail's /root directory, but it can go anywhere.

# cd /home/poudriere/jails/110armv6/root
# fetch

Now we can chroot to the master jail.

# chroot /home/poudriere/jails/110armv6

And install the FreeBSD package manager. We need to turn off signature checking also.

# cd ~

# setenv SIGNATURE_TYPE "none"
# pkg add -f pkg.txz

Install gcc into the master jail - note the actual package name will be whatever current version you've built earlier.

# cd /packages
# pkg add -f gcc-4.8.5_2.txz

And exit the chrooted environment.

# exit

Edit the port Makefiles
You need to add the USE_GCC?= yes variable into the Makefile for each port you wish to compile with gcc. Do this for the poudriere ports tree located at /home/poudriere/ports/default.

N.B. you need to do this on a port by port basis. Putting the generic instructions into /usr/local/etc/poudriere.d/make.conf does not work. You get the error Mega56 complains of in the post mentioned above, i.e.

====>> Computing deps for devel/binutils
====>> DEBUG: devel/binutils depends on devel/binutils
====>> Error: devel/binutils incorrectly depends on itself ...

Which is how I managed to Google that post (and come up with this solution) in the first place.

Build the ports with gcc
Poudriere should now build the offending ports with gcc. I issue the command to build each separately. This just avoids building everything else that may need updating listed in my pkg.list file.

For example:

# poudriere bulk -vv -j 110armv6 devel/libatomic_ops

Once you've successfully built the ports that require gcc compilation, you will be able to build ports that depend on them (with the standard clang compiler) using a standard poudriere bulk build.

Have some coffee
Or a cold beer. You've earned it and your Raspberry Pi should now be able to install your pre-built packages from your poudriere package server.

16 June 2015

My New Career

Following an excellent idea @rhythmnation posted on Facebook last night, I'm embarking on a new career of not doing crime. There is so much more to this business that not people-smuggling. You can extend it to almost any cash crime.

First up, I've invoiced the Attorney-General's department for not robbing the bank in my local shopping mall. Have made the terms COND (cash on non-delivery) as it seems this is their preferred operational method of settling invoices for services not provided.

I'm expecting a government officer with a big sack of cash to come round to my place anytime now. Should I invite him in for tea? Or is it best I keep the relationship purely business?

I have some questions for my accountant friends.

Do you think I can claim a tax deduction for any expenses I've incurred by not robbing banks?

Could I negative gear my investment in not robbing banks by claiming the amounts I don't steal as losses?

11 June 2015

The Responsibilities of Citizenship

The idea that a government might have the power to strip citizenship from a dual national on the basis of criminal suspicion alone is probably the most anti-democratic idea proposed by parliamentarians during my (middle-aged) lifetime. The fact the media and some lawmakers are even discussing doing the same to someone with solely Australian citizenship is frightening. I even heard someone on radio say you could do this without making someone stateless. Which is frighteningly ignorant.

These ideas are wrong-headed and self-defeating. And I can't believe no commentator (to my knowledge) has picked up on the most obvious pitfall scenario. What happens when an exiled terror suspect, stripped of Australian citizenship and never tried in Australian court, ends up in a terror training camp and then carries out an attack on a friendly nation's city? How are the good burghers of that city going to feel and think about Australia in that instance? How much co-operation on the “War on Terror” is that ally going to give us after this event?

Citizenship is an exchange of responsibilities. Those naturalised or born to it have a responsibility to act in the interests of their fellow citizens and can expect to be punished when they break the law, are fairly tried and found guilty. The state, accepting the newly naturalised or newborn, has a responsibility to its existing citizens and the international community to try and punish criminals it has previously accepted as its own. A nation shirking that responsibility is cowardly, selfish and acting against the best traditions of democracy.

A similar act of shirking from a petulant 5 year old would have them in the naughty corner for some time. Why do so many seem prepared to accept behaviour from lawmakers that is unacceptable from children?

My example scenario assumes that the law would only be used against terrorists. However, in the past decade we have seen the scope of terrorism creep. To take advantage of powers granted after September 11 the UK and USA have tried to label normal domestic criminal acts like dealing drugs and even acts of journalism as terrorism. We have no reason to think Australia would be any more responsible in applying the label, which is fairly ambiguous to begin with. The power being considered has the potential to allow a government to exile its opponents and critics. This is unlike any law that exists or has ever existed in Australia.

A revocable citizenship threatens all who hold it. A revocable citizenship is not citizenship at all.

Until this proposal to strip citizenship I was indifferent to Australia introducing a bill of rights, whether constitutional or legislative. I found the argument “to define is to limit” convincing – why put boundaries around something unbounded?

I have changed my mind. Australia requires a constitutional bill of rights defining individual rights governments do not have power to infringe. Ideally the definition should codify the Universal Declaration of Human Rights and additionally include the best descriptions of fundamental rights to be found in democratic legislation and constitutions everywhere. It must include irrevocable citizenship and explicitly state that its definition does not express the complete boundary of rights.