Add a Package to Optware

You have a program you would like to add to the packages area?

Here are the steps necessary.


 * 1) Understand why we are doing the packaging efforts.
 * 2) Check out your own copy of packages source tree.
 * 3) Create your first own package.
 * 4) Request SVN developer access.
 * 5) Check in your package with your commit comments.
 * 6) Mark it as ready for testing.

=Software Packaging Overview=

You must have SVN access to the packages source tree. If you have questions about SVN, the instructions at SourceForge http://sourceforge.net/apps/trac/sourceforge/wiki/Subversion are quite helpful. NSLU2 SVN is available on the web at http://svn.nslu2-linux.org/svnroot/optware


 * To build and test your new application you will only need anonymous read access
 * Saving your changes requires you to have developers access.

Note that packages are normally built on a HOST machine (like a desktop or laptop) running Linux. Some people compile natively on the NSLU2.

You should wait until you have compiled and tested your application before requesting write access. Once you have downloaded the development tools and package sources, and successfully built Unslung and all the packages, you are ready to add your own package.

To get SVN write permission so you can add your package to the Optware distribution, send your request to the developers mailing list (Join nslu2-developers on Yahoo Groups then send an email to nslu2-developers@yahoogroups.com requesting SVN write access, with the .mk file and any other associated source files as an attachment, and you will normally be given SVN write access as soon as an admin sees it. Note that we will not add the package for you - the reason for sending it to the list is so you can be approved for SVN write access - you will still have to check in the package to SVN yourself. Furthermore, as a registered SVN developer, you must be entered into the Trac bug tracking system at http://trac.nslu2-linux.org/ as a developer and every package you add must be added too.

If you are cross-compiling on a normal Linux desktop or laptop (this is the preferred option for packages that can be cross-compiled) then the build system will compile the crosstool package for you, so you don't need to build it yourself any more. If you are native compiling, then see HowTo/NativelyCompileUnslungPackages for details of the native compilation toolchain. Native package building should be possible from beginning to end using the following instructions.

!Check out package sources from svn.nslu2-linux.org !! Checking out package sources before you have SVN write access First, you need to copy the SVN tree. cd /home/slug/        (you can use whatever directory you want, of course) svn co http://svn.nslu2-linux.org/svnroot/optware/trunk optware This will create an optware directory with the whole svn tree. Inside optware/make is our guiding template.mk file. template.mk makes life much, much easier as we only have to edit this file to ensure everything builds properly. For the purpose of the remainder of this article, I'm going to call the package we're building mypackage.

Quick build notes: If you have not yet got a linux build system you can use windows, see below. If you type make in /home/slug/optware then the cross compilation tools along with every package will be fetched and built -- this takes a long time and is likely to break over a missing system package or old hyperlink. If you just want to build the cross compilation tools and obtain a set up that will allow you to build your own packages then take a look in the Makefile, pick a small utility of interest from the package list such as make/which.mk, execute the commands listed below in the section 'Building the ipkg package' and type make . Once its built, have a look at optware/make/ .mk, compare with optware/make/template.mk and follow the section 'Making a package using the template' to start building your own packages.

If you are using a recent version of Ubuntu, the default shell will not allow the toolchain to compile (errors like "csu/version-info.h:1: error: missing terminating " character"), type ls -la /bin/sh and if the result points to dash, then type sudo dpkg-reconfigure dash or simply sudo ln -sf /bin/bash /bin/sh to change the shell. With dash as /bin/sh, it sometimes causes optware perl build to stop with 'You haven't done a "make depend" yet!' error message.

New (windows) build system: If you are starting from scratch with a windows box then the easiest route is install VirtualBox (http://www.virtualbox.org/) and then install a version of linux into that, pick your favourite but I used Ubuntu 7.10. Create a machine with at least 6Gb of hard disc. You will need to install some packages to build with: apt-get install  the following packages: gcc, cvs, flex, bison, make, pkg-config, rsync, gettext, libglib2.0-dev, autoconf, libtool, automake, automake1.9, sudo, patch, bzip2, gzip, wget, sed, texinfo, subversion and while your at it, your favourite editor e.g. emacs.

Note, if you are wanting to build for nslu2 (unslung), Ubuntu 9.10 cannot be used, as GCC 3.3 is needed (3.4 may suffice). Debian 5 still has this in its package feeds.

Developer (with SVN write access) way of checking out package sources
$ svn co https://svn.nslu2-linux.org/svnroot/optware/trunk optware
 * Request and be approved for SVN write access - you will be sent an NSLU2-Linux SSL client certificate
 * Use SVN to check out the optware distribution:

= Building the ipkg package=

The ipkg package is needed as part of building your own packages. Do the following:

$ cd optware; make -target $ cd ; make directories toolchain ipkg-utils

Where can be one of nslu2, cs05q3armel, etc. See optware/platforms/ for available targets.

=Guidelines for package developments=

There is work in progress to show which practices have shown to be successful and therefor encouraged, and which aren't. Go to Packaging Best Practices to see these and add your own commments.

An especially important rule is that all packages must be able to be rebuilt from source from the SVN repository. We do not accept binary-only packages in the official Optware package repository (however, we are happy to point to other third-party repositories).

You might also find it helpful to know how our build machines are set up. This information can be found at:


 * Info.NativeBuildMachine for official native build machine (an NSLU2 dedicated to building optware native packages)
 * Info.CrossBuildMachine for official cross build machine (a PC)

!Making a package using the template

cp make/template.mk make/mypackage.mk

Now comes the bulk of our work -- editing mypackage.mk. Probably you could simply follow the directions in that file and be quite successful, but I'll go through the process is a bit more detail. Two variables in mypackage.mk can be replaced quite quickly -- these are big  and little. Big  and little need to be replaced with your package name in capital and lower case respectively. We can do this quickly with:

sed -i 's//MYPACKAGE/g' make/mypackage.mk sed -i 's/ /mypackage/g' make/mypackage.mk

Or use make make/mypackage.mk as a shortcut of the above.

mkdir sources/mypackage

Edit mypackage.mk with our package specific variables. Open your favorite editor, and look for MYPACKAGE_SITE. This variable needs to be set to the url (usually ending in "download") where your package's source may be found. wget will try to download your package from $(MYPACKAGE_SITE)/$(MYPACKAGE_SOURCE) -- so set MYPACKAGE_VERSION as well. If your file is bzipped, set MYPACKAGE_UNZIP=bzcat, otherwise leave that variable alone.

For the remainder of the editing process follow the directions in mypackage.mk. For the packages I've worked on there are no patches so I commented out references to patch.

The template.mk file assumes that you will be downloading your package's source in tarball format. If for some reason you must download it with cvs or some other revision control tool, you should start with template-cvs.mk instead. Only do this if you have tried and failed to find a usable tarball.

The optware/sources/mypackage directory can also contain files for actions to take after installation (postinst), any init tabs your package might need (rc.mypackage), and any other files that your package might need (note: if you don't know of any, there probably aren't any -- as an example, the bash package installs profile). If your package doesn't have any postinstall or init files, you should comment out the appropriate lines in the (MYPACKAGE_IPK) section of mypackage.mk.

Now it's time for the moment of truth. From your optware/ ( being the optware target) make your package. make mypackage     (this compiles your package) make mypackage-ipk (this makes your package and the ipk) make mypackage-check (this does some sanity check of your new ipk)

You may get errors; try to determine whether the error is occuring due to the makefile (i.e. mypackage.mk) or due to problems in your source program. If you have problems, you can probably find help, or at least sympathy, on IRC.

If you get an error during compilation that says keramik is not found, unset STYLE and try again. Keramik sets the STYLE variable which is also used by some packages.

If you want to know the value of any optware make variable, you can "make query-VAR_NAME" (very useful).

If you get an error building the ipk that tar doesn't recognise the --format option you need to upgrade to a newer version of GNU tar (at least 1.14). If you are running Suse 9.1 it is possible to uninstall the standard rpm (ignoring any dependency violations) and install the rpm from Suse 9.2. Similar tricks may be possible on other distros but make sure you have got the original package available to reinstall if needed.

Error Example 1: "copy and paste" was performed from above instructions to get following error message. www.mypackage.org is a fake one, it should be replaced with a real address and so as "mypackage" pacakge name. [tjyang@dual optware]$ make mypackage wget --passive-ftp -P /export/home/tjyang/slug/optware/downloads http://www.mypackage.org/downloads/mypackage-3.2.3.tar.gz --21:32:51-- http://www.mypackage.org/downloads/mypackage-3.2.3.tar.gz            => `/export/home/tjyang/slug/optware/downloads/mypackage-3.2.3.tar.gz' Resolving www.mypackage.org... failed: Host not found. make: *** [/export/home/tjyang/slug/optware/downloads/mypackage-3.2.3.tar.gz] Error 1 [tjyang@dual optware]$

Once you have it compiling, try those commands again. Nothing should happen (as everything should be up to date). If you find that things are being rebuilt, then it means that the dependencies in your mypackage.mk are not correct - you will need to fix them before your package will be accepted into the official repository.

Once you have your ipk, try to install it (ipkg install /path/to/mypackage). If everything seems to be working, write your project back to svn (and remember, you have to have write access to svn). Update the Optware Patches page with your new package.

=Check in your package sources to svn.nslu2-linux.org=


 * remember to put in commit comments as described here: http://www.nslu2-linux.org/wiki/Development/CommitGuidelines
 * svn add
 * svn commit -m "package: brief description of new package, initial addition"

=Mark your package ready for testing=

When your package is ready to be looked at by a core developer, add its name to one of the READY_FOR_TESTING lines in the root Makefile. In due course, someone will take a look at what you've done. If it works, and follows the Unslung.PackagingBestPractices, it will be uploaded to the feed. If not, it will be moved to a NEED_TO_BE_FIXED line, and a comment entered in the makefile describing the problem.

svn lock Makefile  svn commit -m "package: comments as per guidelines" Makefile

If you are requesting native compilation for your package, please add a comment to the Makefile explaining concisely why the package cannot be cross-compiled. The process for this is:

You would be well advised to be on the #nslu2-linux irc channel when you do SVN commits, especially when you make your package READY_FOR_TESTING. That way you will get realtime comments from other developers.

=References=
 * 1) For more information about the IPKG Package Management System, take a look at http://www.handhelds.org/moin/moin.cgi/Ipkg
 * 2) For SVN introduction, please see https://sourceforge.net/docs/E04/
 * 3) A nice overview howto by jeanfabrice is here: http://wl500g.info/showpost.php?p=14309&postcount=4

Good luck with NSLU2 development!

If you see something unclear in this description, please be sure to update it. !! Note regarding Libtool and Libtoolize

For building most packages, the GNU Libtool and Libtoolize are essential programs that need to be installed on the build system. These two programs interact with the ltmain.sh script included with most program tarballs. There are currently two version streams for these programs, version stream 1.5.x included with most older versions of Debian and Ubuntu etc, this is the version used by the NSLU2 Optware build machines. More recent versions of Ubuntu and other Linux distros have now upgraded Libtool and Libtoolize to the version stream 2.2.x.  Unfortunately many Optware packages will not build with this later version. The reason - version stream 2.2.x is not fully backwardly compatible with ltmain.sh scripts from earlier versions. This backward compatibility problem is well documented on the internet, there are may recommended fixes and patches etc, most are not fully explored,  some may fix one package but break another. To check the version of Libtool installed type at a console prompt : -

libtool --version

For Optware development it is recommended that the version of Libtool is downgraded to version 1.5.26 or lower, example - with a Ubuntu Intrepid build system install the Ubuntu Hardy version of the libtools.

Another possibility is to upgrade the version of the ltmain.sh script included with the program tarball to a version 2.2.x. This seems to work with some program tarballs but not all.

December 2008

Note regarding Build servers
As of 2010-Oct-26, the official builder is still on Ubuntu 8.04.3 LTS.