cross-built linux

A process for building a simple linux installation for a different machine architecture than the one you start with. There's a blog, for those who are interested: http://crossbuiltlinux.blogspot.com/



The architecture that seems least prone to breakage is MIPS, so that's
the suggested architecture to start with. If you have an actual machine
you're targeting with CBL, of course you can use that one instead! But
it might not work as smoothly.

1. Grab all the tarfiles and patch files you need to build the CBL
   system. For your convenience, you can grab `cbl.tar` from
   `http://repo.freesa.org/` to get everything at once (or you can get
   individual files from there, if you prefer; or, for that matter, you
   can get source distribution files from the upstream project sites, if
   you read and understand the IMPORTANT SAFETY TIP below.) Put them in
   `/tmp/cbl-materials`, or in some other location if you prefer.

   IMPORTANT SAFETY TIP: Litbuild presumes that the filename for the
   source tarfiles is the package name, followed by a hyphen, followed by
   the version number, followed by `.tar` and optionally the compression
   suffix `.lz`. (Other compression programs besides lzip can be used,
   but the package-users build script currently only handles lzip.) The
   tarfiles must expand into a directory named package name, hyphen,
   version. This is the convention used by the GNU project, so most
   packages already conform to it. When the upstream project
   distribution has some other convention for the filename or unpacked
   path, you need to fix that before you can use it with litbuild! All
   of the source tarfiles on repo.freesa.org have already been
   rejiggered to conform to this convention.

2. Make sure you have ruby 2.5 or later installed. If you got all the
   raw materials from repo.freesa.org, you have the source for this
   already! It might work best to have your normal (non-privileged) user
   account own this ruby, so you can install gems and things freely. You
   might find a system like `rbenv` or `chruby` handy for this.

3. Install the `litbuild` gem (from the litbuild git repo, or by using
   http://gems.freesa.org/ as a gem source). You need the version of
   litbuild specified in the `litbuild` package blueprint to follow
   these instructions without any changes.

4. Make changes to `configs/amd64-mipsel.sh` if you like. (The
   configuration file shipped with CBL assumes that `cbl-materials.tar`
   has been unpacked in `/tmp`, and defines build and output locations
   for you under `/tmp` as well. You may want to change those locations,
   or the machine type used for the base kernel configuration, or
   something else.)

5. Get rid of the work directory (default `/tmp/build`), if it exists,
   and generate the CBL build scripts with `source
   configs/amd64-mipsel.sh; lb cbl`.

6. If there is a file `/tmp/build/scripts/lb-sudoers`, ensure that it
   is included in your sudoers file (perhaps by copying it to
   /etc/sudoers.d and chowning it to be root:root with a restrictive
   mode like 0400). You can also add the entry:

   (root) NOPASSWD: /usr/bin/sudo -l

   to your sudoers file, so you don't have to type your password every
   time you run litbuild!

7. Run the script `/tmp/build/scripts/00-cbl.sh` that was just produced
   by litbuild.

8. Wait a while to see if you get a COMPLETE SUCCESS.

If you are doing a build with the default configuration -- the host
system is an actual computer, the target system is a QEMU emulated
virtual machine, networking is enabled in the target system -- you'll
need to set up the host system as a DHCP server and gateway for the
final CBL system. (You can refer to the `setup-networking` blueprint for
a dicsussion of all this.)

Here's how to do this:

1. Get rid of `/tmp/build`, if it exists, and generate build and
   configure scripts with `source configs/amd64-mipsel.sh; lb

2. Run the script `/tmp/build/scripts/00-host-dhcp-server.sh` that was
   just produced by litbuild.

3. Wait a while to see if you get a HUGE SUCCESS.

4. If all went well, you can now set up networking suitable for CBL:
   before launching the new CBL system in QEMU, run `sudo
   HOST_TOOLS/bin/launch-target-network`. This will create a bridge
   interface that CBL target machines will use as a switch or hub, a TAP
   interface connected to the bridge that will be used by the QEMU
   virtual machine, and run a DHCP server listening on the bridge. The
   DHCP server will be running in the foreground, so leave that window

5. If you're running multiple simultaneous CBL machines, you can also
   run `sudo /tmp/cblwork/host-tools/bin/create-cbl-network N`, where
   `N` is a number higher than 1; this will create an additional TAP
   device and connect it to the bridge.

6. Run QEMU for the target system with appropriate parameters to provide
   the TAP device to the VM -- this is similar to the QEMU command run
   by the `launch-qemu` blueprint, but with the init command as
   `/sbin/init` and the network options:

   -netdev tap,id=network0,ifname=tapcbl1,script=no,downscript=no
   -device e1000,netdev=network0

7. If you want to have the ability to reach the network via the host
   server, you can also run `sudo HOST_TOOLS/bin/enable-target-internet`.


(As these notes are incorporated (and expanded upon) in the CBL
narrative, they'll be removed from this section.)

- Dependency tracking -- by observing what files were accessed during a
  build, it should be possible to infer what packages that build depends
  on (e.g., do a `touch /tmp/TIMESTAMP && sleep 60` before running a
  build, and  `find / -type f -a -executable -a -anewer /tmp/TIMESTAMP`
  after it's complete). This will be most effective when using package
  users, since looking at the ownership of all the files produced by
  that command will reveal the package responsible for them. So the best
  place to add this feature might be in the package-users build script.

- Bootstrap from C/C++. You have to have _some_ binary code to start
  with, unless you want to do some hand-compilation and hand-assembly to
  get the initial compiler built. The absolute smallest programming
  language footprint you can start with is the language the operating
  system kernel is written in, so that's the best starting point. That
  means any _other_ programming languages and tools that will wind up on
  a CBL system (go, java, docker, etc) have to be bootstrapped directly
  or indirectly from C or C++. (Or copied in as binaries produced on
  some other system, but that's not something that we're going to
  endorse or support.)

  Ideally this would just be "bootstrap from C" (since the Linux kernel
  is written in C) but since modern GCC is written in C++, this would
  require starting with GCC 4.7.x. With that in mind, it might be
  worthwhile to start with an initial GCC 4.7.x in the
  host-prerequisites section -- unfortunately, this would require also
  using an older glibc (as of this writing, latest-stable glibc is 2.27,
  which requires GCC 4.9 or newer to build) and possibly older versions
  of other packages as well.

- Multiple tool sets. GCC is the default C/C++ toolchain, but LLVM works
  pretty well these days and it would be awesome to be able to use
  either of them as the initial cross-compiler, or as the initial
  host-side compiler. Similarly, glibc is the default C library, but
  musl and uclibc-ng are both perfectly good alternatives at this point.
  It would be a good idea to support multiple tools and libraries!

- Single-arch -- the multi-lib and multi-arch systems used by most
  distributions (and enforced by the current GNU toolchain) are messy
  and confusing, so CBL avoids them except where it is completely
  unavoidable. The multi-target, multi-ABI stuff for gcc results in
  horrible complexity and confusion. The specs file for the final system
  GCC should ideally only have the minimal set of options to support a
  single target and C library. Other (single-arch) toolchains can be set
  up to support other targets.

  An idea under consideration: move all libraries and toolchain-related
  files to subdirectories of a new top-level `/arch` directory
  (completely abandoning the `/lib` directory prescriptions of the FHS).