Mk-boot-usb: a Script to Create Multiple-Bootable USB Sticks

Note: this page is for version 08g or older. For newer versions please follow the links in the yellow block at the upper left corner of this page. 中文頁請見頁面左上角黃色方塊內的連結。

Summary

Mk-boot-usb is a perl script to create multiple-bootable usb sticks (some call these usb keys / usb thumb drives). It wipes out an entire usb stick, partitions it, creates file systems on it, installs grub, and installs a minimal linux on it. Mk-boot-usb is meant to speed up and lower the barrier of entry for creating bootable usb sticks. The usb stick will immediately become bootable (using the minimal linux), and more useful distributions can then be installed into other partitions manually simply by (1) copying any Live CD into each partition (2) modifying grub's configuration file. Starting from version 08g, it supports cloning: use mk-boot-usb to create a first bootable usb stick and install mk-boot-usb into it, and then with a simple command you can use it to clone many more bootable usb sticks that have cloning capabilities themselves!

This program will destroy all of your data in the usb stick. It may even destroy all of your data in the harddisk and/or those on any storage device attached to it if you are not careful! Additionaly, this program is written with absolutely no security consideration in mind. Please use it at your own risk.

For non-technical FLOSS advocactes, you can leave the technical details to your geeky friends and skip directly to the section: "How Can mk-boot-usb Help Your FLOSS Advocacy?"

Download

Please download here; md5sum: 637b899ab4d0cb8adbfbe3535da44681; change log; release announcement of newer versions will be posted as comments at this blog entry. This program is distributed under the terms of GNU General Public License, version 3 or later.

Note that starting from version 08g, you can extract all files directly at the / directory, and the executable script will be extracted to /usr/sbin/mk-boot-usb. All other necessary system files are extracted to /usr/share/mk-boot-usb, including ttylinux. (Or rather, we should probably say that mk-boot-usb becomes an unofficial skin for ttylinux.) This avoids lots of hassles including the need for network, failing to mount loop file system on unionfs, and so forth.

Requirement and Limits

The following utilities are used by mk-boot-usb: perl, sfdisk, mkfs (mkfs.vfat and mkfs.ext2), grub.

It is recommended that you also install qemu so that the usb stick can be immediately tested after it is processed.

Mk-boot-usb is developed on Edubuntu, the educational variant of ubuntu. There is report that running mk-boot-usb from fedora successfully installs grub and yet the resulting usb stick does not boot. Please report if you succeed or fail on distributions other than *ubuntu.

This program is written with absolutely no security consideration in mind

.

Creating Your First Bootable USB Stick

It takes two steps to create your own first bootable USB stick, and you need to read this and the next section. Thanks to Eating for posting articles with pictures detailing the entire process: creating linux live usb (older articles: 1, 2). You may want to follow the textual directions on my page and look at the pictures in his articles to make your installation smoother since my page will be constantly updated in accordance with the new releases (and since you probably don't read Chinese :-) ).

First please open a terminal and become root by typing sudo bash. In fedora it is probably just su, but see above note about fedora. Then disable automatic mounting of usb disks. In ubuntu 7.10 for example, you would go to system => preference => removable media, and uncheck everything. In ubuntu 8.04, you need to give the command gconf-editor as user "ubuntu" (not as root), go to apps/nautilus/preferences, and uncheck media_automount media_automount_open.

Say you have a 1GB usb stick and you want to put into the usb stick Damn Small Linux 4.2.5 and slax 6.0.0 (other than the default ttylinux). Mk-boot-usb's default grub boot menu happens to contain entries for these two distributions, and therefore they require the least of your efforts. These are recommended to users who try mk-boot-usb the very first time.

So we will allocate rougly 60MB and 240MB for them, respectively, leaving about 700MB for your usb as an ordinary storage device.

First make sure that this usb stick is _not_ attached to the computer. Then extract the downloaded file at the root directory, and run it:

        cd /
        tar xzf ~/mk-boot-usb-08g.tgz
        mk-boot-usb

Mk-boot-usb will ask you to insert the stick. Please do so, and wait a few seconds for the LED on the stick to flash and stop flashing. If file manager windows suddenly pop up, then you have not disabled automatic mounting of usb sticks. Please press ctrl-c to abort, read the beginning of this section, disable it, and restart again. If no file manager windows suddenly pop up, you can then proceed to press enter and continue. Mk-boot-usb compares your /dev/sd* before and after the insertion to figure out which stick you want to wipe out.

Next mk-boot-usb will display a partial listing of the files it finds in your usb stick. Make sure that this is indeed the stick you want to wipe out. It then determins the true size of the usb stick (in MB) and ask you to type in a list of numbers that add up to the true size exactly. It will refuse to proceed if the sum does not equal to the total size exactly. This is to ensure that you are wiping out the right usb stick. (or at least one that happens to have exactly the same size :-) ) Let's say it shows 968 for the 1GB stick. Then you would type something like:

        668 60 240

Leading and/or trailing spaces are ignored. Once you agree to proceed, it will create a primary vfat partition of size 668MB, where it will install grub and ttylinux, plus two ext2 logical partitions of sizes 60MB and 240MB each. (Ext3 is too slow for usb sticks.) Note: actual allocated sizes may be slightly different from what you specify since space is allocated by units of 8032.5 KB.

Now you can test your bootable usb stick by typing qemu -usb /dev/sdy (or whichever device you are processing) if you have qemu installed. ttylinux's login id is 'root' and password 'root' (both without quotes). Congratulations! Now you have your own first bootable usb stick. Of course, with its bare command line interface, ttylinux is not a very interesting distribution for daily use. That's why we need the next section.

In fact you can use mk-boot-usb in batch mode to avoid all the questions. Please read the first few pages of the source code, where the options are commented. Be very careful in batch mode, especially with the -d option. Extremely dangerous!

Installing Live CD's of Your Choice into Other Partitions

For DSL 4.2.5 just copy everything on the live CD into one of the appropriate empty partition of the usb stick, and modify /boot/grub/menu.lst in the first partition of the usb stick. Note that with grub syntax, the first primary partition is called (hd0,0) when this very stick is being booted, and its first logical partition is called (hd0,4). This is about all that you need to modify.

Ditto for slax 6.0.0. BTW, slax automatically records any changes you make during the session into the /slax/changes directory of its partition. Think about it: you copied from a live CD which packs 400MB worth of software into 200MB space, and yet now it works like it cared nothing about having had to deal with a read-only device in its prior incarnation! Slax just rocks!

Notes about ubuntu CD's: (1) There is a hidden directory /.disk on the cdrom. Make sure not to leave it out when copying it to your usb partition. (2) With a few additional steps described in Live CD Persistence (go directly to the section 'Creating the "casper-rw" File System'), you can make your ubuntu read-write even though it is copied from a live CD. That is, you can install new packages on it as if you were using a harddisk installation! (3) When booting from qemu, an ubuntu partition on a usb stick created by mk-boot-usb manages to boot into the splash image but then drops to busybox. However, it successfully boots on a few of my real machines, including eeepc. Someone from ubuntu should really fix /casper/initrd.gz so that when it drops to busybox, there is at least some log for people to post questions for help. See this for details.

Notes about otg linux: This is a an educational Live DVD from Taiwan which was created using tux2live from a an ordinary, installed ubuntu. (And thus suffers from the same qemu problem described above.) I plan to distribute it to teachers since they compose of the majority of my speech audience. Thus it is included in the boot menu since version 08d. If you like otg-linux but mk-boot-usb does not work for you, you can try its own script for creating bootable usb: pc_hd-otg-ins.sh.

For other live CD's, you have to translate isolinux's (or syslinux's) configuration file syntax into grub's. Use DSL and slax entries as successful translation examples.

You could also do the copying from Windows if you have explore2fs or ext2fsd but I would be very nervous about allowing Microsoft Windows to mess with my linux partitions.

Installing a Live System into the USB Stick

Please skip this section if you are a novice linux user.

Let's say you use Ubuntu daily on your desktop and you would like to install it to a usb stick. First of all you need a usb stick with large enough memory (4GB is a reasonable size) since such a live system is not compressed like DSL and slax are. Secondly you have to boot from some other device (maybe from a live CD, or maybe from the slax partition of the usb stick you just created!) and copy everything from your Ubuntu partition into one of the stick's empty ext2 partitions. (ext3 is too slow) By now you also know that /boot/grub/menu.lst on the usb stick's first partition has to be modified.

The remaining steps involve modifying /etc/fstab of the target partition of the usb stick:

  1. Root device should be specified by UUID or LABEL since you don't know its true name in a mobile environment.
  2. Mount /var and /tmp as tmpfs.
  3. Remove most static entries.
  4. Add 'noatime' to the options.

Anything else? Your comments please.

Cloning a Bootable USB Stick

* * * Before you start, beware of privacy issues when cloning your usb stick for others. Make sure that there is no private information such as instant messaging contacts and/or account/passwords. Information wants to be free. Once it is released, you will never be able to recall it. * * *

If you have two or more identical usb sticks (same model, same size), you can make them all bootable easily. First run mk-boot-usb to make one of them bootable. Let's say this bootable one is /dev/sdx, and the other one is /dev/sdy. You can then use dd < /dev/sdx > /dev/sdy to copy everything, including the boot sector, to the other one.

Does it work more generally, namely on usb sticks of roughly the same size? I tried this on two usb sticks of different brands, both of size roughly 2GB. The result was that grub loads partially and hangs after printing "grub ". If anyone knows why it does not work please give me a hint. Thanks!.

No problem. Starting from version 08g, you can clone usb sticks of different models and different sizes easily:

  1. Boot the computer with a usb stick that has mk-boot-usb on it. Let's call this stick W.
  2. Give the command ls /dev/sd? and observe how many harddisks/memory sticks have been recognized by your system.
  3. Insert another bootable usb stick X. Make sure that it also contains mk-boot-usb so that its clone Y will be able to further clone itself :-)
  4. Wait for a few seconds. (Typically the LED on X will flash.) Then give the command ls /dev/sd? again, and observe what name the system assigns to X. Let's say it's /dev/sdx. This will be the source of cloning.
  5. Give the command mk-boot-usb -s /dev/sdx, and insert the destination stick Y when prompted by mk-boot-usb. After pressing Enter, the cloning process will begin, and you can take a break for 10 or 20 minutes. Get some exercise or get a cup of coffee. If everything goes smoothly, Y will be completely wiped out and become a clone of X when you come back.

Is it possible to skip W? Can we use X to boot and clone X to Y directly? I did all this using otg linux. Recall that otg is created from ubuntu using tux2live (see above), and hence it employs unionfs. I had problem mounting partitions of X from within the mk-boot-usb script and I suspect that it has to do with unionfs. (To linux experts: I am very interested to see whether slax avoids this problem since it uses aufs instead of unionfs. But I don't have time to play with slax.) No problem. You just have to manually mount all partitions of /dev/sdx before executing mk-boot-usb. (To linux newbies who were sent to this section directly from elsewhere: please ask your linux friend what "mount" and "df" mean and how to use them. This only has to be done once. You don't have to repeat this for the second clone, third clone, ... and so forth.)

Additional notes:

  1. mk-boot-usb will cowardly refuse to clone from a larger stick to a smaller stick. So you cannot clone from a 2G stick to a 1G stick.
  2. mk-boot-usb can clone from a smaller stick to a larger stick. Extra spaces are allocated to the first partition, the vfat data partition.
  3. However, if the destination stick is too large (larger than 32GB), then mk-boot-usb will also cowardly refuse to work because it does not want to destroy your system harddisk.
  4. Please make sure that every partition of X (the source of clone) is less than 90% full, because (1) Two sticks may not be exactly of the same size even though both are claimed to be, say, 2G (2) There are still risks of running out of space even if Y is larger than X in storage capacity. The process of partitioning Y is not very precise. Each partition on Y may not be exactly of the same size as the corresponding partition on X. If a partition of X is too full, its corresponding partition on Y may run out of space.

My friend Chungyu says that I have created a Mr. Smith of the Matrix film. I think mk-boot-usb is also similar to the Borg of Star Trek :-) In addition, it is similar to Microsoft Word(s), which propagates itself among the citizens, either legally or illegally. But the main difference from either of these three, is that an mk-boot-usb bootable stick leaves the decision of whether to clone to each human master instead of forcing itself upon the victims like these three. Microsoft tries to block linux and other free software in many ways. But linux, like life, finds its own way, and multiplies itself ;-) Please help it multiply and spread in Microsoft's barren kingdom.

* * * Finally, did I remind you to beware of privacy issues? * * *

How Can mk-boot-usb Help Your FLOSS Advocacy?

Think about your next linux install-fest. Now you don't need to ask people to bring their desktop computer. Instead just ask them to bring a usb stick. And a notebook if they really wish to see it work immediately.

It doesn't even have to be a geek event like linux install-fest. You can demonstrate this in 10 minutes to your non-technical audience. For example, next time when you give a FLOSS speech (even to non computer science students), please ask the event correspondent to ask the audience to bring their own usb sticks. If you are lucky enough to have an assistant who is not too afraid of command line, s/he can help create a bootable stick for every person while you give the talk. Then ask the audience to test this usb stick on new computers they plan to buy. This will have at least two effects: (1) Make sure that the hardware supports linux well, including booting from usb. (2) Show the computer sales what is possible with linux. By the way please also distribute portable apps on the first partition.

That's what I will do anyway :-) That was the major motivation behind this work. Happy advocating.

Discussions?

I have no energy to maintain a full-blown software package and its corresponding maintainence system. However, your comments are most welcome. Please use the blog entry to leave your comments.