How do I install and use flashcache/bcache to cache HDD to SSD?

Question

I want to know if there is anyone who has tried installing and running their kernel built with flashcache or bcache for SSD caching?

How did you manage to do it?

Solution 1

Well, we got a bcache answer but no flashcache answer. I chose flashcache because I already had an existing installation, so bcache was out of the question. To me it seemed easier to set up as well. I chose the DKMS method so I wouldn't be stuck rebuilding the module/tools every time I got a kernel upgrade.

These steps are outlined in README-DKMS on Github.

Building the Module

1. The first thing you do is install the building prerequisites:

   sudo apt-get install dkms build-essential linux-headers-$(uname -r) git
   

2. Then snag the source:

   git clone git://github.com/facebook/flashcache; cd flashcache
   

3. Build and install the modules:

   make -f Makefile.dkms
   

Creating a flashcache device

(In this example I use writeback caching -- the default -- but flashcache can do writearound and writethrough as well)

Caching a non-root (not mounted at /) device

sudo flashcache_create -p back fcache /dev/sdbX /dev/sdaX Where fcache is the created device (/dev/mapper/fcache), /dev/sdaX is the SSD partion (or whole disk) and /dev/sdbX is the HDD partition.

Caching your root device

There are a few more steps if you're caching your root device. You will need an Ubuntu LiveCD/USB for this section.

1. cd /path/to/flashcache/source

2. make -f Makefile.dkms boot_conf

3. Edit /boot/grub/grub.cnf and /etc/fstab to boot from /dev/mapper/fcache (I don't believe this step is really necessary, but it says to). Be sure to write down the UUID of your original root device.

4. Reboot with the LiveCD/LiveUSB.

5. Mount your root device: mount /dev/sda4 /mnt (/dev/sda4 is my Linux root partition)

6. cd /mnt/path/to/flashcache/source

7. sudo apt-get install dkms build-essential linux-headers-$(uname -r)

8. make; sudo make install No need to do the whole DKMS setup in the live environment

9. Unmount the root device sudo umount /mnt

10. sudo flashcache_create -p back fcache /dev/sdbX /dev/disk/by-uuid/[UUID] Where fcache is the created device (it should be the same as you put in /etc/fstab and /boot/grub/grub.cnf, in my case it was fcache), /dev/sdbX is the SSD partition (or whole disk) and [UUID] is the UUID of your root partition.

11. Reboot!

Caveats when using the root device

One minor annoyance when flashcache is used for the root device is that grub-probe fails to detect the root device and can mess up your boot menu so that you have two Ubuntu entires. But, it shouldn't matter which you use in all reality since the make -f Makefile.dkms boot_conf step above installs some scripts in your initrd that will detect and use the flashcache device.

Solution 2

There is a conversion tool for enabling bcache on an existing device (disclosure: I wrote it). It's also the way to go for fresh installs, since current distro installers don't know how to create bcache devices (in which case: start by installing to the HDD).

See converting your root filesystem to bcache in the blocks README for instructions.

bcache requires Linux 3.10 or newer, which is in saucy, and can be manually installed from http://kernel.ubuntu.com/~kernel-ppa/mainline/?C=M;O=D (look for the newest tagged release, currently v3.11.5-saucy, and install the debs that match your architecture).

bcache-tools and blocks can be installed from my storage ppa.

Solution 3

Prior warning: bcache can not be used for your existing Ubuntu installation. If you need SSD caching for your operating system you should wait for a future or special version of the Ubuntu installer which could set up your SSD cache. See this answer by Oli here: Does bcache require a fresh installation?

For everyone else who wants a bcache drive for non-OS data:

The first thing to do is upgrade your kernel if your using 13.04 or older using the kernel mainline PPA. Instructions for using this can be found here:

https://wiki.ubuntu.com/Kernel/MainlineBuilds

Make sure you select the 3.9 or higher kernel when booting or it won't work and your drive will disappear.

Install bcache-tools, at the moment you will have to install from the git repository:

git clone http://evilpiepirate.org/git/bcache-tools.git

If you manage to package bcache-tools for debian, let us know and this question will be updated for you. (bcache-tools's build dependencies are uuid-dev and libblkid-dev. They must be installed, to build bcache-tools.)

Once you have all the tools in place, you're ready to try and follow the guide here:

http://atlas.evilpiepirate.org/git/linux-bcache.git/tree/Documentation/bcache.txt

This guide shows how to format the drives (SSD AND hard drive) and then attach them together to make a new device block using bcache. Post a comment if you find the instructions confusing and I'll update this answer with simplifications.

Solution 4

i thought I would come here and comment on what occurred to me when I installed a new kernel. I was using flashcache for my /home/ folder partition only. I installed a new kernel, booted into it and it stopped during bootup and said it couldn't find /dev/mapper/home_cached, there are 3 choices you have when you recieve this message and I choose the option to log in as root and attempt to fix it. I first attempted to load the flashcache module and it said it couldn't find it. So i figured I would have to compile and install it for this new kernel I just installed. i still had the /flashcache/ folder located within /root/ so I cd'd into it and ran

make -f Makefile.dkms boot_conf

then I ran

make install

then I loaded the flashcache module with

modprobe flashcache

then i just had to load my existing flashcache device which was

flashcache_load /dev/sdb1

then just mount my home partition with

mount /home

then exit out of being logged in as root with exit on the command line and it should continue booting up as normal. all done!

View more solutions

Comments

[Admin]

Good question. I've been suggesting flashcache in some of my answers (1, 2). It seems the tool for the job, next to dm-cache. It's also relevant to all those who are looking for an alternative to Intel Smart Response for Ubuntu/Linux.

[Admin]

I, tooo was looking at it as an alternative to Intel SRT. The arch wiki here says in, in a few more words, that it is not for the faint of heart. The problem is that I am planning to play around with it on my only PC for both work and play :), so was apprehensive and hoping to get a how to here. Do you think it will be possible to run ubuntu on a VM and have one drive(file based) on the disk one on the SSD(again, file based) to test. Willing to give it a go then :)

[Admin]

Nice question, but is an answer that uses VMs as a test bed considered complete? A lot of people don't have the necessary hardware to really answer this.

[Admin]

@AbrahamVanHelpsing For me that would be okay. I think using a ramdisk from the host presented as a virtual disk to a VM can pretend to be an SSD very well.

[Admin]

already tried this for Flashcache? gerrit-tamboer.net/…

[Admin]

@jasmines nice link. Will try and give it a go with a virtual device set up, but I was actually trying to know if it was possible to use the flashcache device the exact way in which intel srt works, in that my entire OS partition, with installed apps and boot files etc is completely cached and gets speeded up. Do you know if this can be done? How to?

[Admin]

@kapad Yes, that is certainly possible. Instead of /home from that tutorial you'll need some more tricks to get this done on /, such as using a Live CD with some extra steps. This is exactly what I would like to see in an answer. But yeah... not much time left to get answers in for the bounty...

[Admin]

hopefully dm-cache will be available in ubuntu 13.10 then.

[Admin]

and just last month I formatted to SSD to move my Swap space onto it. Damn!

[John G]

Is it possible to create a startup disk running kernel 3.9 and then create the bcache drive and install ubuntu on top of this drive?

[Martin Owens -doctormo-]

kapad - it should be possible, but you would have to make sure the ubuntu install had the right drivers available at boot time and you might have to make boot outside of bcache so grub could find it.

[ivant]

I see that at least in raring there are packages for flashcache-dkms and flashcache-utils. How would the installation change?

[Chuck R]

Oh really? I hadn't noticed. I'm running Raring as well but installed it from the Git repository. I'd have to look at the package, but I can imagine that you would be able to skip the first couple sections and still run through the process of booting a root device.

[elementstyle]

ubuntu 13.10's flashcache-dkms package does not include flashcache modules and binaries to the initramfs, therefore if you flashcache your root you will be unable to boot it. I have created a bug: bugs.launchpad.net/ubuntu/+source/flashcache/+bug/1290148 I have built flashcache from source as described above and I was able to boot the following chain: sda7 -> flashcache -> dm-crypt -> lvm -> ext4. Just add the flashcache dev to /etc/crypttab and update-initramfs -u

[Ubu the Tech Guru]

im curious if this still works, i ended up switching and not using flashcache any longer, i just installed Arch on the 64GB SSD (the linux kernel thinks it's only 32GB however) due to the internal firmware. I could see this being beneficial as a caching drive for some steam games.

[Jean Jordaan]

This tool is broken for the current (15.04) version of Ubuntu; see github.com/g2p/maintboot/issues/1 and github.com/g2p/blocks/issues/13

[ntninja]

Actually the tool itself is not broken currently (tested in yesterday on a 350GB hard drive). However current versions cannot be used to convert system disks (/, /home, /usr/, …) while the system is running (as maintboot is broken). As an alternative you may use a live USB Stick, install blocks there and use that for converting your disks. Beware however that this tool only works on primary MBR and GPT partition tables and requires 1MiB of extra space before each partition.

[John G]

can you add a comment with the changes that you had to make to the grub file.

[Chuck R]

I don't modify the grub.cfg anymore, it doesn't seem necessary. However, you do a change like this: linux /boot/vmlinuz-3.19.0-51-lowlatency root=UUID=0a540b91-5990-4df0-bc99-5d6e2cd3fed6 ro recovery nomodeset to linux /boot/vmlinuz-3.19.0-51-lowlatency root=/dev/mapper/fcache ro recovery nomodeset (i.e., change the root= parameter)