Slow SSD + dm-crypt with Luks encryption in Ubuntu 12.10

Question

I have an 128 GB SSD installed in my laptop (Samsung 840 Pro) on a Sata 3 interface. This laptop also has an i5 3210m Ivy Bridge Intel processor and 8 GB of RAM.

I installed Ubuntu 12.10, using the graphical installer to get full-disk encryption. I am kind of disappointed, because I was expecting the specs I have to yield better results than what I get.

While looking at this SSD Benchmarking page it claims that my processor is able to do:

• ~500 MB/s : With AES-NI
• ~200 MB/s : Without AES-NI

Looking at the numbers I get, I think I may not have AES-NI enabled. But first ...

Reading unencrypted data is fast:

# hdparm -Tt /dev/sda1

/dev/sda1:
 Timing cached reads:   14814 MB in  2.00 seconds = 7411.70 MB/sec
 Timing buffered disk reads: 242 MB in  0.48 seconds = 502.75 MB/sec

That's actually close to my SSD's specs of "up to 530 MB/s" and doing a dd test yields similar results to the above.

Writing encrypted data is fast too with dm-crypt (otherwise with eCryptfs the performance on writing is abysmal, lower than 100 MB/s), the numbers being close to the SSD spec (I guess writing is buffered or something):

# dd if=/dev/zero of=tempfile bs=1M count=1024 conv=fdatasync,notrunc
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 2.93896 s, 365 MB/s

Reading encrypted data is however another story:

# echo 3 > /proc/sys/vm/drop_caches
# dd if=tempfile of=/dev/null bs=1M count=1024

1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB) copied, 5.85956 s, 183 MB/s

While writing this message, I actually got lucky to get 183 MB/s, because this number varies. Normally it's somewhere around 150 MB/s, but I also got close to 300 MB/s on a fresh boot, but then performance drops gradually to under 200 MB/s without me starting any apps. Do note that while conducting this test I have no other processes that are doing I/O (as seen with iotop).

Also, here's the test with hdparm which yields worse results:

 # hdparm -Tt /dev/mapper/sda2_crypt 

 /dev/mapper/sda2_crypt:
  Timing cached reads:   14816 MB in  2.00 seconds = 7412.86 MB/sec
  Timing buffered disk reads: 422 MB in  3.01 seconds = 140.11 MB/sec

I also tried the experiment by looking at htop ... not sure how to interpret what I saw, since the i5 processor does Hyper-Threading, but 2 threads out of 4 where going to about 73% of usage during the test, while the other 2 threads where left unused. Indeed, if I start 2 processes that are reading with dd from 2 distinct files (to prevent buffering), then iotop reports a total of approx 400 MB/s. So this definitely feels like it's CPU-bound.

My disappointment comes from the fact that my i5 processor is capable of AES-NI. Unencrypted data is being read at over 500 MB/s using the same tests I did above. So we are talking about an encrypted partition being at least 3 times slower.

I don't really know if my dm-crypt install is using AES-NI. Here's the output of lsmod:

 # lsmod | grep aes
 aesni_intel            51038  35 
 cryptd                 20404  10 ghash_clmulni_intel,aesni_intel
 aes_x86_64             17256  1 aesni_intel

Here's the output of cryptsetup:

 # cryptsetup status sda2_crypt
 /dev/mapper/sda2_crypt is active and is in use.
   type:    LUKS1
   cipher:  aes-xts-plain64
   keysize: 512 bits
   device:  /dev/sda2
   offset:  4096 sectors
   size:    249565184 sectors
   mode:    read/write
   flags:   discards

So, is this expected? Shouldn't AES-NI improve things more than this?

Also, how to disable AES-NI to see if there's any difference? And maybe I should enable it somehow, but haven't found any tips in my searches.

Thanks,

Solution 1

Your Samsung 840 Pro supports hardware AES encryption. If your laptop BIOS supports the ATA security feature mode set master and user passwords, you're in luck.

By setting the ATA master password in the BIOS, and then doing a secure erase of the drive, the drive firmware should generate new random AES keys. After the secure erase make sure that you've set good ATA master and user passwords. As far as I have been able to establish (see my post here http://vxlabs.com/2012/12/22/ssds-with-usable-built-in-hardware-based-full-disk-encryption/ ) the Samsung also encrypts its AES keys with your ATA password.

This will give you full speed AES encryption, no software required.

Solution 2

There is a misconfiguration in Ubuntu that results in the aesni_intel module not being loaded early enough to handle crypto for boot-unlocked devices. I was able to fix this on my machines by doing:

sudo vim /etc/initramfs-tools/modules

Below the last line, add

# enable h/w accelerated encryption
cryptd
aes_x86_64
aesni_intel

Then run

sudo update-initramfs -u -k all

reboot, and enjoy. After this, on a similar SSD, I was seeing 500MB/s read and write with negligible CPU usage.

Full details on this bug are at https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/908387/comments/7