ZFS: best practices for adding cache to pools

Question

sounds like dumb question but there are no ready answers at SF

can the cache pool have different name than the zpool?

eg: suppose there is a zpool "tank" serving NFS and iSCSI

can a new cache pool be added as "xyz1" for it to be serving the "tank"?

or is there a per pool cache concept?

Solution 1

First of all, for general ZFS best practices the Solaris Internals wiki is quite useful: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide

If you want, you can create a separate ZFS pool composed entirely of SSDs. However, this will not improve the performance of any other pools. A more cost-effective approach for most workloads is to use the hybrid storage pool feature; that is, adding SSDs to normal spinning disk in order to improve read and/or write performance.

Key points:

• SSDs can be added in to an existing pool to improve the performance of that pool only. They don't have different names - they are just devices within the pool. You can add multiple cache and log devices to the pool depending on your performance requirements.

• To improve the read performance of the pool, assign SSDs as cache devices ("L2ARC").

• To improve the write performance of the pool, assign SSDs as log devices ("ZIL").

The L2ARC is a read cache that fills up over time and stores data based on a combination of which blocks are most frequently used and most recently used. This provides fast access to "hot" data. It gives almost the speed benefit of RAM at a lower cost per gigabyte.

The ZIL is an intent-log - adding SSDs allows synchronous writes to happen at high IOPS and very low latency, which improves application performance in some cases (e.g. databases and NFS shares).

Solution 2

If you are going to be deploying NFS, a ZIL device or devices may be more important than adding cache devices. Instead of cache devices you can increase the amount of RAM in the system, and instead spend more on write-optimized small capacity SSDs for ZIL. At Nexenta, we always recommend mirroring your ZIL. Not doing so does expose you to a very real possibility of data corruption under certain conditions. With cache devices you would actually look at read optimized SSDs, and no real need for mirroring, because all data and metadata could be discarded at any moment without any risk to the rest of the pool.

Comments

[Philip]

There is no cache pool concept, the two types of cache ZIL and L2Arc are part of a pool.

[Giovanni Tirloni]

I think an important point that people from more conventional storage shops forget is that SSDs can be either cache or permanent storage. If you would like to use SSDs for cache, add them as log/cache devices. But you can also create a pool of SSDs.

[John-ZFS]

when SSds are added as cache or log devices - do they serve all the pools in the system? my question is not about hybrid storage pools.

[Asinine Monkey]

They serve the pool that you add them to. They are not global.

[Naman Bansal]

@John-ZFS - a pool with SSD cache/log devices is a hybrid storage pool. See Adam Leventhal's blog. As @Asinine Monkey mentioned, cache devices are per-pool, not system-wide.

[slashdot]

Yes you are essentially correct. Keep in mind some failures of disks do cause bus failure and resets which can easily cause a crash.

[Naman Bansal]

Thanks. Last year I set up a 7410 with mirrored log devices for essentially the same reason, but I did wonder if I was being paranoid!