BTRFS - One Year Later

It was a year ago when I decided to give BTRFS a try.  Prior, I was using primarily EXT4 partitions, with a few XFS partitions on really slow HDs (4200rpm).

It was also about a year ago that I started using LXC (Linux Containers).  I needed a way to maintain multiple LXCs while reducing disk storage overhead.  Initially, this is what caught my attention for BTRFS.  BTRFS provided a typical journaled Linux filesystem a keen to EXT4, but provided "snapshot" and "cloning" a keen to QCOW2 with KVM.  I could maintain multiple LXC containers, and let them share common files, reducing disk consumption.  With BTRFS, I could clone or snapshot LXC instances and avoid duplication -- only worry about storing the changes between instances.  There are not many filesystems that provide this ability while still maintaining properties associated with a typical EXT filesystem and avoid using a file container like QCOW2.  I immediately started using a BTRFS filesystem for storing all my LXC instances.

Since my LXC usage was recreational and non-critical, I experimented with subvolumes, snapshots (including read-only), compression and a few other features of BTRFS.  Having encountered no stability or reliability issues, in the summer of 2014, I decided to give BTRFS a trial run as the primary filesystem for a new Linux Thinkpad.   BTRFS proved beneficial for me for LXC, and the snapshot functionality could equally provide me benefits from effortlessly backing up and rolling back Linux upgrades in the future.  Prior, whenever I moved to new update packs (distro upgrades) in LMDE (Linux Mint Debian Edition), I would create a clonezilla backup -- obviously an offline process, that required me to boot up with a clonezilla image to perform the backup.  I've had to rollback upgrades previously, for all kinds of reasons.

Like I typically do, I cloned a Linux EXT4 image of a working system (using Clonezilla) to the new system.  I utilized the BTRFS tools to convert the EXT4 image to a BTRFS filesystem.  Since the conversion process doesn't modify the data itself (only the inodes or meta data), it was a relatively quick process (mere minutes) and provides an equally quick process to rollback the filesystem back to EXT4.  I even (successfully) tested the rollback.

It was around this time I had my first BTRFS data disaster.  Ironically, it had nothing to do with the new system but on an existing system where I was using BTRFS only for LXC storage.  I had previously encountered situations where I would find myself with my LXC storage mounted in "read only".  A reboot would cure the issue, but I wasn't sure what was triggering the situation.  I was convinced that hibernating the system while LXC were running may have been causing the issue, so I modified my hibernation scripts to explicitly freeze any running LXC instances prior to hibernating.

One day when the problem had surfaced, I decided to resolve the issue by remounting the BTRFS subvolumes as read-write.  This proved disastrous.

When simply unmount and remounting in rw failed, I decided to try using the BTRFS fsck equivalent to examine the filesystem and fix any errors.  There were reported errors with the BTRFS structure, later which I determined to be false, but I attempted to fix these issues.  That was my first mistake.  I was not aware that the BTRFS filesystem repair utilities are wildly experimental and inherently will corrupt and cause further problems.  Warnings throughout the net, reiterate not to use btrfsck to try to repair filesytems.  Why does the tool exist, I cannot be certain, but definitely one day the tool may provide some kind of meaningful function.  My use of the tool led to a corrupt BTRFS root filesystem, which could no longer be mounted.  By virtue that the root filesystem was corrupt, all subvolumes were inaccessible and considered lost.

I had later determined that my BTRFS filesystem was never initially damaged or corrupt (at least, prior to running btrfsck).  I discovered that whenever I had encountered that my BTRFS had been mounted or remounted as read-only had only occurred after shutting down an LXC.  I quickly realized that the mount scripts installed in some of the init.d routines on a number of my LXC were "successfully" causing my BTRFS subvolumes (or root filesystem itself) to unmount and remount in ro.  I had not seen this behaviour with EXT4.  And because I don't usually shutdown my LXCs without rebooting the host itself, the issue was rarely triggered.  So, the issue was never caused or contributed to hibernating the host.  By simply not shutting down the LXC (use halt or leave them running and let them stop when the host shutsdown) or by removing/disabling these mount scripts, the problem was resolved.  I have not encountered the issue since.

I was very lucky in the disastrous scenario.  I had lost the entire filesystem hosting all my LXCs on one system, but because I had used the send/receive functionality in BTRFS to "clone" or backup my LXCs from the system to a central hard drive, I was able to simply "receive" backups of the filesystems back onto the drive.  My LXCs don't contain data, themselves, so the instances are fairly static.  The disaster let me at least validate recovery of the filesystems from backups.

But from the disaster and my other day-to-day experience with BTRFS on various systems for LXC storage and on the newly setup laptop using BTRFS as the primary drive, here is my list of cautions when using BTRFS:

1) BTRFS is considered "stable" but is still being highly developed.
For this reason, use the most current "stable" kernel as you can.  I would not consider using anything less than 3.13 for day-to-day use, and definitely nothing less than 3.10.  If you don't have other issues or reasons to not use 3.15, I would highly recommend 3.15 be your day-to-day kernel release.  The most current BTRFS changes are in 3.15 and there has been a considerable amount of enhancements and fixes that might prevent unnecessary issues.

2) "Stable" but not necessarily "production" ready.
Considering the repair tools themselves are destructive, I would not use this filesystem for "data" storage.  For example, I would not use these filesystems to store primary or sole backups of data.  I continue to use EXT3 and EXT4 with it's journaling and widely reliable and stable filesystem tools (such as fsck).  Generally, my "data drives" are mirrored across different hardware to account for hardware failure.  When proper processes are followed (performing clean dismounts and syncs), I haven't encountered a filesystem corruption issue that journaling could not resolve.  When that day occurs, the fact I maintain mirror devices will hopefully assist with recovery.  Accounting for the fact that these tools can be considered not production ready for BTRFS, you have no recourse to address filesystem corruption caused by software error, human error or hardware failure.  Even with mirroing, it is plausible that if your only recourse is to restore your data, there is a higher probability of a double-failure, where both mirrors fail, when using a filesystem that doesn't provide for reliable recovery.   Thusly, on systems where there is constant turn of data, I would also not recommend BTRFS.  On systems such as laptops for the primary operating system filesystem, as long as you have backups of the filesystem on other devices to restore from, I would recommend you do included BTRFS for your consideration.  However, if these systems are critical, where downtime is not tolerable, I would approach with caution.  If you are using the filesystem on a primary laptop while traveling, I would make sure a copy of the filesystem or alternative filesystem be accessible either on a memory key or flash device.  If you are traveling (where obviously you may not have your backup drives with you) and unfortunately encounter an issue with your filesystem for which you can't boot or mount your system, you'll need that alternative boot device.

3) Don't treat subvolumes within the same device as your sole-backups.
If you create subvolume snapshots and store them on the same root filesystem, then don't considered these backups.  They'll prove worthwhile for rolling back changes on your primary filesystem, but if you encounter a hardware failure or corrupt your root filesystem, your subvolumes will serve no purpose.

4) Mount subvolumes only, wherever possible.
Instead of mounting your root filesystem to access your subvolumes, consider only mounting your subvolumes.  Human errors such as a "rm -rf *" on your root filesystem will take out all your subvolumes, but the same human error when performed on a subvolume will only affect the subvolume itself.  Likewise, if you encounter a bug or other filesystem corruption, if only your subvolume is mounted at the time, the risk to the root filesystem (and thus, other independent subvolumes) should be unaffected.

5) Use send/receive to store mirror copies of your subvolumes to other devices.
When backing up, generate a read-only snapshot (snapshots are essentially free in terms of disk storage), and send these snapshots to a suitable BTRFS filesystem stored on a separate disk.  Space-saving  properties are maintained when transmitting your backups.  Use these read-only snapshots as a reference point.  Create your read-write filesystems as snapshots of these.

6) Track your snapshots and subvolumes by using logical naming and track hierarchies in a text file.
If you are creating a lot of subvolumes, performing a lot of snapshots and sending/receiving (cloning) your filesystems across multiple systems, it is smarter to ensure the subvolumes and snapshots include references in their name such as source and date.  Also, I recommend tracking this to a text file, to track which subvolume originated from which snapshot and is shared with which systems.  As you start having branches distributed among multiple targets, it'll be harder to trace the origin of a particular subvolume in the future.  I store this information in a text file typically found in the root filesystem so that I don't rely on memory or tracing through output from tools to familiarize or validate the origin of any given subvolume.

7) Use compress with lzo whenever possible.
When comparing performance with EXT4, BTRFS can sometimes perform "on par" but generally can be 3x slower in some real-world situations.  The ability to have inherent compression in the filesystem, coupled with the "lighter' lzo compression algorithm, a BTRFS drive will perform better than EXT4.  Your individual situation needs to be evaluated to determine if the benefits of BTRFS will outweigh the performance risks.  Understanding under what situations BTRFS will perform well is critical.  Generally in random read-write scenarios (most predominately on filesystems housing databases), BTRFS will under perform to EXT4 by up to 3x.  Compression in this case provides little or no benefit because the data tends to be in a state not suitable.  In cases where data is primarily read, BTRFS with lzo compression will outperform EXT4.  In situations such as a primary filesystem for a Linux distro on a laptop, you are generally reading and loading programs, so the benefits of BTRFS will outweigh the costs.

8) Understand how the benefits will actually help you.
For example, of the benefits of BTRFS that I haven't touched on is built in filesystem checksum tracking.  This means the filesystem has a means of checking for corruption from such bit rot, etc.  On an EXT4 filesystem, bits that have changed, files that have become corrupt from reasons other than from journal errors or apparently hardware failures, is detectable as the filesystem tracks the checksum of files.  Although bit rot is not new, it is increasingly becoming more common as society moves to cloud-storage (where bits are stored elsewhere and can be lost or distorted from transmision) and to SSD or other flash devices where data corruption is not caused by traditional hardware failures such as heads smashing into platters, or loss of magnetic polarity / or magnetic interference, but by memory chips failing,  electrical interference, wear-and-tear on the chips themselves or by software error/bugs in the controllers managing the data on the SSD devices.  But you have to understand what the benefit really is.  Bit rot on a EXT4 filesystem will most likely not occurring unless you access the data file and realize it is corrupt (if you even realize it).  It could be as simple as a corrupt number being incorrect on a spreadsheet or an "artifact" in a media file.  You may not realize the data is corrupt (and if you backup the corrupt data to another device, your backup will become corrupt).  You would have to exert ongoing effort to track checksum values on files and determine which changes in checksum values are due to valid file changes and those introduced by bit rot.  However, like on EXT4, identifying bit rot is only part of the solution.  On an EXT4 filesystem, you would replace the corruption with a "known good" version of the file that is stored elsewhere.  Likewise, the checksum benefit in BTRFS is the same.  It does not provide a means to fix corruption, only the means of detecting it.  Throwing away your backups because your filesystem can detect bit rot corruption does not mean it will provide you a means to automatically repair the bit rot.  There is a difference between perceived benefit and actual benefit.

My year-long experiment with BTRFS has been successfully and for reasons already explored in previous posts or ones to be explored in future posts, I'll continue to expand my use of BTRFS in the years to come.

Linux Mint Debian Edition (LMDE) UP8 on Thinkpad X131e

I recently acquired a Thinkpad X131e.  It's the 3372-3FU, which is the AMD -- meaning AMD CPU, ATI Radeon HD 7340, and Broadcom wifi.   I'm used to Intel-based systems and Nvidia Optimus systems (with Intel as fallback).

Having previously used a system with broadcom wifi chipset, I knew I would need to load some non-free, closed-source wifi drivers on this new system.

Getting Started

I replaced a Thinkpad X61t (tablet) system for this system.  The Thinkpad X61t had my standard LMDE install with Update Pack 8.  It had a Intel CPU (core 2 duo), Intel Centrino wireless and  OCZ Vertex 3 60-SSD.

I thought I would at least be able to transplant the SSD and boot into my install (perhaps with no wifi and fallback to default drivers for video).

My first challenge was with the SSD.

Low Profile Drive

I had completely neglected that the industry created a new and shorter kind of 2.5" hard drive standard.  The drives are 7mm tall versus the standard 9.5mm.  My 2 year-old OCZ Vertex 3 drive would not fit in this Thinkpad X131e, that apparently uses the "low profile" 7mm standard drives.

I realized by looking at the SSD case, that most of the cases was likely just "filler".  What stopped me from finding out was a "warranty void" sticker.  Having still 1 year warranty on the drive, I decided to go for it, and void my warranty -- if the drive dies, I'll be eager to replace it by a larger drive anyways.  Opening up the drive, I realized my intuition was correct -- the SSD consisted of a slim board (no bigger than 5mm in depth).  I kept the bottom of the case for protection from the board ever meeting contact with the metal bottom of the laptop, and placed a plastic insulator sheet adapted from an old 2.5" HD caddy on the top side of the drive (to avoid the top portion making contact with the laptop case door).  Warranty issue aside, the SSD issue was resolved.

Wifi causes Linux to hang at startup

I quickly realized that upon bootup that I couldn't make it past the init.d startup scripts without the laptop completely hanging.  I did a trial-and-error, disabling the startup services (by starting up in recovery mode, dropping me to a root prompt before the services start up).  In the /etc/init.d directory, you can simply remove the execute permission on startup services that you suspect are causing the issue, and then trial-and-error until you find the offending service.  I realized quickly that it was the networking service causing issues, and shortly after that, the wifi in particular.  To allow me to at least boot up the system with ethernet, I needed to completely disable wifi.  It was not sufficient to log onto the BIOS and disable the Wireless LAN antenna.  I found I needed to go to IO Port Access and disable Wireless LAN.  Upon doing this, I was able to boot up with a network connection (over ethernet).

Installing broadcom wifi drivers

Fairly simple to do.  With a proper working internet connect over ethernet, I was able to run:

sudo apt-get install broadcom-sta-dkms

This downloaded and installed the kernel modules for the broadcom wifi.

Broadcom STA is a binary-only device driver to support the following IEEE

802.11a/b/g/n wireless network cards: BCM4311-, BCM4312-, BCM4313-,

BCM4321-, BCM4322-, BCM43224-, BCM43225-, BCM43227-, and BCM43228-based

hardware.

After re-enabling Wireless LAN in IO Port Access, I was able to boot up with a wifi connection.

Xserver fails

With xorg not understanding AMD/ATI graphics with a regular install (get the error No Screens Found on startup of xorg/xserver), I realized I needed to install some working graphic drivers.  

Avoiding the debate over installing the latest open-source drivers or the closed-source proprietary drivers, I decided to opt for the proprietary which happen to be fairly up-to-date.

I made my first mistake by trying to install the ATI Catalyst 14.10 and 14.20 beta drivers from the Linux section on the ATI website.  The installer installs the drivers successfully, but the installer isn't actually doing anything.  Trying to run the buildpkg manual method with the software fails, and you end up going in an endless loop of troubleshooting.  I wasted a day after realizing I simply won't be able to use the installer method as its designed to only install to a 32-bit system, whereas I'm using a 64-bit install.

I decided to use the Debian repository instead.  I already had the non-free directive in my apt source.list file.  I simply followed the instructions found on the Debian Wiki for ATI (https://wiki.debian.org/ATIProprietary).  

Making sure I already had the linux-headers for my kernel installed (uname -a showed 3.11-5.dmz.1-liquorix-amd64, so I ensured I had the linux-headers installed by running sudo apt-get install linux-headers-3.11-5.dmz.1-liquorix-amd64).  Next I installed the ATI drivers by running:

sudo apt-get install fglrx-driver

After the kernel driver installs, you need to have the installer configure xorg to understand how to work with ATI.  You would run:

sudo aticonfig --initial

Apparently, if the kernel module fails to build/install/load, the aticonfig changes to xorg are sufficient enough to get X windows to load with a fallback graphic driver.  It will be a display with a slow refresh rate with other performance factors, but it will load.  I realized that the version included with Update Pack 8 doesn't compile the module properly for the Linux Kernel 3.13 that I had installed.  Apparently ATI announced it does not yet support Linux Kernel > 3.11.  I did find references to patches people have created to get the kernel modules to build on 3.14, etc, but to deal with that on another day, I decided to just use Linux Kernel 3.11.  I was able to successfully build and load the ATI Catalyst 13 Beta drivers in the Linux Mint Debian Edition Update Pack 8 repository.  Backlight adjusting via function key was working properly with the proper module loaded -- on the fallback, the backlight function keys don't work, a sign you are falling back. 

Now, if you boot up the system, you should have wifi and proper working X windows.

With the proper driver loaded, 3D acceleration is supported (and the ATI control panel will work). The command fglrxinfo will return a GL version number.  When fallback driver is active, you won't receive a GL number with fglrxinfo.  Further, with the proper drive loaded, if you perform a lsmod, there will be a fglrx driver loaded.

Additional Notes

After some reading, I decided to add the directive "nomodeset" to my grub.cfg.

Compiz wouldn't load correctly (window manager crashed, windows displayed without a window manager, etc).  I expected compiz to work with a near-latest ATI proprietary driver installed and loaded (with 3D kernel module compiled and loaded).  I did read that the latest open-source ATI driver SHOULD have compiz support also working when the Mesa GL lib is also loaded.  Naturally you cannot have both open and closed source drivers installed on the system at the same time.  Once I have things stable, I will troubleshoot further.

Hibernation/Suspend

I use pm-suspend and pm-hibernate setup with laptop tools on all my systems.  Both fail to suspend or hibernate the system.  I can suspend and resume successfully by running as root:

echo "mem" >  /sys/power/state

The same trick for hibernation, 

echo "disk" >  /sys/power/state

results in a system that cannot thaw from hibernation properly.  Running the hibernate command does work though.

Therefore, I use the following two commands

to suspend:

echo "mem" >  /sys/power/state

to hibernate:

hibernate

Adding exFAT support for Linux

I recently stumbled across a need to store > 4GB files on my 32GB microSD card.  It naturally came formatted in FAT32 and I needed to keep compatibility with a device that requires FAT32 or exFAT format types.

exFAT does support file sizes > 4GB, so I needed to reformat the card, but I noticed that Linux Mint Debian Edition (LMDE) or Debian, doesn't support exFAT natively out-of-the-box.

To gain support, install exfat-fuse exfat-utils

sudo apt-get install exfat-fuse exfat-utils


Then, to format your microSD card (or other device):

sudo mkfs.exfat /dev/mmcblk0p1

Rejuvenating the DLINK DNS323 NAS with alt f

I've purchased three of the DLINK DNS323 over the years.  I retired the units a number of years ago because of the issues discussed below.

Problematic support for 4k sector sizes

The user needs to format drives that use 4k sector sizes in a firmwae image that supports it (which is the last, 1.10), but the community of users have pointed out various bugs and issues with 1.10, suggesting users not use this firmware on an ongoing basis.  Therefore, for the purposes of formating (and properly aligning the sectors on these drives), it is recommended the user ugprade their firmware to 1.10 and immediately downgrade to 1.08 after formatting.  You just need to setup the drives on a supported firmware -- you don't have to actually persist using that firmware.

No support for EXT4

No sense staying with EXT3 when journaling is significantly better in EXT4.

No support for hard drives > 2.2TB (such as 3TB drives)

With the official firmware updates retired, no new hard drives will ever be supported officially.

No GPT partition tables (only MBR)

Along the reasons of not supporting newer, larger hard drives.

Formatting the wrong drive bug

Although DLINK recognized the problem in earlier firmware images and says they corrected it, I personally encountered the issue when a pre-existing drive was formatted by the firmware instead of the selected new drive, despite both drives having different sizes, different manufacturers and different models -- therefore, not a mistaking the identify of the correct drive to format.


Last year I came across alt f, a free alternative firmware for these NAS units.  I flashed it on one of the NAS units and never looked back.  It really breaths life into these old units.  I've been making use of my obsolete and retired NAS units.

The primary features that stand out for me are:

• Support for EXT4
• Support for GPT
• Support for larger/newer hard drives (3TB, 4TB, etc)
• Use pre-existing hard drives without reformatting or modifying the partition tables
• Support for NTFS drives in the left and right bay (in addition to USB)
• RAID that includes support for drives over the USB port
• ffp package support builtin

I've put together a video walkthrough below of alt f firmware.

USB Logitech microphone support in Kazam on Linux

I had an issue where my USB Logitech microphone would not show up in the Kazam Screencast application.  Linux sound record could see the microphone, and I could record audio through sound recorder from the microphone, but Kazam could not.

I ended up creating a loopback Pulse Audio device using:

pactl load-module module-null-sink sink_name=Virtual1    

pactl load-module module-loopback latency_msec=1 sink=Virtual1 

In Kazam, I would select Sound from speakers (not sound from microphone):

In preferences, for Speakers, select the microphone.

Now the sound recording will be relayed through the microphone to Kazam.  There won't be any speaker output, so there will be no feedback.

Which filesystems I use and why

I've been frequently asked which filesystems I prefer.  I decided to put together a post discussing my filesystems of choice and general analysis of the criteria.

Windows VMs/KVMs etc

Obvious choice is NTFS.  In KVM, the NTFS filesystem is stored in a QCOW2 container (allowing the filesystem to be easily resized/extended and snapshots/clones reduce data storage consumption).

Raspberry Pi / Beagle Bone Black / Cubieboard

read-only EXT4 with journaling off

I generally turn journaling off (by running tune2fs -O ^has_journal /dev/sda) to preserve the flash and SD write health of my media.  I have literally killed genuine SanDisk premium SD cards with "active" Raspberry Pi systems that had journaling on.

I prefer to make my partitions read-only, with using tempfs to store writes.  If I want write to be persistent, I store the data on a secondary EXT4 writable partition.  However, the root and boot partitions, I keep read-only unless I'm performing an upgrade on the system.  By keeping the filesystem as read-only, I avoid corrupting the SD card with bad writes caused by bad power sources, overclocking or improper shutdown.  I found even with reliable power sources, no overclocking and always ensuring safe shutdown, my SD card would become corrupt over time (just generally less often -- say over 6 months instead of 6 weeks, but corrupt neither-less).  I have seen several exhaustive tests done by other Raspberry Pi users that conclude the same -- that only read-only filesystems on SD cards will prevent corruption.

Debian-based Linux Systems on SSD

EXT4 

I use the mounting parameters rw,errors=remount-ro,user_xattr,noatime,discard that reduce unnecessary update of inodes when accessing files.  The discard enables TRIM support and should be enabled for as long as your drive supports it (hdparm -I /dev/sda | grep TRIM).  If you enable discard and your drive doesn't support it, you'll have data loss.

EXT4 has better support than previous versions (EXT2, EXT3), even though there is increased journaling in EXT4 (that can reduce the lifespan of SSD drives).  If you have frequent writes and have means to mitigate the risk of data loss in system crashes/etc, then consider turning journaling off (tune2fs -O ^has_journal /dev/sda).

I reduce the amount of writes to SSD by using various tempfs (stored in RAM) for logging, tmp and caching.  I use the find -mtime command to identify files that are frequently written to (in most cases, for unnecessary reasons).  I then schedule startup, suspend/hibernate and shutdown routines that sync these tempfs to disk.  I store everything from system logs and google chrome / Mozilla configuration and cache directories to RAM.

Debian-based Linux Systems on slow hard drives

XFS

I have one Sony VAIO system that uses a 1.8" 4200rpm SATA hard drive.  A really slow hard drive benefits from the speed advantages of XFS.  One should note that data loss can become and issue for system crashes/etc.  You can consider mounting the XFS filesystem as read-only, and use tempfs to store writes to be synced to disk in a safe fasion (similar to the Raspberry Pi setup). 

Linux Containers / LXC and Linux Based VMs/KVMs

BTRFS 

For any system that you use kernel 3.9 or greater, you should consider using BTRFS to store your operating system.  BTRFS allows you to "snapshot" and clone your LXC/VM with minimum data duplication.  Similar to QCOW2 (in KVM) and snapshots in VMware, BTRFS will help you reduce your LXC/VM disk consumption.  

NAS units (such as DNS 323)

EXT4 with SAMBA and NFS

I use alt-f firmware on my DNS 323 units, so my NAS units do support EXT4.  A lot of older NAS units or DNS 323 units running on official firmware, may not support EXT4.  In those cases I use EXT3, and as I upgrade firmware on the NAS or replace it with a more capable NAS, I convert to EXT4.  The journaling of EXT4 provides the best means of preventing data loss besides moving to a filesystem such as BTRFS or ZFS.  My NAS units, even with modern open-source and community driven firmware like alf-f, does not support BTRFS or ZFS due to hardware limitation (memory and CPU required to support "tomorrow's filesystems").  I mitigate risk of data loss due to corruption by maintaining offline mirror drives for each NAS drive.

Samba shares enabled for sharing with Windows and XBMC on Raspberry Pi and NFS for mounting the drives on Linux systems.

Comments about BTRFS and ZFS

I really believe that BTRFS and ZFS will be the filesystems for the future.  BTRFS is highly being developed on and is meant to share a lot of the benefits and features of ZFS but without the licensing restrictions.  If you have situations/applications where you can't run newer versions of the Linux kernel, then you should avoid BTRFS -- it's always best to only use BTRFS on systems with at least kernel 3.9.  

ZFS is an Oracle created filesystem that has great data integrity (to guard against bit rot and data corruption), built in encryption and compression and data deduplication (great for snapshoting and cloning).  ZFS has been ported to Linux using either native kernel modules or by FUSE.  There are licensing issues as ZFS isn't licenses under GPL, and therefore, the implementation has stripped out features and is not as stable as the original Sun Solaris implementation.  Being that the original Oracle implementation is 64-bit, the linux native kernel modules are only compiled for amd64, so if you need to support 32-bit systems, you should not consider ZFS.  Further, ZFS needs a fair amount of RAM (4GB) to manage.  Also, if you often run/use the latest Linux kernel versions, you may have issues as you have to wait for the native kernel modules to be updated for your kernel release -- due to licensing restrictions, they don't come bundled with the kernel, so you need to wait for the support community to update the modules afterwards.  I've been using kernel 3.13 for 6+ months, and I still don't have ZFS kernel modules that work on my kernel version.

ZFS also has embedded support for Samba and NFS -- that is, the filesystem can be automatically shared, without any additional configuration, simply by enabling sharesmb or sharenfs.

BTRFS is a good compromise to ZFS -- to gain some of the features without dealing with some of the problems.  Both would be considered "non-production" at this point.  Don't store critical data on them without having good mechanisms in place to safe guard against being inflicted by bugs that often pop-up in bleeding-edge development work.

Map Suspend & Hibernate to shortcut keys

One of the issues I recently encountered when upgrading to Linux Mint Debian Edition (LMDE) Update Pack 8 (UP8) is the loss of functionality of the suspend and hibernate keys.  At first I was under the impression that it was just an issue on my Thinkpad, but I discovered the issue existed on all the system that I upgraded.  It's a known issue and was identified in this bug: https://bugs.launchpad.net/linuxmint/+bug/1276678

There might be other workarounds to resolving the issue, but I took this as a good opportunity to finally mapping out a new suspend/hibernate key combination that I could use across all my systems.  The FN-F4 for suspend and FN-F12 may be universal on all Thinkpads, but my Vaio and Ultrabook use different suspend and hibernate keys, thus my finger-memory sometimes slips up.

To tackle both problems, I decided create a common shortcut key for each function across all my systems.  Because the FN keys tend to be tied to hardware routines, I settled on SHIFT+F4 for suspend and SHIFT+F12 for hibernate.

The first step to tackle was writing a suspend and hibernate script that could be invoked by my user ID without having to provide a sudo password.

I added the following two lines to my /etc/sudoers file.

durdle t420-ssd = (root) NOPASSWD: /usr/sbin/pm-hibernate
durdle t420-ssd = (root) NOPASSWD: /usr/sbin/pm-suspend

Substitute durdle with your user ID, and t420-ssd with our hostname.

You may find that you need to change the file  /etc/sudoers  to have write permissions before you can edit it (doing a  chmod 740 /etc/sudoers before updating the file and a chmod 440 /etc/sudoers after updating the file, will do the trick).

Next, I decided to create shortcut scripts in my ~/bin/ folder.  This will allow me to quickly suspend and hibernate from the command line and allow me to easily tie the two routines to the Autokey program.

My two scripts resemble as follows:

~/bin/hibernate.sh 

sudo /usr/sbin/pm-hibernate

~/bin/suspend.sh 

sudo /usr/sbin/pm-suspend

If you can't use the pm-hibernate or pm-suspend programs to hibernate or suspend, you can substitute your equivalent calls.  Or if you need to add wrappers around them, these shell scripts are the place to put them.  Previous to kernel 3.13, my Vaio need some additional system calls before calling pm-hibernate, in order to ensure a proper hibernate.  I place those calls in my hibernate.sh.

I use a program called Autokey for various shortcuts and keyboard macros.  If you don't normally use Autokey, you will need to ensure that it starts up on boot up.  You can do this by calling it in your "Startup Applications" list.

The program is fairly self explanatory.  You will create two new scripts, enter the following, substituting your username in, and then assigning the Hotkey <shift>+<f4> and Hotke <shift>+<f12>:

for suspend:

import os

os.system('/home/durdle/bin/suspend.sh')

for hibernate:

import os

os.system('/home/durdle/bin/hibernate.sh')

That's it!  Now when you press SHIFT+F4, your system should suspend and when you press SHIFT+F12, your system should hibernate, all without entering a password.