Adding exFAT support for Linux

I recently stumbled across a need to store > 4GB files on my 32GB microSD card.  It naturally came formatted in FAT32 and I needed to keep compatibility with a device that requires FAT32 or exFAT format types.

exFAT does support file sizes > 4GB, so I needed to reformat the card, but I noticed that Linux Mint Debian Edition (LMDE) or Debian, doesn't support exFAT natively out-of-the-box.

To gain support, install exfat-fuse exfat-utils

sudo apt-get install exfat-fuse exfat-utils


Then, to format your microSD card (or other device):

sudo mkfs.exfat /dev/mmcblk0p1

Rejuvenating the DLINK DNS323 NAS with alt f

I've purchased three of the DLINK DNS323 over the years.  I retired the units a number of years ago because of the issues discussed below.

Problematic support for 4k sector sizes

The user needs to format drives that use 4k sector sizes in a firmwae image that supports it (which is the last, 1.10), but the community of users have pointed out various bugs and issues with 1.10, suggesting users not use this firmware on an ongoing basis.  Therefore, for the purposes of formating (and properly aligning the sectors on these drives), it is recommended the user ugprade their firmware to 1.10 and immediately downgrade to 1.08 after formatting.  You just need to setup the drives on a supported firmware -- you don't have to actually persist using that firmware.

No support for EXT4

No sense staying with EXT3 when journaling is significantly better in EXT4.

No support for hard drives > 2.2TB (such as 3TB drives)

With the official firmware updates retired, no new hard drives will ever be supported officially.

No GPT partition tables (only MBR)

Along the reasons of not supporting newer, larger hard drives.

Formatting the wrong drive bug

Although DLINK recognized the problem in earlier firmware images and says they corrected it, I personally encountered the issue when a pre-existing drive was formatted by the firmware instead of the selected new drive, despite both drives having different sizes, different manufacturers and different models -- therefore, not a mistaking the identify of the correct drive to format.


Last year I came across alt f, a free alternative firmware for these NAS units.  I flashed it on one of the NAS units and never looked back.  It really breaths life into these old units.  I've been making use of my obsolete and retired NAS units.

The primary features that stand out for me are:

• Support for EXT4
• Support for GPT
• Support for larger/newer hard drives (3TB, 4TB, etc)
• Use pre-existing hard drives without reformatting or modifying the partition tables
• Support for NTFS drives in the left and right bay (in addition to USB)
• RAID that includes support for drives over the USB port
• ffp package support builtin

I've put together a video walkthrough below of alt f firmware.

USB Logitech microphone support in Kazam on Linux

I had an issue where my USB Logitech microphone would not show up in the Kazam Screencast application.  Linux sound record could see the microphone, and I could record audio through sound recorder from the microphone, but Kazam could not.

I ended up creating a loopback Pulse Audio device using:

pactl load-module module-null-sink sink_name=Virtual1    

pactl load-module module-loopback latency_msec=1 sink=Virtual1 

In Kazam, I would select Sound from speakers (not sound from microphone):

In preferences, for Speakers, select the microphone.

Now the sound recording will be relayed through the microphone to Kazam.  There won't be any speaker output, so there will be no feedback.

Which filesystems I use and why

I've been frequently asked which filesystems I prefer.  I decided to put together a post discussing my filesystems of choice and general analysis of the criteria.

Windows VMs/KVMs etc

Obvious choice is NTFS.  In KVM, the NTFS filesystem is stored in a QCOW2 container (allowing the filesystem to be easily resized/extended and snapshots/clones reduce data storage consumption).

Raspberry Pi / Beagle Bone Black / Cubieboard

read-only EXT4 with journaling off

I generally turn journaling off (by running tune2fs -O ^has_journal /dev/sda) to preserve the flash and SD write health of my media.  I have literally killed genuine SanDisk premium SD cards with "active" Raspberry Pi systems that had journaling on.

I prefer to make my partitions read-only, with using tempfs to store writes.  If I want write to be persistent, I store the data on a secondary EXT4 writable partition.  However, the root and boot partitions, I keep read-only unless I'm performing an upgrade on the system.  By keeping the filesystem as read-only, I avoid corrupting the SD card with bad writes caused by bad power sources, overclocking or improper shutdown.  I found even with reliable power sources, no overclocking and always ensuring safe shutdown, my SD card would become corrupt over time (just generally less often -- say over 6 months instead of 6 weeks, but corrupt neither-less).  I have seen several exhaustive tests done by other Raspberry Pi users that conclude the same -- that only read-only filesystems on SD cards will prevent corruption.

Debian-based Linux Systems on SSD

EXT4 

I use the mounting parameters rw,errors=remount-ro,user_xattr,noatime,discard that reduce unnecessary update of inodes when accessing files.  The discard enables TRIM support and should be enabled for as long as your drive supports it (hdparm -I /dev/sda | grep TRIM).  If you enable discard and your drive doesn't support it, you'll have data loss.

EXT4 has better support than previous versions (EXT2, EXT3), even though there is increased journaling in EXT4 (that can reduce the lifespan of SSD drives).  If you have frequent writes and have means to mitigate the risk of data loss in system crashes/etc, then consider turning journaling off (tune2fs -O ^has_journal /dev/sda).

I reduce the amount of writes to SSD by using various tempfs (stored in RAM) for logging, tmp and caching.  I use the find -mtime command to identify files that are frequently written to (in most cases, for unnecessary reasons).  I then schedule startup, suspend/hibernate and shutdown routines that sync these tempfs to disk.  I store everything from system logs and google chrome / Mozilla configuration and cache directories to RAM.

Debian-based Linux Systems on slow hard drives

XFS

I have one Sony VAIO system that uses a 1.8" 4200rpm SATA hard drive.  A really slow hard drive benefits from the speed advantages of XFS.  One should note that data loss can become and issue for system crashes/etc.  You can consider mounting the XFS filesystem as read-only, and use tempfs to store writes to be synced to disk in a safe fasion (similar to the Raspberry Pi setup). 

Linux Containers / LXC and Linux Based VMs/KVMs

BTRFS 

For any system that you use kernel 3.9 or greater, you should consider using BTRFS to store your operating system.  BTRFS allows you to "snapshot" and clone your LXC/VM with minimum data duplication.  Similar to QCOW2 (in KVM) and snapshots in VMware, BTRFS will help you reduce your LXC/VM disk consumption.  

NAS units (such as DNS 323)

EXT4 with SAMBA and NFS

I use alt-f firmware on my DNS 323 units, so my NAS units do support EXT4.  A lot of older NAS units or DNS 323 units running on official firmware, may not support EXT4.  In those cases I use EXT3, and as I upgrade firmware on the NAS or replace it with a more capable NAS, I convert to EXT4.  The journaling of EXT4 provides the best means of preventing data loss besides moving to a filesystem such as BTRFS or ZFS.  My NAS units, even with modern open-source and community driven firmware like alf-f, does not support BTRFS or ZFS due to hardware limitation (memory and CPU required to support "tomorrow's filesystems").  I mitigate risk of data loss due to corruption by maintaining offline mirror drives for each NAS drive.

Samba shares enabled for sharing with Windows and XBMC on Raspberry Pi and NFS for mounting the drives on Linux systems.

Comments about BTRFS and ZFS

I really believe that BTRFS and ZFS will be the filesystems for the future.  BTRFS is highly being developed on and is meant to share a lot of the benefits and features of ZFS but without the licensing restrictions.  If you have situations/applications where you can't run newer versions of the Linux kernel, then you should avoid BTRFS -- it's always best to only use BTRFS on systems with at least kernel 3.9.  

ZFS is an Oracle created filesystem that has great data integrity (to guard against bit rot and data corruption), built in encryption and compression and data deduplication (great for snapshoting and cloning).  ZFS has been ported to Linux using either native kernel modules or by FUSE.  There are licensing issues as ZFS isn't licenses under GPL, and therefore, the implementation has stripped out features and is not as stable as the original Sun Solaris implementation.  Being that the original Oracle implementation is 64-bit, the linux native kernel modules are only compiled for amd64, so if you need to support 32-bit systems, you should not consider ZFS.  Further, ZFS needs a fair amount of RAM (4GB) to manage.  Also, if you often run/use the latest Linux kernel versions, you may have issues as you have to wait for the native kernel modules to be updated for your kernel release -- due to licensing restrictions, they don't come bundled with the kernel, so you need to wait for the support community to update the modules afterwards.  I've been using kernel 3.13 for 6+ months, and I still don't have ZFS kernel modules that work on my kernel version.

ZFS also has embedded support for Samba and NFS -- that is, the filesystem can be automatically shared, without any additional configuration, simply by enabling sharesmb or sharenfs.

BTRFS is a good compromise to ZFS -- to gain some of the features without dealing with some of the problems.  Both would be considered "non-production" at this point.  Don't store critical data on them without having good mechanisms in place to safe guard against being inflicted by bugs that often pop-up in bleeding-edge development work.

Map Suspend & Hibernate to shortcut keys

One of the issues I recently encountered when upgrading to Linux Mint Debian Edition (LMDE) Update Pack 8 (UP8) is the loss of functionality of the suspend and hibernate keys.  At first I was under the impression that it was just an issue on my Thinkpad, but I discovered the issue existed on all the system that I upgraded.  It's a known issue and was identified in this bug: https://bugs.launchpad.net/linuxmint/+bug/1276678

There might be other workarounds to resolving the issue, but I took this as a good opportunity to finally mapping out a new suspend/hibernate key combination that I could use across all my systems.  The FN-F4 for suspend and FN-F12 may be universal on all Thinkpads, but my Vaio and Ultrabook use different suspend and hibernate keys, thus my finger-memory sometimes slips up.

To tackle both problems, I decided create a common shortcut key for each function across all my systems.  Because the FN keys tend to be tied to hardware routines, I settled on SHIFT+F4 for suspend and SHIFT+F12 for hibernate.

The first step to tackle was writing a suspend and hibernate script that could be invoked by my user ID without having to provide a sudo password.

I added the following two lines to my /etc/sudoers file.

durdle t420-ssd = (root) NOPASSWD: /usr/sbin/pm-hibernate
durdle t420-ssd = (root) NOPASSWD: /usr/sbin/pm-suspend

Substitute durdle with your user ID, and t420-ssd with our hostname.

You may find that you need to change the file  /etc/sudoers  to have write permissions before you can edit it (doing a  chmod 740 /etc/sudoers before updating the file and a chmod 440 /etc/sudoers after updating the file, will do the trick).

Next, I decided to create shortcut scripts in my ~/bin/ folder.  This will allow me to quickly suspend and hibernate from the command line and allow me to easily tie the two routines to the Autokey program.

My two scripts resemble as follows:

~/bin/hibernate.sh 

sudo /usr/sbin/pm-hibernate

~/bin/suspend.sh 

sudo /usr/sbin/pm-suspend

If you can't use the pm-hibernate or pm-suspend programs to hibernate or suspend, you can substitute your equivalent calls.  Or if you need to add wrappers around them, these shell scripts are the place to put them.  Previous to kernel 3.13, my Vaio need some additional system calls before calling pm-hibernate, in order to ensure a proper hibernate.  I place those calls in my hibernate.sh.

I use a program called Autokey for various shortcuts and keyboard macros.  If you don't normally use Autokey, you will need to ensure that it starts up on boot up.  You can do this by calling it in your "Startup Applications" list.

The program is fairly self explanatory.  You will create two new scripts, enter the following, substituting your username in, and then assigning the Hotkey <shift>+<f4> and Hotke <shift>+<f12>:

for suspend:

import os

os.system('/home/durdle/bin/suspend.sh')

for hibernate:

import os

os.system('/home/durdle/bin/hibernate.sh')

That's it!  Now when you press SHIFT+F4, your system should suspend and when you press SHIFT+F12, your system should hibernate, all without entering a password.