Though modern solid state drives can now store a large amount of data they differ from conventional disks in a number of ways.
- There are no moving parts so solid state disks are typically more resistant to physical shock.
- When writing to a solid state disk any existing data must be erased first, unless the sector is already empty, so each write to a solid state disk is actually two write operations.
- Flash memory can only sustain a limited number of write or erase cycles before it fails. It is worth remembering that all solid state drives will fail at some point – a sobering thought if you don’t take backups very often.
While all devices that use flash memory have the same limitations solid state hard disks have more powerful controllers which allow the designers to include a number of different features to mitigate these problems. Manufactures can make the drives last longer by including additional memory that can be used to replace any failing blocks and some drives actually allow you to increase the number of spare blocks, a technique known as over provisioning, in order to extend the operational life of the disk still further.
Unfortunately SD cards have to be much smaller and are not usually designed to be used as mass storage devices, so they tend to be a lot slower and more prone to failure when used in the same way.
In both cases minimizing the number of writes will prolong the operational life of the device, and will also help speed things up if you are using a relatively slow device like an SD card.
- Don’t use a swap partition.
- Disable file access time-stamping.
- Disable journalling or use a file system that does not support journalling.
- Use ‘tmpfs‘ for temporary files.
Each of these techniques can be used on an ordinary computer or as in the examples below a Raspberry Pi but you need to keep an eye on the amount of memory you have available, particularly if you disable swapping.
In order to make any of these changes you need to be running as a super user.
Virtual memory allows the system to use more memory than is physically available by allowing the operating system to free up memory by using a swap file (or partition) to save the least used pages of memory to disk in order to allocate them to another process.
Pages of memory that are saved on disk in this way are referred to as being ‘swapped out’ it isn’t quite as inefficient as it sounds since Linux uses the available free memory to cache disk writes which also allows disk operations to be optimized, particularly when using a conventional disk which is more efficient when writing data sequentially. So when a page is swapped out all that actually happens is that it is added to the list of pages in free memory that are waiting to be flushed to the disk, if the process needs to access the page again before it is actually written to disk then it is simply allocated back to the process and never gets written to disk at all.
However, once saved to disk if a program needs to access that page swapping it back in from disk it will take much longer. If the system starts to run out of free memory then the length of time that a page is unused before the kernel will decide to swap it out gets shorter and it becomes increasingly likely that it will have to be swapped in again. This can lead to a system spending most of the time just reading and writing to the swap file – in this situation the system will become very unresponsive and is said to be ‘thrashing‘.
To see if there is an active swap file or partition and how big it is use the ‘swapon’ command to display a summary of the current swap devices.
If there are no files or file systems shown then you do not have a swap file or swap partition configured, and can skip to the next step.
If the system is using a dedicated swap partition then the type of swap file will be ‘partition’ and the filename will be the path to the device that corresponds to the swap partition.
If the system is using a swap file the type will be ‘file’.
To disable swapping begin by turning it off.
Then if the system is using a swap partition then all we need to do comment out the line in ‘/etc/fstab’ that is used to mount the swap partition when the system starts.
Note – The swap partition may be identified using a UUID rather than a device name but the mount options at the end of the line will be the same. Do not modify any other lines.
You will need to reboot for this change to take effect.
If the system is using a swap file then the easiest way to disable it is to uninstall ‘dphys-swapfile’ which is the package that installs the scripts that set up the swap file each time the system starts. This will also free up the disk space used by the swap file itself (which is normally about twice the size of the physical memory).
Check that there is swapping is disabled.
Disable file access time-stamping
By default Linux records information about when files were last accessed as well as when they were created or modified. This means that each time a file is read the system must update the directory entry with the current access time. If we don’t need to know when files were last accessed then disabling this will make quite a difference to the number of writes.
Note – Even if file access time stamping is disabled the system will still record when files were last modified or created.
To disable file access time stamping we need to add a mount option to the entries in ‘/etc/fstab’ for each of the partitions on the SD card that are mounted when the system starts. This will eliminate the need by the system to make writes to the file system when reading files.
Note – In this example I have already disabled the swap partition.
You should only modify the entries that correspond to a Linux file system on the SD card, the first two entries in the example above do not need to be changed as they correspond to the MS-DOS boot partition and the swap partition.
You will need to reboot for this change to take effect.
A journalling file system uses a journal to keep track of changes to the files system that have not yet been committed to disk. The advantage is that the journal significantly increases the ability of the file system to recover from an unexpected system failure, but at the expense of an increased number of I/O operations. Disabling journalling will reduce the number of writes but will make it more likely that the file system will become corrupted in the event that it is not unmounted properly as a result of a power failure, system crash etc.
To avoid the additional overheads associated with a journalling file system you can either choose a file-system that does not support journalling at all like ‘ext2’ when you install the operating system, or if you are already using and ‘ext3’ or ‘ext4’ file-system you can disable it.
If you are using an ‘ext3’ file-system then you can disable journalling simply by modifying ‘/etc/fstab’ so the system mounts it as an ‘ext2’ file-system.
Note – In the example above I have already disabled the swap partition and file access timestamps.
For this change to take effect you will need to reboot.
You cannot disable journalling on an ‘ext4’ file-system that is in use so you will need to be able to mount your SD card on a second Linux machine. Providing it supports ‘ext4’ it doesn’t need to have the same architecture so you can use a regular PC with a ‘live CD’ if you don’t normally use Linux on anything other than your Raspberry Pi.
It can be a bit difficult to work out which device is which so if you are using a using a USB card reader you can use this script to find out what removable storage devices are available, this should help you identify the right device name to use.
Note – Your system may mount the SD card using a different device, in this case you will need to modify the following commands accordingly,
Use the following command to list all the ‘ext4’ partitions on this device.
The next four commands need to be repeated for each ‘ext4’ partition. The first two will unmount the partition and check it to see if journalling is enabled; if it is then the output from the second command will include the phrase ‘has_journal’ (highlighted in colour); if it isn’t we don’t need to do anything else and can move on to the next partition.
If journalling is enabled then you can turn it off by using the following command, which also lists the features so you can check that journalling has been disabled.
Finally it is a good idea to check the file-system at this point.
When you are finished you can then eject the card, which will ensure any uncommitted writes are completed and dismount any other partitions.
You can also reduce the number of disk writes by using memory to store temporary files, lock files, and possibly log files rather than writing them to disk by mounting the following folders using ‘tmpfs‘ and Although Debian currently doesn’t some distributions do already mount ‘/tmp’ using ‘tmpfs’ by default.
- /tmp – Temporary files
- /var/lock – Lock files used by active processes
- /var/log – System logfiles
Note – Mounting ‘/var/log’ using ‘tmpfs’ will cause issues with some services.
You need to make sure that your system has enough free memory, particularly if it can’t free up memory by swapping pages to disk but providing your system has enough memory then saving any lock or temporary files in memory shouldn’t cause any problems since these files don’t have to be saved permanently and generally don’t take up too much space. Storing any log files in memory will also reduce the number of disk writes, but if your system encounters an error that causes it to reboot the log files will be lost, and it get complicated if any services use separate sub-folder as will also be lost when the system reboots (unless you explicitly recreate them as part of the boot process).
To use ‘tmpfs’ to store these folders in memory you need to edit ‘/etc/fstab’.
The ‘tmpfs’ partition will grow and shrink as needed and can use up to half the physical memory unless you specify an upper limit to the size.
At the moment I have decided I do not need to save the system log files in memory, so that entry is commented out.
You will need to reboot for these change to take effect.
Raspberry Pi is a trademark of the Raspberry Pi Foundation