Skip to content

Flash Degradation

The device stores everything — bootloader, kernel, and root filesystem — in a single NAND flash chip. NAND does not hold data perfectly forever, and the way it fails is gradual rather than sudden. This page explains what is happening, how to measure it, and what to do about it.

What is happening

A NAND cell stores a bit as a small trapped electrical charge. That charge leaks slowly, and it is also disturbed slightly every time nearby cells are read. Over months and years, individual bits flip.

This is expected and designed for. Every 512 bytes of stored data carries an error-correcting code, and the SoC's hardware ECC engine repairs flipped bits transparently on every read. On this hardware the scheme is BCH8: up to eight flipped bits per 512-byte chunk can be corrected. Data comes back correct and nothing is reported to the application.

The problem is that the correction budget is finite. A chunk with two flipped bits is perfectly healthy. A chunk with eight is at the limit, and the next flipped bit in that chunk is unrecoverable data loss.

You may see the bootloader print lines like these on the serial console:

nand: bit-flip corrected @data=168
nand: bit-flip corrected @data=486
nand: bit-flip corrected @oob=10

Each line means one bit was repaired. Seeing them is not a fault — it is error correction working as intended. But they do not tell you how much budget is left, because they do not say whether those bits were spread across many chunks (harmless) or concentrated in one (serious). That is the number you need, and it is what the tool below reports.

Two further points worth knowing:

  • Flipped bits are not physical damage. The charge has drifted; the cell is fine. Rewriting the data restores a clean charge level and resets the count to zero. This is why the fix is simply to write the data again.
  • The root filesystem partly looks after itself, but only where it is read. It is managed by UBI, which repairs a block by rewriting it elsewhere — but only once a read of that block reports corrected bits. UBI never scans the flash on its own. Files that are read regularly are therefore kept fresh automatically, while data that is never read is never checked and can drift unnoticed. libretouch closes that gap with a background service, described below. The areas holding the bootloader and kernel have no such mechanism at all.

Automatic background maintenance

Because UBI only repairs what something happens to read, libretouch runs a small service, ubihealthd, that reads the root filesystem's storage blocks itself. It works through them one at a time and asks the kernel to check each, which lets UBI find and repair drifting blocks it would otherwise never look at.

The service is enabled by default and needs no configuration. It is deliberately slow: one block roughly every two minutes, taking about a day to work through the whole filesystem before starting over. The cost is negligible, and spreading it out avoids doing bursts of work while you are using the device.

To confirm it is running:

systemctl status ubihealthd

This is something libretouch adds. The maintenance relies on a kernel interface introduced in Linux 4.11; systems built on older kernels cannot perform this check at all, and on those the flash is refreshed only where ordinary reads happen to fall. A device that has spent its life on the manufacturer's software may therefore have accumulated drift in areas that were rarely read — which is worth keeping in mind the first time you run the check below on such a device.

The service only maintains the root filesystem. It cannot help the bootloader or kernel areas, which are not managed by UBI.

If you would rather it did not run:

systemctl disable --now ubihealthd

How to check

nandcheck is included in the libretouch image. Run it as root over the serial console or an SSH session:

nandcheck

It reads every block of every flash partition and reports, per partition, the worst single ECC chunk it found. It only reads — it never writes to the flash, so it is safe to run at any time on a working device.

Typical healthy output for a partition looks like this:

=== mtd6  NAND.Kernel-Ping ===
    5120 KiB, 40 blocks of 128 KiB, 2048 byte pages, ECC 8 bits per 512 byte step
    40 blocks: 40 readable, 0 bad, 3 corrected bits total
    worst ECC step: 1 of 8 bits (block 12 @ 0x00180000)
    blocks by worst step:  0:39  1:1
    -- no action needed: 1 of 8 bits used in the worst step,
       7 bits of headroom remain.

Read the worst ECC step line. That is the health figure:

Worst step Meaning
0–3 of 8 Healthy. Nothing to do.
4–5 of 8 Ageing normally. Worth re-checking every few months.
6–8 of 8 Act now. nandcheck prints ACTION REQUIRED and names the partition.
Uncorrectable Data has already been lost in that block. A refresh cannot bring it back.

A summary at the end lists every partition that needs attention, and the exit code reflects it: 0 clean, 1 a refresh is called for, 2 data already lost, 3 a partition could not be read.

nandcheck -v additionally lists every individual block that needed any correction, which is useful for watching a specific area over time.

Bad blocks are also counted. A handful is normal — flash chips ship with some from the factory, and a few more appear over a device's life. They are managed automatically and only matter in large numbers.

Note that nandcheck reads the flash directly and does not go through UBI, so it reports what it finds without causing any repair to happen — even on the root filesystem. It is purely a measurement.

How to fix it

Reinstall the image. Writing the data again is the entire fix: it replaces the drifted charge with a clean one and returns every affected partition to full error-correction headroom.

Follow Install to internal flash. The normal update procedure rewrites the kernel and root filesystem, which is where the wear accumulates and where nandcheck will normally point you. No special mode or extra step is required — a routine update is the repair.

Afterwards, run nandcheck again to confirm the worst ECC step has dropped back to zero or near it.

If nandcheck reports a problem in a bootloader partition, do not treat it as routine. Those areas are deliberately left untouched by the normal update, and rewriting them is a separate operation with a real risk of leaving the device unbootable. Read the U-Boot update mechanism and Unbrick a device before going near them, and make sure you have a working recovery path first.

If nandcheck reports uncorrectable errors, that data is gone; reinstalling the image is still the right move, because it replaces the damaged content wholesale.

How often to check

There is no need to run this routinely on a device in regular use. Reasonable occasions are:

  • After a device has been powered off and stored for a long period. Charge leaks faster with no power and at higher temperatures, so long unpowered storage is the main way devices arrive with a lot of accumulated flips.
  • When the bootloader starts printing bit-flip messages on the console.
  • Before and after an update, if you want to confirm the refresh worked.
  • Once a year on an older device, as a general check.