Rsync of a directory with 2M small files takes forever

The bottleneck is per-file syscalls, not bandwidth. tar piped over ssh moves them as a single stream and is often several times faster; rsync becomes the right tool again on the second run, when only the differences matter.

Almost always a deleted file still held open by a process, so the space is gone but no path points at it. lsof +L1 lists them; restarting the holder releases the space instantly.

An afternoon to do it properly the first time, then minutes when it recurs. Most of the first afternoon goes into discovering which of the defaults do not match your situation.