Communicating relative change
Sometimes I have to compare numbers and communicate their relative differences; say, because a metric changes meaningfully. I routinely forget how to do so correctly and safely and then have to remind myself. This is that reminder.
I set out to write a thorough explanation of relevant concepts when a serendipitous click revealed to me that—naturally—Wikipedia already has that. Then I decided to summarize that article instead, because it’s long and dense and addresses details I haven’t needed, and provide some examples that are more contextually relevant.
Calculator
A change is a , to of the magnitude.
Terminology and concepts
The difference between two numbers \( x \) and \( y \) is given by
\[ \tag{1} \Delta = x - y \]
In the special case where \( y \) is some past measure of \( x \) it can instead be written as \( x_{reference} \)1 and
\[ \tag{2} \Delta = x - x_{reference} \]
may be called the change of \( x_{reference} \). If \( x_{reference} \) is non-zero, the relative change is given by
\[ \tag{3} \text{Relative change}(x, x_{reference}) = \frac{\Delta}{\abs{x_{reference}}} = \frac{x - x_{reference}}{\abs{x_{reference}}} \]
The relative change is a positive number when \( x \gt x_{reference} \), a negative number when \( x \lt x_{reference} \), and undefined when \( x_{reference} = 0 \).
For completeness, in the event that both \( x \) and \( x_{reference} \) are positive, from \( (3) \) we can derive another form:
\[ \tag{4} \text{Relative change}(x, x_{reference}) = \frac{x}{x_{reference}} - 1 \]
We won’t use \( (4) \) again because the formula does not generalize but we show it to make a specific point: to anybody that has heard that “when comparing numbers you need to subtract 1”, that rule comes from \( (4) \).
A relative difference is calculated nearly the same way as a relative change but for various reasons of technical correctness not important here cannot be neatly summarized in a single equation. It does have one important difference: only the absolute value is used.
Relative differences are described with grammatical comparison: \( x \) is harder, better, faster, or stronger than \( y \); and when such language is used, a relative difference is being described.
A less common ratio form expresses the same relationship between \( x \) and \( y \) but uses a different equation and different language. Distinguishing ratios and relative changes is the primary motivation for this post. The ratio is given by
\[ \tag{5} \text{Ratio}(x, y) = \frac{x}{y} \]
and we say that \( x \) is \( Ratio \) of \( y \). Contrast the use of “of” with the comparison form used for the relative difference. Other phrasings can be substituted for “of” but the statement can always be rewritten to use “of”.
Both relative differences and ratios can be represented as percentages by multiplying by 100. Sometimes reporting in percentages clarifies the message, sometimes it obscures. There are no rules for deciding but, as a rule of thumb, double-digit ratios often look awkward as percentages.
Signs play a large but subtle role in what is actually being communicated and are a source of some confusion, separate from the confusion between relative differences and ratios.
Examples
Speed, frequency, throughput
Wikipedia says speed is the rate of change of an object’s position, which is like saying distance travelled over time.
Speed is tricky. We sometimes talk about the “speed” of software or processing units but, as these tend not to physically move, that’s arguably nonsensical. The related concepts of frequency—cycles over time, measured in Hz—and throughput—data over time, measured in bits per second—are often technically more accurate, but all three terms describe some notion of “things per time”. I will use an example that doesn’t fit neatly into either of these definitions and perpetuate this careless disregard for terminology.
Perhaps what makes comparisons of speed tricky is that the distance travelled often doesn’t change; instead, the same distance is travelled, or the same work task is completed, in a greater or lesser amount of time. If a car moves at a greater speed than a bicycle it will require proportionally less time to travel some specified distance—this consequence is sometimes reported as speed but is really its reciprocal, pace: the time taken to travel some distance2, and I address it in the next section. Or perhaps what makes speed tricky is that we often care about timing cycles that transfer data.
Suppose a database query pattern changes such that the number of transactions per second (“TPS”) drops from \( TPS_1 = 1161 \) to a meagre \( TPS_2 = 197 \). By \( (3) \) the database has become
\[ \frac{TPS_2 - TPS_1}{TPS_1} \Rightarrow \frac{197 - 1161}{1161} = -0.83 \]
times faster than before, which sounds bizarre, so we invert the sign and report it as 0.83 times slower than before. It still sounds clumsy, and invites the interpretation that we judged the previous measurement as slow and the new measure even slower, even though we express no opinion on either measurement. Worse, a reader is at risk of interpreting this relative difference as a ratio, in which case the reduction in TPS seems much less severe than it is. Reporting as a percentage doesn’t really help; “83% slower” may sound less clumsy but it is no more resistant to misinterpretation. The best cause of action is likely to report the change as a ratio: by \( (5) \) the database is
\[ \frac{TPS_2}{TPS_1} \Rightarrow \frac{197}{1161} = 0.17 \]
times as fast as—or 0.17 times the speed of—before. This phrasing does not impose judgment, and the smaller number emphasizes the severity of the regression. Here, reporting in percentages, as 17%, can further improve clarity of communication by replacing the subtle decimal place with the visually and audibly more distinct percentage sign.
Alternatively, we can emphasize the regression: speed—or throughput—has been reduced by 83%, or reduced to 17%, of the baseline.
A frantic DBA creates a much-needed index to support the new query pattern and
runs VACUUM ANALYZE, thereby bringing the TPS up to \( TPS_3 = 1800 \). By
\( (3) \) the database becomes
\[ \frac{TPS_3 - TPS_2}{TPS_2} \Rightarrow \frac{1800 - 197}{197} = 8.14 \]
times—or 814%—faster than during the regression, such that, by \( (5) \), it moves
\[ \frac{TPS_3}{TPS_2} \Rightarrow \frac{1800}{197} = 9.14 \]
times as fast as—or the speed of—before. The new query pattern with tuning also runs at
\[ \frac{TPS_3}{TPS_1} \Rightarrow \frac{1800}{1161} = 1.55 \]
times the speed of the original query pattern, which is
\[ \frac{TPS_3 - TPS_1}{TPS_1} \Rightarrow \frac{1800 - 1161}{1161} = 0.55 \]
times faster and an increase in throughput by 55%.
Pace, duration
Instead of measuring the number of operations completed over time we can measure the time needed to complete one operation. It’s merely another perspective on the same measurements but in some contexts a more natural one. Pace makes it easy to determine if one measurement is “faster” or “slower” than another: if the same operation takes less time to complete one way than another, that way is intuitively faster. Yet, “fastness” is a property of speed, not pace, so pace should not be directly reported as “faster” or “slower”—this is the point Bruce Dawson made3 above.
The numbers from the previous section would be uncomfortably small to work with4. Instead we’ll make up some other numbers that are mutually comparable. For instance, I can informally measure that it takes Slack \( t_{Slack} = 7.92s \) to start up and connect to a channel, and that it takes Ripcord \( t_{Ripcord} = 2.05s \) to do the same.
In this example we’re dealing with a general difference, not a relative change. We can still use the base calculations of \( (3) \) and \( (5) \) but we must take the absolute value. We also have to decide which measurement should take the place of \( y \)—and we have complete freedom to choose. If we wanted to promote Ripcord we could say that it starts up in
\[ \biggm\lvert\frac{t_{Ripcord} - t_{Slack}}{t_{Slack}}\biggm\rvert \Rightarrow \biggm\lvert\frac{2.05s - 7.92s}{7.92s}\biggm\rvert = 0.74 \]
times, or 74%, less time than Slack, or in
\[ \frac{t_{Ripcord}}{t_{Slack}} \Rightarrow \frac{2.05s}{7.92s} = 0.26 \]
times, or 26% of, the time of Slack. But Ripcord is neither 74% nor 26% faster than Slack. Recall that pace is the reciprocal of speed; the launch speed of Ripcord, in “launches per second”, is \( 2.05^{-1}l/s \) and the launch speed of Slack is \( 7.92^{-1}l/s \) so Ripcord is
\[ \frac{{t_{Ripcord}}^{-1}\frac{l}{s}}{{t_{Slack}}^{-1}\frac{l}{s}} \Rightarrow \frac{2.05^{-1}\frac{l}{s}}{7.92^{-1}\frac{l}{s}} = 3.86 \]
times as fast as Slack, or
\[ \biggm\lvert\frac{{t_{Ripcord}}^{-1}\frac{l}{s} - {t_{Slack}}^{-1}\frac{l}{s}}{{t_{Slack}}^{-1}\frac{l}{s}}\biggm\rvert \Rightarrow \biggm\lvert\frac{{2.05}^{-1}\frac{l}{s} - {7.92}^{-1}\frac{l}{s}}{{7.92}^{-1}\frac{l}{s}}\biggm\rvert = 2.86 \]
times—286%!—faster: literally the amount of extra times Ripcord could start up while Slack starts up once.
Size, mass, weight
Size and mass are straight-forward absolute values. Weight is trickier: what we commonly report as weight is actually mass, while weight is a force exerted by gravity; and because gravity can vary, so can weight. But if that distinction were important to you, you would already know. I won’t talk more about mass and weight because that wouldn’t contribute anything.
If OptiPNG reduces the byte size of a picture from \( S_1 = 18332b \) to \( S_2 = 15940b \), by \( (3) \) the picture is now
\[ \frac{S_2 - S_1}{S_1} \Rightarrow \frac{15940b - 18332b}{18332b} = -0.13 \]
times larger, or less awkwardly \( 0.13 \times \Rightarrow 13\% \) smaller, and
\[ \frac{S_2}{S_1} \Rightarrow \frac{15940b}{18332b} = 0.87 \]
times, or 87% of, the original size. If bytes had mass you could substitute “heavier” for “larger” and “lighter” for “smaller”. That’s really all there is to say about size.
Multiple dimensions
The calculations don’t generalize to multiple dimensions. This may not surprise but it’s important to remember.
While a change in resolution from 720p to 1080p could be reported as
\[ \frac{1080p - 720p}{720p} = 0.5 \times \Rightarrow 50\% \]
larger, that wouldn’t be the whole truth. Although the image is indeed 50% taller, both of those values are really terms for the two-dimensional values of \( 1920px \times 1080px \) respectively \( 1280px \times 720px \) and that 50% increase only accounts for one of the dimensions. The amount of pixels on the screen has increased by
\[ \frac{1920 \times 1080 - 1280 \times 720}{1280 \times 720} = 1.25 \times \Rightarrow 125\% \]
When dealing with multiple dimensions it’s important to be explicit about which dimension is being reported.
- Alternatively, \( x_{reference} \) can be written \( V_1 \) and \( x \) can be written \( V_2 \). [return]
- Pace is itself confusing, being sometimes reported as “high” to mean “in less time” even though a smaller rating implies greater speed. Pace is more like “slowness” than “speed”. [return]
- Not entirely successfully, I felt. [return]
- But they could be normalized to the time needed for \( \alpha \) transactions with \( \frac{\alpha T}{T/s} \), e.g. \( \frac{1000 T}{1161 T/s} = 0.86s \) for 1000 transactions. [return]