Here's a lesson in Murphy's Law (along with a PSA).
Last Thursday I held a session at the Softwerkskammer München on mutation testing. This included an exercise in which participants took a program with 100% code coverage through tests and tried to find changes in the production code which broke the tests.
This doesn't work so well when the tests are already broken, as they were for at least three of the participants.
Of course I had made sure that all tests run smoothly beforehand. They were all simple unit tests, not depending on any exotic machinery or external systems.
So what happened?
The failing test in question was (roughly) of the following form:
result = process(..., LocalDate.of(2015, 12, 31));
assertThat(result.getDateAsString(), is(equalTo("31-12-2015")));
Nothing special here: the date is formatted to a string and put in a data structure. But on some (but not all) systems, the test failed with the message that the string in question was "2016-12-31".
Huh?
It turned out, after a bit of frantic investigation, that the systems on which the code worked properly were set to the German locale, while those on which the code failed were in the English locale.
Why does this matter? Let's take a look at the formatting code:
getDate().format(new DateTimeFormatterBuilder().appendPattern("d-M-Y").toFormatter())
Turns out the problem was with the date format pattern. Looking at the Javadoc for DateTimeFormatterBuilder.appendPattern:
... y year-of-era year 2004; 04 ... Y week-based-year year 1996; 96 ...
So... the capital "Y" is not actually the year, but the "week-based year." In Germany, the week begins on Monday, so the week-based year of 31 December 2015 is 2015. In English-speaking countries, the week begins on a Sunday, so the year is 2016.
Ugh.
The fix is easy: just replace "Y" in the format string with "y". Given that this kind of bug leads to wrong results just a few days a year, and then only in some locales, I can't imagine how many programs got it wrong.
Some takeaways:
- This is, frankly, bad API design: "Y" and "y" are far too easy to confuse, especially given that they differ on only a few inputs.
- Make sure to check the format strings of your programs for this error!
- When testing any code with dates, make sure to include edge cases, like the last day of the year, and to test with different locales.
- A program which works for you may fail for others for the strangest reasons. Murphy's Law holds.