Whenever we make a choice, we must also decide how much time to spend making it. Many theories of decision making crucially assume that this deliberation perfectly balances the costs of time expenditure and the benefits of better decisions. However, might we “overthink” or “underthink” decisions? Here, I propose and implement a method to precisely determine whether people are optimally spending their time on deliberation, accounting for individual preferences. This test evaluates consistency of underlying preferences for time when incentives change, which is a necessary condition for optimality. This enables a more comprehensive analysis of rationality in a range of contexts. I measure choices and response times of human participants in motion-discrimination tasks, and use the test to reveal significant departures from optimality when task difficulty and monetary incentives are varied.