Measuring energy access in developing countries involves much more than simply recording whether or not households are connected to the grid. Both international organizations and scholars now recognize the importance of reliable electricity supply for achieving positive development outcomes. Yet, measuring reliability is much more difficult than measuring the existence of connections. We propose an economical croudsourcing method for measuring reliability, and compare this method to energy monitor data for 122 households over 12 months. The results suggest that, while far from perfect, crowdsourcing provides a reasonably accurate method for monitoring the reliability of access over time, especially when modeled as a non-linear relationship. We apply these findings to model energy reliability in a broader group of villages across Uttar Pradesh, India, demonstrating the existence of disparities between urban and rural reliability and seasonal fluctuations in reliability. The system laid out in this study can be utilized by government and non-government organizations to quickly and cheaply monitor energy reliability.