We quote our NILMTK paper explaining the need for a NILM toolkit:Empirically comparing disaggregation algorithms is currently virtually impossible. This is due to the different data sets used, the lack of reference implementations of these algorithms and the variety of accuracy metrics employed.