In this work, we demonstrate a method to quantify uncertainty in corrections to density functional theory (DFT) energies based on empirical results. Such corrections are commonly used to improve the accuracy of computational enthalpies of formation, phase stability predictions, and other energy-derived properties, for example. We incorporate this method into a new DFT energy correction scheme comprising a mixture of oxidation-state and composition-dependent corrections and show that many chemical systems contain unstable polymorphs that may actually be predicted stable when uncertainty is taken into account. We then illustrate how these uncertainties can be used to estimate the probability that a compound is stable on a compositional phase diagram, thus enabling better-informed assessments of compound stability.