Over the last two decades, coherent risk measures have been well studied as a principled, axiomatic way to characterize the risk of a random variable. Because of this axiomatic approach, coherent risk measures have a number of attractive features for computation, and they have been integrated into a variety of stochastic programming algorithms, including stochastic dual dynamic programming (SDDP), a common class of solution methods for multistage stochastic programs. However, it has been shown that they lead to inconsistency if agents care about their state at the end of the time horizon, but control risk in a stage-wise fashion. The more general class of convex risk measures includes the entropic risk measure, which does not have this shortcoming. We discuss how to incorporate general convex risk measures into an SDDP algorithm, exemplifying our approach with the entropic risk measure. We illustrate the advantages of the entropic risk measure with an example in transportation.