If you need to take time to understand the code before you start refactoring

then the refactoring step you plan to do is probably already too big.

The understanding of the code is built as we refactor to the same extent that we refactor as we get a better understanding of the code and domain.

Limiting time for ourselves to understand a piece of code can act as an enabling constraint for finding a smaller refactoring step we could make.