That would be a solution indeed, but it would be way too tedious to achieve, since I have 100+ folders.
Anyway, for what it's worth, I've just found a little trick ! Instead of opening folders by clicking on the small "+" icon, it's way faster to double-click the folder name. Might sound silly but I had never thought of doing it that way !
Then, to browse back to any parent folder, simply pick the one you want in the dropdown list above.