“I store all my location data and I see no problem because it's stored locally” is the new “I store all my passwords on a post-it and I see no problem about it”.
The more you store, the higher the risk, simple as that.
You have a convincing argument for not taking photos and not writing notes down. In fact, why write anything down? Remember everything like Socrates asked people to.
The more you store, the higher the risk, simple as that.