Read Reflections on Trusting Trust to understand why having little bits of binary blobs sprinkled all over your compute arch is actually a major problem. Just because it’s a hard problem doesn’t mean we’re gonna pretend it’s fine.
The general sentiment is that you cannot trust code you did not write yourself and that we need to be able to trust the person who did, but you can form your own conclusions about how that fits into the modern tech landscape.
One of the points made in that paper is that you can't even trust the compiler, even if you write the code yourself. I think this is one of the stronger points as it shows you it is unfeasible to require everybody to audit all source code before running it. Be pragmatic, know your threat model, decide who you trust and move on with more important things in your life.