There's a blue button. If you press it, it kills you, unless >50% of the population also presses it.
Why would you press it? Of course you don't, it's a suicide button. It kills you. You don't need to kill yourself just to hope that >50% of the planet also does it.
There's a red button. If you press it, and >50% of the population also presses it, it kills everyone who didn't press it. Why would you press it? Of course you don't, it's a genocide button. It kills people. You'd only need it at all if half the people on earth were willing to kill off billions of people for no reason, and they aren't.
Both of these lines of reasoning are completely coherent , correct, and reasonable. Both of them are logically equivalent to opposing viewpoints of the same question.
This is what the trolley problem was always about. When it was proposed, it was proposed BECAUSE it would be blindingly obvious that you should pull the lever. That's scenario 1. The point of the trolley problem is the other two scenarios.
In scenario 2, you're standing on a bridge, and you can push someone off onto the tracks. The trolley operator will see them fall onto the tracks and apply the brakes, which won't be enough to save the person you pushed but will save the five people farther down the track.
In scenario 3, it's the lever again, but the track loops around, so it's the actual 1 person that's stopping the trolley.
Clearly, in scenario 1, you should pull the lever. Clearly scenario 3 is the same as scenario 1. But in scenario 2, you won't throw the person off a bridge to save five others.
(This is true for a vast majority of people surveyed. You, personally, might disagree, but most people don't.)
Why not? It's the same dilemma, isn't it?
The trolley problem is a proof that our desire to not subject ourselves to the act of killing someone is a far bigger motivation than any altruistic goals.
The median human doesn't care about people nearly as much as they simply think murder is gross. (Are we willing to accept that fact? In many cases, probably no.)
So the button problem just comes down to which button you're attributing "agency" to. Is the red one the one that's killing people, or the blue one? Obviously, that's a silly question, but we still ask it and we still answer it, in our heads.
If the red button kills people, then pressing it is a genocidal choice, and pressing blue is a non-genocidal choice where you risk yourself to save people.
If the blue button kills you, then pressing it is straightforwardly suicide, and red is basic self-preservation.
If the killing is done by both buttons, then you're choosing between genocide and suicide, both of which are bad, and there's no answer you can argue for at all.
This isn't a reasonable way to think about things, but it is the basic core concept of human morality- killing a person is worse than letting them die, including yourself. Even though they're the same action.