Not too sure I understand. Could you detail that a bit?

I select a group or some parts, then click the Select button to define a specific origin I'd like to set. Instead of that origin being selected, the part owning that origin is being selected. I click exactly on the origin point, not next to it (which would select the part clicked on).

"Select" button is actually "selecting" (moving the gizmo to) the saved or default point of a group or part. This is the "set" button that saves the selected point as the origin.

"Select" buttons is used to show you what is the origin.
"Set" is used to changed that origin.

For groups, the default origin is the first point of the first part of that group (including sub-groups).

This is actually how I understood this feature. It didn't always work that well for me, though. But in a complex scene selecting points is difficult anyhow, so it's possibly rather that cause. Please consider this one closed for the time being.