I am French, so I am not going to answer to the last part about Keir Starmer, but I can help with your first question.
The executive branch of the French government is two-headed, with a President and a Prime Minister.
The President is the head of state, as such they are elected by universal suffrage, and are viewed as an arbitrator. The President is the commander in chief of the French military, and acts as the head of the French diplomacy.
The Prime Minister is the chief of the government. The PM is nominated by the President, to form a government or cabinet of several ministers. Together they are responsible, before the parliament, of implementing policies in the best interest of the nation. The parliament can revoke the PM and its government.
That's where it gets messy :
if the parliament's majority party is the presidential party, then the President can nominate a political ally as Prime Minister, and in practice, the center of gravity of the President-Prime Minister couple will move towards the president, who will be free to enact his policies.
However, if the majority party in the parliament is opposed to the President, then the President can only nominate a PM from the majority, otherwise the parliament will promptly revoke the PM. This situation is called a cohabitation, and it happened only 3 times under the 5th French republic. With a President and PM in opposition, the center of gravity is usually tilted towards the PM.
Now what happens when there is no clear majority party or coalition in the parliament? That has never happened before, not at the scale we are witnessing today. And there are no clear guidelines : presumably, the President will nominate a PM from the party that came out in the lead in these parliamentary election. But if all the other parties reject the new PM, they could oust them. A multi-party coalition could be created to reach the absolute majority, but lately the major political formations have been sternly opposed.
The coming days are going to be very interesting, and a good stress-test for our institutions.
The closest thing we have to American bureaus are ministries and "secrétariats d'état", and yes they are managed by the PM.
The budget is created by the government, under the PM's management, and ratified by the parliament.