IDK about setting clockspeed in an Arduino, but with a raw atmega328 it's definitely something you'd do. IIRC the atmegas uses fusebits which has to be set during programming. I suspect that you'd at least need to access the ICSP pins instead of programming over USB. At which point you could just build the needed part of the Arduino on perf board and ditch the atmega16u4 and LDO.
If you don't mind 4MHz @ 3V, then the datasheet claims that will draw 1.5mA https://ww1.microchip.com/downloads/en/DeviceDoc/Atmel-7810-Automotive-Microcontrollers-ATmega328P_Datasheet.pdf but if you refer to section 28 in the datasheet you'll see that you can go as low as 2.7V.
If you really have your heart set on the PP3 battery then sure, but I'd probably see if I could use 3xAA instead. I just spent 10s googling, so my research may be lacking. An energizer PP3 lithium ultimate claims 750mAh, if we, for the sake of the argument, don't care about discharge voltage curves and just use the nominal voltage, that makes 6750mWh. Varta has a 2900mAh AA, and 3 of those then become 13050mWh. I know that it's a crude comparison, and you may not get twice the capacity, but still.
If you went for the AAs then your buck will be more efficient as well. Buck converters become less efficient the higher your input voltage https://electronics.stackexchange.com/questions/528284/why-does-converters-efficiency-degrade-with-higher-v-in so you can double the capacity and increase efficiency with AAs.