I would indeed add addr:housenumber= to each unit, there is also addr:unit= which I would prefer over constructions like "addr:housnumber=123/4". Drawing a polygon around the lot for the housenumber sounds usable, for which I would use the residential that should already be there but, preferred method for that (afaik) is a singular address node.
There is probably a section in the wiki that describes, or at least proposes, a setup for precisely this use case.
addend: Another adjacent tag to addr:unit= would be ref=, though with buildings this also is often applied to buildings that already have differend housenumbers.