Science

Particularly, I worry that CO₂, plastics, uranium, HCFCs etc. are just the first of many problems we'll have with breaking down these materials. The non-biological elementome will not degrade, at least not without leaving non-biological elements behind.

That can be fine. Rocks generally don't participate in the biological cycle either and they don't bother anyone.
But for example, plastics are practically rocks in funny shapes, which float out into the ocean. Even just that tiny difference causes problems for maritime wildlife. Other super-durable materials will produce different rocks, which may cause problems in new and innovative ways.

And of course, not everything we use is a rock. Some materials will genuinely just interact with our surroundings in destructive ways. The hope is that they do then degrade.