I tried what you are describing: Using a Pi4 and loading lineageOS (KonstaKANG) on it. Before that I tried libreELEC with Kodi or something like that.. Internet via WLAN. Both setups suck. Everything is slow, input with my cableless keyboard/touchpad is laggy and I am not using the TV because of it.. Watching jellyfin on my smartphone is more convenient.
But I am new to all this stuff, so maybe I missed some ways to optimize this setup. Next time I will just take a long HDMI cable and connect the TV with my main PC.