We noticed with size 6 , very large shadow fish we do better in hand held mode. We tried two different switches and several different TV's and multiple controllers. There seems to be a very slight delay in sound using HDMI on every TV we tried, even in "Game" or "PC" mode.
We experimented and learned using the headphones in addition to HDMI solved the issue. We are using 2 different switch setups so I used a cheap manual 3.5 mm a-b toggle switch and 2 cables to feed an old set of powered computer speakers. Perhaps people with better hearing and faster reaction times the slight audio delay is not enough to affect their game play but if you do find you miss the larger fish too often when using HDMI consider try using headphones.
.