Not trying to be facetious, but you just kind of do it. I think it might be something that you just subconsciously keep track of once you really become aware of it. I remember it seeming like magic until I was maybe 15 or so, and then I had landmarks for each direction in my mental map and could figure things out in reference to them. After a bit of that, I could mostly stay oriented when traveling by land, and now it's not an issue even when I fly somewhere. I went to England for the first time last year, and I had the cardinal directions sorted probably by the time I'd walked from the train to my hotel.

Once you've got it down, you just sort of do it on autopilot.

I think it helps that I've been a pilot since I was a teenager. Spend some time where you can see a third of a state at a time you'll just develop this sense. You get a bigger picture of how things are oriented relative to each other that's sort of like, wherever you are in your home, you can probably work out which way the road outside goes, likely parallel to one of your walls. I can do that over much greater distances. If you've ever stood in the middle of a parking lot in a strip mall, and gone "the highway is over there, the Belk is over there, the J.C. Penney is that way, the furniture store is down on that end and I know the Red Lobster is just on the other side of it though I can't see it from here," I can do that with the major cities in my state.

Orienting yourself if you've gotten turned around is another habit to build up. Yes "the afternoon sun is in the West" but also if you're in the Northern hemisphere, your shadow will point North at noon. I also have a pretty good picture of the highway system in my head and can orient myself by knowing the general heading of a nearby highway.

From both my time as a pilot and as an amateur radio operator I'm familiar with the various towers across the state. I've used those to work out my approximate location and heading both in the air and on the ground. In medium sized cities often there's a city center with a few tall buildings that can be seen for several miles around, orienting yourself to them can help you develop a sense of direction. I've started doing that almost subconsciously.

Now if I were to wake up in a cave my gyros would be tumbled until I managed to get out. I don't have an actual built-in compass. But it wouldn't take me long to orient myself seeing how the daylight hit the cave entrance.

You track you direction throughout the day. My road runs east/west. If i leave my house and turn left, I'm going west, turn right im going east, I keep this in mind as often as I can.

I also know my general area pretty well and what direction each town/city is in relative to my home, which helps me find my direction if i lost it. I also like to remember which direction each major intersection goes as that helps me keep track as well.