I'll split my answer into 2 for Motorsport and Horizon, its not fair to compare them both!
MotorsportI agree FM4 probably was the peak for me. It had the best custom options available for competitions and the single player mode was rewarding without being as endless as I remember FM3 being. I lost interest in the motorsport series after that as the horizon series came out.
HorizonFH2 for a long time, even after FH3 and FH4 came out, was my goto racer for multiplayer. It was a big improvment physics wise from FH1 and didnt have the emphasis on getting skill points to win that FH3 does. It's also got one of the best locations in the Italian/French Riverias.
FH4 is my favourite though overall. Its set in the UK, mostly around Lake District/Peak District/Edinburgh. The variety of stuff to do is off the charts. Eliminator is giant game of cat + mouse accross the whole map, there are new championships and challenges weekly with their own rewards, stories that have (hilariously badly acted) people giving you silly things to do, hourly there is a Forzathon which is a quick challenge with random people to earn rewards. Then of course there is the racing, single player is great fun with the new weekly events keeping it fresh and multiplayer has a large variety of modes to pick. I personally favour team unranked as it splits between S2-S1-A-B Road/Offroad racing fairly evenly, you can setup custom events for yourselves if you wish.
Theres are couple niggles - Ranked is too slow to find matches and is currently bugged for me, and the UI/Interface is horribly slow sometimes but neither stop me playing the game.
FH4 is also cross play PC > XBox and you don't need XBL (apparently) for PC. Potentially for a game night?
Here's a picture of me in an Aston Martin Vulcan racing a totally not a lookalike Vulcan Aircraft (-;