Paul - here is my 2 pence worth.
Modes are an amazing and deep hole of wonders. I wrote an extensive topic on them here: Modes Parts 1 - 9
But I would argue they are not the path you seek for your specific need.
Modes (exclusing Ionian and Aeolian - major and natural minor respectively) are peculiar beasts and most conducive to being played over one or two chord grooves and vamps. Sometimes three or four chords but with a heavy proportion of the tonic and one other chord and often with slash chords that sees the tonic note in the bass of all / most chords. Their use as scales for improvising and melody then leans heavily on the flavour notes that make those modes unique.
In Lydian, the #4 is a target.
In Dorian the b4 and natural 6 are targets.
Etc.
You seem to be referring much more to longer chord progressions containing many chords - mainly but not exclusively diatonic.
That is what jazz players aim to do. I am a million miles form being a jazz fan let alone a jazz player.
But I do believe their approach incorporates arpeggios, triads and targeting chord tones, chromatic runs and more. I do not imagine they are switching modes as a conscious process.
I recently saw a video of a session player (I forget which, sorry) saying that he doesn’t know any player in Nashville who would approach a regular chord progression with modes at the forefront of their mind - it would be chord tones, CAGED, triads, pentatonics and arpeggios.