I too came from different brewing methods (plunger, aeropress) with a decent grinder, before deciding to purchase an entry level machine. Im happy with my EM5900 for now, because for <200 that was the best I could do, brand new. But, that said, I had only intended it to further hone my newly-acquired espresso-making skills before plunging head on into the real world. I do detect a major difference in taste when switching between brew types, even though my espresso machine is far from top notch. So in that sense, Im still quite satisfied (until I can save up for an HX and feel confident enough that I could handle it).
Seeing that youre already comfortable with an HX machine, you might not be in the same boat as I currently am. So maybe a second hand prosumer-level machine would be better for you, or maybe stick to stovetop for now.
That didnt help, now, did it? :)