You can back the grinder off to wherever it suits the flavour of the resulting plunger brew...the only reason people are sold really coarse "plunger grind" in this country is that someone (murphy ???) seems to think the grinds should not be able to get through the filter.
The reality is the filters used in plungers havent got a hope of stopping much except for the most coarse of particles...so it doesnt matter how fine / coarse you grind the coffee. So by all means, grind to suit your taste buds!
The Germans for example, almost pulverise their plunger grind, and live with the fines getting through...
So, if you wish you may even try the same grind you use in your silvia, and see how that suits. It *may well be you have to back it off a bit, but it most probably wont have to be backed off too far.
Essentially, grinding coffee coarse to try and stop fines coming through the plunger filter just robs you of the extra coffee flavour & body that you could have enjoyed. In Oz, this is called robbing you of value for money!
Try combinations of diff coarseness/fineness Vs amount of coffee per cup.
Hope this helps,