You need to leave the plastic part in place unless you only make single shot drinks (which is all that this machine is really capable of).
If you choose to leave it in, you will need to remove it for cleaning as old residue will build up underneath on the actual PF