The flowering probably stops other new shoot growth, but your plant still needs to grow bigger, and your plants have a lot of flowers, so it may stay at that size if you leave the flowers.
Also if any fruit is set, it will probably be too heavy for newly grafted plants, it may put pressure on the new graft union ? so unless you really want to get fruit, flower, pollen or seed from any newly grafted plants, I would remove the flowers.