You can use them whenever you feel like it. You stand outside of the building of the starters you want and hit the activator.
example:
stand outside of elm's building, press start and select, walk inside and up to where you chose your starter, press a and choose another starter
you can do this as many times as you want with all 3 sets until you get the ones you want
Also, where is the in party IV modifier code posted (what page)?