By default, any existing Shifts you have already published will have GigSmart selected as the sourcing option, meaning they'll continue to source from your full pool of eligible workers rather than a specific Worker Group.
If you want to update or change this, you can do so by editing the Shift and selecting Edit next to the “Fill Strategy” section of your Shift details. From there, you can keep Smart Hire and choose to exclude specific Worker Groups (like Favorites or My 5 Star Workers) from sourcing, or switch to Custom to manually set Shift access to only the Worker Group(s) you select.
Please note: changing this will not impact Shifts that have already been published; it will only apply to future posts, so that workers are sourced according to whichever Fill Strategy and Worker Groups you select going forward.
To learn more, see What is a Worker Group and why should I use one? and this blog post.