The group used to schedule the volunteers was a group type of Serving Team. This group type had RSVP and scheduling enabled. On the Group Type, RSVP settings for the communication template and the number of days for the reminder were set. This means it was not editable on the group level. If a person set their preference to "No reminder" they would not receive a scheduling reminder, but they would receive an RSVP reminder.
In order to leave the ability for RSVP on this group type but to fix the reminders from not sending for the schedule groups, I left RSVP to enabled but removed the settings for the communication template and the offset days. This made it editable on the group level. If the setting is left blank on the group, the volunteer will not receive a reminder.