I think in the context of the day, in which the Church was struggling against the Judaizers in regards to how Gentiles were to be received (as seems to be the background of many of Paul's epistles), these seem to be JEWISH days that Paul is referencing.
Certainly, there may be an application to those observing Christian holidays, if those are doing so for show, out of habit, or subconsciously trying to score brownie points with God. However, I don't see anything particularly wrong with a day set aside, at the beginning of Lent (which commemorates Christ's fasting in the wilderness) to be reminded of our mortality: "Remember, O Man, from dust thou art and to dust thou wilt return". I see the Church's Liturgical calender being a wonderful teaching aid for growing in the faith as one meditates on the life and work of Christ (and remembers those who have been sanctified by Him), particularly when combined with the Scriptures specific to those days.
Those of us who follow the 'Daily Office' find it to be a 'wonderful teaching aid for growing in the faith' as well. Your Book of Common Prayer has a lectionary for those who pray using the 'Daily Office'. I think the BCP is one of the 'gems' of your church.