I would say that it depends greatly on the comparison between official stop spacing (or theoretical stop spacing) and the amount of actual stopping that a typical bus does (or practical stop spacing). Busses aren't trains; they don't stop just because there is a bench and a sign on the side of the road. Eliminating stops will only improve speed if it means that a typical bus is actually stopping less often. I'm not sure if that would really be the case for very many routes.
I used to ride the #3's southern portion quite extensively and still have some experience with it. Each bus seems to already skip a large proportion of stops.
So, for instance, if we call the stops A, B, C, D, etcetera. Bus 1 is stopping at ACDFILMOQT..., then bus 2 is stopping at BDGIKLPR..., and so on. Will eliminating half the stops really make much of a difference or will it simply mean that every bus is now stopping at ACEGIKMOQS...?
That said there is definitely room for weeding out existing stops. Stops simply should not be less than 200 meters apart and even less than 300 meters should require some justification. Furthermore, as a general rule I think 400 meter spacing on local routes (normal city busses) and 800 meter spacing on express/mainline/crosstown routes (essentially what we have been calling BRTs) is an ideal worth striving for. Commuter routes can have more distant spacing (past ten kilometers given circumstances that make sense) but no line that is intended to service pedestrians should have spacing much greater than 1000 meters.
I would also be more open to the idea of significant stop elimination if it meant a real change in the nature of the stops. Something along the line of replacing a large number of bus "stops" with a smaller number of bus "stations" that involves an upgrade in stop amenities and a corresponding change in the zoning. Not to the extent of a C-Train Station but definitely something in between.