Bigger sprites == more collision points to handle... In the end, it would be almost the same of handling many little objects.
See WWF Wrestlemania Arcade. When you have 4 "big" sprites on screen you have a lot of slowdown. Then you TKO one of the enemies and the framerate gets a huge boost, while the sprite is still there, on the floor... The collision detection is what causes slowdown in situations like that. The AI is a problem too...
VDP would contribute more to the flickering than slowdown IMO.
And you'll not have less enemies on screen while playing alone or in 2p due to the multitap support. The 3 or 4-player mode just need to set a different enemies on screen limitation, it's just a single additional line on the source code.