Hmm, it shouldn't really matter. There is some little housekeeping associated with all expansions but just starting the app and starting a new game shouldn't take longer. Of course it's different if you actually load a save which has been to many expansions (it still shouldn't be that noticeable but it takes a bit more time).
Syncing save files and logging into Google Play has nothing to do with how many expansions are installed. So this points to something else slowing down your device.
Still, reinstalling the game and leaving out not used expansions certainly wouldn't hurt. Just be advised that any save you created with expansions installed will require all those expansions to be installed. So if you uninstall expansions you can only start a new game (as none of your current saves would load).
You might try to delete and reinstall everything. Be sure to copy your save files to safe space before. (Here is info about the save game locations:
viewtopic.php?f=3&t=4624)