removeField don't remove field from palettes
Example:
->setTable('tt_content')
->setType('image')
->load()
->removeField('image') // this field alone and remove correctly
->removeField('imagecols') // this field is in the palette 'gallerysettings' and don't removed
->saveToTca();