During a rebuild of the Search Service Application at a customer site, we’ve encountered a problem with the removal of the Search Service Application. We tried to delete the application in Central Administration but the process never seems to end and actually delete the application.
So we tried to do it from the Sharepoint PowerShell:

First we found the GUID for the Search Service Application.

Get-SPServiceApplication -Name “Search Service Application”

After that we just used the GUID of the Search Service Application to delete the application:

Stsadm -o deleteconfigurationobject -id id-of-search-service

This successfully removed the Search Service Application from the farm.