Google doesn't remove 404 pages from its index. Reason can be if your site is accidently broken do you want Google to remove all your indexed pages? Generally 301 is the best solution for removing pages from the site. 410 status code is also supposed to work for removing previously indexed pages but I haven't tested that. This is true for all pages and not only amp pages.