At the time crawling those freshly found URLs, they evaluate Every single URL's HTTP reaction code. Should the URL returns 404, bots do not add the specified URL to their index and also take away that URL from their index if it had been previously indexed previously. You may also paste text t