It is a common problem with many websites: crawler taking it weeks to update the index coverage and showing you “crawled –currently not indexed”.
Before looking for the solutions, you need to understand the problem on the grass root level. The delayed indexing happens because of many reasons, such as:
• Google has been struggling to eliminate spam pages, content and structurally ordering them. Some e-commerce sites are so badly structured that Google faces a problem in ordering and moving ahead.
• There are a lot of pages to index, and hence Google sometimes takes even as long as a month to index them, as it has to go through all of them and approve their indexing.
• With the increasing number of pages, the number of duplicate pages also increase. And if your page lacks canonical tags for every page, Google is going to take a lot of time to make sure your pages are not duplicated.
• Google does not index every page of your site as some are not of much importance according to Google, some are duplicated, and some simply have low-quality.
To avoid delay in indexing you should take care of certain things, like:
• Enhance and upgrade the quality of your site in terms of structure and order, instead of making useless efforts towards getting as many links submitted and indexed as possible.
• To improve the number of pages indexed you need to either improve pages so they are no longer duplicated or add canonical tags to help Google attribute it to the correct page.
• According to John Mueller:
“If you have a website and you realize you have low-quality content on this website somewhere then primarily of course we’d recommend increasing the quality of the content if you really can’t do that if there’s just so much content there that you can’t really adjust yourself if it’s user-generated content all of these things then there might be reasons where you’d say okay I’ll use a no index for the moment to make sure that this doesn’t affect the bigger picture of my website.”
This means that using “no index” can work for your site’s low quality content as it is difficult to get that indexed.
• Prevent your site to generate low-quality signals on the basis of which Google indexes your pages:
#Spam comments
#Content lacking EAT: Expertise, Authority, and Trust.
# Poor page design
# Malicious harmful or deceptive practices detected
# No website contact information
# Auto-generated content
# Keyword stuffing
# No website contact information
• To improve the quality of your website you can inculcate the following:
#Up to date copyright notifications
#High-quality main content
#Accurate and trustworthy information
#Topical authority of the page author
#Customized and helpful 404 pages.
#Awards
#Evidence of expertise
• Minimize the production of thin pages you produce on your site.
• “Use the robots.txt file on your web server to manage your crawling budget by preventing crawling of infinite spaces such as search result pages. Keep your robots.txt file up to date”, says Google itself.
