A sitemap is a hint, not an instruction. Listing a URL does not get it indexed and omitting one does not get it removed. What a sitemap does well is tell a crawler which pages you consider canonical and when you last changed them, which matters most on large sites where discovery through links alone is slow.
Only canonical, indexable URLs belong in it
The single most common sitemap error is including pages that the site then tells the crawler not to index. A URL that appears in the sitemap and also carries a noindex tag is a contradiction, and Search Console will report it as one. The same goes for URLs that redirect, URLs that canonicalise elsewhere, and URLs that return anything other than a 200.
Tag archives, author archives and paginated series are the usual offenders on a WordPress site, because the default behaviour of several plugins is to include everything and let you opt out later. Decide what you want indexed first, then make the sitemap reflect that decision rather than the other way round.
lastmod is worth getting right
Google confirmed it uses lastmod when the value is consistently accurate, and ignores it entirely when it is not. A site that stamps every URL with today’s date on every build has taught the crawler to disregard the field, and there is no quick way back from that.
The value should change when the content changes, and not when a footer widget changes. If your build process cannot tell the difference, it is better to omit lastmod than to publish one that lies.
The fields that stopped mattering
Priority and changefreq are still valid sitemap elements and Google has been clear for years that it ignores both. They cost nothing to leave in and gain nothing either. Time spent tuning them is time not spent on the two fields that are actually read.
Size limits and index files
A single sitemap holds up to 50,000 URLs and 50 megabytes uncompressed. Beyond that you need several files and a sitemap index pointing at them. Splitting by content type rather than arbitrarily is worth the small extra effort, because Search Console reports coverage per sitemap file, and a file that contains only product pages gives you a coverage number that means something.
Submit the index file in Search Console, reference it from robots.txt, and leave the individual files unsubmitted. Submitting both produces duplicate reporting and no additional crawling.
Reading the coverage report properly
The number that matters in Search Console is not how many URLs you submitted but the gap between submitted and indexed. A large gap is a content problem wearing a technical costume: pages that are thin, duplicated or simply not worth a slot in the index. No sitemap change fixes that.
Work the exclusion reasons in order of volume rather than in the order they appear. Discovered but not currently indexed usually means crawl budget or quality. Crawled but not currently indexed almost always means quality. Duplicate without user-selected canonical means your canonical tags and your sitemap disagree, and the sitemap is the easier of the two to correct.
Leave a Reply