Sitemap for a Large Website Shouldn't be Created Manually

An SEO professional wanted to create a sitemap for a website with over 2 million products.

As you can imagine, the process is so tedious that they didn’t know how to go about it. Thus, the SEO took to a Reddit thread on the r/SEO subreddit to ask for help.

The thread reads:

“I have a client in a very technical, industrial niche. They sell industrial parts that are similar but very specific. They have 2 Million + SKUs in their inventory. My challenge here is to create a sitemap. Because there are so many pages, I have to create it manually.”

The SEO further stated that the dev team had collected a list of URLs based on the products and categories. They broke the 50,000 URLs — as much as a sitemap can contain — into 37 excel sheets.

The SEO further delved into details of how they intend to tackle these challenges. However, that became irrelevant since Google’s John Mueller advised against creating the sitemap manually.

The webmaster trend analyst at Google simply stated: “Don’t create a sitemap manually like that.”

Here’s what Mueller suggested doing instead.

Get a Sitemap from the Content Management System

Since Mueller’s advice was the most upvoted comment, it obviously echoed the Reddit SEO community’s thoughts.

Instead of going through the tedious process of creating a sitemap manually, Mueller suggested getting it from the content management system (CMS). “You need to get [the sitemap] from the CMS or the underlying database directly,” he said.

Another Reddit user also suggested getting a sitemap from the API, where the dev team pulled the URLs list.

Why are these recommendations better than creating it manually?

Aside from the enormous work that goes into creating a sitemap manually, updating it would also be logistically impossible.

The website has a massive product catalog, which comes with adding and removing tens of thousands of products every day. Moreover, manually creating a sitemap is impractical when automated solutions exist.

In the end, the SEO who started the thread wrote:

“Thanks for the wake-up call, guys. I agree. This was the plan of action laid out for us, and it didn’t sit right with me from the beginning.”

Read More: Google’s John Mueller Says Google Ignores Paid Links