What are indexing spiders searching for on your website?

what are search indexing spiders searching for on your website

In the old days, when search engines sent out indexing spiders to gather data on your Web site, their job wasn’t simply to look for relevant keywords; they also checked the density, frequency, and prominence of those keywords.

Keyword density played a role back then, but now relevance to the searched term is more important.

Keyword density measured the number of times a keyword or keyword phrase appeared on a Web page, compared to the total number of words on the page.

To determine density, you take the number of words on the page (say, 1,000 for a long page) and the number of times that the word appears on that page (maybe 23 times). Divide 23 by 1,000 to get a density of about 2.3 percent.

Keyword density used to be one of the factors a search engine spider looked at when determining whether a Web page was relevant to that search. These days, it has to be super relevant, available in multiple formats and the overall authority of the site plays a huge role in ranking.

Keyword frequency was a factor that SEOs looked at. It’s simply means how many times a word appears on the page; in this case, 23 times. The combination of frequency and density is the prominence — higher density and more instances lead to greater prominence of the term.

These factors collectively are why it’s important to have searchable text on your Web page, and especially on each landing page. That doesn’t mean you have to write a novel on your landing pages.

Search engine spiders generally put more weight on the first 200 words on a Web site, including words in your navigation, headings, and so on. It’s important to make sure that your keywords appear throughout the page but especially right up front so that search engines and your visitor know what the page and site is all about.

Content depth is important, but not as important as it once was. Nowadays, if you answer a question in three lines of text, if it’s relevant enough, it can reach the holy grail position “0” for voice search.

We used to fret over content length and people wrote books on blogs to try get rankings, but that just wasted people’s time and frustrated us when we wanted quick answers.

If you want to rank now, think in terms of the searcher, what is their intent, how relevant will your answer be, and can you cut out all the fluff, to give them the answer they want in as few words as possible.

The spiders aren’t there t read millions of rows of text. The algorithm wants relevance to match back to the query. The most relevant, to the point answer wins!

Leave a Comment

Your email address will not be published. Required fields are marked *