What is Keyword Research and why is it so important?

In the world of SEO, Keywords are the terms that people use to search for things on Google, Bing, and other search engines.  These terms are normally typed into the search bar, or increasingly spoken now that speech recognition allows voice search.  Knowing what key search terms people are using to search for your products and services is a vital first step of any SEO strategy.

So-called ‘Keywords’ are normally more than just one word, normally they are several words combined into a small phrase, but are still normally referred to as ‘keywords’ not ‘key-phrases’.  A ‘Search Term’ is essentially the same thing as a ‘Keyword’.

If you don’t know what search terms your target customers are using (which are often very different to the words that you use) then you are likely to head down the wrong path, and rank highly for search terms that your target customers are not often using.  You may feel you’re doing well having several terms in the top position, but if these terms are not the most relevant you will be missing out on valuable website traffic!

Having said that…

After reading the above you may decide you’re going to get your Keyword Research 100% correct, but a word of caution here.  Keyword research is not an exact science.  There are so many factors to consider (see below), and the weight or priority that should be given to each factor is debatable, so there is never a 100% ‘right’ selection of Keywords to pick…  The best approach we find is to set a sensible length of time for such an important project (at least 4 hours), and then do the best you can within the time limit, then make a start with the keywords selected.  It is not until you move on and start optimising your keywords that you will find out exactly how valuable they are.  The key is to do a good and thorough job to start with but remember you’re not going to reach perfection so don’t waste days over it, then give the list a go, and then go back and monitor the results a month or 3 down the line, and see if the list needs modifying.

You may find a few months later that some keywords are bringing you massive volumes of enquiries, but these enquiries are not too relevant and are actually wasting your sales-people’s valuable time.  Or conversely you may find that some keywords are only bringing you 1 or 2 enquiries each month, but those enquiries are so massive, so valuable, and so in-line with what you offer, that you may decide that these low-volume keywords are actually the most important on your list…

Where to start your Keyword Research

First quickly draw up 3 lists of keywords straight from your ‘head’ without doing any in-depth research, just based on your knowledge of what products and services you provide.  These lists should be under 3 headings:

1. Commercial keywords with buyer intent – these are keywords that you think people looking to buy your product or service may use.  For example, if you sell men’s shoes, you may think up terms like ‘buy black shoes’ ‘hand-made shoes for sale’ etc etc.  These are also often known as ‘transactional keywords‘
2. Informational keywords – these are keywords that people use to research your products or service, but higher up the sales funnel, often quite a while before they’ve actually decided to purchase your product or service.  They are often in the form of a question.  So if you’re an opticians you may include terms like ‘why do I get headaches using my computer’ or ‘what risks are associated with contact lenses’.
3. Navigational keywords – these usually include a brand name, e.g. your company name, or your product’s brand name, or a competitor’s brand name.  These are the terms used by people who already know what they are searching for, and they are just trying to find the website that they already know exists.  These are usually the least valuable to target specifically as you will likely come top of the results for these anyhow if the rest of your SEO is reasonable; however, there are cases where a little known brand may be able to piggy-back some website traffic on the back of a well-known brand by ranking highly for the competitors keyword and making it clear they offer an interesting alternative.  For example if you sell a revolutionary new vacuum cleaner you may want your post that compares the latest dyson model with your model to rank highly for people searching for ‘dyson hoovers’ (just by the way, Hoover is actually just a brand name for a vacuum cleaner 🙂 ).

Next use tools to expand your ‘Keyword Shortlist’

At this stage you want to go broad and consider as many potential keywords and key-phrases as you can.  There are multiple helpful tools out there, one of the most useful for those Commercial Keywords is the Keyword Planner within Google Adwords (you will first need to set up a Google Adwords (PPC) account in order to use this tool, but you can pause your campaign indefinitely immediately after setting up your PPC account if you want to use the tool and not spend anything on Adwords).  Some of the best other tools include https://moz.com/explorer, https://www.semrush.com/features/keyword-research, https://answerthepublic.com (excuse the bearded guy on this one 🙂 !), and many many more.

Google adwords is the best tool for commercial keywords, they tend to focus on these because these are the ones people are prepared to pay for.  Using their ‘suggested bid’ for each keyword you can get a rough idea of the commercial value of each search term – the higher the suggested bid amount, the more other people in your industry are prepared to pay Google to rank for that term.  One awesome feature of the Google Keyword Tool is that you can put your own website address in, or a competitors website in, and leave everything else blank, then click ‘Get Suggestions’ and you will get a list of what keywords are currently bring traffic to your own website, or your competitors website.  This doesn’t necessarily mean that these are the keywords you should focus on, but you can get helpful ideas from seeing what keywords are bringing traffic to your competitors websites, and also knowing where your own website is currently getting it’s traffic from is helpful too.

The Moz tool tries to pull together a whole range of considerations like the click through rate, the search volume, the difficulty to rank for that keyword, and your own personal value of what you think that term will bring you, to give you an overall priority score at the end.  This is easiest to use if you’re on their paid subscription, see this example.  Don’t get too carried away with the priority score on this tool – the the only way to truly assess the priority is to actually test the search terms and see what quality of enquiries they bring in in real life.  Treat the priority score as interesting and helpful information only, and remember that some valuable keywords to you may be given a low priority score – just like any tool, it can only do the best it can with the information it has available.

The SEMRush tools is a good all-rounder and gives a balance to the other tools – they all pull up slightly different results so it’s worth using a few and then amalgamating the lists, if you have time.

The ‘Answer The Public’ tool is an excellent tool for long term keywords and also informational keywords – simply put a short 1 or 2 word search time, maybe just the name of your product or service, and it will instantly come up with long-tail search terms, informational search terms, and alternative terms used to search for the same thing, all arranged into neat categories of similar terms.  See here for a more in-depth overview of the pros and cons of Answer the Public.

A word of caution regarding ‘search volumes’ data that some of the above tools will give you.  This data typically relates to the exact search term used, and yet if you rank highly for that term, the chances are you will rank highly for similar terms, so the traffic from that term may actually be a lot higher.  For example if you have a page that ranks highly for ‘black brogues’, the chances are that someone searching for ‘mens black brogues’ will also find your page.

Estimate £ values for each keyword in your shortlist

Using the (sometimes overwhelming) mass of data obtained using the tools above, now put all of your keywords into 1 long merged shortlist, ideally on a spreadsheet like excel, and try to come up with a ‘Keyword Value’ expressed in £ per month, using a spreadsheet similar to the below example:

Select a Cognate list of Keywords to actively target, track, and review

Now it is time to hand-pick a small, manageable selection of search terms to target, keep regular tabs on, and monitor to track the quality of enquiries they bring.

What number is manageable depends on your available time resource, but generally speaking you’d want at least 3 so that you can capture a variety of search types, and up to 50 is usually quite manageable with the help of tools such as Rank Tank – this awesome free tool lets you instantly check the ranking of up to 100 keywords in your particular country, as well as showing you useful info about who ranks number 1 currently, and who is immediately above and below you, and what page or post is ranking for that search term.

You also ideally need to then set up Google Analytics so that you can see traffic relating to each search term and what pages are generating what and where website visitors are going after arriving on your website, etc.   This will help you to work out which Keywords are bringing you the most relevant website traffic and enquiries, and generally performing the best for you.

Keep your ‘Top Focus Keywords List’ refreshed and manageable as you review it over time – the danger with adding to it over and over again without purging out the lower performers is that you will lose focus on the keywords that really matter.  You will probably find yourself focusing on the lowest ranking search terms on your list and optimising those, but if these terms are not particularly valuable, it would be better not to have these on your ‘Focus List’.  Moving a valuable keyword from position 3 to position 1 on page 1 of Google is likely to bring much higher returns than moving a low value term from page 2 to page 1.

Keep the entire shortlist and bear this in mind when writing content

Whilst we advise above to select a small manageable list to actively track and manage, it does make sense to keep the entire list and review it whenever writing new content, especially when writing content designed for long-tail searches.  If you weave in those long-tail keywords on your list into your content writing, the chances are you will come high on the search rankings for those terms without any further optimisation, because the competition for long-tail keywords is generally so much lower than for short-tail.  Your Google Analytics will still pick up if any of these long-tail keywords happen to drive unexpectedly high traffic levels to your site, and you could at that point add them to your ‘focus list’ of top-targeted keywords.

Summary

I believe that the reason many people find Keyword Research so difficult, is that to do it well you need to combine your gut instinct with accurately researched facts and data – most people are inclined to use one method or the other, combining both methods doesn’t come easily to most people.  The secret is to alternate between using your instincts and then using tools to gather accurate data to support, validate, or disprove your instinctive ‘guesswork’ as to what will work best.  The process goes something like this:

1. Use your intuition to dream up an initial list of keywords that you ‘think’ are valuable
2. Use tools to see how the facts and data available support these initial assumptions (or don’t support them!).  Use tools to expand your initial lists with further suggestions based on publicly available search data.
3. Again use your instincts to select a cognate list of keywords to focus on, bearing in mind that tools don’t always get it right as they don’t have the full insight into your business that you do, they only provide basic information to help you decide
4. Use tools and Google Analytics to gather data on your selection and see which keywords are great and which are not so valuable
5. Drop off any low value keywords from your list if the real-life data shows them not to be valuable, and refresh the list with new keywords as you repeat the process above

Bear in mind that when assessing the performance of Keywords, remember that there are other factors that may influence the data.  User experience (UX) on the page and Conversion Rate Optimisation (CRO) can both have a big impact on what your website visitors will do after landing on your site.  For example, if a prominent call-to-action link at the top of a page is broken, the data from Google Analytics may suggest that visitors are being drawn to this page by a keyword and then ‘bouncing’ (leaving the site without visiting further pages).  You may assume from the Google Analytics data that the Keyword isn’t relevant or helpful due to such a high bounce rate, whereas in actual fact your website visitors may be just getting frustrated with the broken link and abandoning your site for that reason.  Fixing the broken link in this instance could give a very different set of data as to what visitors are doing when they land on your webpage.  This is another example of the need to combine analytical data with common sense and business intuition to get the true picture 🙂 .