Welcome back to aimClear’s continuing coverage of #SESSF. Keyword research is a primary cog in PPC and SEO systems alike. There is no question, leveraging intent-based semantic insight for prospective customers can directly influence lift in the bottom line. Over the past year, though, the addition of Google Instant and intent-based Bing search results has made it necessary to revisit the keyword research process and adapt it to meet new standards.

Bill Hunt, President of Back Azimuth Consulting in his solo session Advanced Keyword Modeling looked at using keyword research data oftentimes pinned against organic and/or PPC keyword performance to determine success and opportunity. He examined several reporting formats and concepts that showcase keyword data in a way that yields results. At a very high level, this process is about conceptualizing how the data you have available whether from keyword suggestion tools, historical PPC performance or the recently altered search engine result pages (SERPs) can be organized, segmented and classified to best give insight into new opportunities. Read on for the full scoop.

When viewing the short and long tail keyword spectrum, it quickly becomes obvious there  is a lot going on with the keywords searchers use to find what they are looking for. Keywords are more than just a list of words.

Advanced Keyword Modeling
All too often there is a mismatch between the ad and what the searcher wants. Bill referred to a scene in film Joe Dirt that highlights this mismatch between the brand and the consumer.

Understanding the search terms and analysis allows you to understand the needs, wants and intent of the searcher.

Understanding the “Voice of the Consumer”
This next generation of analysis enables you to:

• Effectively map your content with query and query intent
• Identify new products and services
• Create and influence PR and social media opportunities

Types of Keyword Models

• Missed opportunity matrix – this is a calculation that examines:
  • What is the opportunity for a given keyword?
  • How many people are searching for it?
  • What share of that traffic are you getting?
  • What could you stand to get if you made improvements?
• Critical keyword performance monitoring
  • How are the words that important to you really performing?
• Co-optimization modeling
  • Paid and organic synergies
• Preferred landing page monitoring & optimization
  • Identifying the set of keywords you want to rank for a particular page
• Snippet optimization
  • The description being served in SERPs for a given keyword or phrase. Is the optimal snippet showing?
• Integrate popular keywords into content pages
  • This drives content creation. Find the popular words and create content around.
• End of life (EOL) product handling
  • Many times products that have reached EOL are removed from the site. Why not create content around this?
• Integrate relevant keywords and landing pages into PR and social media
  • Making sure the list of pages and keywords you desire to rank for are passed through the organization so all departments are on the same page and can leverage the data.
• Product market research

Understanding Searchers
There are typically three types of query types:

• Informational - “search marketing conferences”
• Navigational - “SES San Francisco agenda”
• Transactional - “Register for SES San Francisco”

Purchase Cycle & Intent

• Research product use and information?
• Looking for features and functions?
• Comparing brands?
• Looking for discounts or promotional codes?
• Have they already purchased?

“The average site is seeing a 65% decrease in homepage traffic. People are coming into every nook and cranny of your site directly to what they want through search engines.”

Understanding Qualifiers
Qualifiers are those words that help us isolate a specific opportunity, interest, audience or need. In Bill’s experience he found keywords containing “strategy,” “implementation,” deployment,” and “cost” was most likely a CIO or CTO. What words tell you who your audience is? Connect with the qualifiers searchers are giving you. Begin to mine those keywords and try to understand how many searchers are using those words in your keyword universe.

Brainstorming Keyword Research
In the brainstorming portion of your keyword research it is important to ask questions!

• Why did somebody do this query?
• How do you differentiate?
• What page do you want them to go to?

Brainstorming words is the starting point. Take these top, general keyword phrases and use them as the basis of the next set of actions.

Research Demand & Opportunity

• Leverage one of the keyword research tools available out there to understand variations of keywords and relative search volume for each phrase.
  • Google external keyword tool
  • Wordtracker
  • Wordstream
  • Trellian Keyword Discovery
  • Google trends
  • Google insights for search by city
  • KeywordSpy
  • Spyfu Recon

Keyword Prioritization
Prioritize buckets into high, medium and low categories and use these categories to:

• Control SOV (Share of Voice) levels on specific segments in paid search
• Identify priority words for SEO work
• Advise content developers on best words to include in new content

It is important to allocate effort and budget to keyword prioritization. Bill recommends the following allocation:

• High priority – 60% budet
• Medium priority – 30% budget
• Low priority – 10% budeget

The New Generation of Keyword Research

• Map keywords to predefined action segments
• Use a brainstorming session to discuss and segment words into appropriate purchase lifecycle segments.
• Map to specific relevant pages
• Maximize keyword modeling and search intent
• Create searcher intent models (researching potential searchers keyword variations and assigning them a classification based on what the searcher may be thinking when they do the search)

How Do We Use Searcher Intent?

• Leverage searcher intent modeling for paid search campaign launches
• Leverage data to inform keyword selection at the brand level
• Leverage data when looking to identify content development opportunities
• Leverage data for product identification

Influence Design & Content

• Sites need to use the words their consumers are using
• Create a taxonomy that can be translated into a content & search strategy

Align Keyword Data to Social Media

• Add tier 1 keywords to monitoring
• Understand the “sentiment” of keywords
• Understand the “context” of words
• Compare search volume

Combine Search & Social Data
You may find recurring transient discussions happening far before you expect them to. This may give you an opportunity to drive the conversation in social media or in paid space.

That about does it. Bill proffered some great information regarding keyword modeling. If you have a chance to see him speak in the future, I would highly recommend it. Stick around aimClear blog for more coverage comin’ atcha from #SESSF.

AdWords Keyword Tool may be viable again for SEO. The debilitating short-tail-only problems seem to have been fixed. Long live SEO!

On October 7th 2010, we published a post in aimClear Blog entitled, “R.I.P. Google Keyword Tool. Long Live SEO!” Baris Gultekin, Group Product Manager, Google AdWords, Google, Inc. had just dropped a big stink bomb at #SMX East, confirming what many of us had already figured out.  The omnipresent AdWords Keyword Tool had changed and only was returning keywords Google deemed “commercial.”

We understood the meaning. Modified Broad Match, AdWords newest match type, may have  rendered the need for exhaustive long tail PPC research less important. The pervasive search suggest box was herding users like cats to dumbed down inventory.   Arguably PPC-only marketers didn’t need to see the long tail as specifically any more.  At the time, the most respected PPC trainers in the world were quietly preaching a de-emphasis on intensive keyword focus. Still, a lot of professional search marketers, PPC and SEO alike, were pretty bummed out at the KW tool changes.

Google Listened
Wednesday, December 08, after weeks of complaining from accomplished users and bloggers, Google’s Inside AdWords blog, published, “The Keyword Tool – when you talk, we listen.”  This’s official Google blog for news, information and tips on AdWords, offered,  “We want to thank you for all the feedback you’ve given us on the Keyword Tool.” “The first thing you told us is that you want more keywords that are better targeted to your account and searches, and we’ve made improvements in this area.”

Upon a review of changes, made to Adwords Keyword Tool earlier this month, we’re happy to report that the most dumbass problems seem to have been fixed, making AdWord KW Tool a viable SEO go-to utility once again.

They’ve seemed to have removed the lame-sauce, making the tool serious once once,  by adding depth back into the tool.  Now marketers have access to a relatively deep slice of Google’s mid and long tail search inventory.  Thanks for listening Google. Good decision.

Back in October, the newly modified KW tool only returned 12 semantic permutations worldwide of “Facebook,” logged in, at phrase, broad and exact match. You heard it right…the entire world.

October 7th, 2010

Now using the same settings,  AdWords KW tool yields over 800 keywords in the United States alone, with plenty of search frequency to spare. Whew, that’s a relief.

December 21, 2010

There were a few other improvements announced in the December 08 Inside AdWords “we listen and care” post.  Google added stars to keywords and a useful “View as text button.” These tweaks are nice but not nearly as significant as Google’s sensitivity to negative buzz, responding and dialing the mid and longer-tails back in.

It seems Google even has Feedback form for marketers wanting to contribute to the AdWords Keyword Research Tool.  I plan on using it. Thanks Google. It’s good listening to customers, given the incredible gobs of money we spend on AdWords.  The benefit to SEO is an added bonus, and we’ll take it.  Thank you for listening Baris.

Of course AdWords KW tool is only one of many sources SEOs shold use to locate choice keywords.  Stil it’s nice, once again, to have deeper insight into Google’s search inventory. Long live SEO.

photo credit: jaci XIII

Compare that to what they tell the average user:

When the Prius first came out many owners were slapped with the tree-hugger hippie label. A marketer could assume the same for an electric car, such as the Leaf. What’s wrong with being a hippie or treehugger? Nothing. They’re usually passionate & enthusiastic about their causes, and the products that facilitate them. Find the people on Facebook who are proud of that label and rock their worlds .

Following that “personal label” train of thought… identify people charged with caring for the environment as their profession and the obvious workplaces:

Now that the AdWords keyword tool has been dumbed down, we’ve received many inquiries seeking updated advice regarding keyword research best practices.  For those depending solely on Google’s free tool, the new reality is stark.  In truth, it’s not that radical a change for many pros, because we’ve always used an assortment of tools– mild to wild–to build out our KW baskets.

Over the coming weeks, we’ll be sharing alternative keyword research tools and techniques. This post is the first and highlights a radical, arguably black hat, tool that blew the audience away when I demonstrated it live at #PubCon earlier this month.  So, hold onto your hat (whatever color it may be) and let’s have a look at ScrapeBox’s fabulously intuitive UI for mashing up mainstream search suggest boxes query results for keyword research.

Search Suggest Keyword Research
These days, from iTunes and Facebook to Google, Bing, Amazon and YouTube, search engine proprietors have learned to herd users. Based on popular searches, “search suggest” boxes push users towards the keywords engine owners want users to click.

At search marketing conferences, speakers have encouraged us for years to keep search suggestions in mind for keyword research. However, I’ve never seen any speaker actually show a tool that does so, past screen captures and typing keywords into spreadsheets.  That’s because there were none, commercially available. The most advanced marketers in the world needed to build custom data extraction agents to mine various engines’ suggest boxes.   Proprietary tools are not the stuff a speaker gives away.

That’s all changed now with the discovery of ScrapeBox, a killer-app’ dark arts tool programmed and maintained by a polite and mysterious person in Australia, who won’t tell me his or her first name. That’s probably because while his or her Windows-only app’ has robust suite of white hat SEO, PPC and social media tools, other modules are best described as “black and brilliant.”

We are sharing it because ScrapeBox solves an important problem: easy access to engine-suggested keywords.  You can draw your own conclusion regarding ethical applications of other application features.  aimClear® has no interest, affiliate or other incentive to promote ScrapeBox

How to Mine Suggest Boxes
In the main ScrapeBox UI, select “Scrape.”

Choose the engines. Let’s work with Google and Bing, the two largest mainstream search engines.

Click the “Scrape” button to see suggestions from both Google and Bing. Yes, this is affected by your location and other personalization factors.  However, it’s pretty easy to fool ScrapBox by IP proxy spoof, install it far away computer run by remote access, etc. Keep in mind that not all suggestions are personalized.  The topic of suggest box inventory is worthy of a blog post on its own.

Next, click, “Remove duplicates”  to get rid of overlapping keywords between Bing and Google.

Let’s get fancy.  Transfer the cleaned list back to the left box and run the suggestions through the suggest tools again.  Now we’re really leveraging the genius of this UI.

The scrape offers lots of insight as to the keywords towards which engines herd users.  What makes this so interesting is that we know these keywords get traffic and are, “commercial.” Let’s go deeper. Remove the duplicates once again.

The list is not truly useful until we get some indication of the keywords’ value, in terms of search frequency. Google AdWords keyword tool would work, since we’re dealing nearly completely with suggested terms. We can also use WordTracker. Criticism we’ve heard about WordTracker over the years is not completely fair. WordTracker is a sweet tool for some things, including our purpose here: checking the relative (weighted) values of bucketed KWs.  Let’s check them at phrase match.  The words on the left come from ScrapeBox:

The “Searches” column is not daily searches, but rather a weighted scale, reflecting how many times the keyword shows up in WordTracker’s sampling.

Google indicated last month’s search frequency for “binoculars” was about 33K– knowing this puts the rest of the WordTracker proportional weightings in perspective. Now we understand the relative search frequency for our KWs.

Next, let’s take our keyword bucket in ScrapeBox and find out where there might be YouTube suggested KWs we can market to. Uncheck Google and Yahoo and choose YouTube suggestions.

Transfer the KW list mined from Google and Bing suggest. The list on the left is from our original run of “binoculars” in Google and Bing.  The right hand list consists of the YouTube suggested KWs we mined. Next, as indicated by the arrow, take the larger YouTube list, move it over to the left and run it. We’ve not included the list generated by ScrapeBox, but you get the semantic possibilities.

ScrapBox  rocks product search. Here we run “Nascar” in Amazon, Google Product Search, shopping.com and Yahoo! Shopping Suggestions. This is a list of Nascar products the major engines have blessed with suggest box user herding:

Run that list in YouTube suggest to find associated YouTube keywords we can market the products to  as promoted video or organic YouTube community management.

The Adventure Starts Here
ScrapeBox costs about $100 and is licensed to a single user/computer at a time.  In addition to the software’s more… ummm… colorful features, the ability to easily mine Google, Bing, YouTube and other keyword suggest boxes is priceless. Now, marketers who don’t have access or expertise enough to build their own custom extraction agentshave a way to easily mine and mash suggestions.

Watch for the next installment of Radical Keyword Research Hacks. In Part Two, we’ll demonstrate Wikipedia stemming hacks that will curl your toes.

Attendees sat stunned at #SMX East as Baris Gultekin, Group Product Manager, Google AdWords, Google, Inc. clarified that the ubiquitous AdWords Keyword Tool now only provides keywords Google deems “commercial.” It’s been obvious anyway by degraded results over the last few weeks. This represents a shift of seismic proportions to those who utilize the tool for demographic research purposes, in and outside of AdWords. Here’s an overview of what we just lost and how to get the data anyway. [Author's Note 1/1/2011: The KW tool has been updated. After reading this post, head over to our coverage of AdWords keyword tool fixes.]

Google Backpedals
Google owns the largest first-hand sampling of query data in many parts of the world. Now, breaking rank with 15+ years of holistic search engine open-source tradition, Google has decided marketers should no longer have easy access to the long, or even mid tail data (search inventory), unless the keyword has demonstrated “commercial” characteristics historically.  Effectively, in tandem with the well-established user-herding “suggest” feature, this move consolidates Google’s semantic semi-monopoly into fewer keyword SKUs.

Sadly, this evolution has shut off our ability to mine much meaningful semantic data at all from the tool. This may be great for Google’s bottom line, or not. One might assume financial motivation. It’s bad for marketers who use the tool for a wide range of activities, none the least if buying AdWords and optimizing sites for Google’s organic crawl.  Some would say it’s an incredibly selfish move by Google, yet more evidence that big G’ wants us to optimize AdWords accounts and organic pages, but not too much, too easily or too quickly without spending wads of cash first to figure things out.

This harsh reality has immediate repercussions for how marketers, from small business to large agencies, might leverage their understanding of Google’s inventory for search and business in general. Many posts will be written over the coming weeks illustrating just how lame the tool has become for serious research.  The lack of detail in output is mind-numbing. Choose any query to test and it becomes readily apparent that there’s a lot less beef in the hamburger.

“Facebook” Must Not Be “Commercial” Then
Check out the keyword tool in action below. Google represents the output includes “all keyword ideas, including “adult ideas” and everywhere in the world. Could it be true? Do Google users really only articulate 12 semantic permutations of “Facebook” at phrase, broad and exact match? Eeesh… Obviously that’s a laughable proposition. These 12 keywords are what Google wants to sell as they productize Facebook related queries into AdWords inventory.

Reality Check
Trellians Keyword Discovery’s global premium database suggests nearly 2,000 keywords at phrase match, over 10 pages of output. Even keeping in mind that Trellian is working with a  smaller sampling, it’s clear that our friends at Google have cut us off.

Not For SEO Any More?
For the better part of two decades, marketers have used paid inventory tools like Overture, Panama and Google KW tools to gain insight as to search engines’ inventory.  Many of us thought these insightful oracles would never go away.  After all, the engines need to show us their inventory in order to sell us clicks, right?

Retired Overture Keyword Tool

Paid inventory keyword tools have always been the easiest and most direct method to learn what users search for, those optimizing pages for organic search, used the data to define SEO. This is no longer possible with the AdWords tool, straight out of the box.  Incredibly, Google has found a way to circumvent needing to reveal its inventory in specificity, even to those parties interesting in purchasing AdWords search PPC.  This author never thought we would see the day where such inventory is not available.

Does this mean that big G’s KW tool is no longer reliable for SEO? We think yes. Though some might say that practitioners should now make a point of optimizing for people-herding suggest terms, there will be huge organic competition for these words. It is likely that this new short tail may be unattainable organically for all but the most authoritative and powerful sites.

This serves double duty for Google in that that many advertisers will be forced to buy more AdWords. This rush to purchase the limited “suggest” inventory may drive the cost up even further.   For Google it’s a win, win, win and then win some more.

Not Good for Much of Anything
Savvy marketers have also used search engine paid inventory data for many on and offline purposes. For instance owners advise print ads on how people search. Companies and products are named based on keyword research.  Universities design curriculum based on the subtleties of what students search for.  Detroit uses keyword research to name alert lights in dashboards in the manufacturing design process.

The revolution of search has traditionally been centered on our ability to discover how humans search, the words they use, in what order, the frequency thereof and other subtle semantic permutations.  Unless we want to subscribe (in resignation) to Google’s dumb-ass shortening of the known language universe, this tool borders on useless.  No more will Google’s sampling of human queries be easily available just for the asking.

Dumbing Down Users & Content
Regardless of one’s philosophy of creating content, it might be hard to find an SEO who would suggest that only having access to the brain-dead commercial short tail is a sufficient enough a dataset to advise content creation.  Remember the task is to write content both compelling to humans and diverse enough for optimization.

The suggest feature in Google’s search box, has herded searchers to Google’s prescribed short tail now for a couple of years.  Though this data can be highly personalized based known and black box variables, the suggestions roughly correlate in their obvious banality, to the queerly useless results in the KW tool.  It is a sad world when organization of the planet’s content is thinned for maximum profit.

There are other places to cull the long tail. This much is true. aimClear is switching to Yahoo and Bing’s combined inventory today.  [Author’s note: Trellian, WordStream & WordTracker must be jumping up and down today going “yippee!”] Long live SEO. Here are a few analytic processes and tools, of many, to mine the mid and long tails.

Buy the Data From Google
It’s possible to purchase insight into Google’s mid and long tail inventory.

• Run Google PPC, at modified broad match,  for the keyword tools’ output, in single keyword AdGroups
• Overbid, and only run the keywords just long enough to cull a large enough sampling.
• Take note of inbound paid queries using analytics.  Many of these keywords won’t be revealed in the KW tool.
• Run a query report. The query report has never been fully transparent, in that not all queries are revealed. It is reasonable to think that Google will dumb this report down even further, since it’s a leak of inventory information.
• It’s important to note that there is no search frequency metric attached to organic analytics. This method gets us the keywords, but not the relative important of the KW, ability to sort by monthly searches, etc…

———————–

Microsoft AdLabs
Microsoft has a real opportunity here. Since Yahoo rolled search into Bing, the sampling is much larger.

———————–

WordStream
WordStream may have just become became more important. The methodology and toolset for mining keywords is an interesting approach that many.

———————–

Trellian
Trellian Keyword Discovery is a classic alternative tool, with various databases.  Many swear by the data.

———————–

WordTracker
WordTracker is a cool and campy keyword tool. Mined from meta search engine Dogpile and other sources, WordTracker yields quirky and powerful results from a relatively small sampling.

photo credit: wikioticsIan

Deeply rooted in the same keyword research that advises SEM, nestles the discussion of classic query types and user intent. This conversation has bubbled under the online marketing industry for nearly a decade,  continuously evolving in both sophistication and scope.  The reason is obvious. Leveraging search terms common to consumers is only half the battle-  now more than ever it is crucial to understand nodes of user intent before you shove transactional content at merely walk-by traffic.

While this fascinating conversation can be traced back nearly a decade, the concept really gained traction in 2006 with the boom of social media and a new emphasis on the human behind the keywords. Now we’re talking about ascribing intent to “social segments.” This dialog has proven so important to our daily demographic research that we think revisiting the history of thought-leadership on-topic is in order. Each post below is a worthy read that helps lay groundwork to this fascinating aspect of our semantic craft. We hope you enjoy this walking tour.

25 Classic Query Intent Posts From The Vault
December 18 2006
Why Do People Google Google? Understanding User Data to Measure Searcher Intent
Search Engine Land | Bill Slawski

May 28, 2007
User Intent and Characteristics of Search Queries
SEO by the Sea | Bill Slawski

October 30, 2007
Search Engines and User Query Intent
Search Engine Watch | Eric Enge

February 1, 2008
Two Approaches To Determining Intent: The Wisdom Of Crowds And Personal Values
Search Engine Land | Gord Hotchkiss

February 27, 2008
Create Landing Pages Based on Keyword Intent
CDF Networks | Chad Frederiksen

March 10, 2008
Keyword Intent: Tidy Campaigns Avoid the ‘Dump and Chase’
Search Engine Land | Andrew Goodman

July 17, 2008
Google On User Intent in Search Queries
Search Engine Watch | Nathania Johnson

May 4, 2009
PPC-Man Drowning… Too… Many… Keywords…
Search Engine Land | Andrew Goodman

March 23, 2009
Understanding Your Customers’ Language
Clickz | Mike Grehan

May 27, 2009
The Search Taxonomy: Getting Inside the Mind of the Searcher
SEO Book | Peter Da Vanzo

June 17, 2009
PPC Segmentation: 4 Tips To Better Target Customers
Search Engine Land | Michelle Stern

August 31, 2009
New Research On AdWords Broad-Session-Based Match Type Results
Search Engine Journal | Brian Carter

September 29, 2009
Keyword Commercial Intent – Choosing Keywords That Convert
SEO Wizz | Tim Grice

September 30, 2009
Where to Kickoff Your Keyword Research
Outspoken Media | Rhea Drysdale

November 16, 2009
Leverage AdWords Impression Share Data to Improve Your PPC Performance
Clix Marketing | David Szetela

November 24, 2009
SES Berlin: Google and Microsoft looking for user intent
SearchCowboys | Bas van den Beld

December 7, 2009
Customer Insights via Search Engine Tools from SES Chicago ’09
Search Engine Roundtable | Chris Boggs & Marty Weintraub

December 14, 2009
How Search Engines might Divine the Intent behind Regional Queries vs. Global Queries
SEO by the Sea |Bill Slawski

March 2, 2010
Mining Subtle Query Intent for PPC Conversion
aimClear | Lauren Litwinka

March 3, 2010
Trends, Zombiebait & Sideways Keyword Research
aimClear | Matt Peterson

March 9, 2010
Targeting Multiple Keywords vs. Singular Keyword Focus
SEOmoz | Rand Fishkin

February 12, 2010
Understanding Content Relevance in Relation to Keyword Intent
KOMARKETING ASSOCIATES | Derek Edmond

And last but not least…

March 25, 2010 (upcoming)
The Evolution of Search: End Users Signal the Way
SES NY Keynote | Yusuf Mehdi

Though this little gem hasn’t taken place yet, we look forward to learning more about Bing’s take on query types and user intent when aimClear attends Search Engine Strategies in New York City a week from today. You might say we’re a little excited for the trip. You might say that.

photo credit: huangjiahui

There has been much written surrounding classic query types. In review, conventional wisdom dictates that there are basically three kinds:

• Informational - suggests the searcher is on the broad quest for knowledge about a topic of interest (medical symptoms, film trivia, tips on housebreaking puppies, etc.).

• Navigational - indicates the searcher is on the hunt for a specific website, usually dedicated to a specific brand (a search for Pepsi usually means the searcher’s destination is Pepsi.com).

• Transactional - connotes the searcher is ready to make a purchase online (typically formatted like so: “buy [product keyword] online”]).

However, things are not always that cut and dry. In these days of four and five word searches, there are subtle shades of intent that aren’t always obvious to the marketer.  For PPC jockeys, spending cold cash to separate out live customers can be a crucial matter of budget.  Let’s take a look at four ways to identify and leverage some subtle intent words out there to help you focus time and money on PPC keywords that should stand a decent better of converting.

Background
When crafting PPC campaigns, it can be tempting to choose any keywords that seem highly relevant to your client’s products and services. Targeting people who search for what Client X has to offer makes sense at face value: doctor is to “surgery” as restaurant owner is to “Indian food” as clothing store is to “yellow sweater,” right? Tight industry-associated  keywords reach out to every Tom, Dick and Harry interested in your commodity, and the glistening pot of search volume they generate is enough to make any client glassy-eyed with glee. But as the marketer-of-record (he/she who will be judged on actual ROI) are you in tune with how deep in the purchase funnel these searches are?

4 Types of Keywords that Help Define User Intent
The objective here is to get a handle on the level of intent behind a person’s search.  There won’t always be blatantly obvious “where can I buy a” or “where can I find a” transactional prefixes to distinguish walk-by traffic from potential customers.

1) Profession-based keywords. Determining user intent can be as simple as separating commodity-based keywords from the people who provide them. For example, someone searching for “gynecology” may have a very different end-goal compared to someone searching for “gynecologist.” The former is more informational whereas the latter suggests a sense of direction, of urgency, of preparedness to take next steps further into the funnel. This works for a healthy chunk of industry-based descriptive nouns and their performers:

2) Clarification keywords. The keywords people tack onto informational short tail queries can help define intent, too. For example, “kidney stone” as a stand-alone keyword is too generic to convey intent one way or the other, but consider:

• kidney stone doctor
• kidney stone specialist
• kidney stone medical center
• kidney stone hospital
• kidney stone emergency clinic
• kidney stone 24 hour clinic

The bold terms up the intent value of the basic keywords tremendously. This person is no longer searching for information about kidney stones… they’re looking for someone or someplace where they can find help.  These are “buyers,” considerably deeper in the purchase funnel, and as such, worth marketing to.

3) Transactional keywords. Terms such as “cheap,” “discount,” “clearance,” and “new” convey intent in a very direct way. A query for “discount digital camera” immediately places the searcher deeper in the buying funnel as opposed to simply “digital camera.”

4) Geo-specific keywords. Tacking a city or zip code to a generic keyword also strengthens the level of intent.  A search for “Indian food New York City” is much more telling of a person’s intent as compared to simply “Indian food.”

Once you identify your profession-based keywords, clarification keywords, transactional keywords and geo-specific keywords, dump them all into a tab on an excel sheet. For the sake of this post, let’s run with the example of a client who happens to be a veterinarian. Here’s some examples of correlating intent words:

Incorporating Intent Keywords into your PPC Ads
Either manually or with a keyword list generating tool, mash up carefully thought out intent words with any keywords that apply to your industry.

Run the mashed up keywords though the AdWords Keyword Tool to obtain literal search volume on terms that indicate a readiness to convert:

Now head further out on the long tail by mashing up three buckets: industry words, intent words and geo-specific words:

Segmenting the Results
Export the keyword mash to an Excel spreadsheet:

Create tabs in the spreadsheet as you see fit:

From there create keyword clusters with correlating data- for example, a chunk of the spreadsheet dedicated to search terms with the intent word “specialist” and a chunk for “hospital.”

Armed with these intent-driven keywords and their literal inventory, you’re ready to create PPC ads that specifically target people ready to convert.

Sure, adding the the keyphrase “new york” as a qualifier to “veterinary hospital” whittles the global search volume from 368,000 down to 390, but those 390 people are your new best friends (and potential customers). The last thing you want to do with your PPC budget is spend it educating the sweaty masses at the top of the funnel who aren’t ready or even looking to convert. Your goal should be to identify those in the prone-to-purchase sweet spot and pounce with your effective PPC ads.

Let the record show that it certainly isn’t a  waste to market to informational keywords so long as the process is undertaken in a very intentionally way. PPC ads to informational  searches are terrific for engaging users, making friends, building brands and even selling at a reduced conversion rate and (often) higher CPA. On the other hand, marketing specifically to intent words will filter your keyword bucket to such a great extent that you can expect fewer but better converting searches.
photo credit: AlishaV

SEO’s cut their teeth in the keyword research process. The first few months of an SEO’s development is often spent in the “saltmines” researching and testing nearly endless variations of words and phrases in Keyword Discovery, Wordtracker and Google’s keyword tool. Are users searching for “trash cans” or “trash bins”? We learn that the spectrum of keyword inventory has both a head and a (sometimes long) tail. Title tags, meta descriptions and page copy alike are thoroughly advised by massive excel docs, organized by topical nodes and user search volume.

We strive for a golden ratio of content relevance, old-school ad copy “hook”, and maximized search inventory weighed against attainability. SMX Advanced’s Keyword Research Artistry session promised to examine  keyword research as a holistic process.

Session Moderator Chris Sherman of Search Engine Land explained that while past keyword research focused heavily on tools, best practices keyword research requires examination of user behavior. This session was about exploring the  ways of discovering the unique & compelling keywords that people search for. Presenting first was Christine Churchill of Key Relevance with Tips to Expand Your Keyword Galaxy. She explained her presentation was not just about tools but techniques. Christine asked how many audience members regularly look at site search data.

She is an advocate of examining site search data to see what users are looking for and what they can’t find. Google Analytics allows you to incorporate Site Search, although many other analytics packages allow for integration. Christine really likes keyword trend tools, especially Google Trends. Watch for changes in the search volume index in step with events like the economic downturn. She gave “coupons” as an example.

If you are signed in, Google Trends allows for granular examination and keyword extrapolation, and most people don’t go this far. You can even examine keyword frequencies by city area. She said to examine the top searches but pay special attention to rising searches.

Export it all to a CSV. Google Hot Trends is another tool with valuable search user insight. Christine gives an example of looking through Hot Trends results to advise blog posts, examining branching topic buzz surrounding the Preakness winner.

She also likes Yahoo Buzz as a one stop shop for advising research based on the most popular topics being searched for on Yahoo. She also uses Facebook Lexicon, which gives amazing insight to follow trends being written about on user Facebook walls.

Christine also likes several Twitter tools, and while she acknowledges their keyword tools are in their infancy, she expects great things in the near future. Twitter search is real time search results and trending topics. 3rd party tools such as Twitscoop do real time graphical topic trending and weighted tag clouds. Tweetvolume provides insight on the “phrasology” people prefer to use on twitter.

She did a search on Hybrid car vs Hybrid Auto vs Hybrid Vehicle and she found the results to be surprising. Site Volume aggregates trending data in Digg, Myspace, Youtube, Flickr and Twitter; it’s a valuable tool for monitoring brand-specific chatter as well.  Other briefly discussed tools included TweetBeep, Twhirl, and Tweetdeck. Christine presented some of her favorite Wikipedia tools. Wikirank displays the most popular trends in Wikipedia.

It also allows you to key in phrases and it will display how many times a topic was viewed during a period of time. VisWiki and Eyeplorer are visual tools to provide additional insight with a graphical display of  your keywords. Christine stressed that you shouldn’t just use these keyword tools in isolation, develop a process of combining multiple tools to feed each other for additional keyword inspiration.

Next was Gab Goldenberg, owner of SEO ROI Services. Gab wants to take us through long tail keyword research, stuff that doesn’t show up typically in keyword research tools. Gab says he will trade insoles for constructive criticism on his presentation, genius. Gab presented a case study on a client of his in the scrapbooking industry.

The average CPC for competitor’s was in the $1.00-2.00 range,  but Gab found long tail keywords that brought his averaged CPC to around $.24. How did he find these keywords? Gab explains, “he’s literate”, he actually reads client industry publications and product pages for keyword insight.

Gab used the exact same ads on Google search and the Google search network and he got identical positions, yet he had a 3x higher clickthrough rate on the Google search network. Gab provided his longtail research techniques:

• Gab showed a picture of a red Ducati 1098 motorcycle and asked the audience  to spend a minute describing what they saw to their neighbor. Each person on the panel had different ways of describing the bike, Christine saw shock absorbers. Use this technique to tap into long tail keywords with low competition.

• Find user generated content sites and just read them. A more efficient way of doing this is SEOmoz’s TermExtractor Tool, you can make a custom scraper to identify these longtail words on a large scale.

• There is a hidden market for keywords, much like you can find in Compete and SpyFu. However, Gab says they suffer from “broken telephone syndrome” , something is lost in translation. You can solve this problem by going direct for user perspective. Buy, trade and barter with indirect competitors, bloggers and non-profits. Make sure you pay these people a decent amount and provide an NDA.

• Be aware that some people give away their data, look at popular items and related topics in Amazon.com. You can create search operator footprints for popular searches, bestseller lists, and hot trends in Google.

• SEO-Savvy competitor’s navigation – look at what categories your competitor’s use, what the top content is within competitor’s sites and publications. You can use the Google keyword tool to extract terms from top content URL’s to gain insight.

Speaking next was Taylor Pratt, search marketing specialist with nFusion. Taylor presented a 6 part recipe for the keyword research process. Taylor puts things into perspective by explaining that every company, be it Microsoft or a small non-profit needs to do keyword research.

Ingredient 1 – Start with the right ingredients Look at the following keyword research metrics: Search volume - how often is a keyword searched for per month? Relevance – if a user clicks will they find what they need? Competition – what can be realistically attained in the SERPs? Taylor gives an example of his algorithm where he ranks each metric for a specific keyword on a 1-10 scale and then gives an average score for that keyword.

Ingredient 2 – Use the right cookware Taylor likes Raven SEO Tools, Google Adwords Tool, Google Suggest and SEOBook. You can export these keyword lists to an excel spreadsheet and then run them through the algorithm. Ingredient 3 – Check the Oven: Analytics Look at the keywords sending the most conversions, examine your long tail organic and paid keywords. Examine user behavior with your site’s internal search. Watch your top referrers and look at your top content.

Ingredient 4 – Who’s at the Table? Use Yahoo Answers to see how your audience describes your product. Listen to how users on your Facebook group describe the product to their friends. Try and leave the branding stuff at the table, you’re going to rank for it anyways, don’t waste the precious real estate.

Ingredient 5 – Kitchen help: Customers and Employees Talk to your customer service employees and tech support. Ask them what language and words customers used to describe their issues. Don’t overlook the knowledge of people that deal firsthand with the customers. “Contact us” forms are also a goldmine for insight into how users communicate their needs. Run user tests, offering a 30 dollar gift card for valuable user perspective is a bargain.

Ingredient 6 – Did it taste all right? Use PPC testing results to examine success in conversions across the entire targeted keyword spectrum. Look at time on site within pages, how long are people interacting with the site per keywords?

The Number One ingredient to avoid is professional blindness: experts in the industry alienating search users with specific acronyms and cryptic industry language. Show the results of your tests to steer your clients in the right direction when it comes to these keywords. I found the Keyword Research Artistry session to be an extremely interesting blend of classic SEO tool techniques with fascinating conceptual practices for understanding what your customers are saying in real time. A truly holistic process.

These days predictive utilities like Trellian, WordTracker and AdWords are only half the keyword research story. State of the art processes combine these standard search predict tools’ output with “what we already know,” by mashing in site-specific datasets at the keyword level.

Additional keyword data might include statistics culled from organic analytics (KW traffic, conversion, behavior criteria), PPC history (CPC, CTR, conversion, etc…) and SEOmoz’s Linkscape (SEO competitiveness-predict).

We all know it’s theoretically useful to “advise” the keyword selection process by aggregating datasets. Very few SEMs actually pull it off. This article will guide you through the process of using spreadsheets to pilot advanced techniques, for API automation later.

What Is Keyword Research For Anyway?
Classic keyword research is all about selecting words for SEO, PPC and offline clarity. However there are other compelling objectives attainable by analytic mashups.

This might include wire-framing SEO based on page strength, taking  SERPs competitiveness into account,  discovering natural search clues for more efficient PPC and a multitude of  seriously useful applications. At the end of the day keyword research is about increasing ROI in All Channels. The approach proffered here can yield extremely deep insights.

Not Your Mother’s Keyword Basket
Using multiple datasets relies on the power of  mashing data together and stacked sorting routines. Even within themselves,  most web based tools only provide a single dimension of data sorting.

For instance AdWords Keyword Tool lets us sort by any single attribute at a time, like estimated average monthly searches or CPC. Creative demographic research artists rock these tools by exporting to .CSV, adding additional keyword-level metrics and executing multiple sorts.

Example: Intersection of Low PPC Cost & Search Frequency
Start with an AdWords keyword basket You’ll find available dataset attributes, in the show/hide columns menu.

Export keyword Basket to an Excel .CSV

Now use the data sort functionality in Excel to identify the intersection of low PPC cost & search frequency. This particular sort is not possible in the AdWords tool’s web UI, because it has 2 steps. That said, it’s super easy in Excel.

The Sort Reveals 269, 400 Annual SEO Keywords, estimated  @ $.05 CPC.

Sorting Analytic Mashups!
Now it’s time to aggregate  keyword predict, organic analytics & PPC datasets into a single spreadsheet for cross channel data sorting. It’s very simple actually. Just create new columns on the spreadsheet and enter each keyword’s data next to the keyword basket’s output.

Import/Collate Each Keyword’s Organic History (Remember, this data is from your site’s web analytics like Google Analytics, ClickTracks, Enquisite, etc…)

• Traffic Frequency
• Conversion Frequency
• Conversion Ratio
• Engagement (Time & Page Views)

Import/Collate Each Keyword’s PPC History (This data comes from your PPC Accounts)

• Click Through Count
• Click Through Ratio (CTR)
• Actual Cost Per Click (CPC)
• Conversion Count
• Conversion Ratio
• Cost Per Conversion

Import Average mR of Each Keyword’s Top 3 Organic SERPs
mR means “MozRank,” which is the LinkScape analogy for Google’s infamous PageRank metric. It’s is SEOmoz toolset’s contribution to measuring page value and organic competitiveness.

Query each each keyword and take the average mR of the top 3 organic results with personalized search turned off. Enter the value into the spreadsheet next to each keyword, as a viable estimate of the SERPs competitiveness for that keyword.

Ready to fly? Now that you have the the predictive, organic & PPC historical data for each keyword on one spreadsheet, there’s  awesome collating power at your data-sorting fingertips. Here is a partial list of “sorts” to cross-advise organic and PPC marketing efforts

High Organic Traffic / Low Organic Conversion
Use this report to identify where organic traffic intersects with low conversion. Obviously the list of  keywords portends site-needs, in order to convert traffic.

Higher Predict Frequency / Lower Organic Competitiveness
This handy little sort finds the intersection of AdWords search volume predict, with easy organic SERPs for the keyword.

Lower Predicted PPC Cost Per Click / High Organic Conversion Ratio
Find low cost keywords that are proven to convert well organically. This is a classic aggregated dataset keyword research report.

High PPC Conversion Frequency / High Organic Traffic / Low Organic Conversion
This triple-sort identifies deficiencies in organic conversion, for keyword where PPC has proven the site converts.  It’s especially useful to seek targets for improvement where there’s already good organic traffic to the site.

Other Fun Keyword Attributes For Research Mashups
There are any number of other (provocative) attributes which comprise organic and PPC historical datasets. They can be mashed into our aggregated-sort approach for spectacular clarity.

• Behavioral Triggers & Funnels
• Users’ Proclivity To Socialize Content
• Geographic Data
• Page Strength (Page on Which The SEO Is Placed)

Pioneering algorithms using spreadsheets is easy.  Practicing these techniques without a number crunching army is quite another story. After refining advanced keyword research sorting with exported data, bring the processes to your development team for automation on the various APIs that comprise keyword basket, organic analytic and PPC reports.

Keep in mind that not every dataset is available for every website. Use what’s available to make keyword research even more valuable. These days predictive keyword basket tools are only part of the show. Also mash in keyword-level metrics for cross-channel sorting, using both what is predictable and known.

While classic keyword research counts phrase permutations users search for, Buzz Pocket Mining measures the individual and community-wide social graph. The good news is, incredibly powerful BPM tools are totally free!

“Social graph” simply means categorizing definable pockets of members’ participatory chatter as folks interact, share content, make friends and live lives together. Often expressed by a tag cloud, Buzz Pockets are how we index topical interests, as expressed by users.

The deliverables for BPM research include content & category ideas, keyword baskets and insight regarding influential users. Find content by tag and scrape it for keyword clusters using Google’s External AdWords Keyword Tool, to build potent keyword lists rooted in actual search frequency. It’s money in the bank.

PPC & SEO Keyword Research
We use insight gained from Buzz Pocket Mining to advise traditional keyword research processes by discovering uber-thesaurus stemming ideas, targeted to the collective’s most passionate interests as expressed by behavior. Confused yet? Here’s a great StumbleUpon Buzz Pocket Mining tutorial to make things clearer.

BPM datasets are priceless because, as a starting point, listening to what folks actually care about in specific networking and bookmark channels inspires content that’s easy to promote. All the while BMP research provides an insider-peek of each community’s authority figures. They’re likely to enjoy (and propagate) content we already know they’ll like. It’s a win/win deal.

Use Paid Advertising Tools for Free!
Just as with traditional keyword research, killer BPM tools can often be found at the demographic targeting front end of paid advertising platforms. Ever since Overture’s now-retired keyword tool, we’ve been using search engines PPC advertising interfaces to research SEO (non-paid). Facebook and StumbleUpon are terrific examples of using front end paid platform tools to gain insight for organic social media optimization.

20 Totally Free Buzz Pocket Mining Tools
StumbleUpon Recently & All Time Most Popular Hot Tags
This is one of my all time favorite drool-tools. Though SU’s traffic driving propensity is seemingly being siphoned off by rampant migratory Twitterers and Facebook users, the 7 million person sampling still provides very useful insight. Create content advised by this tool and check out how easy it is to market all over the Internet.

Also drill down into any tag and take note of authority users, with influence enough to push content hot, should they take an interest. Come back to them later to share the content you know they’ll like.

Facebook Lexicon
Lexicon is a Facebook tool following language trends all over Facebook. Looking at word and phrase usages on profile, group and event walls, compare the popularity of several words on Facebook walls.
The example compares relative wall buzz for Obama, Sex, Football and the Beatles.

Twitter Search Engines
Twitter has touched off a micro blogging revolution and is rapidly heading mainstream. Mine real time buzz about topics, brands and people in this flammable crucible of human participation. It’s a marketers paradise for insight and influences. This is true and street-level word of mouth marketing. Because the Twitter API is so accessible, many third party tools have cropped up free of cost. (follow us @aimClear).

• search.twitter.com: Now with people search.
• Green Tweets: Tracks green and environmental tweets, relying on semantic natural language analysis as opposed to keywords. Look for other niche’ Twitter search engines to emerge over time.
• Searching for Interesting Topics on Twitter: Seek out interesting users and comments. This is a cool tool to locate like minded Twits.
• Twellow.com: A People Directory: Yellow pages approach to categorizing Tweople. Create an account to maintain your own record.
• Twitturls: Links, pictures and videos hot on Twitter.
• Tweetag: Watch trends in nested tag clouds.

Technorati Blog Search
One of the original BPM tools, use Technorati to find content related to any topic or URL. Though Technorati has gone though several generations of unfocused evolution, it’s still a cool stop to gauge the blogosphere’s buzz. Sort results by authority, tags, language, videos, etc…

Google Advanced Blog Search
Some believe that Google Blog Search has dealt a knock out blow to Technorati. No doubt blogsearch.google.com is a very cool conduit to identifying blogs and bloggers that might matter.

Facebook Create Campaign Tool
In the finest tradition of using free campaign creation tools to advise content creation, this tool is a gem. In the example screen capture here, I want to reach people between 18 and 40 years old in the United States who graduated from Indiana and like Basketball, Playing Basketball, College Basketball, or Play Basketball.

StumbleUpon Create Campaign Tool
Be a paid advertising UI squatter and mine buzz pockets for free. Just act like you’re going to create a campaign. Flipping through the category and subcategories, select the user interest groups you want to target.

StumbleUpon Discovery Bookmarking Tool
Join SU, install the toolbar and be the first to thumbs-up content.

The dropdown menu is among the most colorful category suggestion tools in existence. Search SU (or other communities) on suggested tags to find wonderful content, blog posts and authority users in the keyword space.

LinkedIn Answers Categories (must be logged in to LinkedIn)
Looking for a terrific business categorizations? Check out the way LinkedIn organizes businesses. While the “Answers section” is a walled garden, scrape the pages, post as your own private HTML and crawl with Google Keywords tool to mine outstanding keyword baskets by social input to each business category and subcategory.

Google Hot Trends
Google Trends provides insights into broad search patterns. Take note that approximations are used for extrapolating these results. I love starting with trends-results and going back to the SU socially advised search function to find associated tags. Find content to scrape using Google AdWords Keyword tool and kaChing.

TrackUr: (OK not free but great) From tool maker and rep’ management authority Andy Beal, TrackUr is also killer as a buzz tracker front end for PPC content network laser-targeting. Search the indexable Internet on a small grid of keywords to locate every page holding the words.

Export the list to .csv and paste them into Google Content Network’s site targeting tool, to reveal which ones are AdSense publishers. Hat tip to David Szetala over at ClixMarketing for his content network magic.

Google Insights for Search
With Google Insights for Search, you can compare search volume patterns across specific regions, categories and time frames. Narrow data to specific categories, like finance, health, and sports. Insights can help you determine which messages resonate best over time and geographically.

Yahoo! Buzz – Top Overall Searches
Another social search engine in any category. A subject’s buzz score is the percentage of Yahoo! users searching for that subject on a given day, multiplied by a constant to make the number easier to read.

Del.icio.us Popular Tags
Delicious is a classic social bookmarking service allowing users to tag, save, manage and share web pages from a single source.

Amazon – Most Popular Tags
The Amazon tag cloud depicts users’ categorization of products. It provides cool shopping insight.

SEOmoz Popular Searches Tool
Rand Fishkin’s free tool is groovy in that it archives Google Hot Trends, Yahoo Buzz – Top Overall Searches, Del.icio.us Popular Tags, Amazon Popular tags and other BPM tag clouds. Search each of the channels by date for perspective on the evolution of hot.

Buzz Pocket Mining
While classic keyword research indexes keyword permutations users search for, Buzz Pocket Mining measures individual and community-wide interactions. Seriously powerful BPM tools are totally free. Happy mining everybody, and don’t forget to put on the hip waders!