It’s 3am on a Sunday morning, and I just wrote a 20 question survey that I want to share with you to better understand the agony and the ecstasy of one piece of the insight value chain – qualitative research.

I believe in the value of talking to real people about the products and services I help my clients develop and sell. I believe in being a member of a reality-based community. And I believe that evidence and empiricism often lead not only to incremental improvement, but also to great leaps of imagination and invention.

I believe in it so strongly that I’m building a business around it, and working only with clients who are willing to give evidence a shot.

Still, I feel saddled with a tradition of “research” for gathering this evidence.  As a former partner in a research-based brand consultancy, I can tell you that the standard operating procedures of most researchers are built on tradition, on a desire to be taken seriously as something professional and almost science-y, balanced with a need to be flexible and creative and responsive.

But tradition is not the only reason that the standard operating procedure has become standard.  The biggest reason we cling to the 8 person focus group, conducted in a grey room in front of a grey mirror with grey people, is that we are dependent on a research supply chain that is broken.  And there are four reasons, all mutually dependent and a bit circular, why this chain is broken.

How Sample is Collected is Broken.

Random sampling of people walking past you on the street corner, mall intercepts, fliers on the bulletin board at the community center or grocery store, direct mail solicitation, robo-calling, or registering online – these are all legitimate, time-tested means of collecting sample.

Most recruiters focus on the most general of general population folks when they are standing in malls, or calling people at dinner time.  There’s never any trouble finding stay-at-home moms, retirees, students or the marginally employed (which are, admittedly, a plentiful piece of the economy these days).  But start getting specific – not just Walmart Moms, but Walmart moms who own an iPhone and a Prius – and things get tricky.

Or start looking for people who aren’t at the mall at 3pm on a Wednesday: office managers, shipping managers, IT directors, CEOs, lawyers or accountants or doctors.  Or for people who aren’t at home in front of the TV at 10am or 10pm – college students, retail workers, bartenders, people with social lives. It starts getting tougher, and then you have only two choices – pray for a list, or staff up to start dialing the phone book in search of the people you need to talk to.

Maybe you’re lucky, maybe there is a customer or prospect database. Many clients aren’t direct to consumer sellers, so they often don’t have this data.  Yet even when they do, you begin to rethink the definition of luck.  Dummy phone numbers, misspelled email addresses, out of date addresses, incorrectly entered data.  I’ve had to quote clients a 1000:1 ratio on some lists for number of calls we’ll have to make to find a single respondent. Because even if the data is good, you don’t know when is best to reach them, if they’ll be interested in participating, if they’ll be available on your schedule, or if they’ll even qualify once they’re put through the screener.

In other words, sample is tough to get, and even tougher to ensure is of quality.

Screeners and Profiles are Broken.

If you’ve ever drafted a screener you’ll know that they seem to go in one of two directions: too specific or too simplistic.

The too specific screener may actually get you exactly the person you think you want to speak to (short of having an accurate, up to date, well tended customer or prospect list).  But it will strike fear and panic into the hearts of any researcher or planner to have to sit patiently as the days go by without a single recruit – because of the sample problems I outlined above.

The too specific screener also begins to set up every good-enough respondent as a scapegoat, branded with the “Not the Target” mark of the client or agency who is looking more for validation than for a learning experience.

But on the other hand, the simplistic screener is a blunt instrument: “do you buy this product and are you available on Monday” may not give you enough information about the potential respondent to know whether they’ll be right for the kind of research you want to do.

Regardless of whether the screener is good or bad, the biggest problem with it is that it is a script from which we do not allow or trust our research partners to deviate. There’s no improvisation in recruiting – we quite literally say “TERMINATE” on screener questions where the wrong answer leads to disqualification. Not only is that a sudden stop to a phone call from a stranger, one that ends in what is unmistakably a rejection, but it’s also a phone call the recruiter doesn’t get paid for.  Recruiters get paid by respondent recruited, not by time spent calling people or by effort. Therefore, recruiters want the most relaxed criteria they can get – ensuring they have to make the fewest number of calls to “fill the groups” – and therefore to get paid.

Project Management is Broken.

Because of all the difficulty in getting a large enough sample to recruit from, and in defining a subset of that sample that you want to include in your research in a way that is specific enough to get you who you want, but not so specific that you can’t get anyone, researchers, strategists, and people like me are constrained in multiple ways.

• It’s batch-and-queue, baby. Screeners must be drafted and approved by clients. Then they have to be handed off to the recruiter, who will inevitably spend a day asking more questions to clarify the screener so they can preemptively reduce errors when the phone bank gets to work.  Recruiters will almost uniformly tell you that all recruits take 2 weeks, some longer.  This isn’t strictly speaking true, because they will almost always be able to “fill the group” – whether you give them a shitty list of 2000 names and 4 days, or no list at all and 2 weeks.  They will generally recruit up to the last minute – and often don’t put the screener in field right away if they feel they have enough time to spare.
• It’s a black box. Once the screener has been “programmed” and the call center activated, there’s no transparency into progress.  Once a day, beginning on whatever day they get the first confirmed respondent, most recruiters  will begin to share an excel spreadsheet with respondent names and their answers to the screener’s questions.  If there are people being ‘terminated’, you don’t see them.  Only if lots of people start to disqualify on one particular question will the recruiter call their client to talk about those disqualifications and ask to “relax the recruit” in order to make numbers.  As a buyer of field services, I can’t see how those criteria are affecting the recruit, and so I can’t take action to help my recruiter course correct. In the meantime, I’ll have nervous clients or colleagues wanting updates on the recruit, and I simply have to wait until the recruiter calls me back or emails me an update that is, almost by definition, an incomplete picture of the situation. But it’s no wonder that recruiters only give you daily updates – they’re working the phones too hard to get in touch on progress or check in with ideas about how to improve the situation.

Qualitative Research Design is Broken.

This is a topic I can go on and on about. But for the purposes of this post – study design is broken because it is created with the realities of the research supply chain in mind, and this can trump quality of the learning experience.  Rather than thinking through the right kinds of people to meet and learn from, we start thinking about cities: Which town will have early tech adopters aplenty?  Where do people tend to shop at big box stores?  Which cities over-index on soap opera viewership?  The question of location is dictated by two considerations: the need to do face to face research that lots of people can observe, and the need to find a recruiting partner who has a sample database with our kinds of people in it.  At the same time, we’re thinking about segments that are distinct enough from one another, but still all within the reach of our clients, from whom we’ll learn something useful.  They’ve also got to be distinctive enough that they look like different groups when you’re sitting on the other side of the glass, but not so unique that they’re a needle in a haystack.

In other words, we’re trying to guess what our recruiting partners can get us that is also relevant to our clients.  And we default to using the recruiter’s crappy facilities because you want to keep them on side by paying them a rental fee plus the head count fee, and besides, in for a penny, in for a pound.  Plus it’s easier to corral our clients into a dark room with bowls of Chex Mix and a mini-fridge full of sodas than it is to have them tag along for every site visit, ride along, and in-home interview.

How We Treat “Respondents” is Broken.

Even the way we treat respondents is batch and queue. It begins with a phone survey, followed by an email with instructions for getting to and preparing for a group; showing up 15 minutes before a group begins, filling out more forms, sitting in a waiting room; names are called and 1 or 2 people are left behind; they’re directed into a room where they’re told to turn off cell phones and sit in a chair and put on a name tag.  A moderator comes into the room and asks questions – depending on how good she is, it’ll either seem like an interrogation or a conversation.  Sometimes it’ll be fun.  We’ll provide sodas and food, but not schedule a break in the 2 hours for a trip to the bathroom.  The moderator will cut people off if they talk too much, and then when the time is up, that’ll be it.

Respondents have the vague sense of being watched, but mostly forget about what sits behind the giant mirror behind the moderator, unless of course there’s a tap at the glass, or a note is passed in, or she asks someone to speak up because the microphones aren’t picking up soft or low voices.  Then they’re ushered back into the waiting room, asked to sign for their ‘incentive’, and head home, not sure whether this was helpful to anyone, or what will come of it.  And this is how to “get paid for your opinions.”

We spend about $100 a head just to find these people.  And then we haggle with them about how much it’s worth to them to spend 90 minutes or 2 hours with us discussing topics ranging from nearly irrelevant to them to deeply personal and private.  We assess their value – the 22 year old part-time employed mom is worth $75 for 2 hours, the 35 year old IT director is worth $150. This isn’t about their value to our clients, but about their value in the world.  We reason, that mom would be lucky to make $35 an hour; whereas that IT director might actually need to be paid a bit more to show up if his title is senior enough.  We don’t stop to think that the mom spends thousands of dollars every year at her local grocery store, while that IT director may not actually be the one who signs off on the purchase of a new CRM system.  Her value to a consumer packaged good brand is definite; his is tenuous.  But we don’t think about them as valued customers or prospects, we think about them as short term employees.  We should be engaging them as collaborators that our clients don’t otherwise have (or think they have, or want to have) direct access to.

Trust is Broken.

Why are we doing things this way? It’s a problem of trust. Clients don’t trust agencies to do ‘unbiased’ research on their own ideas. Agencies don’t trust researchers not to kill a good idea. Researchers don’t trust recruiters to get them the ‘right’ respondents. And we don’t trust respondents to be smart, creative, collaborative, or frankly, even experts in their own lives. So we over process the process, we constantly question and negotiate the investment of money and time, and rather than a true best effort, we do what we think is possible, rather than what is best.

So why do we do keep doing it??

Look, we – planners, strategists, designers, makers, brands – still commission qualitative research, we still write screeners, use recruiters, hire moderators, sit in back rooms and listen as questions on discussion guides are asked.

After all, we still need evidence. We need that gut check, that reality check. We need to learn *something* or risk making unfounded decisions, decisions purely on personal taste or ego.  While we all seem to have come to consensus on the notion that you can’t ask consumers what they will want in the future, we all seem to also agree that the person who comes up with an idea is inherently biased in its favor, even when it’s shit.  So we hope for the wisdom of the crowd in adjudicating the value of an idea.

Our path to this adjudication strips crowd wisdom of most of its value. It’s too stressful, too opaque, too costly, too time consuming and inefficient.  So by the time everyone is huddled around laptops and gobbling down fistfuls of M&Ms on some Tuesday evening in Cleveland, it’s no wonder they’re not really listening.  They’ve spent all their energy worrying about the recruit, fretting about the screeners and the guides, trying to keep costs down, and herding people onto planes to come watch dull people in dull rooms talk about dull things.  The people we’ve recruited to participate are kept in the dark about our intentions, treated as ‘respondents’ rather than as partners or collaborators.  And the people who recruited them are hidebound by their own business model, with little incentive or opportunity to collaborate, and a strong incentive to appear regimented when they’re really just tapdancing as fast as they can.

I – and my team – want to change that. We want to find out where the value chain is broken for you, and where there is still value in gathering evidence for insight and inspiration.  We want to understand it from the perspective of time, money, satisfaction and utility. So that’s why I wrote a survey. I’d love it if you could fill it out or share it with others. When we get the results, we’ll share them with this community of people who do and buy qualitative research.

And we’ll keep thinking about the insight value chain, especially as it regards innovation and product/service design, and I’ll keep writing about stuff that pisses me off. You can be sure of that.

Two weekends ago, when I was working on client things and trying to unravel the mystery of addiction vis-a-vis Civilization V, there were some ladies getting righteous in my twitter feed.  Responses to Ashkan Karbasfrooshan’s post to TechCrunch, “Who Will Be the Next Talisman of the Tech World?” had lit up my feed and set my alerts to pinging – mostly because people were DMing or cc’ing me in outrage.

Perhaps because it came on the heels of two other lists that conspicuously omitted female names, this one seemed at first glance like ‘yet another list where great women in tech are ignored.’  I’ll confess that my first response was to feel exhausted, and my second was to invade Edinburgh.

But I did eventually get around to reading Mr. Karbasfrooshan’s post.  I thought it was an interesting list, this guess at who could be the ‘next Steve Jobs’.

After all, it makes for excellent link bait to write about Who Will Be the Next Steve Jobs.  It supports the folklore of Silicon Valley to speculate on his heirs, intellectual, aesthetic and otherwise. Search for “who will be the next steve jobs” on google and you get 846 million results. Everybody’s doing it. I’m thinking of going as that question for Halloween, in fact.

So, who’s on the list?  Scott Forstall, Tim Cook, Jonathan Ive, Larry Ellison, Marc Benioff, Bill Gates, Michael Dell, Mark Zuckerberg, Sean Parker, Evan Williams, Sergey Brin and Larry Page, Jeff Bezos, Elon Musk, Max Levchin, Peter Thiel, Janus Friis and Nicklas Zennstrom, Mark Andreesen, and Jack Dorsey.  The list is very journo- and reader-friendly – we’ve heard of these guys.

It’s my opinion that Mr. Karbasfrooshan didn’t even consider the matter of gender when he typed up this availability-heuristic-based, link-baiting listicle of guys in tech.  But it’s the ‘guys’ part that pissed people off. Where were the women? Did Karbasfrooshan mean to suggest there were no worthy women in tech?  Or worse, that there was no place for women in tech?

How did Karbasfrooshan respond?  In the immortal words of my best friend’s ex-husband, he found himself in a hole and kept digging.  Mr. Karbasfrooshan defended his post by saying, “That’s because the media remains biased against woman (fairly or unfairly). This list is largely about whom the media will turn to, most women are IMHO both put on a pedestal when it’s convenient and then viciously and unfairly attacked otherwise.”  In the comments on the post, the twitter exchanges that followed, and in a follow-up post , he suggests that if Charlie Rose wouldn’t book the talent they weren’t worth putting on the list (in what I shall now and forever call the Charlie Rose Booking rule).

Because data is helpful as evidence in an argument, I did a quick search of charlierose.com. Only half of the list have ever been on the show.  Jeff Bezos has been on 6 times, Larry Ellison and Michael Dell 3 times each, while Evan Williams, Sergey Brin, Larry Page, Elon Musk, Peter Thiel, Jack Dorsey and yes, Steve Jobs were on only once.

The others on the list don’t seem to have passed the Charlie Rose Booking rule; Jeff Bezos, on the other hand, practically has his own chair.  So the rule seems to hold.  In fact, in his follow-up blog post, Mr. Karbasfrooshan mentions a few women – “Catherine [sic] Fake, Sheryl Sandberg and Marisa Mayer.”  First of all, Ms. Fake’s first name is Caterina. Second, Ms. Mayer’s first name is spelled Marissa. Third, Ms. Mayer is the only one to have been on Charlie Rose – and she’s been on three times. (So why not include her in the  list if it’s really all about ‘the media’?)

It isn’t worth it to engage the kitchen sink arguments that Mr. Karbasfrooshan employs to defend his position: there aren’t enough women with enough experience, men in the media will overlook them, women in the media will overcompensate for potential perceived favoritism by excluding them, if they made it into the media we’d probably ogle their breasts, what successes they’ve had will either be criticized or minimized, oh, and tokens & Uncle Toms are bad.  Seriously, he covers all that in one post. It’d be impressive if it weren’t so bizarre.

In short, the lady (here, played by Mr. Karbasfrooshan) protests too much.  But he quotes Gloria Steinem several times, perhaps as some sort of innoculation from outraged women.

I leave it to any intrepid reader to find his self-defense plea tiresome, outrageous or both.  Because despite the author’s apparent lack of a criteria for assembling his list (other than the Charlie Rose Booking rule), there was a common thread among those who made the list – and it wasn’t just that they are all men.

What struck me as the true criteria was that the men on this list (with a few exceptions) are inventors.  Take a look at the list this way:

OS & SOFTWARE INVENTORS

• Bill Gates is a software developer who invented MS DOS and Windows.
• Larry Ellison is a database developer who invented Oracle.
• Scott Forstall is a software engineer at Apple who helped develop OSX and iOS.

INTERNET INFRASTRUCTURE INVENTORS

• Marc Andreesen invented the first web browser, Mozilla.
• Sergey Brin is a computer scientist who co-invented Google.
• Larry Page is a computer scientist who co-invented Google.
• Elon Musk is an inventor with an interest in physics & engineering who co-invented PayPal.
• Max Levchin is a computer scientist who co-invented PayPal.
• Niklas Zennstrom is a business development guy with a tech background who co-invented Kazaa and Skype.
• Janus Friis is a network developer who co-invented Kazaa and Skype.

SOCIAL WEB INVENTORS

• Mark Zuckerberg is a coder who invented Facebook.
• Sean Parker is a hacker who invented Napster.
• Evan Williams is a programmer who invented Blogger.
• Jack Dorsey is a programmer and software designer who invented Twitter.

E-BUSINESS MODEL INVENTORS

• Michael Dell turned a side business upgrading computers into a vendor license into a business model – the no-overhead PC manufacturer, selling direct to their customers.
• Jeff Bezos is a network engineer who invented Amazon.com, transforming the way you buy books, music and video content.
• Marc Benioff worked at Oracle for 13 years in sales, marketing and product development before creating his own SaaS, cloud-based business, salesforce.com.

VISIONARY ENABLERS

• Peter Thiel was a hedge fund manager and venture capitalist who saw the potential of PayPal and other start-ups.
• Jonathan Ive is an industrial designer who redesigned Apple.
• Tim Cook is an operations expert who reinvented the way Apple makes and sells its products.

All but 3 are inventors of the products their businesses sell.

And this is the real problem for women in tech.  It’s not (just) that the media don’t like us or sex sells or that bias and sexism exist.  It’s that we don’t have enough women who are true inventors in our midst who take their inventions and turn them into multi-billion dollar businesses… And either stay on to be CEOs or sell the business to a bigger fish.

The sad truth is we don’t have enough inventors right now, especially in the US, where enrollment in STEM degree college programs (which would at least give you the basic skills and knowledge for inventing physical things – or say, getting a job even in this economy) is down across the board.

Even those with an interest in engineering don’t get degrees – 1/3 of the list Karbasfrooshan assembles didn’t finish college, much less get a computer science degree.  So it’s not required to have a STEM degree to invent something, but in terms of skills acquisition, women are poorly represented in the shrinking population of those who do study science, technology, engineering or math.

Perhaps more telling however is how few engineers rise through the ranks of existing companies to be CEOs.  In most of the biggest companies in the world, STEM degrees are not tickets to the boardroom.

If you look at the Fortune 500 for 2011, 18 companies are helmed by women: a media company (Gannett), food and food production companies (Campbell Soup, PepsiCo, Kraft Foods, Archer Daniels Midland), a cosmetics company (Avon), a pharmaceutical company (Mylan), retail & wholesale companies (TJX Companies, BJ’s Wholesale Club  finance and insurance companies (Guardian Life Insurance, KeyCorp, WellPoint), energy and fuel companies (Sempra Energy, Sunoco), and yes, tech companies (Xerox, Hewlett-Packard, IBM).

These CEOs are distinguished women, many graduating with honors.  Most of these women went to prestigious schools. 10 have at least one post-graduate degree.  The vast majority have been with their current company at least 5 years, some as many as 30.  They have worked hard, risen through the ranks, worked for increasingly prestigious brands, working their way up their industry food chain.  But while 6 do in fact have STEM degrees, of those, only one seems to have held a related post, the newly named CEO of IBM, Ginny Rometty. Others, despite their mechanical, civil or electrical engineering degrees, came up through operations, finance and marketing roles.  Even Ms. Rometty went from systems engineer to the consulting arm of IBM, and from there worked her way up. Meg Whitman says she abandoned a math & science degree in favor of the more lucrative and employable economics degree.

While these women have much to be proud of, not one invented the product their company sells or have revolutionized the businesses they helm.  They have made them profitable, made interesting acquisitions, improved productivity or efficiency or morale.  But they haven’t utterly transformed the way people think about packaged food or cosmetics or pumping gas.

But here’s the thing.  Most Fortune 500 CEOs are not the inventors of their products, not the visionaries, not the game-changers.  So this is not a female problem.  It’s a CEO problem.

Some of the tech brands on Mr. Karbasfrooshan’s list are on the Fortune 500: Apple, Microsoft, Dell, Amazon, Google. And you’ll find companies from tech booms past: H-P, Intel, Cisco, eBay, AMD, Yahoo, alongside those old stalwarts Xerox and IBM.  But those companies are now starting to look more like their colleagues in consumer packaged goods, finance/insurance/real estate, media & marketing, and so on.  They hire CEOs, they acquire new technology, they maximize for productivity and cut for efficiency.  They answer to shareholders and the Street.

If his logic holds true, we won’t count the women OR THE MEN of the start-up scene until they have invented and grown their businesses to the size that makes lists like these – household names even Charlie Rose would book.

The companies not on the Fortune 500 created by his tech talismen are an interesting mix: Meg Whitman acquired both PayPal and Skype (Skype is now owned by Microsoft). Google bought Blogger. Best Buy bought Napster. Comcast bought Plaxo. Netscape belongs to AOL. Facebook could IPO any day, they keep saying, while Twitter continues to seek a business model. So perhaps the other future for a product inventor is to exit well and become an investor or serial-entrepreneur.

In other words, it’s about that vision thing.  Karbasfrooshan didn’t omit women because of sexism and bias and discrimination – at least not directly.  He omitted women because there just aren’t any playing at the level these very few guys play at who are visionaries about new products and services built out of technology.  There aren’t enough women who are inventors and cultural visionaries or industry game-changers… because there aren’t enough of those kinds of people, full stop.  They are, almost by definition, rare.

As ever, I come back to the wise words oft-repeated by Cindy Gallop: you can’t be what you don’t see.
Clearly there were women in the 70s and 80s who had engineering degrees but who either could not, or would not, or didn’t know how to put those degrees to use in a way that would serve their considerable talents and ambition.  But it wasn’t just the women who struggled.  If we’re seeing enrollment in STEM programs decline it’s because the business culture makes the case that a strategist at McKinsey, or a trader at Goldman, or a lawyer at White & Case will make the big bucks; the media culture makes the case that a pro-ball player, or a rock star, or an actress, or a reality show contestant will be famous; and both still seem to believe that nerds who invent stuff lack the necessary skills to be either rich or famous, all evidence provided by the dudes on Karbasfrooshan’s list to the contrary.

There aren’t enough of them to really break the mold – so far, these 20 guys just have cracked it slightly, proffering the exception that proves the rule (think of all the nasty comments and characterizations of Bill Gates or Mark Zuckerberg as unattractive or socially awkward, or of Steve Jobs as a cruel egomaniac). It’s upsetting, yes, that we don’t have more female role models in tech.  But what’s more upsetting is that we have so few role models in tech altogether.

Still, Mr. Karbasfrooshan should have accepted responsibility for his words instead of playing the ‘blame the media’ card.  He should have simply said, “I was talking about inventors and visionaries who are inventing what’s next, and can helm businesses and stay relevant.  I was talking about Steve Jobs, not Steve Wozniak.” That criteria makes it hard to think of anyone you’d add to the list, male or female. He could have simply played it as it lay.  But he took the bait he no doubt unwittingly set, and now looks like a fool or a jerk or both.

So, who are the women (or the men we haven’t heard of, for that matter) who are inventing new OSes, software that changes the way you interact with the world, social platforms that alter the infrastructure of the internet, technologies that enable new kinds of transactions and business models, boxes of wires and silicon that transmit and calculate data in new ways?

If you know who they are, please say so in the comments here, and I’ll follow up with that list.

In the process of thinking through what kind of business I might like to start for myself, I chose to start with a survey conducted by RSW/US, a ‘matchmaking’ company that brings clients and advertising agencies together.

As usual, satisfaction with current agency ratings are quite low, at about 41 percent, and nearly half wouldn’t ask their current agency back in a review.  Clients don’t believe traditional full service agencies have the expertise to deliver great digital ideas; but they’d rather give their business to a full-service shop.

It’s the following two charts in the report that are the most illuminating:

Right time-right place, recommendations, and just ‘fitting the bill’ – being the right kind of agency – are the most influential factors for clients selecting agencies.  Awards, search, trade associations, matchmakers – these are minor considerations for clients.  They want to have heard good things about you, and see you on their doorstep when they need you.

So why do they change agencies?

The biggest issues that lead a client to put the account in review are about their perception of the quality of the work: either the strategy isn’t there, or the creative isn’t.  In my mind, I’d combine “Lack of proactivity” and “Worsening relationships with agency team” because they both seem to be essentially about perceptions of service – together these would come in to tie at second place.  Costs are in a distant fourth place.

If clients put their business into review because the work wasn’t up to expectations, this puts the offending agency at risk for future business, since so much hinges on word of mouth.  So I spent a great deal of time this morning and afternoon thinking through the main categories of dissatisfaction, and matching those against the chief complaints I’ve heard working with or for advertising agencies about clients.  There’s a rather shocking alignment of concerns that express themselves as a kind of he says-she says account of the apparently worsening relationship between agencies and their clients.

This is step one in a process – my intention is to spend some time over the next few months examining what a Lean advertising agency would be like, perhaps even for a book (!).  I’m curious about where clients see the value coming from in what agencies are, or should be, offering.  I’m equally curious about testing the belief that while you might be able to build cars in a lean way, you can’t make a TV commercial using lean principles.  But today we’re going to just start by looking at the problems.

Product-Market Fit

Talk to any venture capitalist or start-up founder – they’ll spend a fair amount of time describing the importance of product-market fit.  This isn’t about changing the product to fit the market, or even changing the market to fit the product – though it could ultimately lead to either outcome – but rather about positioning the product in a way that demonstrates value to the potential market.  There are a lot of start-ups that fail not because they’ve made a bad product or because there is no natural market for their product – but because they haven’t clearly and compellingly articulated why that market might want or need the product.

It seems that agencies are facing a challenge in this regard as well.  Over the hundred or so years that advertising agencies have existed, the mission has become rather fuzzy.  At the outset the role of an advertising agent was to purchase advertising space on billboards or in newspapers or magazines on behalf of businesses.

After awhile, customers started asking for more: ideas about how to produce advertising that stood out and increased sales.  The agent, not wanting to lose a good customer, hired a typesetter and an illustrator, maybe wrote the ad copy himself or hired someone to write it for him.  He called up his contact in the publisher’s office of the newspaper and this time sent over an ad along with the money for the placement.

The agent became an agency.  The customer became a client – he’d be back, because the agency knew how to get this kind of work done.

And this is how advertising has evolved, as an industry of agents designing, producing and placing advertising on behalf of the client.

What the client used to want? An advertisement that drove sales.  Send us five cents and we’ll send you a Sunkist juicer so you can make juice out of Sunkist oranges.  Write to this address for a complete Sears catalog of all our tools and parts.  Visit this Ford Motor showroom to test drive a new car.

But the market changed. It became more complicated – more brands, more media, more channels. It seemed you had to spend more and more to see that sales curve lift. It became a kind of arms race between brands to see who could reach the most people.

Clients still want to see the sales curve go up.  Read the feedback at the end of the report and you’ll see a lot of concern with ROI, effectiveness, sales and volume increasing, business results, and so on.  But unlike those early days when the guy with a few dollars in his pocket to place an ad in the Gazette sold more widgets than the other guy who hadn’t thought to, the ability to track results against communications activities has become diffuse.  A campaign lives in simply too many channels; the drive for differentiation now means that there might be no direct, trackable call to action; other economic factors (pricing, distribution, competition, etc.) cause signal interference for brands that advertise widely, sell multiple product lines, distribute through multiple sales channels, and face many competitors.

It’s complicated.

But clients don’t feel sympathetic towards advertising agencies for having to muddle alongside them through all this complexity.  Increasingly, this survey suggests, clients feel a great deal of disappointment and bitterness about the failure of their agencies to clarify and simplify the complexity, while still bringing home those obvious, measurable results.

“What do you want from me? Fine writing? Or do you want to see the goddamned sales curve stop moving down and start moving up?” — Rosser Reeves

Looking through the complaints I jotted on post-its as I read through this report and the complaints I heard working with and for agencies, I noticed that a lot of the problem seemed to rest under a few overarching concepts.

Strategic Expertise

Clients complain that agencies don’t think or act strategically enough, that they don’t come to clients with a case about how the proposed approach will help them gain market share, increase volume, or otherwise steal sales from their competitors.  They complain that agencies don’t know their consumers, their market, their competitors, their sales and distribution channels.

Agencies complain that sales numbers and ROI are all that clients think about – they don’t ‘get’ marketing, they don’t appreciate the concept of building a brand.  They complain that the client’s goal or definition of ROI is unrealistic.  They complain that clients are not transparent about their business objectives, that they don’t engage agencies as partners, or include them in future plans.  They believe that clients sometimes perceive themselves as competing against the wrong people, or that they want to target the wrong consumers.

Transparency & Accountability

Clients complain that agencies are bad at strategy and analytics.  They believe that agencies lack the competence to effectively measure the results of what they produce, or worse, conceal those results from the clients. They want more accountability from agencies – for agencies to follow-up on the results of a campaign, and to report these results to the client.  And they want these results to be tangible – sales, volume, measurable ROI.  I’ve had some clients who’ve wished that the agency could offer a theory about what success would look like if it isn’t going to be sales or volume or share price.

Agencies on the other hand, feel that there is more to advertising than analytics, or feel that campaign measurement is more complicated and nuanced than mere sales figures.  Agencies often feel that the work of planning and account management – where this strategy and analytics would likely be managed – are under-valued by clients, who care (they believe) only about the creative and the costs.  Often, agencies feel they are provided with instructions from a brand manager or CMO that are unrelated to the company’s business goals.  They worry that clients test ideas to destruction, that they make their goals moving targets, that they aren’t transparent about their actual business goals, and that they don’t give credit to agencies when they do have a successful campaign.

Creativity

Clients say they want – but do not get – enough new, fresh, innovative ideas from their agencies.  They say that they are often not happy with the creative output.  They say traditional agencies lack digital skills.

Agencies say clients lack taste and sophistication, are scared of new ideas, are overwhelmed by fresh or innovative technologies, don’t ‘get’ digital/social/mobile, and prefer the safe, me-too route. Those who do ‘care’ about creativity are just in it for the awards or the chance to rub shoulders with celebrities.

Trust & Service

Clients complain they don’t feel agencies listen to them, they don’t get enough attention, they get pawned off on junior team members.  They complain agencies do not bring them unsolicited ideas; they don’t bring enough ideas; they don’t bring fully realized ideas - overarching strategies with executions for each of the channels.  They complain that agencies set themselves apart and above the internal client marketing team; that they do not recognize the capabilities, talent and expertise that clients have about their own business and market.  They complain about being condescended to.

Agencies complain that clients are not loyal – that they lost business to other agencies or competitors without getting a shot at it first.  They complain that clients don’t trust them – that they don’t share information, or set up ‘gotcha’ scenarios where the agency is being tested rather than engaged in a collaborative way. They complain that when multiple agencies are used on a single campaign, the client plays favorites, undermines some players, elevates others.  They complain that clients are not responsive to requests for feedback or approval, that they unnecessarily delay signing off on scope of work agreements. And they complain that clients can be abusive to some members of the team.

Costs & Capabilities

Clients complain that agencies nickel & dime them for basic service that should just be ‘included’ in the project – for advice, revisions, or projections.  They complain that agencies are bad at projecting costs and managing them throughout the lifetime of the project.  They complain that agencies can’t tell them what the results based on spend will be.  And they complain that agencies can not do everything well, or respond nimbly to their changing needs.  They want a more innovative full service model.  Clients don’t actually want multiple agencies – they want one agency to handle all of this on their behalf, to coordinate the production and placement and management of an integrated campaign for them.

Agencies say that clients want more and more work for free – work that is outside of the agreed-to scope of work.  They say that clients don’t pay enough (in that commission on placement and production structure that most agencies still use), or negotiate them down on the agency fee for coordinating and managing the campaign, thereby making every additional request from the client an opportunity to lose money on the project.  They say that clients lack the internal structure to implement or manage the approval process for integrated campaigns.  They say that clients start with one budget, set goals and KPIs off that budget, and then cut once the project is approved – but still expect the same ROI as the higher spend.  They say that they are the last to know when the client’s needs change. And they say that the era of the big account or project seems to be waning – that it’s tough to find a true ‘full service’ piece of business these days.

What’s it all mean?

It’s pretty simple, when you look at the complaints all lined up on a wall, as I did today.  Clients have business needs, challenges, and goals.  They try to solve for these through advertising.  They believe, however, that agencies are poorly equipped to help them achieve any of these goals or surmount any of these challenges. So the two have become antagonists. – people who do not trust each other.

I think the real culprit is procedural.  Sales and marketing may not be sharing goals with one another.  Marketing may not be incentivized by sales goals.  Marketing commissions advertising based on what some in the Lean Startup movement call ‘vanity metrics’ – likability or brand perceptions or awards.

Advertising agencies aren’t paid to become experts on the client’s business, so they become experts on advertising.  They don’t have the time or the relationships to go deep on the structure of the market; they aren’t rewarded for challenging the client’s assumptions.  They don’t have access to the right people in the client teams for the information they need to make the best recommendations and the best work.  They don’t have the budgets to get that information on their own.

“Good” advertising doesn’t have a clear call to action, isn’t direct marketing – it’s lifestyle based, taps into hidden desires or unspoken needs, creates cultural icons or foments generational movements.  They fear research as a sure-fire path to killing their creative darlings. They limit their financial exposure by bringing fewer ideas to the table.  They defend them fiercely, even at the risk of seeming condescending, because it is the only asset they feel they have.

And because they fear research, and lack the client’s willingness to invest in it, they do not gather data on the efficacy of a campaign, or do not gather the best data, and therefore, do not learn from one campaign how to make the next one better. But then, often, neither does the client.

In the end, relationships falter because of hurt feelings, unmet needs, disappointment, and an erosion of trust.  These come from a misalignment of expectations with capabilities.

It’s hard to say for certain who is in the wrong here – are clients asking ad agencies to solve problems they can’t solve?  Or have agencies intertwined themselves too tightly with the services they provided as agents, and not the outcomes those services were used to create?  In other words were clients buying something agencies didn’t fully realize they were selling?  Is what clients want from ad agencies not really advertising, per se, but increased revenue, sales volume, or market share?  And in today’s complicated world, is advertising always part of the solution set?

It seems to me the fundamental problem is that advertising agencies have thought, this whole time, that they were in the business of selling access to the development and placement of advertising, while their clients were trying to buy increased sales.

Perhaps clients don’t really need advertising agencies anymore (though they will still need creative production and media placement/negotiation).  Maybe they need business-model-seeking agencies that create roadmaps to carry out consumer, product, channel and marketing strategies.  Maybe those agencies facilitate the creation of assets that are placed into those channels or campaigns on behalf of their clients.  Maybe they are paid to be trusted experts who guide clients through the ever-evolving landscape of their market.

Maybe.

Just take a look at Mashable’s partnership with Microsoft BizSpark – you’ll find a running list of a great many start-ups and their products or services, many of them have some kind of venture capital or angel investment backing their pursuits. Lots of them are also-rans, many have no idea how or when or if they’ll make money, and most will fail.  Despite this high rate of start-up failure, investors don’t appear to be slowing down in the cash injections they’re pushing into the scene.

But when it comes to who founds and funds these ventures, as has been oft-noted, the flow of cash does not run in the direction of women.  When women do found start-ups that get noticed, they tend to be part of what I’ve come to think of as the 4Fs: fashion, food, family or feminism.

Recently, Bloomberg featured some of these very successful founders in a profile called, “Stilettos Invade Startups as Niche-Shopping Sites Attract Women“.  And while any woman would find the stories of these successful women founders to be inspiring as we start to eke our way through the ‘narrow sunroofs’ that Sheryl Sandberg and others in Silicon Valley think have supplanted glass ceilings, there is also a cause for concern.

I wonder if other women entrepreneurs sigh a little when they read sentences that identify these women as part of “a growing group of women at e-commerce companies tailored to specific areas, such as food or fashion, where the female perspective is seen as vital.”

Maybe it’s because I both work in advertising as a strategic planner, the ‘voice of the consumer’ in the agency, and as an entrepreneur myself. Maybe it’s because I love technology as much as I love my shoe collection, now numbering 60 pairs. But it seems to me that limiting the market sectors to which we apply women’s perspectives to food or fashion is not simply sexist, it’s stupid.  When you leave women – who control upwards of 80% of consumer purchases – out of the conversation in nearly any market sector, you are leaving an enormous amount of money on the table.

American women control or influence 85% of purchase decisions, and globally spend nearly $20 trillion annually.  That’s a lot of money to spend on lipstick, nylons and sewing notions.

It’s a less surprising number when you consider two simple facts: women make up half the population, and they are citizens with equal political and property rights (e.g., they can enroll in universities, hold jobs, own real property, engage in contracts, open bank accounts, loans and lines of credit, and even vote). They therefore have income and spending power – and there’s lots of them.  Lots of us.

What do we spend all this money on?  According to research compiled by She-economy.com, we were influencers or purchasers of 91% of new homes, 66% of personal computers, 92% of vacations, 80% of healthcare, 65% of new cars, 89% of bank accounts, 93% of OTC pharmaceuticals.  So why aren’t we thinking of women’s perspective as vital to businesses in the areas of construction, financial products, computers and electronics, travel, healthcare, automotive and pharmaceutical?

As a planner, I moderate or attend a lot of focus groups; indeed, in six years in brand consultancies that underpinned our work with primary qualitative research, I literally moderated hundreds of groups and interviews across a variety of market sectors. There are certain areas where we brief our recruiters to supply a high quota of women: consumer packaged goods, TV shows, fashion, and pharmaceuticals.  Products aimed at children mean we talk to moms; food and CPG manufacturers and retailers, like Kraft or Wal-mart, are almost exclusively interested in moms, who wield the check book and steer the shopping cart.  It’s understood that women are the ones buying the bologna and the body wash; we take for granted that moms handle family health decisions, take the kids to the doctor’s office, and make the trips to the pharmacy.  We know they watch a lot of TV, and do most of the clothes shopping. We know that women are also the primary influencers of their husbands, friends, children and parents. And most importantly to our business, we know that the clients in these sectors want to hear from them, because they want to market to them.

But there were plenty of categories in which we are at best sneaking in women, usually with the instruction under the gender field, “recruit a good mix.”  Logistics and shipping, enterprise software, automotive, consumer electronics, even some liquor brands, are categories in which the marketing and advertising world simply presume men are the primary consumers.

Often, as strategists spending most of our work lives talking to people out in the world, we know that women are significant purchasers or influencers in these categories as well.  However, our clients base many of these targeting decisions on consumer segmentations; often, a 55/45 male skew is enough to suggest that they need only address or hear from men.  Worse, targets that are defined by job titles like ‘small business owner’ or ‘IT decision-maker’ or ‘business decision maker’ (usually meaning the C-suite), or even ‘shipping manager’ are assumed to be male; we typically don’t even screen on the basis of gender, and rarely meet a woman as a result.  The presumption becomes the rule.

Our clients don’t have a problem with that. It reinforces the picture in their heads, and our lack of an attempt to talk to women keeps that picture in focus, because qualitative research in focus group facilities, behind one-way mirrors, is often the only occasion our clients ever have for seeing consumers “up close.”

But there is another side to these stereotypes.  Gilt Groupe, One Kings Lane and Rent the Runway are regarded as successful businesses (in the e-commerce spaces of daily deals and fashion/design, of course)… now. When Susan Lyne left Martha Stewart Omnimedia for Gilt Groupe in 2008, Patricia Sellers wrote: “Since Susan Lyne made a big name for herself at the top of ABC Entertainment and then Martha Stewart Living Omnimedia, her move to the CEO position at tiny Gilt Groupe seems to be a head scratcher. Have you heard of this year-old startup? I hadn’t.”  While the article goes on to speak positively about Gilt’s prospects and business model, it wasn’t exactly a ringing endorsement of a move to Gilt as a career move. One Kings Lane, among others, benefited from Gilt’s early and continued success in defining a business category.

In late June, Business Insider ran a story about a new start-up, under the headline: “Here’s Proof That Any Ridiculous Idea Can Get Funding Right Now“.  The article is much more positive than the headline about Naturally Curly Network, a social network for people with curly, kinky and wavy hair to connect with content, products, stylists, and each other. However, Business Insider seems baffled: “What’s more astounding is that NaturallyCurly says it was profitable and cash-flow positive last year.”

By the way, this is not the first round of funding for the site (really four sites), who obtained $1.2 million in the second round, after a $2 million initial investment. It’s an established business – and a profitable one.

According to the company, a global consumer packaged goods brand is sponsoring the launch of their mobile app. With a million unique visits to the sites each month, clear tie-ins for product sales, brand sponsorships, and local business promotion, Naturally Curly Network (while not helpful to my stick-straight head), presents a clear value proposition to its investors.

In the US, hair salons alone generate $19 billion in consumer spending; personal care products are a $40 billion industry, of which hair care products make up approximately a quarter of that market. This start-up, admittedly, is aimed at people with curly, kinky and wavy hair, but that’s not a small population – Naturally Curly claims it’s more than half of Americans.  Products for curly hair clearly sell – as evidenced by the empty spaces left by the few options that are chronically understocked at my local drugstore.

And it’s not simply women who struggle with styling and caring for curly hair, but men and women of a variety of ethnicities and hair types.  Retail sales of ethnic health and personal care products in the US are about $1.5 billion – but have you ever seen the ‘ethnic’ section of the hair care aisle at your local drugstore?  It’s usually at the end of the aisle, near the floor.  So in this case, we’re not just leaving money on the table by mocking investment in a service that could connect people to products and services in a multi-billion dollar consumer sector, we’re literally leaving it on the floor.

There are a few brands and businesses who aren’t going to leave that money on the table, and investors and entrepreneurs should take note.  If large companies see the potential in products and services for women and other under-served communities, then there is serious opportunity for start-ups who can, theoretically, move more quickly and be more responsive than their much larger, older cousins.

I spoke at a TIAA-CREF Forum event in April (The CMO, CTO and COO of TIAA-CREF, a retirement and financial services company, are women). The keynote speaker at the event was Joseph Coughlin, head of the MIT AgeLab.  He spoke engagingly and at length about the rise of the 50+ woman in America: she starts more businesses, controls more consumer spending, engages online  more than her husband.  Women don’t retire – at least not the way men do.  In the recent recession, women lost jobs at a slower rate than men (though they now appear to be regaining jobs at a slower rate, too). And TIAA-CREF, rightly, sees this as a huge business opportunity.

So congratulations to the start-ups, founded by women, who are finding responsive customers and investors and building successful businesses serving women and other under-served communities.  We need more investors – and the business press – to take their market segments and their customers seriously.  But wouldn’t it also be amazing if real estate, finance, automotive, technology, business-to-business were areas that investors feel are ‘where the female perspective is … vital’?

And what would it take to get there?

PSFK gives and gives and gives. so a shout-out today for adding fuel to the fire, as i’ve been thinking about retail quite a bit of late.

first up is using google maps to browse a store: businesses can add layout/floorplan and images from the interior of a store to their location on google maps.  helps to get a sense of what’s there.  makes me think of this great map of London high streets – came in quite handy when i needed a pair of boots for a lunch meeting in Paris. oh, i’m such a frakkin’ jetsetter.

then there’s ‘next-besting‘ – a more optimistic yet still pragmatic version of trading down. i know i’m finding myself browsing shopittome a lot more than nordstrom. online retailers who make this easy are winning.

and finally, how the iPad is going to change the (user inter)face of online retail.  i’ve seen the gilt app and it’s beautiful, but apparently the gap is also doing some good work for this device. again, where the hell is my iPad??

[gratuitous use of a puppy is within my rights]

i’ve been working in and around advertising for 12 or 13 years.  i’ve been a copywriter and a web designer and a planner and a strategic consultant and a qualitative researcher and an innovations lead and a ‘corporate intellectual’.  yesterday i was described on a phone call as an ‘expert on brands, strategy, research methodologies and implementation. and she’s a wild blogger.’  practically feral, i’d say.

but wait – “implementation.”

now that takes me back to the beginning, when i was making websites and ads and games.  and it reminds me of a really early conversation i had with a client who wanted to get into the e-commerce world.  this had to have been 1999, it was Ron Herman, who owns the Fred Segal store on Melrose.  he was turned on by Helmut Lang‘s website, but also by the Gap. but he didn’t have the fulfillment capabilities to ship everything anyone wanted, and he didn’t have the inventory system to know what he had and link it to his stores in both a virtual world and a brick-and-mortar one (remember how we all used to say that? i’m so glad it’s gone).

what he did have was a line called RH Vintage – which still exists, though at the time it was basically bedazzled vintage bought out of poundage.  the line was comprised of jeans or cords, t-shirts, and belts.  no two of anything was alike.  the prices were comparatively reasonable for a Fred Segal shopping trip.  we thought – let’s experiment:  let’s put up your jeans, your cords, your tees and your belts.  you have three choices to make as a customer: which of these 4 categories do you want to shop from, are you a guy or a girl, and what’s your size.  tick those boxes, and the good people at RH Vintage will pick out your clothes and send it to you.  it’ll probably fit.  it’ll probably be what you want.  it’ll definitely come in a branded bag, with a branded receipt.  it’ll make you think that you actually got in your car and went to Fred Segal.  you can pretend to your friends at Brown and Wesleyan and Amherst that you shopped there (and you sorta did), and you’ve got the threads to prove it.

Ron loved the idea – i’m not sure what happened next, but here was an answer that wasn’t an ad.  it was a micro-model for doing business.  it was a branded product line with a branded distribution system and a branded user/shopping experience.  yes, it would have a website.  probably taglines would need to be written and designs made &c.  but it wasn’t an advertising idea – it was a business idea.

the best stuff i’ve ever done in anything related to advertising has always been ‘this is what you should do‘ not ‘this is what you should say‘.

i was talking to Seth Wolk the other day at Saatchi about who in the business is making things.  (he’s so great – really smart and candid and clear and open.  frankly, a rarity.)  but he did cut to the quick: maybe i’m barking up the wrong tree.  maybe i’m expecting places who don’t, as a matter of course, do what i do, to want to do what i do – and to know how to package it, sell it, and implement it.  maybe people like me need to find a new roof. or build our own house.

it’s really important to not only think about what you do as the definition of who you are, but to make sure you’re in the right place, the place that will not only let you be you, but wants you to be more of you, is hungry for you, is receptive to you.  people like me should be in places where people say ‘this is what you should do’ – and then adds, ‘we’ll build it for you.’

and here, then, is my question:  where are those places, really? lots of places claim to be doing that, but are at heart still ad agencies or branding companies.  is it, as Seth suggested, media properties and platforms?  is it tech startups?  who are the companies that are looking at brands on a holistic level and then suggesting – and implementing - action instead of talk?

You’re definitely going to see this as a key theme to what I hope to explore here – that in the quest to get ahead, most companies and brands find themselves mired in cultures and processes that actually can only truly accomplish keeping up.  And keeping up is what they can do when the engine is firing on all cylinders.

As part of their quest to keep up, most companies and brands spend significant resources on ‘keeping their finger on the pulse’ of their consumer.  This is usually done with only the best of intentions, and with a goal of responsiveness.  But responsiveness is taxing and time-consuming.  What trends do you watch?  Which consumers do you track?  How do you know what’s important to respond to and what is not?  It’s classic Stephen J. Covey stuff – what is important, and what is urgent?

For many brands this results in what I think of as chasing consumers, following them around.  A lot of fads in strategy, planning, research and brand management are reinterpretations of ‘responsiveness.’  The strongest brands – and we all know who they are – live and breathe the worlds their best, most influential, most discerning customers experience, they interpret the signs and signifiers of that world, and they think deeply about how that leads their product development and how it leads the consumer to the brand or product.

In other words, they anticipate problems and solve them; they anticipate changes and adapt to them; they, simply, make good things and treat people with respect.

My friend and former colleague Susan Coghill (now of The Campaign Palace in Sydney) noted this WARC article about Coca-Cola’s quest to get ahead of the curve and to ‘shape change.’

“It’s not about following the change as quickly as possible – that’s being reactive. It is about helping your company to shape change.”

In order to achieve this goal, consumer insights specialists will need to radically rethink their traditional techniques, which typically emphasise understanding previous or present behaviour.

“We spend 80% of time on ‘rear-view’ research – brand-health tracking, validation, and risk-avoidance research,” Sthanunathan stated.

“On top of that, we spend 80% of our remaining time debating report cards. And, whether it’s good data or not, it’s all about explaining the past.”

“No company has become great by using the past to predict the future. Companies become great by dreaming of the future and then taking the company there.”

Definitely worth a read-through – but more importantly, worth a meaningful debate about what it will mean, day-to-day for brands and companies.  Time to get beyond whether to do things differently and start figuring out how.  And then of course, you have to commit to it.  But that’s a post for another day.