Online recruiting for surveys, market research and user interviews

February 28, 2023

I will outline here a method for online recruiting that is interesting for:

• if we exhaust our cold calling list and wish to automate the recruiting process

• and/or to investigate multiple countries simultaneously (using the same questions)

• and/or acquire larger samples and get statistical data for more confidence (quantitative research)

• or to bypass the secretary gatekeepers more easily

Online participant recruiting is interesting for recruiting users, getting surveys completed or organising interviews. You can target consumers (B2C) or specialised professionals (B2B). In the past I had success in recruiting users using Prolific.co  which offers a EU-wide pool of 120k active participants. Generally, you are more likely to find higher educated like research/marketing managers, but administrative staff is also well represented (installers probably less so).

How does it work. First, you can filter out people matching basic criteria (e.g. demographics, company size, “food” industry or other) entirely for free. Then you invite those to fill in a screener survey, such as if they are involved in pink ice-cream production. For such invited surveys, I estimate $10 to invite 10 people. Probably only 1 out of 10 qualifies, so after $100 you have 10 matching people. Let’s say 5 out of 10 would be open to a follow-up interview or detailed survey. So basically, you payed $100 to find 5 qualified people who are up for an interview.

Once you have found good matching participants, you’d need to pay them for a telephone interview. In our case you’d ask about $20-40 for a remote 30 min interview (depending on their role).

Bottom line: ($100 + 5 x $30) / 5 = $50 per telephone interview.

Nice: it’s also possible to automate scheduling the interviews in our calendars, by hooking it up to Calendly. 

Alternatively, instead of telephone interviews, you can invite targeted users to complete a detailed online survey for about $5-10 per completion (incentive is up to you, but length of survey and role plays a role in motivation), plus an online survey licence for $0 to $25 per month (depends on needs). It takes time to get a well-designed fully automated survey up and running. If we’d require their company name or plan to record interviews, then I’d need to check the anonimity and GDPR laws about how we handle their data. Btw, for online surveys, I wouldn’t worry too much about bots, liars or students, as I found with a well-done survey 98% answered honestly and accurately. If it is a concern there are ways to deal with it (identity checks, attention checks, repeating questions), but Prolific.co generally gets quality participants as they have good rejection/blacklisting mechanisms in place. For Dutch speakers there are also services like respondenten.nl but they tend to be more expensive, besides I don’t expect English language to be a barrier in the Netherlands. Finally, to open up to non-Prolific audience, here is the alternative option of using targeted Facebook/Google ads to attract and acquire interested interviewees, but my guess is this is more work for the effort.

My feeling is to stick to telephone interviews while we can, but consider online recruiting as interesting depending on how many and which people and countries you plan to interview.


Usability research

December 29, 2008

Where to find (free) research into usability?

Pick your topics

http://uxmyths.com/ ++  (convincing)

https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/visual-aesthetics

..and other chapters of the-encyclopedia-of-human-computer-interaction-2nd-ed

Google for: Research-based guidelines for <keyword>

Search spiders

http://tc.eserver.org/
ACM Digital library (access via university library, not all are free)
Search field of hcibib.org

IEEE HCI? See Encyclopedia of HCI (link above) for more popular academic HCI research resources

CiteSeer(x)(Journals, periodicals, articles (…), (university) research groups, conference papers, special interest groups SIGCHI, theses, blog posts)

Directories

http://degraaff.org/hci/publications.html

Research tools

How to organise your (online) research.

http://researchresources.blogspot.com/

HCI Research bodies

http://www.surl.org/ (Uni WI)
http://www.microsoft.com/usability/publications.mspx (few broad articles by microsoft)

Notes:

  • use it to gain buy-in power
  • beware of the applicability of usability research (Dutch)
  • Applicability of the research source (before+after reading sth):
  • – what can you do with the information? can you study/learn, derive or directly apply the research outcomes?
  • – is it relevant? User performance (user speed, efficiency and ergonomics) goal versus business conversions goal. Though related, a design that focussses on the most efficient may seem more ‘usable’ (less prone to error, faster completion, faster to learn) but might not necessarily be the most desirable design at first sight.
  • Context, audience and cultural differences

Will it make a difference? Asking the users (surveys) gives a skewed reality. Statistics can be damn lies. Usability research on prototypes gives you more confidence. But for the real truth, only getting it live (lean) can tell. Then do A/B, usability, heatmap and satisfaction testing – whatever data you need to get the job done.

 

The value and credibility of research

(coming up)

Cooper: break with convention only if…

A fruit juice may be healthy and good for you, but is it also ‘lekker’?

Research should adhere to these criteria:

Accurate

Compared to a gut feeling, research is often a better representation of the truth. However, a research by UIE.com found that using gut feel by an experienced usability practicioner about a given research question performed just as effective as full-on research.

Applicable/relevant

It holds true for whatever you’re working on or trying to find out.

Credited

You can use CiteSeer, a cross-reference tool, to check how many times a certain publication has been cited by other publications. There are also ways/tools to find out how much a person is cited (userati) or paper referenced from blogs or the web.

Valid/trust-worthy

The method of research, calculations and measurements involved are correct. There is adequete confirmation from a trusted source (of experts).

There are sufficient authors referencing or linking to the source, and few critiquing it.

Resources

Goal; Types of sites with research sources which are suitable for this; Actually URLs to sites to ‘raadplegen’ for reference

Table:

Best practice in business ;  Pattern sites  ;  Forrester

Improve Customer Support ; E-commerce/CRM software vendors ; egain.com (see ‘Research resources’), RightNow

…. ;  ….. ;…..