On a scale of 1-5: a comparison of user survey tools (part 2)

In part 1 of my ‘comparison of user survey tools‘ I explained the events that led me to look at the 3 different services. In this 2nd installment I thought I’d share the criteria I used to grade the survey tools and how well each site performed against them. To quickly recap, the websites I compared included SurveyMonkey, Google docs and Survs.

Summary

Criteria

Each survey tool was scored out of 5 against 10 criteria, which were divided across 3 key sections. The sections were:

  • Getting started
  • Running a survey
  • Results and reporting

Getting started

1. Sign up
All 3 survey tools performed relatively well. It was very quick to register a free account with SurveyMonkey, however, once you have completed registration you are taken straight to the default dashboard which gives very little assistance. The table of surveys is empty but for a small instruction to click on a link at the top of the page. [Score =3]

Google Docs was slightly harder to measure. As an existing Google Docs user it wasn’t particularly difficult to setup a form for use as a survey although I imagine for a new user it would not be as easy. Google Docs is a completely different service to both SurveyMonkey and Survs but none the less it can be used in the same way so can be just accordingly. [Score =3]

Due to its current beta status, and by that I don’t mean Googles definition of beta (see GMail) but real life beta, you have to request an invite to use Survs. Once you receive your invite its simply a case of filling out a short registration form. Signup lead to a welcome screen which tells you pretty much everything you need to know to get you started, thanks in part to a screencast of it in action. A nice touch is the added ability to ‘turn off’ the welcome tab once you’re comfortable with using it. [Score =4]

2. Survey creation
SurveyMonkey Feels clunky during setup. It has far more functionality than the competition including the option of 16 different question types, although this almost feels like too much. Editing the survey takes time, there is an abundance of advanced functionality which means very little first time users. There is very little consideration for the learning curve of users, instead they are thrown straight in at the deep-end. [Score =3]

In comparison Google Docs is incredibly easy, although compared to SurveyMonkey it is a far more rudimentary system. Options are limited to a range of 6 question types. There is the ability to simply add help text. All in all the system is quick and very intuitive. [Score =4]

The only one to include a video tutorial up front was Survs. You’re given a fantastic level control when creating a survey. Its not as intuitive as Google but carries with it far more options. There are fewer question types than SurveyMonkey but the free account includes the ability to add ‘logic’ into questions. Survs does a great job of hitting the middle ground between the advanced functionality of SurveyMonkey and the intuitive but basic Google Docs. A new user can easily pick up the basics and successful create a survey without worrying about some of the more advanced features but as your knowledge of the system grows there are plenty of hidden gems to play with. [Score =5]

3. Cost
SurveyMonkey does offer users a free account but it is very basic and functionality is limited. Full functionality can be gained for either $19.95 a month or $200 a year. More information on SurveyMonkey pricing structure. [Score =2]

Google Docs as expected is totally free once you setup a Gmail account. [Score =5]

Survs is also free but it is still in beta so I’m sure in time it’ll be fee based. [Score =5]

Running a survey

4. Integration
Each of the tools give you the ability to link to them from your own website. SurveyMonkey give you the ability to link directly to the survey in the form of a popup, link to a popup invite or alternatively as hosted html. Each solutions feel quite cumbersome and I dislike the fact that you are effectively forced into linking to popup windows, which in my experience frequently come off badly in usability studies. [Score =3]

Google on the other hand feels incredibly easy. The options are simply kept to a hosted html page which you can link to or the ability to include the survey within your own website in the form of an iframe. [Score =4]

Once again Survs seems to sit nicely in the middle. There are popup options, both directly to the survey or in the form of an invite but also the option to include the survey on your own website through the use of an iframe. [Score =4]

5. Customisation
The amount of customisation within SurveyMonkey is limited based on your account type. As I only have a free account there’s not much I can do to style the surveys. There are predefined themes to choose from but none of them stand out as particularly nice. You have to pay to add logos or edit themes.  [Score =3]

You can’t customise any aspect of Google Docs. It forms are effectively white labelled and as you can include them in a page as an iframe you can pretty much fit them within any pre-existing design. [Score =1]

Survs is fully customisable AND for free! You can change colours, fonts and even add imagery. nice. [Score =5]

6. Promotion
SurveyMonkey gives you the ability to send out emails direct from the admin area. You can import contact lists and edit them once saved.  [Score =3]

Being apart of Google Docs means the survey tool has full integration with your Gmail account. [Score =4]

Survs has the ability to add multiple emails but  there’s no upload feature which could be a real pain if you have a large list of customers you want to contact. [Score =3]

7. Response
SurveyMonkey is easy enough to complete but the form elements are not standard which is a strange thing to do in my opinion. Why SurveyMonkey, why? on the plus side you can fully customise the ‘thank you’ message. [Score =3]

The Google Docs survey feels familiar. The form is simple and intuitive. You can also fully customise the ‘thank you message’.  [Score =4]

Much like Google Docs the Survs survey is very intuitive and you have the option to customise the ‘thank you’ or forward onto a url. [Score =4]

Results and reporting

8. Format
The format of the results within SurveyMonkey are very detailed but you’re unable to filter results without first upgrading to a paid subscription. [Score =3]

Google on the other hand provides a nice clear, easy to read summary and also gives you the ability view as a Google spreadsheet.
[Score =4]

Survs produces a detailed graphical summary which can be filtered without the need to upgrade. Respondents can also be viewed in isolation. [Score =5]

9. Portability
When it comes to analysis you have to be able to take the data away with you. Unfortunately SurveyMonkey only lets you save your results if you upgrade to a full account. Notice a pattern arising here? [Score =2]

As the results from the Google survey are a Google document you have plenty of options. CSV, html, txt, pdf, xls, take your pick. [Score =5]

Survs gives you the option of XLS or CSV with the ability to limit the detail of whats exported. A nice little bonus is that you can print the results as a PDF thanks to their print stylesheet. [Score =4]

10. Access
The concept of portability focused on the reporting side of things were as my thinking behing  ‘accessibility’ with regards to online surveys refers to the ability to share the raw data with others. Once again it appeared that this was available through SurveyMonkey but without upgrading I’m not 100% [Score =2]

As with most Google docs access can be granted in a couple of different ways. You can assign ‘global access’ which means anyone who is given the correct URL can view it and pass the URL on. Contacts can be personally invited but they can only view the results if they login with a Gmail account. Althernatively the results spreadsheet can be published as a html page. [Score =5]

Similarly to Google docs Survs has the ability to invite people to view the results or send out a link that can be openly shared. A nice little feature is the ability to password protect the results. [Score =4]

Conclusion

Out of the 3 survey tools Survs easily came out on top. Its a relatively new product but works so well and its free! The only downside is that its still in beta which can cause problems. I’m under no illusion that sooner or later it’ll be a paid for service but for the time being I’m enjoying the innocents and freedom of it being free and pimping it whenever I get the opportunity.

Free versus paid

I don’t have a problem at all with webapps/services charging, how else are they expected to survive? As a designer by trade I am totally against spec work, an issue that seems to be doing the rounds at the moment, and I gave up downloading music for free years ago for the same reasons. But I disagree with the way SurveyMonkey go about charging their users. They are clearly the most established survey tool on the market at the moment but the idea of limiting so much functionality unless you pay seems ridiculous. How can someone get a clear understanding of what the tool can do if half the decent functions are blocked?

Personally I prefer the idea of trial periods (e.g. 30 days free) or limiting activity. Websort, the remote card sorting app does this perfectly by limiting their free account to 1 sort at a time and also limiting the number of respondents.

Old tools

My main problem with SurveyMonkey is that it just seems so dated. The admin interface looks old, is far more complex than it needs to be and to me feels clunky. The actual surveys that are created look just as dated and the form elements are just weird, where is the benefit in not using standardised controls that everyone is familiar with?

On the other hand both Survs and Google are fresh, modern and intuitive.

Basic versus complex

Initially I thought Google would win hands down but its still relatively basic. There’s no customisation or ability to brand the survey itself, plus the survey generated doesn’t cope well with too many questions as there’s no facility to include pagination or ‘logic’. That’s not to say Google  forms isn’t a good tool. Its great for knocking up something basic in a hurry. To give you an example I quite regularly use Google forms to help back up research findings or present clients with an opinion supported by the findings of a small straw poll. However, beyond this I can’t see much commercial use for the Google forms service at the moment but then its Google so who knows what will happen in the future.

The winner

Taking this all into account Survs.com is easily the best tool in my opinion. If you’ve made it this far through this post without  losing the will to live hats of to you. I hope in some way this review has been of help to you but ultimately the only way to truly find out which webapp is best is to get out there and try it for yourself.

If you have any opinions on this post that you’d like to share please leave a comment I’m always eager to find out what other people think. If you know of other services that you believe are better or should have been considered along side these 3 add a link to in the comments.

Thanks for reading.

Related links

2 thoughts on “On a scale of 1-5: a comparison of user survey tools (part 2)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s