Testing Infinite Scenarios

I have been involved with implementing integration into a mapping service.  I am thankful we were not responsible for testing the accuracy of the third-party GIS (Geographic Information System) service itself, but only the integration of the service with our solution.

However, in testing we did notice a few problems with some of the information we were receiving.  Some of our client properties were not displayed correctly.  They were *close*, but not exact.

When I read this article yesterday, I was reminded of an anxiety I had when I tried to place myself in the shoes of those implementing and testing the GIS itself.

Apple Map flaw results in drivers crossing airport runway

As test engineers know, most test configuration matrices are massive (exponentially affected when adding new configurations), but the GIS testing poses a truly staggering challenge.  Nearly infinite scenarios.

In these times, I use my handy-dandy guide to prioritizing test cases:

1. Do an equivalence analysis (what configurations/scenarios are the same for testing purposes)
2. Do a risk/impact analysis (what happens if something goes wrong?  do people die?  is revenue impacted?)
3. Do a change set analysis (what has recently changed?)
4. Prioritize your configurations (what are the most common configurations?)
5. Prioritize your functionality (what is the most commonly used functionality?  usage statistics are very handy here)
6. Identify the complexity and time-to-test of the different configurations (prioritize complex tests lower, all other factors being equal)

However, I feel like something is missing in my list above, with regards to this particular GIS problem.

How would you approach GIS testing?

Aggregating data in useful ways

This really interesting image shows an aggregation of many famous starships in fiction, shown to scale for comparison.

Every Sci-Fi Starship Ever (not really, but close enough)

While fascinating, it also brings up that personal weakness of mine- taking the time to manually aggregate data.  If a report is not one-click away, then I avoid it.  However, I really shouldn't, as data good for assessing quality is usually hard to get.  Many of the tools we use are created with project management or development in mind, NOT QA.  (See Rally, JIRA)

I have to block time off in my calendar to go mining for data.  It takes discipline, but it is worth it.

Now- taking that data and presenting it in a meaningful way... that's a whole other bucket of beans.

Kudos to the person who put this Starship comparison chart together.  It took a lot of manual work, I'm sure, but it can be appreciated by many, and tells a story.

Ctrl + Alt + Delete was a mistake: Bill Gates

... but I must admit, I've developed some fairly impressive finger positions to do the one-handed login.

Finally:  Bill Gates admits Ctrl + Alt + Delete was a Mistake

h/t funnylifestories.com

Get your jargon straight!

An oldie but a goodie.  This is sort of the opposite of the Urban Dictionary.  This is guide comes in handy when navigating the Sea of Wordplay and Meaning in the land of Corporate Culture.

Jargon Watch

A fun fusion of gaming and career

One theme that keeps popping up in my study of user experience is that of achievement reward systems.  You look at any task you would like a user to perform, and, no matter how mundane, you reward their achievement in some way.  This can even be something as simple as a meaningless list item in a long list of meaningless list items.

Companies have flirted with using this approach in real world activities, such as teeth brushing and bike riding.

In my mind, the three most important characteristics of a successful achievement reward system are:

1. Show users what is possible to achieve (list the achievements possible, and a clear explanation of how to achieve them)
2. Show users where they've been, what they've achieved, in a way that is satisfying.
3. Make sure the achievements can be obtained at a reasonably consistent and fast pace (you don't want people getting bored while trying to achieve one thing... keep the rewards flowing)

With this in mind, I got a kick out of the following link.  It is a list of achievements toward being an expert web developer, presented like a role-playing game skill tree.  You developer yourself along this tree's branches, with the end goal of becoming the web developer you desire to become.  This page even lets you choose your character portrait and name.  Great stuff.  I hope you enjoy.

Becoming a web developer via RPG skill tree

Warren Spector's Commandments of Game Design

A wonderful list that is well worth examining.  Warren Spector is highly respected in his field, and has been involved in development of many successful games such as the Wing Commander series, the Ultima series, and the Deus Ex series.

Warren Spector's Commandments of Game Design

With regards to quality, we can take some good lessons away from this.  When we analyze a design for any type of user experience, we look for things like rewarding the user, making the goal visible, giving multiple ways to achieve objectives, avoiding unnecessary steps.

But, even more than these:  when game design meets user experience, it can be an amazing thing.  You tap into people's motivation to help them achieve their goals in a pleasant way.  Every solution should strive for this result.  It reminds me of an article I read recently where developers are bringing video-game-like-achievements into real life. 

If you can tap into people's motivations, not just their behaviors, you have created a successful user experience. 

You've gained the 'Floss your teeth' achievement!