Remembering Why We Are Here

Details

Written by: Gary Elsbernd

Published: 23 August 2007

I was reminded yesterday of the dangers of developing tools for the sake of tools.  Back in the day when I developed websites for folks, I would be so excited by a new technology or widget I'd found that my instinct was to include it so I could play.  I would add a bit of DHTML or javascript because I found it, not because it did something I wanted to do.  To some extent, I still do that on elsbernd.com, because it's my hobby site. The challenge when working for a client or employer was to ensure the design had as little as possible while meeting the goal, and no more. Anything else is a distraction that wouldn't help achieve the goals of the site.

"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away."  --  Antoine de Saint-Exuper

The actual issue yesterday was a project management process.  The process to ensure quality and governance was starting to shape the software development process, when it is supposed to be unobtrusive and stay out of the way of getting the work done.  The software design process was being asked to change to fit the documentation and reporting requirements set by project management.  It didn't help create better code, just better reporting.  The focus was going away from developing useful, bug-free code on time and under budget to completing deliverable documents and passing through review gates.  The project management process was becoming the focus of the activity, instead of being in an oversight and supporting role.  The software development needed more priority, as it was actually developing code for use by the business.  Likewise, the software development process should bend to service the business, not stay so rigid that the business has to adapt to IT processes.

The focus needs to be on achieving the goals of the organization.  No organization has a goal of "developing software" -- they either want to use the resulting software to conduct business or sell that software.  The development process is just enabling the goal, not meeting the goal.  Likewise, while the project management controls are in place to ensure efficiency and effectiveness of the development, they are not an end in themselves, but rather serve the larger goals.  When the wrong things take priority, the goals are not being met.

And lest you think this only applies to software design, remember the same thing in your own life.  My life is not all about designing cool sites.  In part, I do this because I enjoy it, but at the root, my life is about my family and serving the Lord.  I work so I can afford to do those things.  When the job takes me away from the family more than necessary, or when I start to obsess about my job when I'm at home, I'm no longer serving the goals of being with my family or serving God.  The priorities have to be in the right place and remember why you're doing things, or you will be focused on glittery distractions and useless features in this life.

Wow, that got deep.

 

Performance Centered Portals

Details

Written by: Gary Elsbernd

Published: 20 August 2007

In late 2001, after starting my own consultancy, I decided I needed my name out there, so I wrote an article for Performance Improvement magazine.  It was about portals, which were somewhat new at the time.  While the technology had been around, they were still abused and poorly designed for users in many cases.  Though I didn't know much about the technology, I knew how to design for performance, and wrote the article.

I went on to present this topic with Matt Hummel at ForUse 2001 in Portsmouth NH.

If I were updating this article today, I would do many things differently, but then again, I have been working within the IBM Websphere Portal infrastructure for four years now and know how many of my ideas were naive.  :)

Here it is, warts and all.

Performance Centered Portals

 

That Old Black Magic - Identifying and Dealing with Superstitious Users

Details

Written by: Gary Elsbernd

Published: 20 August 2007

This is an article/presentation I developed with Matt Hummel for a usability conference back in 2003 that I never got to give for various reasons.  We never developed the obligatory powerpoint, since our trip was cancelled before that was called for and we didn't believe in doing things before deadlines :).

I still believe in the content of the article and have used much of the information on mental models in my upcoming article on planning ahead.

 

Abstract:

The success of a system depends on the mental model of the user accurately matching the tasks represented. Users whose existing mental model is based on superstitions, hearsay, or faulty assumptions can defeat the best-laid plans of even experienced usability professionals. System usability professionals can support success by identifying and preventing mismatches.

Introduction:

Superstitions have been employed through out history to explain the unexplained and to assign reason and rationale to events. Superstition spans the spectrum from the mystical (spilling salt is bad luck unless you throw some spilled grains over your shoulder), to the comical (a winning baseball pitcher who won't change his socks), to everyday methods and mannerism we don’t even recognize as superstitions. Unfortunately, computer users also develop superstitions to deal with the seemingly irrational behavior of some computer systems. These superstitions can decrease productivity, increase frustration or even prevent the user from performing the task. How do superstitions form and, more importantly, what can we do about them?

What is a superstition?

Dictionary.com defines a superstition as follows:

su·per·sti·tion

An irrational belief that an object, action, or circumstance not logically related to a course of events influences its outcome.

In other words, believing an action or set of actions will result in a specific outcome, even though there are no facts supporting that belief or the belief is just plain false.

Superstitions in everyday systems

In addition to affecting how human beings behave on their own, superstitions can also influence how humans interact with mechanical and technological systems.

One notable example illustrates how we unknowingly apply superstition to the systems and technologies we use in our everyday lives. In his book, The Psychology of Everyday Things (1988), author and usability guru, Donald Norman, references how most people interact with heating and cooling systems. Whether using an electric range, a refrigerator, or the thermostat in a hotel room, when a rapid change in temperature is desired, users crank the temperature control to one extreme. While the user does this thinking it will make the device heat or cool faster, the truth is the device will heat or cool at the same rate regardless of the temperature setting on the control. The control merely tells the device when to stop heating or cooling.

The disconnection in this example is the user’s “mental model” for the heating or cooling device. Because the user has been rewarded in the past with a cooler room or a hotter range, he or she has formed the belief that turning the control further will speed the process. Those who adjust their mental model remove the superstition and simply set the control to the desired setting. Users who maintain the superstition continue to take this incorrect action and can receive unintentional results. (This is demonstrated in conference rooms across the country where someone is constantly turning up or turning down the climate control.)

What is a mental model?

Mental models are the representations people have of themselves, others, the environment, and the things with which they interact. People form mental models through experience and accretion (when new information fits into an existing schema, and thus is quickly comprehended). A mental model can be developed, added to, or modified any time a person’s behavior and the subsequent results are connected in the person’s mind. In reality, users interpret results and sometimes form an accurate model of what is being experienced, sometimes the results are correlated but without a true cause and effect relationship, and sometimes the results are a coincidence. Regardless of the how’s and why’s, if the actions and outcomes are understood by the person as linked, a mental model can be formed or strengthened.

How do mental models apply to software systems?

Because there is more than one role involved in the design, development, and use of software, more than one mental model is imposed upon the system. Norman identified this and began to label the different models associated with software. Germane to this discussion include:

• The Design Model is the designer’s idea of how the system works and represents the true nature of the tasks.

• The System Model is how the system actually behaves and represents the nature of the tasks.

• The User’s Model is the mental model developed through interaction with the software (which incorporates aspects of both the Design Model and the System Model).

The designer expects the user’s model to be identical to the designer’s model, but if the system model is not clear and consistent, the user will end up with an incorrect mental model.

Because much of today’s software is complex and designed to aid in a wide array of tasks, it is becoming increasingly difficult for users to generate “neat and tidy” models (DiSessa 1986). In most cases system training is limited and knowledge around the work domain is disassociated from the system itself. Users, left to identify, structure, and associate the entire knowledge set (including business rules, best practice decisions and procedures, exception handling and system know-how) necessary to perform his or her work tasks, are increasingly challenged to create sophisticated mental models. Without guidance, users can quickly develop narrow, shallow, misconstrued, or inaccurate models. With an inaccurate model and no mechanism for remediation, users’ performance suffers and the business pays the consequences of lost productivity, poor decisions, mistakes, and loss of customer satisfaction.

How superstitions are formed

Human beings constantly seek meaning in their world. When an accurate and complete model is not provided, superstitions are a strategy to construct meaning where there is none, or none is perceived. Frequently, this assemblage occurs with whatever information is available. Most users will build the model with the information at hand and without regard for their understanding of the information, the information’s accuracy, or the other contexts not currently in use. These model constructs are not necessarily accurate or complete. To accommodate the delta between what is understood in the model and what is not, users develop superstitions.

Most superstitions are rooted in the disconnection between what someone perceives to be true and the actual truth, several factors contribute to superstition development and preservation:

Coincidence

If a good or bad result happens to coincide with another event, the user may associate that event with the result, even if there is no causal link for the correlation.

For example, a user sends several files to the printer at one time and the printer jams (due to too much paper in the paper tray). The user may clear the jam and resend the documents one at a time. Because the user reduced the amount of paper in the feeder when he or she cleared the jam, the documents now print fine. The user may make the coincidence of sending multiple documents and an overloaded feeder a superstition and decide to only send one document at a time in the future.

Lack of domain knowledge

Without the complete knowledge and know-how associated with a job, users will develop superstitions around processes, procedures, and decision making. For example, if a customer service representative has a large FedEx package returned because it was sent to a PO Box, he or she may refrain from sending any large packages (regardless of the carrier) to a PO Box.

Tribal learning

When a person who has a superstition teaches another person how to perform the work, a superstition can be passed on without question. Work around solutions, once required by limitations of the software, hardware or management policies, may continue long after the limitations are resolved simply because that was how the user was trained.

Poor usability

Failure in certain user interface heuristics (such as lack of goal establishment, lack of task structuring, not answering descriptive and functional questions) can contribute to superstitions being formed. These force users to “invent” a process regardless if it is the right process.

Mission-critical tasks

When the work is critical or the consequences are significant, users are less inclined to vary from a process. Even if the process seems redundant, illogical, or inconsistent.

Infrequent tasks

If a task is rarely performed, the user may not perceive benefit from trying alternative methods or questioning an approach.

Problems associated with superstitions

Learning today’s software systems can place a significant burden on a user. Often a user must change how he or she works just to match how the system was designed. This forces users to construct their mental models while under considerable and varying pressures. When a user mental model is inaccurate errors are more likely to occur. The more abstract or mismatch the model, the more likely problems will be frequent or have significant consequences.

When a superstition is entered into a work process several negative results can occur for the business:

• Increased time/loss of productivity and efficiency

• Inaccurate results or responses

• Redundant processes

• Inappropriate actions based on the circumstances

• Inappropriate or poor decisions

• Increased learning curves and time to competency due to incorrect assumptions and associations

• Incorrect procedures and processes followed

• Frustration for the user of the system

“Sometimes the result is a minor frustration or inconvenience, such as changes not being saved to a file. Inaccurate mental models of more complex systems, such as an airplane or nuclear reactor, can lead to disastrous accidents.” (Reason, 1990) Superstitions when ingrained into the model of the system are very difficult to eradicate and persist even after the user has been given concrete evidence to disprove them.

Once these superstitions have led to bad habits, remediation becomes even more difficult.

How to identify superstitions

In addition to the negative impact superstitions can have on businesses, they can also be troubling for usability professionals trying to conduct analysis or assessment. Unless the usability professional is also a subject matter expert, it can be difficult to identify the facts from the fictional superstition. Knowing what user behaviors may indicate a superstition’s presence can improve the likelihood of identification. Being mindful of the following behaviors when working with users can aid superstition identification.

Lack of rationale for behavior

Statements like; “I always do it this way.” or “The person who trained me taught me this method” are frequently signs that can indicate a false model, particularly if the behavior seems odd or overly complex. Such behavior should be noted and investigated further with other users and subject matter experts.

Rationale that is overly complex or simple

If the reasoning appears illogical, very complex, redundant, or unassociated, it could be a signal. Likewise, if there is no rationale for suspicious behaviors, the reasoning seems much too straightforward for the situation, or the response is simply illogical, the behavior might be influenced by a superstition. In either case, the observation should be validated by other users and subject matter experts.

Overly complex, tasks and processes

If the work the user is performing appears excessively complicated or complex, users may have false work, task, or system models. This dimension can be difficult to identify. Because so many work environments today incorporate “band-aid” solutions, cobbled together legacy systems, and non-integrated support resources, even best practice methods can be elaborate. Probing questions can be helpful to assess the situation. If responses to how and why are confident, explanatory, or logical, a superstition is not as likely at play than if the answers indicated the behaviors previously mentioned.

Arcane support resources

When the only way to accomplish a task is through the use of a checklist, notes or procedures, the system model is not understood. While not technically a superstition, this points users to mindless repetition of steps that may or may not be required by the task at hand. Their belief in and reliance on the support resources replaces any independent understanding of the system.

Unnecessary or repetitive actions

Compulsive, repetitive, or unexpected actions (such has pressing the save button multiple times) often indicate a superstitious user. Question the user regarding their behavior to determine if there is a logical explanation.

Avoidance

If a user ignores system functionality, features, or alternatives it may be due to a superstition. Find out why the user is avoiding the function. It may be due to a logical reason such as an actual problem with the function, too little training/support on the function, or the function is unnecessary. However, it may also indicate a superstition where the user is fearful something bad will happen.

 

Usability professionals must be mindful of these behaviors and thoroughly investigate situations where superstitions are suspected. Inquiry involving multiple users across demographics and validation with subject matter experts is prudent if superstitions are to be identified.

How to prevent superstitions

Averting superstition can be accomplished through proper analysis and design techniques. Usability professionals must go beyond traditional human – computer interactions and focus on the cognitive aspects the work involves. It is equally important to identify “how” users think about the work as it is to identify what the work is or how it should be performed. Structuring the work for the user, providing a clear mental model the user can embrace, and embedding knowledge and support into the context of the work can dramatically reduce the cognitive burden and prerequisite knowledge that often results in superstitions.

Specifically, the usability professional must:

1. Perform the correct types of analysis to identify any existing superstitions and gain enough research to make the work process structured and evident. Areas where users need extensive training, job aids, or performance barriers exist need to be targeted for integrated support. Due diligence and blended approaches must incorporate the discovery and validation of requirements as well as the conceptualization of reengineered processes and models. A sampling of techniques includes:

• Contextual Inquiry – Observe actual users, doing the actual work, in the actual work environment and interview them to detail the observation data.</li>

• Focus Groups – Meet user groups and subject matter experts to validate observation and interview research and flesh out processes and models.</li>

• Collaborative Design – Work with actual users to extract their existing mental models. Sketch ideas to represent the system in a manner consistent with how the users “think” about the work. (Ariel PSE 2003)</li>

2. Design interfaces using solid design heuristics. A few superstition-busting heuristics include:
   
   

• Metaphor – An appropriate metaphor can provide the user with a mental model that relates to their work and is extensible for new ideas and functions. Be careful the metaphor doesn’t promote superstitions by implying behaviors or functionality that are not present.

• Establish goals – Recognize the intents the user will have while doing the work and provide clear matches to their goals.

• Structure the work – Clearly orient users to the best practice tasks and process steps. Mark their progression through the work and keep them aware the work completed as well at the remaining steps.

• Answer descriptive, functional, and procedural questions – On each display clearly communicate information that clarifies the work, explains what must be accomplished, and describes the correct approach.

• Provides advanced warning and feedback – Proactively “coach” the user about their decision or its consequences. Be sure to provide clear recognition as to the task status.

• Consistent – Rely on consistent approaches, concepts, and controls. This allows the user to more easily develop and extend their mental model.

• Interpret – Ensure the information provided to the user requires no translation. System responses displays and response should present information in a clear and natural manner for the user. (Ariel PSE 2003)

3. Provide a model for the user. Work with user to identify any existing models or work with them to develop one that is obvious and intuitive. Providing a clear model straightforward model for the user is better than leaving them to develop superstitious ones.

4. Embed knowledge, tools, and other support resources. Follow design approaches such as performance-centered design that go beyond system usability to develop fully supported work environments. The goal is to support the user “intrinsically” with expert advice, best practice methods, and embedded knowledge and know-how. (Gery 1991) Because users can perform the work with less experience or training and receive task related information within their work context, superstitions are less likely to be constructed.

5. Perform user assessments and iterate the design. Let the user be the design’s judge. Construct test based on real-world task scenarios. Observe when the user struggles and investigate the cause. Iterate the design until its use is obvious and intuitive to a new or experienced user. Care must be taken to ensure existing superstitions don’t influence the test. Awareness of the behaviors listed in the “How to Identify Superstitions” section of this paper can help bring attention to mismatched models.

Conclusion

Superstitions can introduce issues and results inconsistent with intended system models. Users experiencing superstition are likely to be confused or misguided in their understanding of methods and procedures. Superstitions can lead to a breakdown in the user’s understanding and trust, further alienating them from an accurate mental model.

Designers who practice diligent, thoughtful analysis and design, incorporate proven techniques and heuristics, and assess their design with users can limit superstition development. When a clear model is provided for a user, the system will be more readily adopted and used productively.

If the user interface is designed to successfully communicate an appropriate model, users interacting with the system will be less likely to construct superstitions and subsequently perform their work with the system more successfully.

 

 

References

1. Beyer, Karen and Holtzblatt, Hugh (1998). Contextual Design - Defining Customer-Centered Systems. San Francisco: Morgan Kaufmann Publishers.

2. Constantine, Larry L. and Lockwood, Lucy A.D. (1999). Software For Use - A Practical Guide to the Models and Methods of Usage-Centered Design. Reading MA: Addison-Wesley.

3. Cooper, Alan (1995). About Face - The Essentials of User Interface Design. Foster City CA: IDG Books Worldwide.

4. Davidson, M.J., Dove, L., &amp; Weltz, J. (1999). Mental Models and Usability. Unpublished doctoral dissertation, Depaul University.

5. DiSessa, A. (1986). Models of Computation. In Norman D. A. &amp; Draper S. W. [Eds.]:User-Centered System Design: New Perspectives in Human-Computer Interaction. Hillsdale, NJ: LEA.

6. Gery, G.J. (1991). Electronic Performance Support Systems. Gery Performance Press, Tolland, Mass. (Previously published by Ziff Institute and Weingarten Publications, 1991).

7. Gribbons, Dr. William M. (1999). Knowledge-Infused Design ( The “Ultimate Solution” to Product Usability. In Help 99 Proceedings. November 1999, 153-156.

8. Nielsen, J. (1990). A Meta-Model for Interacting with Computers. Interacting with Computers, 2, 147-160.

9. Nielsen, J. (1993). Usability Engineering. San Diego CA: Academic Press.

10. Norman, D. A. (1988). The Psychology of Everyday Things. New York, NY: Basic Books.

11. Norman, D. A. (1986). Cognitive Engineering. In Norman D. A. &amp; Draper S. W. [Eds.]:User-Centered System Design: New perspectives in human-computer interaction. Hillsdale, NJ: LEA.

12. Norman, D. A. (1983). Some Observations on Mental Models. In Gentner, D. A. &amp; Stevens, A. A. [Eds.] Mental models. Hillsdale, NJ: Erlbaum.

13. Norman, D. A. &amp; Draper, S. W. [Eds.] (1986). User-Centered System Design. Hillsdale, NJ: LEA.

14. Payne, S. J. (1992). On Mental Models and Cognitive Artifacts. In: Rogers, Y., Rutherford, A. &amp; Bibby, P. [Eds.]: Models in the Mind: Theory, Perspective &amp; Application. London: Academic Press.

15. Payne, S. J. (1991a). A Descriptive Study of Mental Models: Understanding Users' Understanding. Behaviour and Information Technology, 10, 3-21.

16. Payne, S. J. (1988). Methods and Mental Models in Theories of Cognitive Skill. In: Self, J.: Artificial Intelligence and Human Learning: Intelligent Computer-aided Instruction. London: Chapman &amp; Hall.

17. Preece, Jenny (1994). Human-Computer Interaction. Reading MA: Addison-Wesley.

18. Reason, James (1990). Human Error. Cambridge UK: Cambridge University Press.

19. Sasse, M.A. (1997). Eliciting and Describing Users’ Models of Computer Systems. Unpublished doctoral dissertation, The University of Birmingham

20. Tognazzini, B. (1991). TOG on Interface. Reading, MA: Addison-Wesley.

21. Treviranus, J. (2003) Mastering Alternative Computer Access: The Role of Understanding, Trust, and Automaticity. Unpublished doctoral dissertation, University of Toronto