September, 2002 – Even Excellent Sites Benefit from Expert Reviews

Hi! Welcome to the latest usability web cast sponsored by Human Factors International. Today's web cast topic is Even Excellent Sites Benefit from Expert Reviews. During this web cast we will provide insights to the following questions:

• What is an Expert Review?
• How can an Expert Review enhance usability in your site?
• In what ways can even the most well-designed, award winning sites benefit from an Expert Review?

This broadcast will follow our main format of a main presentation followed by a question and answer session.

It is now my pleasure to introduce our main presenter, Dr. Eric Schaffer. Eric is the CEO of Human Factors International. He has been professionally involved in the field of usability for 25 years. Eric has completed hundreds of projects for Fortune 500 companies and is the author of the Schaffer method and synthesis of best practices for optimizing the user experience.

Dr. Eric Schaffer: In our last web cast we talked with Colin Hynes at Staples about the ROI of doing usability work. And we saw how usability can make them millions of dollars. And we did a lot of thinking about that and you might want to take a look at our website, because you can now download that archived web cast. But you know what, people are actually now realizing that usability is not only just nice to have, it is not only something that has a good ROI, it is something that is essential. Be honest with me first I think and just think about how many times you've done an ROI calculation on having the database design professionally done on an application? You just want to see that, right? It was obvious. On doing application, if it is worth doing, if it is worth having database that works well. If it is worth building an application, it is worth having that application to be usable. And people are seeing that and as a result, they are moving more and more aggressively to understand how do you go about making something usable. And it is tough to do that.

I'd like to show you this next slide that's coming up. I'd like you to look at it and I'd like you to take a look at that puzzle and see if you can find the crayon. Okay. Did you find the crayon? It takes a while to find the crayon, doesn't it? Okay. If you can't see the crayon, look by the giant's right hand right at the top of the steeple. Do you see the crayon now? That's good. Now look at that puzzle and try to not see the crayon. You can't not see the crayon because your perceptional system is built so that it shows you things that you kind of know are there. It can resolve those kinds of images for you. They just jump out at you. And that's a fundamental problem that we have in psychophysics that makes it hard to build a usable site. It is a problem because when you design that site and you know that this is what it means and you know that this is how it is structured, you look at it and it makes sense. And as a result of that it is core that you get feedback. Now, there is a great way of getting feedback. Next slide.

And that way of getting feedback is usability testing, okay. Protocol generation (inaudible) think out loud kind of testing is really the acid test of a good site. And it is a great way to find things that could be better. And one of the things that happens to me on a day to day basis, people call in because they want things to be usable and they say okay you know we'd like you to run a usability test. They think you do usability and that means usability test. But in fact, usability testing is one key method. If you don't need to do testing you are certainly not going to have a great site. But before you do usability testing, there's another method that I recommend almost always, okay. Usability testing is great to do, but if you look at the next slide what you'll see is the idea of doing an Expert Review. Now what Expert Reviews do is before you do usability test they allow you to take advantage of the literature in the field. Take advantage of the insights of another expert who hasn't seen the crayon yet. It allows you to very quickly and inexpensively, Expert Reviews, we have Expert Reviews that can take overnight or 2 weeks and they could be very, very effective in terms of time, in terms of cost. And so what you can do is get from that Expert Review the things that are obvious from that point of view from an expert perception, from the literature in the field, from a systematic methodology of analysis. And then you can fix the thing that you can fix right away. And then do that before you build a usability test. Now there's one thing I'd like to make clear and that's that when you're doing an expert review in the way that I talk about it, it's something very different from something called a heuristic evaluation. Okay. People talked about heuristic evaluation in the usability field for a while and what they mean is looking at a site in terms of a set of rules. For some they may have may be like 10 different things like okay I need to make sure the user gets feedback and maintains contact. So and the problem is that the usability field is not that simple. When you do an Expert Review and when we're talking about an Expert Review we are really talking about going through it systematically using what we call an algorithmic evaluation where we take a particular persona and take them through the site (Inaudible) scenario that's appropriate to them and identify that how the site will look to that user population. It also means that we're not looking at 10 principles but thousands of principles of layout, and wording, and color, and highlighting that are well-established, research-based principles. We don't need to do the usability test to know that all capital letters will cross 14% reading speed to fix that. You don't need to do a usability test to know that you need a 90% difference between the color in the front and in the back ground or it gets to be hard to read. So it very clearly (inaudible) that we do this kind of usability testing as a way of improving a site before we do usability test. And that kind of Expert Review is actually something that's worthwhile on an ongoing basis. And what we did to illustrate that is we took some really expert advice from user sites that are excellent because many of them have usability engineers. In fact, you'll find that there's one of those sites that's the site that we actually worked on. And they're also actually because they have brilliant designers, because they are lucky they have a good business model, but their site does work really well. And we took those sites and we did very briefly did an expert review of each of them. And what I'd like to do is show you the results of that because what you're going to find is I mean you might think, well, that we're pretty good. May be we don't need to do an Expert Review. Our site works pretty well. May be we won't benefit from expert review. And what you'll find is in fact, virtually every site benefits from an Expert Review. Certainly in the design process it's key, but also in the process of continuing upgrading and evaluation. So let's take a look at the (inaudible) coming up here. What I'd like to do is share with you the results of those Expert Reviews and then we'll be coming back after that and taking questions and chatting and then Dr. Phil Goddard will join us to help field the questions you've got.

Hi! I'd like to welcome you to the main portion of our web cast. And I'd like to remind you if you look on your screen, you'll see a button that you can use to enter questions, and issues, or comments. And please do that because we'll be talking about those at the end of this presentation. You know, I've been doing website design and various kinds of user interface design back to the main frame days really for 25 years. And I am supposed to be pretty good at this stuff. I've designed may be 300 or 400 major interfaces (inaudible) thousands. I've got a Ph.D. in the field and am board certified. I am head of the largest company in the world that specializes in usability. But I've never once sat down at a desk and designed a screen and got it right the first time. It just doesn't happen. And that's something about best people in this field. They know that it doesn't happen. They know that interface design by nature is an (inaudible) process, which is they design something and then get feedback on it and they design it again. That's what makes good interface designers. And organizations are the same way. They design web sites and they get feedback and the great organizations take the feedback and constantly incorporate it. And each time they get feedback, they get value. It's remarkable. And what we want to do today is show you, we have a list here of some of the very best websites in the whole world, 11 Webby award winners, each one a spectacular success. And yet what we're going to do is look at those sites and we're going to systematically identify three examples of improvements that could be made based on usability engineering principles.

Let's take a look at the first site, which is Google.

Google is the Webby award winner in best practices category. Google offers a targeted search engine. The search engine (inaudible) and ranks websites according to the number of links leading to that site. Over a 150,000,000 searches occur per day on the site. Google also licenses its technology to more than a 100 companies including portal giants Yahoo and most recently America On Line.

Dr. Eric Schaffer: So Google's a pretty awesome search engine. It ends up as the home page for many users. They have a unique search strategy that's powerful and works better than many of the other. But may be more importantly for this discussion, they are known as being ergonomically excellent. Their interface is clean. They've taken off so much content that it's just simply a very small amount of material. Doesn't take long to download and it doesn't confuse many users. It works well for experts, because they can go there and quick downloads and immediately enter just what they need. It works with novices, but they are not confused most of the time. But there are still things we can make better. And here's one example. When people come to a site like this, they're looking for information. And one of the things that Xerox part developed and (inaudible) study extensively that we know is that they are looking for a scent. The smell of the information. They are looking for something that seems like it might give them the answer. And certainly searching the web might give them the answer. Searching images might give them the answer. Well, we have two selections here where I can search Groups and Directory where the scent may not be working for many users. They may not know what Groups means. They may not know what a Directory is. They are both to some extent jargons. It's a little bit like I had one bank that I worked with. And they had I was looking for Visa cards the whole data in Visa cards. And they had it listed it under Affinity Cards. Affinity Card is a term that makes sense to somebody who works in the credit card industry. It wont' make sense to customers. So Groups and Directory are terms that may not work for many customers. And Search Solutions is another example of one that may make sense entirely, but it can also be thought of by many people as another way to get their search done, which isn't what they intend. And you'll get an effect called garden passing where people go down the wrong path and find that they can't get what they want, which is a negative experience. Another example is this one where we have two terms that are very similar. Psychologists of web studying (inaudible) hundred. We take two (inaudible) to make it just slightly different and we measured the JND the just noticeable difference, how much weight does it take in order for people to be able to tell the difference? It's the same thing here. Here we have two terms that are pretty similar. Will people be able to tell the difference between advertise with Google cost per click ad words select and advertise with us? Is it different? Is cost ad words select not us? People may be confused and we're confusing people that we really want to sell to through Google.

Last example I'll show the third one for Google is this one. And this is a navigational example. And I believe that 80% of usability comes from good navigation. If people can come to the site, understand what the purpose of the site is, find things without training, and navigate efficiently (inaudible). And here is a case where the navigational continuer the structure is broken. It doesn't work in a consistent way that a user will expect. I have a left navigation area and sometimes it works the way most people expect. I click on that area, and I see the right kind of material on the right of that left map. Other times I click on the left map and that left map goes away. And I'm left in a whole another space. So I have an inconsistent left navigation operation and there's a real benefit to fixing that. Okay. So that's three (inaudible) for Google excellent and take a look at Idealist.com.

Idealist is the Webby award winner in community category. Idealist.com is the most comprehensive directory of non-profit and voluntary resources on the Web. The site is a project of the organization actually without border, a global network of individual organizations dedicated to finding practical solutions to social and environmental problems. Non-profit career centered, offers hundreds of jobs, internships listings, and voluntary opportunities.

Dr. Eric Schaffer: And by the way people always look at it as Idealist.org. Anyway, here's a really good site. Not only is it altruistic, but it also is designed well in a lot of ways. It's got very clear indications of who it's for. It has sections for organizations, sections for kids and teens. That can be a very good thing. It also gives you indicators of the size of the site by saying there are 29,456 different organizations. That really cues the user in how awesome this site is, how much value there is (inaudible). And there is also what would seem to be a purely low level of clutter on the page. There's a lot of content, but because it's well organized, because it's well aligned, it doesn't appear cluttered. And so they've done a very good job from that point of view. But there are some things that could be better. And the first thing is something that I always look for whenever I am evaluating a site. When you go to this page, will people understand what this page is about? Will they really go, okay I understand what this organization is about and what I can do here? I don't think that would be the case here. I think that it really isn't obvious what this page is offering, what it's for, what the organization is for, and they'll get some drop off as a result. People will come here and not be sure they're going to get the kind of content they want and go somewhere else. So the first thing I would do is make sure we had some kind of indicator of what this organization is about and work on making it obvious what the page and the site is for.

The second thing is we have a set of images at the top and those images are roll over. But I suspect the user will know that they are roll over. It's a little bit like we create a whole world where the only way that we can find it all, we can't look around and find it all, but we have to feel along the wall until we find the door. And we make site hard task when the only way that we can operate the site is by waving the mouse around until we find something that moves or where the mouse changes. And that's a real problem. And the designers of the Idealist.org knew it. And the way I can tell they knew it is they put in click on the pictures above to visit some of the organizations. Okay. So they knew that that was a problem. So first of all, I will fix the problem. (Inaudible) they can click on. That will make it obvious that I can click on it if I am going to use that kind of technique I probably wouldn't use roll overs. And the other thing is if I am going to have an instruction where I click on the pictures, I'm not going to put that instruction down below so that I see the pictures, I go down, and then I see the instructions, and then I can move back on the page. I design pages starting the top left and scan across it down. So if I was going to have a click instruction, I would put it next to or may be even above those images. I am going to try to make the images so I don't need that.

Another example is the hidden function. And the people at Idealist.org spent some serious time and effort developing a advanced search capability, which is a good thing. But the problem is the only way you can get to the advanced search capability is to do a search that fails to find anything. So you can't go directly to advanced search. And so in a sense it's a hidden function. And we would strongly recommend using a more conventional method where we have a simple search and a link that says advanced search, where the users can go to it if they want. Because here what's happening is not only have I hidden the advanced search, but I've also given it to you where you may not want it. Because if you didn't find anything there's a pretty good chance you just spelled something wrong. And you really don't need that advanced search for it. So a more conventional way is going to make sense. Let's go on. That's three and I want to talk about the Library of Congress.

Library of Congress is the Webby awards winner in government and law category. The largest library in the world it is considered also the research arm of US Congress. The collections include more than 18 million books, 2.5 million recordings, 12 million photographs, 4.5 billion maps, and 54 million manuscripts. The library's mission is to sustain and preserve a universal collection of knowledge and creativity for future generations.

Dr. Eric Schaffer: So we click this one in and I want to share something with you. It's one of the sites that HFI has done extensive work on. And we've done structural work, we've helped with some detailed design, and we put it in here and we left. And the Library of Congress staff are watching me right now. But in fact, they've done a great job. This moderation is so much better. It's got a navigational structure which is much more obvious than it was before and which has a much more optimal type menu. I don't have to go to it so many levels. I can go right to where I want. It's organized in a way that people will find things. And we know that because we've done the studies to find that out. It has search immediately available so you don't have to go through levels to find it. And the graphic work is also really good. And we know that because we've validated that through the users. So it's a good site. It's a site HFI worked on. I am going to share three things that could be better.

So the first one I'll show is there's an example where we have an advanced search link. And if you pick advanced search link, you go to a search page that has an advanced search link. I think we have a little bit of a bug in the flow. And that kind of thing leaves the user kind of scratching their head. How did I end up having advanced search that gives an advanced search? So let me get that fixed.

There's another example where here's America's story. This is an area which clearly hasn't been changed for a while. It looks different from the graphic pattern that was set. But it's got may be a little bit more of a concern. Because once you get into this area, we can't find the way to get back to the Library of Congress. I can get back to the America's story home page, but the Library of Congress link isn't clickable. And so I need to have conventional way of getting back. And by the way, we have some good pictures where people expect to look. Put things like a home button, which is at the top left and we know where people look, we want to put it there. So I need to be able to not get trapped down here and be able to get back to Library of Congress main page. So last example is one that I am concerned about and this may be an example where the developers and the team of Library of Congress wanted to add a little Flash to the site and it goes against what? Usability principles (inaudible). And if you go to click on any of the selections at the top like America Remembering or International Horizons, there's an image that appears on the right illustrating it. This is a bad thing to do. Because we have jumble (inaudible). Human beings evolved not really for reading web pages. They evolved for walking around the jungle. And accepting and using appropriate vision in the jungle, you want to look at it immediately. Because it could be hungry. Or it could be tasty. But in either case, you want to look at it right there. Okay. And so we have a reflex built into our visual system called the orienting response. It's a spinal reflex just like if somebody banged on your patella tendon on your knee a ligature, if something moved in your peripheral vision, you'd look at it. And so what happens, if I am going to click on American Memory and something moves over here, I'll take a look at it, which is a significant distraction. So while it may make the site more flashy, it distracts the user from what they're trying to do. That's three for Library of Congress even if HFI has helped, an extensive work, and a dedicated team, there are still things that can be better and an Expert Review gets significant value. Let's go on and talk about Epicurious.

Epicurious is the Webby winner in living category. Epicurious is all about fine dining known for having the world's greatest recipe collection, provides restaurant review, cooking instructions, and everything you need to know about fine wines. The site is also a place to shop for cook books and cook ware. It is the perfect website for anyone interested in world class culinary experience.

Dr. Eric Schaffer: So here's a site where I (inaudible) we talked about you look at it and you immediately have an idea of what it's about. The scent, and in this case scent is almost literal, jumped out of the page and you really have a sense that you can go to this to learn about food, to get recipes. It's very obvious. Just a quick look at the page and it's an example of success from that point of view. It's also successful, we think, in terms of the tone of the site. It's a up beat tone. It's an involving tone. The graphics coordinate with the color. And all of that comes together to make a site that's really special and successful. But we have some ergonomic issues.

One of the things that they've done to somewhat reduce the clutter on the page is to use drop down menus. Don't do that again. Okay. Because drop down menus are not really appropriate in this case. Now in a graphical interface, you look at it and you see file, edit, view, and you know right away that's a drop down menu. In the web what's the convention to use that tells people you have a drop down menu? You haven't got the convention. And because we haven't got a convention, there's no way to make it obvious to users that the drop downs are there. So it's going to be a surprise for the users and they may not find it. Serious problem.

Second thing is drop down menus are actually bad both in the (inaudible) and the web world for novice users and expert users. For novice users, have you ever seen usability tests with drop down menus and the user finds them? And a novice user will pull down search. Which means they go it's not in there, it's not in there, it's not in there. Because we've hidden those functions with the pull downs. So making it so they can easily stand is far better. For expert users it's two clicks to get anywhere. So what we'd like to do is avoid using pull down menus. Now there is a reason for using it. And that's when I have a large working area and I want to hide a lot of functions in a very small space. And in that case, it works. And a good example of that is Word Processor or a Spread Sheet. This is not that. When you want to type in, have a space people can work in, and particularly have more dedicated expert users, and people aren't going to become expert users here and the pull down (inaudible). So let's not use pull downs where they don't fit.

Here's another example and it's a great example of how designs can increase the likelihood of an error. Now in the, the newsletter sign up is really important, because it gets people back to the site. Newsletters areas great. You should sign up for the HFI news letter at humanfactors.com. (Inaudible) and the problem here is that in the text field, where I have to enter my email address, it says enter my email address. So it says enter email address. And if I don't erase that, I'm going to end up with the text enter email address still in the middle of or at the end of my email address, which may spell it up like this. And so I would want to not have it in there partly because of the error likelihood and partly because even if there is no error, the user has to highlight all that and then erase it.

I also want to show you that we had increased visible effort in signing up for the news letter, which would slightly increase the likelihood of drop off and people getting off. Now it's a very slight effect, but you need to fix that up. Here if I want to sign up for recipes, wine, and shop, I will need to put in my email address three times erasing the text three times. What I really want to do is allow me to as you see our example of redesign, enter the email address, and then just check off the boxes for the things I want. So that would be a more efficient, simpler, and less error down there in doing things. I may be looking for that on the site next week.

The next thing I want to show here at Epicurious and the last one is the fact that they use a lot of pop ups. And they use pop ups in two ways, they use pop ups where I pick a choice and I get another browser and since this is a full page browser (inaudible) and the problem there is that the user doesn't often realize that they got a pop up. The only way to really tell us that you look at the start menu at the bottom or you know that the back button doesn't work anymore, which is the more common place. And so now all of a sudden, things aren't working the way I expect and you're confusing the user and slowing them down. So using new browser instances is something really to be avoided. There are only very special circumstances where that makes sense. The other thing they are doing is using pop up ads. And there's an interesting thing going on in the industry today. We all know that banners have tended to become something that people were not to look at very much. Okay. So what we do is we start having pop ups and pop ups initially were very successful because people hadn't habituated to them. They hadn't gotten used to them. They hadn't started automatically just clicking (inaudible) and getting rid of it by buying a pop up stopper. Actually you can get that free. Pop up stopper gets there and makes a (inaudible) when it gets rid of the pop up. So these are reminders for people and very quickly you'll find here people who will learn the pop up comes up just a close it. So that's a strategy which is temporarily good, annoys customers, and will go away soon. Let's talk about Yahoo Finance.

Yahoo Finance is the Webby award winner in finance category. Yahoo is the world's top internet portal. It has a lot more than 220 million visitors each month and offers websites in 25 countries. Yahoo Finance is the company's information site for all things financial.

Dr. Eric Schaffer: So Yahoo Finance is a really powerful website. And I've done studies in the banking environment showing customers liking working with Yahoo Finance much more than many of the paid and members only banking and brokerage sites. It's a great site. It's a site which is laid out well, because it gives you the content you need right away in a (inaudible). In my (inaudible) the control that I need to keep the things in a straight level are right in front of them. The circuit breaks there are here somewhere. The gas, I want to change the gas sites couple of times so they are down one floor. So that's a good design and done by Human Factors Experts. And we use the same principle in website design. We take the controls, we take the features that people need all the time and put them right in front of them. That takes some analysis and they've done a good job at that. They've also done a good job of personalization partly because they've done it in a way where the user doesn't have to go and fill out a long form in order to get some of the benefits of personalization. It happens automatically. It doesn't get in the user's way. And the last thing is they are using graphics in their market summary to let them see the big picture, to see the trend, pattern, and relationship. But let's talk about somethings the Yahoo Finance can be better.

So one of the things is as you look at the home page, a remarkable amount of space is actually wasted. The top area has a lot of just empty space. And there's an enormous ad remember when (inaudible) is what it says this time. And it's taking substantial space that could be used for content. So using that much space for a banner ad really doesn't make sense, I think, in terms of optimizing what the users' experience would be and optimizing space above the fold.

Another example on the next page is the principle of left justification. I'd like you to think of every book, newspaper, and magazine you've seen in your whole life. How many of them have right justified text and a jagged left edge? May be some poetry. But you don't see that and you don't see that for a good reason. You don't see that because it's physically hard to scan down. As your eyes scan it has to find a different place to start reading each time. We also see that because it gives you a false sense of hierarchy. We indent to show hierarchy and here yet in a right justified text we indent just as a artifact, but it's not going to work. So you want to left justify our labels and left justify our data fields. And that hasn't been done here and it gives this page a real sense of clutter. That's not a good thing because this is a page where we are getting people to sign up and we want to make the process look easier. There have been lots of studies done, the goodness of page layout, using (inaudible) analysis, and stuff like that and what they are really doing is measuring the number of left margins on the page. Fewer is generally better. And here with the right justification you have just a ton of left margins. And that gives a sense of clutter.

And the last example I'd like to show of Yahoo Finance is this one. We have a entry of a password required and that's fine. But it's case sensitive. Now case sensitivity adds a little bit more security, but is it worth it? Is it worth it in terms of the frustration of the users. The number of times they'll get called because the caps lock was on. There's a whole nightmare about capitalization for passwords, which isn't necessary for a site like Yahoo Finance or frankly or any site on the web. In very secure monetary operations may be, but even there, you're talking about the trade off between the error rate and the frustration and the amount of work to reset passwords and all of that and the increased security. That trade off is generally not worth while. Let's talk about Emode.

Emode is the Webby award winner in rising star category. It is considered number one destination for self discovery. Emode site has text contained on topics such as relationships, personality, and career. Emode also has information on and links to self improvement programs. Emode's vision is to substantially improve the way that people discover and connect with the important ingredients in their lives: entertainment, information, experiences, and each other. This site has already over 80 million strong.

Dr. Eric Schaffer: Lot of stuff. There's a huge amount of content. And (inaudible) the depth of the content, they are using a two level tab structure, which is okay. People are used to the two level tab structure and it's very obvious what's going on. So that's a good thing. They also have tremendous content. Tremendous amount of content, good demonstrations so you can see what it is before you become a member. So it's a wonderful site. But they have some serious issues with the navigational structure. And again navigational structure is 80% of the challenge when it comes to usability. For e.g. here I'm in the test tab. Okay, I'm in the test tab and then I look down and I have the headers in the main area and it's called self improvement program. But there's another tab called self improvement programs. Where am I? So the user's like to confuse by that. And it turns out as you get deeper it doesn't get better. So if you look at the information architecture, and information architecture is basically the issue of how I do my categorization. A lot of the good methods for that come out of library science. And it's a small part of what a usability engineer has to worry about. But here as we look at that structure, as we go into the self improvement programs, wait a minute, I have a test and self improvement programs. But within self improvement programs, I have a clear personality test. I have the emotional IQ test. I have the ultimate personality test. I have the ideal sexual partner test. Interesting, but if I have a category for tests, I expect the tests to be under there and not to be in two places or not to be over in a place called self improvement programs. So you've got a deep level of confusion in the informational architecture of the site. And that will take some serious work to get fixed. Ideally, the navigational structure of the site, the informational architecture should mirror user's natural mental model of the content. And that's why doing methods of in depth interviews, card sort techniques, those are all methods that we use to get at that deep mental model of the way that people naturally think about this content.

Another example we have here is a place where error handling is going on and it leaves the user little bit at a dead end at a place where he wouldn't want. So if I've gone in and I've picked click here to sign up, I go over to the my account page automatically and that link gives me an error message. I just ask to sign up and it says oops you must be an Emode member to see this page. So I think the user's going to feel like that they have a (inaudible).

Let's talk about AOL.

AOL is the Webby award winner for top US property category. This is the world's largest internet access provider with more than 34 million subscribers. 45% of all time spent on line and on the web is spent with AOL network. 1.25 billion instant messages are sent each day through AOL's buddy link, AOL instant messenger, and ICQ programs.

Dr. Eric Schaffer: So pretty major site. And it's got some really good things about it. One of the things is that as you look at the page, it jumps out at you in terms of what the benefits of membership are. It's not hidden, it's not confusing, and they get their message out really well. So from a marketing point of view, they give us some communication has going to really make sense. They also (inaudible) immediately. You'll see so many sites where you have to pick, you know, click here to log in and then you go to another page and only then you can log in. And here I can log in right away which is really good. So good site, but let's talk about some possibilities to make improvements.

One of the things is that we have two capabilities, a tab at the top and a search field at the bottom that both say search. Now you got two things that are operating the same function. Which do I pick? You know there is a law in psychophysics called the Hick-Hyman law and the Hick-Hyman law says that the action time as a function as the number of choices is (inaudible). What it says is that if I have one light and I pick any button and this light goes off and press the button, you're really fast. If I give you two lights and two buttons, you're slower. Four lights and four buttons, you're slower. It does mean the more ways of doing something it takes more time, because I have to make a decision between them. And so we want to minimize that. I remember when I started working on the Dell.com site, and they had 12 buttons on the home place that all went to the same place and only two remained the same (inaudible) and buy a Dell was the same as let us get one for you. And the user is going to wonder where to click and that's a bad thing.

Another example is AOL gives you a great introduction when you get there it plays the wave files that says you've got mail and you still wonder whose voice that is. But the you've got mail message, which has become kind of the logo of AOL is confused by users and they think that they have mail. And it may not be true because if you get enough spam, you're probably always going to have mail. So it gives you a false message. And so being very careful that we don't take out the user with a marketing message that's interpreted as part of real content. You want to make sure messaging is clear. Okay.

We also have a case and this is something I see as a (inaudible) and it really bothers me and is a bad thing, if you look at the tabs on AOL, none are selected. So where am I? If you're using tabs, stay in the (inaudible) of the tab. If you are in tab, one of the tab is always selected. Because if you think about it, if you have something being displayed, and no tab is selected, when I pick a tab, how do I get back to the thing that was looking at in the first place? I can't. So what we want to do is make sure that we stay in the paradigm of the tab. Let's talk about the BBC.

BBC News is the Webby award winner in news category. BBC News is the world's most recognized online world news source. It provides up to the minute news, sports, weather, and financial news. The BBC has nearly a hundred experienced journalists working around the clock to put together texts, audio, and video, and the vast range of material that comes in from around the world everyday.

Dr. Eric Schaffer: So the BBC site is really good. Obviously it has great news and it is seen as a world leader in terms of objectivity. It also is an indication of a principle of something that we have from the research that density can be a good thing. We've seen studies that compared different levels of density and they compare say filling out a form. And you can have that form broken into three pages or crunched a little bit into two pages, or really pretty tight in one page. It works faster in terms of preference and in terms of performance. It is actually better to have them more denser to the extent that it surprised a lot of usability engineers in the early days. Because here's an example where we have a very dense page, a lot of content, but that's okay, because it's giving real value and because it's organized very well. So it's an excellent site from that point of view, but there are still places to fine tune it.

One of the areas that concerns me is again in the navigational structure (inaudible) 80% of usability is a function of good navigational structure. And they have a global menu, news world, weather world, and so I go from news to sports and the menu is still there. Okay. But then when I go to weather center, I don't go news, sports, weather I go category TV, radio, and I don't go adv and that has an index. And so suddenly my global navigation that should be consistent, suddenly it's changed. And one of the things to know about people is human beings remember things like special location. That's to say that we're not very good at remembering file names that were good evolved or remembering where we left our lunch. And so if things lose on me, if things aren't stable, then I have a problem. If I keep things stable, I remember things because I know that news is always at the top left. And that's a good experience to give your users. That's why consistency is so important. We also have a subtlety here, and this is one of those details designs. And detail design and wording, layout, color, highlight, it is not 80% of the issue like navigation. It's like fleas on a dog. The dog has one or two fleas, you wont' notice it. If it is tense and itching and if it has 300 fleas it's dead. And your site's like that. If you have a couple of wording and layout problems, it's not a big deal. If you have a lot of them that add up to create an experience of cogging and friction. Cogging and friction just means it's hard and frustrating to get through that site, because there are little things that you may hardly even notice, but you keep getting slowed down. And it's not a place you want to stay. So if you look here on the right, they have a listing and that listing is right justified and seems like the left justified. It also has two clear logical groups and just like putting in a little it of white space it will help break up those groups to make it easier for people to understand because (inaudible). It's a small simple thing to do and we'll be looking for it next week on the site.

Another example is part of the categorization and again this is a small thing where we have a section called services and under services it has feedback and help which really don't fit too well under services. I'm not sure (inaudible) and I'll leave it. You want to make sure that again our information architecture works that we have been categorizing it where the headers will tell me where to look for things. Let's talk about ESPN.

ESPN is the Webby award winner in sports category. ESPN is a super star of the sports broadcasting world. The leading table sports broadcaster with 6 domestic networks that reach more than 85 million US homes. ESPN also creates content for TV and radio and operates one of the most popular sports site on the internet.

Dr. Eric Schaffer: So here's a site which has good information architecture. And it has two main ways of access. One of it is it has a different types of sports. So if you look on the left, you can go if you were uninterested in golf, uninterested in football, and it's immediately obvious where to go. It has a second way of looking at it, which is the headline. And so it has two structures which is not too many. Now if you get three or four or five, or six, you're really over the top. You don't want a lot of different ways of getting at the material, but two different ways is okay. And these seem to me like some of the reasonable approaches with this user population. So good site in terms of informational architecture.

Now if you happen to be flying and using a reader, this site is a terrible failure. It violates most of the principles for acceptability. And that's one of the things you need to really think about whether it's important in your site. Whether you want to consider following the section (inaudible) government guidelines, whether you want to consider reaching those kind of users. It's something that I think will be much more important in the future when we start accessing web pages over cell phones probably the cell phones built into our teeth. But in the mean time, it's something that I'll consider to at least get a level A accessibility in your site and here for e.g. the all tags are pretty much all the same image, image, image. It makes it very difficult for somebody with a reader to understand what's going on in the site. Some people who can't see might be interested in sports.

Another issue on the site is while we have good informational architecture, we have some graphic problems. And the graphic problems result in a real sense of clutter on the page. There's a combination of lots of different colors, lots of very small graphical embellishments, and probably worst of all animations which will tend to trigger the orienting response and all of a sudden you are looking at a site (inaudible).

Last thing I'll talk about is target resolution. And one of the things that we know is that when you are building for the public web, you want to build for an 800 x 600 resolution. This site is built for 1024 x 768. And that means that a substantial part of your population will have trouble with the site. They are going to lose that right command area. So we build our site for 800 x 600 with elegant degradation of 640 x 480 and of course it will be legible. And it may even be affluent (inaudible) change to be an even better in 1024 x 768, but the target for the general population today we have lots on data on this is still 800 x 600. Let's talk about Amazon.

Amazon.com is a Webby award winner in commerce category. World's largest internet retailer sells books, CDs, DVDs, toys, tools, electronics, home and garden supplies, health and beauty products, and prescription drugs. Millions of users in more than 220 countries log to Amazon.com.

So Amazon is an incredible site in terms of sales. They developed a one click (inaudible) a brilliant method so that they hold on to your information. You don't need a wallet. You can just go in look at a book that you want and click and you are done. And I have bought many books that way and I think many of us have. Another thing that's cool about the site is that they understand the flow of their users' work and in particularly the book area the vast (inaudible) of users come in knowing the title and the author. And the search is right at the top left and user can put in (inaudible) what they want. So it's a good site from that point of view as well. It's also an amazing site in terms of their ability to use personalization without getting in the way it knows who you are and it shows you things that might interest you. So it's a functionally wonderful site, a very successful sales site. But there are still things that might be better.

One of the things that we see is that there is a tab at the left called welcome. And the population stereotype would be to name that home. It's probably not going to damage anything terribly, but people expect things to work in a certain way. They learn that things work in a certain way. And if we follow those conventions it puts less demand on the user and they don't scratch their head wondering what you decided to do this time and we'll try to make all the pages welcoming.

Another example is and probably because there are probably dynamic with creating their pages, there is some inefficiency in the (inaudible) that makes pages much longer than many today. Here's an example where they are wasting a lot of real estate and they could just take the special features area and move it to the right and have plenty of room for this page. So if you want to limit the amount of scrolling. Now it used to be in the early days of the web only 10% of users would scroll on a given page. It's not that bad any more. But I'd certainly think users fail to scroll on this really valuable information. Readers tend to scroll all the way to the bottom would then scroll part way down and then stop and the link that they need is just below it means that (inaudible) get down there and they often don't. So, (inaudible) usability testing staff and kind of reading on (inaudible).

The other thing is an information architectural issue. And we want to have consistency of the levels and the sequence of the groups and the categories that we are working with. And we can see here how we don't have a consistent sequence between the tabs at the top and the link to the left. And we don't have a consistency in terms of the levels that they are at. So making things consistent will help the user, because they remember things by organization by logic and by station location. The Onion.

The Onion is the Webby award winner in humor category. (Inaudible) are new which is updated weekly. This website was launched in 1999 with a 1.5 million readers visit this site every week to start their day with a laugh.

Dr. Eric Schaffer: So, here's an example of a site which uses a classical newspaper grid and that's very effective. The grid gives you left justification. It gives you organization. It gives you a consistency from one edition to another. And so it worked in the newspaper world and it's been brought into the web successfully. So that's a good part of this site. Another good thing is that they are providing versions that offer mobile access from a PDA. And certainly that's something that you're just beginning to see and I believe we'll see that as a growing trend. The other thing is the site's down side.

But there are some things which can be made better. One of the things, kind of interesting, William Albert from Lycos published a paper at the UPA 2002 this last fall. And the paper was called Do Web Users Actually Look at Ads? Which is a core issue for Lycos. And I went through this data replicated, because we are so interested in it. But the data was really startling. What they found is that if you take a banner ad and you have it at the top, the way it shows here, and when you compare it with that banner ad shown below the logo, the Onion's logo and closer to the actual content, you get 7 times the visual access to that banner ad and yet measurable increases in brand awareness and click through. So while people are clearly habituating to banners, if you put the banner not above as it's here, but down lower, you get a really big difference in terms of the efficacy of the banner. And it's seen that from the data so far that it doesn't interfere with the user experience and you don't see (inaudible) or anything like that. So this ad could be smaller and it should be down lower.

Another issue is we have again animation. And we have animation in this case while I am trying to read a paragraph. And that really makes reading paragraphs difficult. And so it's I understand it's important to try to get people to pay attention to the banner ad, because it's business model, but they make it really hard to read paragraphs. In time what's going to happen? This is a good example where the banner ad is out here and it's just annoying me. And I kind of get used to it. But what you'll see in the future is the ads are incorporated into the content. And so just as you move you have (inaudible) and the Coca Cola bottle shows up in the movie itself as part of the spot. You'll see that actual ad content showing up as part of the content of the site. I think you could take that to the (inaudible) and you'll find that a lot more effective way of getting brand awareness and getting a quick user platform.

The last thing I'll mention here is the idea of affordance. What affordance means is people should be able to tell what they should click on in the site. And on the left we have pictures and they are not selectable. On the right we have a picture which is selectable. And there is no difference between the two and so the only way I can tell is by trying to click it. Waving my mouse over it. I don't know why I want to click the picture on the right, but if I did, I might not be able to tell that I could. So we want to make sure that affordances are carefully designed so users can tell what they click on. And we have very easy tests that we can run to tell whether the affordances are working. You can actually run them with a cookie test. In a cookie test what we do is we go on to a cafeteria and we set up a table and we ask people come up and we show them a website and say circle the things you think you can click on. And when they are done we give them a cookie to thank them for doing it. Which is why we call it a cookie test. And then you see what percentage of things people think they can click on. It's a great way that you can measure affordance. Okay.

So stay on line with us here. What we're going to do we are going to take your questions and there's time still to put in a few more. And we're going to chat about the ideas you got, the comments you've got, and the questions. So stay with us.

Welcome back. We hope you enjoyed our main presentation. For the next part of our web cast, Dr. Phil Goddard will be joining Eric to address questions and comments sent to us during the presentation. Phil is HFI's Managing Director and Chief of Training. Phil has worked close to with Eric over the past 12 years on the Expert Review of hundreds of user interfaces. As HFI's Chief of Training, Phil was instrumental in the development of HFI's certification program for usability practitioners.

Dr. Phil Goddard: Thank you. Thank you. Great to be here.

Dr. Eric Schaffer: So, we got questions and by the way, if you still have a couple of questions, on your mind, we're still taking questions. So just go ahead and press that button and type in your question, okay.

What role does server log analysis take place in your testing? Doesn't log analysis give more true feedback than what users tell you in a test in a much larger base size? In my experience, combining user tests with log analysis provides a powerful toolkit for improving the user experience.

Dr. Eric Schaffer: Okay, so role of log analysis but first of all do (inaudible) users tell you in a test? Yeah. Well, I mean part of it is that it is important. Just I want to make sure this is clear that when we do testing with users, they tell us things by their behavior and we care a lot about that. And when they tell us things about the design or what their intention is, we care a lot less about it. For e.g. if you ask a user whether they care about privacy, you'll get a very large percentage of people saying yes I care about privacy. And then you go and you look at their actual behavior in a shopping experience and see how many click on privacy policy. It's not quite numbered. It's close to may be a few of them will (inaudible). So I think that's a point all of you bring on.

Dr. Phil Goddard: Okay. But I think yes. I think the answer to this question is yes. Server log analysis is a great compliment. You don't necessarily get direct usability data from a log. But you will get some interesting information that compliments the usability strategy. For e.g. where are people visiting the most? I mean that is telling you are you doing right. Right, and you want to probably amplify that traffic, if you can't...

Dr. Eric Schaffer: (Inaudible) site map. It tells you (inaudible)

Dr. Phil Goddard: Yeah, where people are dropping off the site. Those are obviously a benefit. It can also help determine you know the demographics. Some of the, where people are coming from and this we've actually used to determine how we are going to sample in terms of for e.g. international usability test. We've got people coming from different countries and we're developing an international strategy that can help us determine you know how important it is to go in and test the site at various locations. So I agree. I think it is a great compliment. It is very difficult to search through a log so you know you have to spend a lot of time fine tuning the strategy in what to look for. But I would agree it is a good strategy to compliment.

Dr. Eric Schaffer: It is a compliment but you can't just use it because you know for e.g. you might find a page that has a huge drop off rate. But is that because that page satisfied the user's question or pissed them off? And you don't know from that.

Dr. Phil Goddard: Right.

Dr. Eric Schaffer: So really what you do is you're taking your usability testing data, your Expert Review data, you're taking all those different kinds of information along with that and get a very good picture of what you can do to make that site better. So, good.

Who should conduct expert review, that is, what skill sets should be involved?

Dr. Phil Goddard: Oh that's a good one.

Dr. Eric Schaffer: Yeah.

Dr. Phil Goddard: Because what's interesting from the training point of view is what we find is there's (inaudible) in this case. There is always a point at which a specific (inaudible) breaks down. And so a lot of the value of using the evaluation of Expert Review is knowing how to balance the trade off. When teach you know in a web class heuristic, I am always happy to try the heuristics out and I have to speak a little louder here. Try the heuristics out and most everybody enjoys using them. They find them a beneficial in guiding their directions in terms of MOIs in site. But the value of a good Expert Review is as you said in your white paper is on how you prioritize and solve specific problems. So it is one thing to identify problems and a very another thing to know how to actually turn that problem into a solution.

Dr. Eric Schaffer: Yeah, I kept wondering if the white paper was very good. The thing I would say is that as you go to getting somebody to do an Expert Review, you want somebody who has an in depth of the literature, of the principles, of the methods, and of the philosophy of software ergonomics or usability engineering. Somebody with good qualifications in that area is essential. You can get somebody to do an Expert Review just doing it based on just common sense. You'll get limited value off of that and in fact it will be wrong a lot of times. So if you want to use the most current literature, if you want to use a systematic methodology of review, it's something that's non trivial. So anybody and you can go to (inaudible) say go find something running on the screen while I drink wine or (inaudible) or something, tell me get up and you know yeah review the site. They will tell you some things that are wrong with it, but could get somebody to systematically go through the site and really bring out a good chunk of problems and opportunities that takes somebody who is really professional in the field.

I heard you say something in the beginning about the difference between Expert Reviews and Heuristic evaluations that the difference is with Expert Reviews you can use personas and scenarios to evaluating sites. But I have not heard any reference to any specific scenarios or personas absolutely to any of the recommendations. Please explain how the (inaudible) you are evaluating against relates to personas and or scenarios.

Dr. Eric Schaffer: Very good. I'm doing a review of a financial site right now. Some of the folks there are probably with us right now (inaudible) and it's a fascinating site because it uses like two very different groups of investors. And one group of investors are working with investment advisors. The other group of investors are just doing (inaudible) they are using very similar interface. And for one group the site is just way off and the other group, for me, it's more better. So you can never look at a site without understanding user task and environment. And we always do that. Now in this web cast you will find fitted into half an hour. We end up 51 minutes and so we didn't have time to go through the full logic of what we did. But in an expert review, even in a page review, we do work of particular users doing a particular task. And by the way, we have lots of questions coming in. So very good. We'll keep going here.

What about using photographs for branding purposes such as to create a warm supportive image?

Dr. Phil Goddard: I think it's a natural part of brand strategy. Photographs are you know are graphs and you have to ask yourself from a functional point of view what is a function of your graphics. So we think a graphic having different functional purposes and photographs add to brand appeal and brand definition. But you have to recognize if you are using images just for branding they can get frustrating after a while. So what we look for is how much branding you need to do. If it is a service site for e.g. Eric talked about a financial site he has been working on. You might want to brand a little bit on the top level to create that who we are and what we are about. But as you move down on the surface service area to the site, you want a strategy that starts to minimize or removes the size of the graphics and that whole emphasis is getting the brand.

Dr. Eric Schaffer: Please make sure the graphics are useful for that purpose. They're not just (inaudible). Now there are some graphics that are really bad like you see a lot of sites that use a huge amount of real estate on hero shots. Hero shot is a shot that you really take from the top and (inaudible) okay I do really need to get that downloaded. Sometimes I got a site I am reviewing right now that shows a woman getting her (inaudible) by a dog and it is taking up like half a screen (inaudible) they don't need that. So you want to make sure the graphic's useful. The other thing is we want to empirically test whether we're getting the image we want. So it is not enough to say (inaudible). Okay, we'll go out. But when we do our testing, we actually systematically lay out the different design options we've got and one test to tell us the impression it is giving people. Okay.

How much time does an Expert Review take on a per page basis?

Dr. Eric Schaffer: Okay. So the thing to know is that there are different kinds of reviews. And there are some reviews which are really just focusing on the page design and looking at it in terms of the detail design, is it following principles of lay out and wording and color and highlighting and graphic usage. And they are very valuable. We got some clients that literally won't put a page in front of customers until they've had a page review. And you can have page reviews on our home page. And page reviews on our home page you can get them if you give to us by 9 p.m. Central time, we give it back to you by 7 a.m. the next day and we charge $640 and we do it over night. So it literally isn't time (inaudible) and if you wait 3 days $260. So that's something which is very, very practical. It doesn't get any simpler task at all. On the other hand, you know, for Expert Review we were looking at strategy, and structure, and really getting into depth, into understanding the personas. That's another level of value and that takes a couple of weeks.

Dr. Phil Goddard: You know an actual (inaudible) question we had earlier about personas and scenarios and this is a good point to make because there are certain kinds of as you showed today, certain kinds of page level particularly you can do you described and that's a (inaudible)

Dr. Eric Schaffer: Well, yes. There are details of (inaudible) and we need the people and we need some basic scenarios. But we can't get into the kind of depth on those that we do in a real Expert Review. We will come there and some times.

Dr. Phil Goddard: Right. But to do a good review on the information architecture, navigation, and consistency issues across pages, it is difficult to do on a pager per page basis. Because the question you are asking is about interaction across pages. It's got to be focused and you have to think about it slightly differently and I always say that because that's where the values, scenarios, and personas come in to help you envision a complex interaction sequence moving through the site in a specific way.

Dr. Eric Schaffer: Yeah, and I sometimes have people coming and say I have a site that has you know 50,000 pages and I'd like you to review it. And I go okay, I can do that in 2 weeks. And they get absolutely worried, no, no, you don't understand. If what we're looking at is how effectively the users would have structure and we sample the pages of each of the design, we can pretty much do any reasonable size in 2 weeks or so.

Dr. Phil Goddard: That sounds right.

Dr. Eric Schaffer: What is the demonstratable ROI for Expert Reviews?

Dr. Eric Schaffer: Okay. Well, I guess first of all Expert Reviews are core to the overall process of usability. And as I said, for me it's not an issue of ROI so like if it's worth building it is worth building right. It's worth building so it's usable. On the other hand, it is easy to find pages where you just notice some point in your site where there is an instruction that's confusing where there is a link that goes to the site you wouldn't expect and you just are going to lose all the customers you got at that point. I did one Expert Review for a credit card company some years ago. And I was sitting with one of the top executives and she said you know this usability stuff seems like it is pretty important. I said let me show you something. And we went through the main page that they used for answering questions coming in on the phone. And I showed her how our Expert Review had identified three unnecessary tabs. And four unnecessary data entry (inaudible) and we added it up and the (inaudible) was that was tens and thousands of dollars a year.

Dr. Phil Goddard: Yeah. I am thinking about that one large ecommerce site we worked on where the first page you see is a zip code entry field.

Dr. Eric Schaffer: Yeah.

Dr. Phil Goddard: And they were getting a huge drop off rate because down the left hand side there were a lot of links inviting them to leave this site. Before we did any structural redesign of the website, we said let's just improve this one page. Do you remember what the ROI impact was?

Dr. Eric Schaffer: It was a million dollar increased sales a month. We got that.

Dr. Phil Goddard: So this brings out the point that not all fleas on the dog are treated equally. And you know some of these things can have huge impact, but what we need to do on demonstrating ROI is we need to benchmark. We need to have some kind of a criteria against before comparing our design evaluation impact.

Dr. Eric Schaffer: But I think if you have any kind of understanding of the routine value of usability, then you don't need to do a big ROI study. We looked at it so many times. It's worth it.

Will this web cast be archived for future viewing?

Dr. Eric Schaffer: Yes. It will take us about a week to 10 days to get it on our site. And you'll find a link to it from our home page and you'll see the link to the previous web cast on ROI.

In your opening remarks you seemed to minimize the value of a heuristic evaluation versus your suggested systematic approach. Were do you see heuristic evaluations most beneficial? Based on your presentation, I'm not clear on the distinction you can make between an Expert Review and an heuristic review. Most of your points – left justification, surfacing advanced search etc... seemed heuristically based.

Dr. Eric Schaffer: I guess part of the issue is what you mean by heuristic. If you look at the classical example of heuristic, you get 10 principles and left justification probably isn't principle, it's probably design the page so that is easy to visually access. It's something like that. And that's a very different level of look. So let me try to lay these things out a little bit. And the way to look at all the messages feedback can be good, but as we go on we want to use more and more powerful methods that get us more and more fine tuned data and so a heuristic review is something that somebody who's taken our course perhaps and has a set of heuristics and he comes in and he'll experience if he can look at the pages and look at the site and say gee this is sort of following the kind of things that I'd like. And it's a reasonable thing to do very early on, but don't rely on it because we have good data that it doesn't catch you know all that many things. And it isn't terribly reliable because it's not specific.

Dr. Phil Goddard: Yeah, I think that here's my comment, okay. Heuristic has been traditionally understood as second order principle. If you look at the things Nielsen's Heuristics or even Gerhardt-Powals' Heuristics now that has evolved that research based heuristics. They would apply to any technology whether it's webGUI, hand held PDA that means that there are basically coming from human factors point of view to understanding human capabilities limitation and creating general rules.

Dr. Eric Schaffer: Very general.

Dr. Phil Goddard: The difference I understand, Eric, between the heuristic and our algorithmic reviews is we've taken the second order principles and applied them as first order level. That means two web site designs, two GUI designs and come up with very specific guidelines that actually are easier to follow and more clear in their specificity. For e.g. information architecture. Creating distinctive labeling systems that follow parallel construction. That's not a heuristic.

Dr. Eric Schaffer: Parallel construction means that I don't do that (inaudible) I don't have some verbs and some nouns. Because if I have something like a word processor challenger spread sheet, no I'm sorry. If I had something like hired employee, faster input, and spread sheet, then spread sheet can be read as verb and people will like (inaudible).

Dr. Phil Goddard: Yeah.

Dr. Eric Schaffer: The spread sheet. A principle like that doesn't ever show up in heuristic study.

Dr. Phil Goddard: Right. I was just talking about it earlier about the same. And (inaudible) has popularized. He applied that to link design. And there's a whole set of guidelines now that we know (inaudible) more descriptive links on idea of linking, linking material work. These aren't heuristics anymore. These are very specific algorithms for design.

Dr. Eric Schaffer: The other thing is you have a distinction that I made between a page review which is more like an ergonomic editing process and an expert review with really a global look at how the site is working, what the conventions are, what the process is, and those are two very different levels as well. So you got a number of different tools you can use. And I talk to people everyday to kind of sort out what the right (inaudible) this would be given the application. And it's a little tricky to do the right one, but it's really critical. Two (inaudible) is that we'll cost effectively get you to the design that's good. Okay.

So what is the solution to the drop-down menu? Can't drop-down menus be effective in portal spaces where the bulk of the page should be devoted to the user experience, not navigation?

Dr. Eric Schaffer: Well, wait a minute. What do you mean by the bulk of the page devoted to the user experience not the navigation? A portal page is mostly about getting used to the things that are there. By nature it says this is I mean it is sort of the main menu. May be the menu in your internet or something like that. And so I want to use, and I have seen cases I love it and this exactly was the point. Okay, what we are going to do is we are going to have a little area with a pull up. And you can't really tell that it's the pull down. And you won't notice unless you are fairly constant (inaudible). And you know that people will start reading (inaudible). And then you have this great big image they can't do anything with it. Okay, I have my experience, thank you. And so the thing is that drop down menus are tools that can work in very specific circumstances. That doesn't sound like one to me. And it tends to be most useful for expert users who are doing things where you need to hide a lot of functions and gives them a lot of working area with valuable content. Now if you have a portal where I'm looking at my account and then I can't possibly fit all the functions I need to go and do things with links within that this way and with a set of tabs or section of tabs (inaudible) okay, may be that.

Dr. Phil Goddard: Well, can I?

Dr. Eric Schaffer: Yeah, sure.

Dr. Phil Goddard: I've done a little work lately with enterprise portals and we had used drop downs effectively there. And I think the key trick is to consider this pattern. What a drop down does as you suggested is for the expert users and it slackens the information hierarchy actually. And what that means is that you can show two levels of a hierarchy without actually showing them. Which means if you are down to level two, you need to get back to level one without having to go to the back button. You can get to any level from level two. So they have the notary attribute of being good because they hide functions and bad because they hide them.

Dr. Eric Schaffer: Right.

Dr. Phil Goddard: So for expert users I found them useful. So the question here if you mean by user experience what we are saying is making a dent to say at the top level that provides the interaction that the user (inaudible) the information I think it's a good thing.

Dr. Eric Schaffer: Yeah. But if what you mean by user experience is I'm just looking at some images or something like that, then you don't need a pull down and so you really push to do that by the depth of the information.

Dr. Phil Goddard: Yeah.

On the newsletter signup example – your recommendation makes use of a non standard html button (aqua in color). Are you recommending moving away from the standard html button for aesthetic reasons?

Dr. Eric Schaffer: I guess what I would say is that aesthetics will make you where, obviously you can use a standard button. That's a good thing. You get to save download time and all of that. On the other hand, you may need a non standard button in order to get the right kind of semantic impression and so it's a balance from that point of view. All things being equal, use the standard controls. But as you really have the desire to create a specific impression and you want to test that you've done that, then you may be willing to handle an additional mode of (inaudible) for the custom control customize it.

You mentioned in the presentation that the labels should not be justified to the right since it makes it less readable. How can this be better when most of the applications and operating systems that run in our computers use the same type of format such as when right clicking and viewing changing the property of the file?

Dr. Eric Schaffer: You see right justification?

Dr. Phil Goddard: I'm not sure if I get that one. I think the point that Eric made is a certainly good one. Basically a measure of complexity is the number of unique horizontal lines and points on a page. Vertical lines and points. It takes the eyes longer to (inaudible) a jagged left edge than a solid left edge. Our research has found that experts prefer left justification on field labels and left justification of edit fields. What was funny about that page is I was thinking well, may be they are trying to satisfy a localization challenge.

Dr. Eric Schaffer: That is one thing that pushes you a little the other way is if you're trying to allow you know an easy just automatic globalization.

Dr. Phil Goddard: Technique for handling an extension of that.

Dr. Eric Schaffer: But you really at that point you know that getting a real legibility problems where I have you know long labels and short labels and I don't find it.

Dr. Phil Goddard: Right.

Dr. Eric Schaffer: Scan that is really (inaudible).

Dr. Phil Goddard: Right. But I'm not sure about the right mouse click example here. May be that means the dialog that pops up.

Dr. Eric Schaffer: I don't know.

Dr. Phil Goddard: I'm not sure yet.

Dr. Eric Schaffer: But on the other hand it gets you what you want to do with quality usability principles is just look at Microsoft here in a lot of trouble. It certainly limits (inaudible) that are not really fully out optimized. None of them are really fully optimized.

When is it appropriate to launch a new browser window when the user clicks on a link?

Dr. Eric Schaffer: Oh, great question. I guess the primary obvious case is when I'm working on some objects and I want to get information that refers to that, I may want to take the risks of popping up, now the problem's I got another pop up stopper running and I click on it and nothing happens. The pop up stopper will beat that pop up.

Dr. Phil Goddard: Are you sure? Is that true? The pop up stopper will stop in your own...

Dr. Eric Schaffer: Oh sure. Any pop up...

Dr. Phil Goddard: Your own native...

Dr. Eric Schaffer: Yeah.

Dr. Phil Goddard: Secondary windows.

Dr. Eric Schaffer: In an html environment, yeah.

Dr. Phil Goddard: That would be a dumber.

Dr. Eric Schaffer: Yeah. It actually stops your http pop ups in a very different way sometimes.

Dr. Phil Goddard: (Inaudible) not to use that.

Dr. Eric Schaffer: Yeah, absolutely. But you may take the risk if I'm working at, let's say I'm working on putting in trade and I click on cell and I go wait a minute. How much have I got of that security? And I don't remember. Or how much cash have I got in my account? I may want to be able to top up so I could go check it and then put it away and then go ahead. Or help (inaudible) where I may need an explanation I don't want to (inaudible) so I click on that, I get help. Okay. Now I still don't understand it (inaudible).

Dr. Phil Goddard: Right. Secondary window but it serves the primary window.

Dr. Eric Schaffer: Right.

Dr. Phil Goddard: Right.

Dr. Eric Schaffer: And that would be the primary fix. This also an argument when you are going to another site, it will reference and the person (inaudible)

Dr. Phil Goddard: The third one which is a little bit tricky to deal in a transactional environment when you're trying to hide all the options to navigate elsewhere and you want them to either delete a record in the database, get a new database record and you want them to finish that function before they do anything elses.

Dr. Eric Schaffer: Do we not have any backup?

Dr. Phil Goddard: So it's kind of like an open record, you're trying to preserve that motto (inaudible) projects because you don't want to move around in the middle of the trasaction.

Dr. Eric Schaffer: But can't you just go to another so if I click on so let's say if I have a menu, I click on the menu and when I get done of transaction, I go back to menu.

Dr. Phil Goddard: You could (inaudible)

Dr. Eric Schaffer: Yeah. Or not even (inaudible) just stay in the mean browser. And when you're done with the transaction take them back to the (inaudible) I prefer to do that unless I'm really pushed which can be (inaudible). Good question.

How important is it to have consistency throughout the complete site? That is from a corporate site to regional and office sites for government agencies?

Dr. Eric Schaffer: Very good. I think we talked a little bit about standards (inaudible) 150 plus custom standard by now. So let's not (inaudible) about it. The answer is that as you go from different sites within an internet or within an e-governance environment, you will have the need to have different brands, different unique selling propositions for the people coming in to it. You'll have the need for different functionality. But you know that you won't have the need for you won't need to call the button that you use at the end of a form submit in one and say okay in another and go ahead in another and cool let's go on in another. It can just be one. And so while it's necessary to have different brands and purposes and functions. You should have a standard set of page template that ensures that for the kinds of pages that you proliferate all the time (inaudible) forms and search and it is just that you'll find 15, 20 different kinds of pages. There ought to be a standard set of rules about how those would be designed so the customer, the user, and (inaudible) now what does fetch mean? I've seen find, I've seen search, but what does fetch mean.

Dr. Phil Goddard: Yeah. Any time that a user is going to automate it for office, they're going to create a mental model for how to navigate. You want to make that consistent model. And the general rule of thumb is what you need to cross site is the content. But the same should be the navigation mechanism in the way that something's (inaudible)

Dr. Eric Schaffer: Because I don't want to think abut how to navigate. I want to think about time.

Dr. Phil Goddard: Right. By the way, can I just say that that is one of the greatest things about an Expert Review since the word did come up consistently. The research I see is that that is the one incredible value add with a usability test on that. In other words, to begin to look for that one thing that differentiates an Expert Review is the issue of consistency. Because experts are extremely good.

Would you talk about the differences between websites and web applications in terms of usability and the use of design convention?

Dr. Phil Goddard: A million dollar question. Go for it?

Dr. Eric Schaffer: Okay. So the first thing I would say is that website, web application there's almost a fuzzy line between the two. You can go all the way from (inaudible) to a very complex (inaudible) application. And the answer is that usability principles apply to all of them. But as you get more complicated, as you get to a web application which has a lot of interactivity, a lot of different material in it, a lot of different states, the usability challenges are greater and greater and you need a lot more force power from usability engineers (inaudible) to succeed there. So I would say that it's the same usability principles. I still want consistency, I can (inaudible) methodology, I still want usability testing in the process, but the stakes becomes in some cases higher and harder to achieve as you get the complex applications. So in terms of our practice for e.g. it's relatively rare that you would do just (inaudible) website. But when you start getting more interactivity, when you start having a site which is also very much an application, then we can say (inaudible).

Dr. Phil Goddard: I would say, actually the way I think about it is websites are a lot easier to design because they basically feel natural in a browser. A web application doesn't feel very natural.

Dr. Eric Schaffer: Yeah, it's hard to make it work.

Dr. Phil Goddard: So what you're doing in an application design is you're trying to build sub-sufficient pages that are independent of the browser that create a strong mental model for staying within the page and not using the browser whether they are intended to be used.

Dr. Eric Schaffer: Yeah. And you are also struggling with you know I can't (inaudible)

Dr. Phil Goddard: Technology issue.

Dr. Eric Schaffer: Okay. It's fine with them.

Dr. Phil Goddard: And here's the one case where you might start using frames a little bit more than we would in a normal content based site that we're trying to create (inaudible)

Dr. Eric Schaffer: Yeah.

Dr. Phil Goddard: And transactions that keep the user in a context. So our approach is very different. And the convention for buttons versus doing trying to make it look like a view versus look like a website is a tricky one.

Dr. Eric Schaffer: Yeah.

Dr. Phil Goddard: It's an excellent question.

Dr. Eric Schaffer: Very good.

Where am I going to find a systematic methodology that you are referring to for expert evaluation?

Dr. Eric Schaffer: In our classes we teach some of the principles and some of the methods, but HFI actually has a proprietary set of tools that are processes that we use. Now we do have a tool called usability central which contains the entire Schaffer methods and much of our Expert Review process as well. So that's probably the best, that's resourceful.

I have a lot of people contacting my company saying they can do Expert Reviews. How do I sort out who's real and who is not?

Dr. Eric Schaffer: Yeah, that's an interesting problem, because indeed the good thing from our point of view is we see a lot fewer companies since the tech bubble burst who are running around and going we're usability experts and you know we've got human factors people here and (inaudible) that they were human factors specialists. That's good and what's your background? Well, I am a human. So I must be a human factor specialist. It can be really a problem. So clearly it makes a difference to get really good qualified people. And you can go to a company with long experience in the field, credentials, and good staff, and good process, and that's probably your usability standard.

Dr. Phil Goddard: I think so. In the Expert Review is basically the weight of the value of an expert review is in prioritizing and guiding someone actually in identifying the problem and where they go from there.

Dr. Eric Schaffer: It's certainly that.

Dr. Phil Goddard: And the sign of that expertise is to differentiate and the companies that do a lot of website design and have good web sites (inaudible)

Dr. Eric Schaffer: But certainly, it's not just the design from graphic artists and system folks. You need people who are actually usability specialists and that means hopefully they have degrees in the field, look for certifications like our certified usability analysts are definitely a cut above. Certified professional ergonomists. Those are also very good type of certifications. So getting people who are really qualified in the field is crucial but on the face this is still a field where you can easily get somebody who's really just looking at your site based on common sense and they're not going to do a very good job for you.

During your analysis, animations have been heavily disapproved. Is there an appropriate use of Flash, movies, and animations?

Dr. Eric Schaffer: Yes. When that Flash movie or that animation is the central focus of attention and is giving me the content I care about. So if you want to watch animation, it's fine. Okay, there's nothing wrong with watching animation. But if you're trying to work on something in one place and there's something going in the periphery, it's pretty much never good.

Dr. Phil Goddard: Yeah, I agree. I have used Flash the same way I've used the use of graphics and what their functional purpose is that they are used for content delivery they're very powerful. It's a multimedia tool. But if is used to simply add humor or attention-grabbing effect, then they can deteriorate (inaudible).

Dr. Eric Schaffer: And specifically in (inaudible) may be that they evolve a usable set of controls and (inaudible) to work on the web and that would be wonderful that you know give me (inaudible) give me slider, you know we're all looking forward to that day and we're hoping there'll be high quality and that will be great. But if it's just stuff we're doing up in the periphery, if it's I want to give you an example. I was doing an Expert Review of a site that was for weight loss recently. And when they were explaining their system of weight loss, they had pictures of food. And the (inaudible) they had was if you wanted to find out how much that food cost you that you wanted, you would have to click on the food to see the number. Now wait a minute. Why don't you just show me the number. Right? I mean clicking didn't help me so it is very important that we want to make sure that the animations really help and that the users take it with a focus of things.

Our last question is at what point does an Expert Review leave off and I should do something more? Or is an Expert Review all I need to do for my site?

Dr. Eric Schaffer: An Expert Review is a great way of starting a process, of making a site useful. It's a great x-ray to tell how to do things going forward, to validate that it's worth doing more work. But really if you make a site usable, there's a whole process that you need to go through. An expert review perhaps usability test. You want to make sure you go through a systematic process of designing the navigational structure based on the organization's vision. And based on the customer's deep mental model and needs you want to validate that structure. We want to make sure that we have standards in place and templates so that we have a consistent design. We want a detailed design and theoretical testing to fine tune. There's a whole (inaudible) in the Schaffer method and these are internal methodologies that we developed over the last 20 plus years for doing that. Our user centered design process is really core. And that means we are designing for the user first, optimizing the user experience and letting the technology make that happen as opposed to letting technology drive the design and hope the user can fit themselves around it. So, well, thank you for joining us for this web cast on Expert Review. And we'll look forward to (inaudible).

Top