May, 2006 – Usability Operations: If you're not global you are NOT competitive

Eric Schaffer: Hi there! Welcome to this new edition of the Usability Broadcast Network. My name is Eric Schaffer and I am the CEO of Human Factors International. This is Apala Lahiri Chavan and she is the Vice-President of all Asian Operations of HFI, which means that there are 140 people who report to her.

Apala Chavan: That's right!

Eric Schaffer: You can see her shoulders are not broad but very strong, sometimes tense I think, but she has really been the pioneer at HFI in building up our global capability particularly in Asia and that's what we're going to talk about today. There is a latest interaction edition, Interactions is the magazine of the ACM in the computer interface area, most of you should know about it. I was asked to be the guest lecturer and we did it on offshore usability. Now you can download it from our website and I suggest that you do because it's a pretty interesting article, it's a pretty interesting set of articles in edition and we've had a lot of input about it. We've had a lot of people with active discussions very excited about it. So this webcast is a sort of follow-up on that issue of interactions. Now I'd like to remind you as we go on that there's a button on your screen where you can ask questions. We're going to talk for a while and then we'll take the questions that you send in. So we're looking forward to those questions, please do include them. So what we're going to be talking about today is as we start to think of the globalization of usability, clearly usability people have to work with offshore development teams but that really wasn't the thing that we wanted to talk about. What we wanted to talk about was how usability work could be done on a global scale. How we can have usability people distribute around the world, and there are some really good reasons for doing that. So what we're going to do today is talk to you about the new way of doing usability work. It's no longer something which we're going to do in a little garage somewhere in Cincinnati it's something we can do on a large scale. It's something where we need to have a perspective that is global and where the economics and practicality drive us even if the user population isn't global. To use global resources to do the very best we can for the dollar and for just the practicality of getting things done. So HFI has been innovating very hard in this area and we're excited to share with you the insights of our work, really since 1999 in how to best do usability. So first thing we are going to talk about is the drivers that cause us to think about usability on a global scale with teams that are distributed, with teams that are not all in the United States but perhaps in Asia and the rest of the world. So to start with that, one of the perspectives that I think that we have, is that we often end up needing to develop systems that are used around the world and as you do that, things start to get a little different. I would like Apala to talk about what it is like to build systems with users all the way around the world.

Apala Chavan: Thanks, Eric. Well, it is a major challenge, you know, if you were designing a product or a software application or even a website that's going to be used by users from maybe urban areas like the areas that you live in but an urban area that's like thousands of miles away in a completely different set of cultural parameters that shape the users' mental models, their expectations, their motivations, their needs and you know, you could be often designing, as we often find ourselves doing, designing for populations that are remote, rural, illiterate with many constraints like no electricity, no you know, a lot of dust, appliances can't be maintained very well but we have to design for those users. Unless you can be in the context or have the resources to understand that context, it is almost impossible to do a good job to design for all these diverse users. The other important issue is, as I mentioned, the cultural dimensions. You know if you take cultural dimensions that are used by very popularly cultural researchers and designers, these dimensions determine in some ways how a user in a particular culture is going to perceive a particular system or product. So if you design something that is meant for a culture that is say, very individualistic, like the United States of America and that product is then transported to another culture which is very collectivist, which is the opposite. It's not going to work very well. Just as we found with cell phones, you know, the cell phone was really designed in Finland for use by for example, an individual. One person, one cell phone. The idea, that suddenly a cell phone in India was used by a whole family changed the whole dynamics of the way the product was going to be designed. So it's very important, therefore again, for someone to be very deeply immersed.

Eric Schaffer: But how is it different? If you have a cell phone and one person is using it and you have a cell phone and a whole family is sharing it, so what?

Apala Chavan: Well, just imagine, so the whole family shares the cell phone. Now the father, he is the patriarch who decides everything and on certain occasions, he gives the cell phone to his kids – when they're going out for a movie or with friends and so the son takes the cell phone very happily and goes out. The problem is when he comes back home. Now he has been talking and SMSing his girlfriend and should his father come to know that, oh my God, it's the end of the world!

Eric Schaffer: It's recorded on the phone?

Apala Chavan: Of course!

Eric Schaffer: Oh my God!

Apala Chavan: You can see the numbers he's dialed. You can see the SMSs. He has to sit there before he gets to the house and hands over the phone, he has to delete everything. He has to make sure there's nothing there. It's a huge problem and we found out this when we were doing a contextual project for a large cell phone manufacturer. The kids were like, "Please can you do something where I can, you know, take something out from the cell phone and that is my little private database and dad can't see it?" From there came, you know, the SIM cards and all sorts of things which protect your individual entity on the cell phone.

Eric Schaffer: I have seen SMS systems that have passwords now.

Apala Chavan: Yes, absolutely, it's just for that.

Eric Schaffer: The father will break the password just like the kids do.

Apala Chavan: So, anyway, so that's you know, very important. Then, as if that was not enough, you may think that you have you know, a sort of market segmentation that is used in your particular context. So market researchers would often, sort of slice-and-dice a particular country into different qualitative segments of the population and those segments are found largely in all countries. So you know, you do catch your product or your website and say, "Well, I'm designing for the trendsetters. If I'm designing for the trendsetters in the US or Japan or India, why do I have to worry? I'm not designing for the whole population." Look at Jessica Myers, trendsetter in the US, she is 20-something, she is a blogger, she is savvy with technology, she you know, has opinions on religion and spirituality, and she hates the fascism of designer-labels, she hates designer wear. Everything is about her individual identity and so she puts together stuff from you know, maybe Wal-Mart and other small stores, and she creates an identity for herself. That's the trendsetter in the US. So you're happy, you've designed something for Jessica, now it's going to Yamamoto San in Japan who's a trendsetter. She's 50-something, she loves designer labels, her whole life revolves around Louis Vuitton and Giorgio Armani. She can't think of putting stuff together from small stores. She doesn't like technology very much except cameras, she loves cameras. You know digital camcorders, webcam, she loves to travel, very up-market, sort of luxurious sort of travel. She is the trendsetter.

Eric Schaffer: But wait a minute, is there really any, if you build an e-commerce site, will there be a difference between these kinds of people? Can we just build one site?

Apala Chavan: No, unfortunately not. Well, just think of this, in China most of the only retail stuff originates online. That is about the only thing that is the same as anywhere else. The transaction is completed often in street corners where the buyer and the seller meet. The product is then handed over, inspected and cash is handed over to the seller when delivery has been taken. Okay, that's different. Now go to Japan, which isn't very far away from China. In Japan, the entire hub of e-commerce retail, it centers around the convenience store. The convenience stores with their major distribution network and the very strong IT infrastructure, they are the places where consumers would order online but they'd go to a convenience store, pay the money, the convenience store owner would then transfer the money into the online vendor's account, and the consumer will pick up the product from the convenience store on his way back from work. Okay, just more, I have to tell you. South Korea-all major e-commerce happens where? Via these huge cyber shopping malls and these shopping malls are all affiliated to physical shopping malls. I could just go and on but I won't.

Eric Schaffer: Okay.

Apala Chavan: You have to pay attention to the differences because otherwise it really is a big waste of money and resources.

Eric Schaffer: You can just look this up.

Apala Chavan: I wish I could, you know, give you some easy and positive answers on that but that the problem is that cultures change a lot. Cultures are dynamic. So you could look at some research or textbooks which were written maybe, you know, ten years or twenty years ago but when you go into the culture, you find that many things have changed. So you know, we were recently in China, doing a study for sort of use of media, media entertainment in the home. As we went in there, we found all the assumptions that we had sort of got, based on all the secondary research, and a lot of it exists, many of those assumptions were simply completely destroyed because obviously, the cultural revolution in China has made society very different over the last few decades. So you have the children, because it's a one-child family that the nation has, every family has one child and the child is the most important decision-maker. Now nobody really told us the impact of this and if you weren't there and you just depended on secondary research that has been done before, it wouldn't help. So the important thing is, it's good to know what the cultural ideas are for each culture but the trick is to find the tensions or the gaps between what the ideal is and what is the current practice? When you find that, when you find the gap, you have a million things that you design even design ways that you can help Chinese to flirt.

Eric Schaffer: You know, the thing is that in addition to these kinds of wider issues, and the work is being done by Apala and her team, in ethnography and contextual invasion, we also have localization and often when we build software and tried using it in other cultures, it bombs for very practical reasons. We have one ATM machine in India and when we tested it, it worked fine except the icons, and we made a brown hand which is nice but the icons showed a person holding money which was green and money in India is not green. So everyone said, "Why do you want me to click on something that gives green paper? I don't want green paper!" Then this is an example from IKEA and IKEA does very well in localization as you can see. The one on the left is for Italy and it has 3 guys on some furniture actually one guy and three women in a bed which works for Italy but it doesn't work for Saudi Arabia as you can see they have a different design. So adapting in terms of sensitivity and adapting in terms of conventions. So one of the things that's very important to know if you're doing, say icon design, is that any position you put a hand in can mean "up-yours" in some culture. So we have to be very careful about the meaning of hand positions, the numbers and colors so there is all that too as we localize in getting that right.

Apala Chavan: There is also the issue of what methods you use, when you do usability work. The methods are often methods that would work in the culture where it originated but unfortunately all the methods don't work when you take them to a completely different culture. That's something we have been looking at very seriously over a period of time. Looking at how something as routine as say a think-aloud protocol that you would use very normally in a usability test here very successfully or a questionnaire with a semantic scale that you couldn't even think twice about using, you know what these things don't work in Asia, in India, it doesn't work. In China, it doesn't work. And it doesn't work because you design these instruments, these methods based on the kind of cultural characteristics of the population that you are interacting with but when the cultural characteristics are different then the basic assumptions of those instruments and techniques fall through. So, to give you an example, we were, you know, we've been using the semantic differentiated scale for a while, while we were doing evaluation in India. Repeatedly, we have the same problem where we found that the participants were not able to really understand how to use the scale, in spite of all the explanation. They would always choose the two, either of the two extreme points of the scale and we could see that, just before that when we were trying to use the interface that they were trying to say, you know, express their satisfaction with on a semantic differential scale, their experience was not you know, either of the two extremes, there were things in between but they weren't able to express that on the scale. Then we realized that the representation of the scale was counter to the way they thought of gradations, the way they thought of going from a low value to a high value. It wasn't a straight line. So we looked at their own culture and took what they used very well for this kind of thing that is a radio-controlled knob, you know to control the volume? Which is very, very common in India and immediately they understood the whole concept of going from one end, one attribute to the opposite attribute and all the points in between without having to explain to them. So a lot of that work needs to be done amidst you know global competency and global resources.

Eric Schaffer: So as we talk about making our designs usable, we talk about usability operations, we need to have the ability to have this global presence partly to kind of understand these kinds of differences and how they are significant.

Apala Chavan: Correct.

Eric Schaffer: Partly also because there has been such a change in the usability industry and I see the usability industry perhaps more than most anyone else will want. I constantly travel around the world, our offices, the sun never sets in HFI and we've seen this change take place. We're really and truly in the third wave of the information age. The first wave was getting the hardware to work, it was a big struggle but that is totally a commodity now. We just buy it. Then we saw, really software being the focus and companies like Microsoft or SAP, they just built functions which if it was hard to use, it was okay. But today, software is a commodity and it's not something-not everyone is a differentiator, no company wins because it can build software. My kids can code in HTML it's not a big deal. The differentiator is who can build practical, useful, usability-satisfying applications and that's being looked at. You know, there are so many questions about the ROI of usability. Wait, why did those questions come out? It's a stupid thing to ask about the ROI of usability. Did you ever see somebody who said, "What's the ROI of getting a professional to do database design?" We could get high-school kids – that would work but you never have that question because it's so obvious and isn't it obvious that we want build things that are usable? Isn't it obvious that you want to be a professional? The question came up for an interesting reason and the question came up because we are making a transition now. We're really through the transition and I think the question is becoming less and less because there is a new kind of person buying usability work. We've gone from usability that was piece-meal and that was very much an early-adopted phenomenon, "Try it for a little bit, if we find a project, we'll do it" to serious main-stream usability that's made a huge change. That means that it's becoming main-stream. At HFI, just 6-7 years ago, maybe 5% of our clients had internal usability groups. Today 90% have internal usability groups. It's amazing to see how the field has become so powerful and so normal and that's great. User-centered design is the way to do it but there's a little problem with it. If you think about it, if we wanted to use a user-centered design, on all of the software that's getting built that needs it, it's about a $55 billion / year industry! We need about a million people doing usability work worldwide and what do we? 20,000. That's a pretty significant challenge. So one of the reasons that drives us to global usability, drives us to offshore usability is that you just can't get enough people. If you try to hire enough people to do the work, it is really, almost impossible to get that number of good people and then the cost is really wild. If you think about what the costing is for doing usability work, it becomes prohibitive but what happens, and these are rough numbers for a consultant, but what happens is that as you look at it offshore, usability in operation, the cost plummets and that reduction in cost makes it possible for us to put in usability into the mainstream basis in the way that we really want, in the way the vision that usability people have had for years and years calls for. So it's a requirement if you're going to get enough people and it's a requirement if the costs are going to be such that it's just even possible to do it. Now one of the things, and again this is in the article and I hope you take a look at it, is that we look at costs around the world and we have some very rough numbers here and you will see that they really vary a lot. So if you can look at somebody in the US at $80,000 a year and somebody comfortable in India at $24,000 a year, well, you can do the math. That's driving things a lot and now one thing that I am not saying and we will get into this later is that all the usability people in the United States and Europe are out of business. In fact, we're hiring, at HFI, we're hiring in the United States, and we're hiring people in Europe as fast as we can. You can send an e-mail to recruiting@humanfactors.com if you're interested. It's very much something we benefit by that difference. You know, the other thing is you look at the value of usability work and one of the things that it does, in addition to save on our costs to do usability work, is that it reduces the cost of development. Part of which is if the usability team creates a good spec, then that can be given to offshore developers and it makes it much more practical to use those kinds of developers because they don't necessarily do good interface design but if you give them a spec, they code it really well because they're CMM Level 97. So let's talk a little bit about what is allowing us to go ahead and do usability work on a global basis because there have been some changes that have been really significant in terms of the way usability work gets done. I remember starting doing usability work almost 30 years ago and it was just like software done in a garage. It was done in our basements and you had somebody who was a guru. I would go sit and listen to Bob Belier, Gary Hill or Gary Briggs or one of those, those old guys who knew so much and learn from them and then I went out and I was the guru and then I got my brown robe with the hood and beads and all that and I could sit there and it was great fun and I loved it. But that's not the way we need to do it now because we need to change completely because, and there's a book out I wrote "Institutionalization of Usability". It's in its second printing now, my mother bought the whole first printing. The book is really about how to build usability into an organization on a mainstream basis. As I thought about that, I realized that we had to change to a process-driven approach. Process-driven usability means we're going to operate just as IT moved from the garage to these huge operations with a systematic process, we have to do that too and we defined a process now where what we can do is take an organization and set up a usability factory. A usability factory means that under the auspices of an executive usability champion, you have to have that, if you don't have an executive champion, you're not going to build a usability factory but under the auspices of an executive usability champion, we're willing to set up the infrastructure and have the organization and staffing to make usability in a repeatable, reliable and cost-effective way. So that's really the key change that is happening. It's no longer a field where it should be some amazing god or goddess of usability who comes in and waves the magic wand. It has to be done on a routine basis as part of the normal operations and we're set up to do that work because we know how to do usability. We have a process and this is the Schaffer-Weinschenk method, version 4 has just come out actually. Actually it's a pre-version, this one I can see. This is something that is not new, we're not creating a new, amazing method, this is just user-centered design and we have a few tricks but really we understand how to do usability work. We don't have to invent it. Once you understand how to do it, once you have the methodology, then you can do it on a process-driven basis. Apala has led, setting up the first ISO certified usability operation on earth, our Mumbai office is ISO certified and that means, it means that you did a lot of work.

Apala Chavan: Exactly.

Eric Schaffer: And it means that the people are selected and trained and certified and they follow process and they document the process and we review it and we get feedback and we pay them.

Apala Chavan: Exactly. It is possible because of the ISO process to actually catch problems that may exist in the whole delivery mechanism very easily and you know, make it improved further and further and not leave it to chance for somebody to discover that "Oh this didn't work and everybody is unhappy." It's not – before it goes out, there are in-built mechanisms in the process and in the beginning it seemed like, "oh my God, why are we going to do all this?" but it helps because it just streamlines everything.

Eric Schaffer: And it means that we make the process better. It's interesting, when I was a kid, in New York City we would buy shoes and these shoes were no longer made by cobblers, they were made by factories and these shoes always fell apart. It would always open, the sole would come off and you'd have to bring it to a cobbler and he'd glue it and sew it back on. Today that doesn't happen very often because what's happened is that through an ongoing process improvement, companies figured out how to make better glue. So we are really at the very beginning of the maturity of this usability field. This is from a company, Sobha Renaissance, which is going down this path of maturity in the IT area and you can see that they've got a process that is good and they are ISO certified at the very beginning of what they are doing. They have got a security certification and we are doing that now.

Apala Chavan: Right.

Eric Schaffer: But it's a long pathway. It's a long pathway to finding all those better kinds of glue that will make the process really reliable but you can't really start doing that in a serious way until you have mature usability. So it means having a process, it means training. We have a public training offering that people go to that teaches the fundamentals of how to do this. In India we have about 2 1/2 months of training that we give our staff that goes far more into depth than even this. There is a certification and the HFI certification has really become the de facto certification in the industry. It's been a surprise to me how that's happened. Initially, I was hoping that UK or one of the associations would come out and say, "okay, we'll do a certification" but I think that they are still not close to getting there but ours is amazingly effective. We've got about 1000 people there certified and we know that it's been effective because people lie about being certified. They do. Or we're still confused as to whether they really are certified.

Apala Chavan: Imagine that they are.

Eric Schaffer: Yeah. Another thing is that we have to have templates and tools that help us to make the process better so this whole conversion to a more process oriented usability operation is really required if we're going to do things in a distributed offshore way. We can't do it at a piece rail ad hoc at a guru sitting there basis. We also have techniques and technology driving us to be more effective in an offshore environment. One of the great things is remote testing. We can sit in Bombay and in a single day test people in France, in Sweden, in United States, in Africa and we can use remote testing technologies and it's amazing what can be done. Let me just show you an example. This is from ARINC. This is a screen that gives you an example of the on-screen drag-and drop card sort tool that we use and this is from the training screen. So we had a design that was a concern in terms of the information architecture so we went out and here's the actual set up with the terms that were being used and asked users to go ahead and drag them into boxes, title the boxes and this is something very easily done remotely without having to be there. No travel costs, very, very efficient and the results, this is an actual piece of data. You can see how we can have the same exact kind of solid cards with data you can get if you were looking at in-person tests. The results were very interesting because in just 3 weeks they were able to do this whole project, do this re-design that we are looking at right now and before they had a 45% success rate in sorting things and after, they had 96%. So this is an example of how we can use these powerful methods to very quickly improve the design, at least the information architecture part of it. So things have really changed. When I first went to India in 1999, it was very, very different. We had the cost of a phone call back to the United States was $0.40 and it's $0.03 now. We have, at least at HFI we have built every kind of method that you can think of in terms of web conferencing, extranets and FTPs and so we can be connected and we can work in a collaborative way and it just does work.

Apala Chavan: And also, there is you know the issue of what are the challenges, what you really need to look for when you are thinking of setting up a global competency.

Eric Schaffer: This is the role of the expert.

Apala Chavan: We've learnt it the hard way sometimes.

Eric Schaffer: I think so.

Apala Chavan: First of all, you need really different sort of people. I don't mean, just like these guys are dressed in our office.

Eric Schaffer: This was fancy dress, this wasn't normal.

Apala Chavan: Yeah. This was not normal. Different because we're not using IT to do the design. We have to get people who either have a background in product design or a background in cognitive psychology. So you need to find those people and you need those guys to be there.

Eric Schaffer: The infrastructure, oh my goodness.

Apala Chavan: Yeah. Infrastructure is so very important because one of the, one of the- you know, paradoxes is that while you save a lot and you get a sort of get this economy to scale and you have to give infrastructure. But the infrastructure provided by the country, by the government, is often really poor. So you have to make up for it by having a lot of backup and redundancies built into your infrastructure. We have this huge 15,000 sq. ft. facility in Mumbai and we've got complete backup generators so that there is no power outage in the office even if the city is sort of in darkness.

Eric Schaffer: It has never happened. Okay, it does happen.

Apala Chavan: So you've got to have you know, internet connectivity, not just one ISP so you've got to have redundancies built in there so if one goes down, which never happens.

Eric Schaffer: Never.

Apala Chavan: You have another one so you're never ever going to be in a situation where you can't deliver or something because you know the online connectivity is so very important. You've got to have an infrastructure which also has facilities that allow training to be handled. Just as Eric mentioned, you know we have an intensive 2 1/2 month training course without which no staff member is allowed to be in the consulting pool but we need to have that training infrastructure there because our instructors are often around the world. So we've got to be able to have them instruct in a very remote way. We record them but using facilities that help in sort of reproduce the realness of being in a class and being with an instructor. We've got facilities for doing usability testing or a room with a one-way mirror and staff can be to be trained on usability testing and because of the facility being in house, we do projects with the facility too.

Eric Schaffer: So having, I think having a whole world team with the same language and infrastructure has been very good.

Apala Chavan: Very good, the consistency, the reputability of the way we do our projects has increased; I think many-fold of putting this sort of infrastructure together. Having a method that is common across the company, you know for everybody, no matter where they are, where they work, also means that you have a common platform. You have one consistent way in which you're going to talk to clients or you are going to talk to customers and obviously it also helps establish what the expectations that you have regarding the deliverables are. You give the activity that you follow in order to arrive at the deliverables, what is the timeframe – all that becomes a predictable controllable phenomena instead of everyone inventing their own way of doing the projects and you just can't have an offshore operations unless you have a very solid methodology. Not just a theoretical method but buttressed with you know, resources, templates, tools that make it possible to apply the method otherwise just having the theoretical method doesn't really help when you want to scale up operations in a concern. It's also very important just as we have the method, it also makes it clear where, in which you know phases of our design process we need to travel and be wherever users are. There are some cases.

Eric Schaffer: Yes, there have been some extenuating cases where if you don't travel you are going to crash and burn.

Apala Chavan: Absolutely.

Eric Schaffer: And you have to know that and there are cases when you don't have to spend the money.

Apala Chavan: Correct and if you know that because it's already built into the method, it makes you plan better, it makes you know when you need to travel and who needs to travel. There are unpredictable situations that often happen.

Eric Schaffer: You know, one of the things, if you look in the article, one of the pieces is when you don't want to use an awful lot of resources. We have a planned a decision table that will help.

Apala Chavan: Yeah so you can have an unprecedented flood that happened in Mumbai the 26th of July last year where it was just very heavy rainfall. The whole city was drowning.

Eric Schaffer: It was one meter of rain in 24 hours.

Apala Chavan: Right and what you do, you've got to be prepared. This sort of public infrastructure is going to fail in those situations so you've got to be prepared for that. Or when a veteran film actor passed away recently in Bangalore, the entire city came to a standstill. Everybody was stranded wherever they were, so on and so forth. You've got to be familiar and build in for these sorts of situations.

Eric Schaffer: And you know there may be cases where you don't want to put it offshore because you can't afford that one day or two days when there is a strike when everything shuts down to mourn the actor's passing away or whatever it might be.

Apala Chavan: Yeah. So perhaps, you know, Eric can tell us a little more about what are the situations when it's actually you know, very good to have a blended team?

Eric Schaffer: I think that really, for me the best part at Human Factors is the blended team. There are cases where because of high-security requirements, because of the instability of the design process because it isn't a mature process, where you wouldn't use a blended team. There are also cases where it makes sense to do it completely offshore where the cultural issues are not very great, where the cost issues are not too high. So both cases, but 80% of the time, we do things in a blended way and there is a good reason for that. Let me just give you an example. So if you look at an expert review, if Dean is an expert reviewer and it takes him about ten days that costs about $2,000 a day, and that's about $20,000 to do the expert review. Just as rough numbers. So Dean can do one expert review in two weeks for $20,000. So that's the basic math of the way we've done traditional usability work. Now look at a different way of doing it. If we do it with a blended team, Dean goes in and maybe he has a meeting to learn about the overall situation to look at the product to get it aligned. He then communicates that meeting including a very supporting the offshore team, get a couple of people offshore. But you can see that the time to do it has now gone over ten days, we now need 13 days to do it but they're done concurrently. So the real time is still ten days but the cost is now half. Instead of Dean doing one expert review in two weeks, he can do four or five. So that's in my mind, an unbeatable way to do usability work to support with an offshore group. You just can't, can't beat it but you do have to do it right. This is a chart for our process in an expert review. So once you know how to do that right, it's an unstoppable combination and the quality of it, I have to say I'm really, really proud of it. I can hold it up against any expert review in the world, it's absolutely world-class work. So the answer, my kids are both in school learning to be usability specialists and they will be usability specialists in the United States or India or Japan but that's going to be the business to be in, usability is a good field to be in no matter where you are because I believe even long-term, we're going to need local resources and the trick will be harmonizing them to work effectively with global resources and that would be truly the state-of-the-art. So we started out as we were working, really just having the global team, being a support facility. So they would often do low-level work putting things together in the background. That's not the way we do it today. What's happened today is that we really have the global team working hand-in-hand with the local team interacting with stakeholders, interacting with users, travelling as necessary but be really as a full partner. So it's no longer a subservient situation but a team working together and that's really the right way to do it with that kind of collegial effort. So that's I think, the future of usability. So now what I'd like to do is take a minute and just give an example of one of our staff who works here in Atlanta and listen to Sally as she talks about a couple of experiences she's had working with an offshore team with the support of that team.

SALLY: Recently I had to conduct a usability test and I had a week to turn it around. In other words, I think we finished testing it on Thursday, on Wednesday or Thursday and the following Thursday I had to fly to the client's place and deliver the presentation. So what we ended up doing for that usability test is we recorded the session because we were testing all day during the day and we didn't want the India staff to stay up all night to do the usability test, that wouldn't be fair to them. So we recorded the session and then they were able to, while we were sleeping, download those recordings, review the tapes, analyze the data and then conduct the analysis and review with me when the times would overlap. Review with me what they had learnt from the data and compare notes, construct the presentation and again, because we were working 24/7, it was working well during my hours and they were working during their hours , we essentially were pulling a double-shift and were able to finish and turn around that usability test in a week very successfully. So that was one good example. Another good example, I just actually last week, finished a presentation for a client. We did a large data gathering effort. We interviewed many users who were all over the country as well as other countries and because we had such a largeness of data, we had to have a number of people on our side doing the interviews and then analyzing the data and then creating a story around that data to be able to explain to our client, "Here's what we learned about the users of your website." Again, by having a number of people that we could apply to the project and because we did have the 24/7 staff working in different hours we were able to conduct the international interviews more easily. We had so many more minds coming to help at the table strategize about what we had learned and come up with a way to best explain it to our clients what we learned about their users. So I think those were both really good examples where we really applied this strength that we have of having a staff in India, applied it really to our best advantage and it made, in both cases, the deliverable, it helped us to deliver it more efficiently and made a better deliverable in the end that we wouldn't have if we hadn't that staff there and with those people involved in the project as well.

Eric Schaffer: Okay, so that was Sally on the webcam sharing her experiences and I'd like to remind you if have questions please send them in. We've got some that people have sent in already ad we'll start with those and please feel free to send your questions and we'll look forward to answering those.

QUESTIONS AND ANSWERS:

Eric Schaffer: So the first one is – there has been a lot of talk about the global team, that's for sure, but how does someone who is not developing products for an emerging market use an India team? Where is the cultural connection?

Well, if you're building just for people in Cincinnati, that's okay. If you think about the work that HFI does, many, many projects are still just for the American market, just for the European market, that is absolutely routine but we still do the work using a blended team. That means we may have the advantage of a global perspective though you don't need that advantage if we are just focusing on that user but still the cost advantage, the efficiency advantage is completely there even if you're just building some software that is going to be used very locally. So it is still a best practice even for a local facility, even for an in-house facility that is used only in America to use a global team. We now have many, many companies which now have full-time global teams with HFI, at HFI who work just for them we call them centers of excellence and their usability people are supported by the HFI offshore team.

Do the rough costs that Eric showed for "in-country", near-shore", and "off-shore", do they assume that "in-country" is the US? Would the cost be different, the same for Europe, what about Latin America?

Yes, they are different. In Europe, you may see that the "on-shore" costs will be a little higher and in Latin America it would be lower. If you download from our website, the re-printed articles from Interactions, what you would find is a survey that gives you a comparison. It's a rough survey but it gives you some idea of what those local costs would be.

I've had a difficult time getting my management to buy into having any usability staff in place, not to speak of an offshore team. How can I convince them to move towards the blended model you propose if they don't already understand the value of usability?

Well, they're not, forget it. The next question, just kidding. The thing is you've got to first get management to understand the value of usability and you do that by showing the problems that you are having and the opportunities that exist. We often do expert reviews to help with that or a usability test which can shows how, you know there is nothing like a video tape of a customer weeping and tearing her hair out to get management attention. But once you get that attention and it's pretty easy today because you know, your executives are hearing again and again about user experience, the economy and all that stuff so people are getting it, executives are getting it but the thing is once they get the vision, it's not that they're going to "yeah, yeah, we're really interested in usability" and you say "well you know we need 10% of the development team on usability and you know and we need $90 million..." and he'll get his ticker shot. So one of the things that could help at the point where they ask that good question "okay but how do I do this?" show them the book and that will show the road map but when they look at the staff costs, when you look at 10% of the development environment being dedicated to usability work and if you have to multiply that times the local staff costs, it's pretty shocking but if you can use the offshore to offset it more practical and easier to sell. So I think it's good for that. I think you'll find management more and more tuned in to the issues of usability and even we have some clients where the high number of executives is compensated based on user experience numbers.

What are your thoughts on addressing issues and challenges related to privacy, security and intellectual property in remote testing?

Well, you know that's an issue with offshore, frankly. In the offshore environment, we've had cases, an active case where people have revealed information, sold information and so it's something that we have to be very aware of. We, in our security certification, have had to re-build through that exhaustive process of looking through security profiles, responsibility, controls, and if a company isn't doing that, I would think twice of using them at all because today, security is just a concern. If you have very secure information, military kinds of secrets or something, I wouldn't put it offshore. Just won't do it.

Apala Chavan: Absolutely won't do it.

Eric Schaffer: But if what you're doing is you're building an intranet or building anything that's reasonably not secure then it's fine. When it comes to remote testing, I'm not sure if there's any special concern.

Apala Chavan: Yes, it's different from the general security concern.

Eric Schaffer: Right, you're having somebody go over the line, well, okay, is somebody going to tap into that remote WebEx connection basically? That's nothing I would lose sleep over. For me the other pieces are what I would worry more about. So having a company that thinks about security, worries about it does the kinds of things in terms of firewalls and permissioning and controls, that's really what you need to look for. It's a serious thing.

Apala Chavan: You know, we have, and we are just in the process of setting up an elaborate security management system which really examines to the level of detail of which part of our facility will have a sort of uncensored access to the internet.

Eric Schaffer: It's a very small part, right?

Apala Chavan: Exactly. In which areas you would not be able to access the internet without authorization, the chat facility is a special facility within the company so that nobody can you know send any files across using MSN chat to their friends etc. The level of detail and effort we have had to go through is unimaginable but at the end of that all, you can guarantee a level of security of that information that is going back and forth our office and other places.

Eric Schaffer: It seems like usability is something that by nature has to be conducted face-to-face. How have you been able get around that?

You know, it's true that you can do usability work a 100 per cent remotely. In fact as you saw that chart that Apala showed with the travel, you saw that people had to travel and so if you think that you are going to do even a big part of your usability completely remotely with no travel, it's not going to happen. Can we do an expert review without travel?

Apala Chavan: Absolutely.

Eric Schaffer: Absolutely. Can we do remote test without travel? Absolutely. Can we build an entire system using a user-centered design process without travel? No. So knowing how to put that together, it's a very cost-effective process, we've proven that. If you look at how HFI has been growing, you can see that's has been successful. But certainly, you need someone to travel and you need to be face-to-face to have that level of understanding, empathy and trust into understanding that person's world view. Once you have that, can I test whether people can find things on an interface? You bet. Can I test for perception? You bet. Can I do detailed design? You bet. We do that all the time.

What are the key differences between offshore usability and offshore coding and development?

Apala Chavan: I mean they are two different fields altogether.

Eric Schaffer: It's kind of like making shoes offshore and doing accounting.

Apala Chavan: Making socks!

Eric Schaffer: Okay, socks it is. So you can buy socks at all...it's completely different isn't it?

Apala Chavan: It is. It's completely different.

Eric Schaffer: You need different kinds of people, different kinds of processes; the whole environmental culture is completely different. If you walk into our office, it doesn't look anything like the coding shops – they look very sterile and controlled and you look at our office and there are colored banners everywhere, there are statues, it's more like an ad agency than one of this field. So it's a different kind of place. So in many ways, it's completely different. On the other hand, there have been a set of things that have been successful in companies such as WIPRO, InfoSys and so on, have done offshore work. I'll admit that I have watched those and copied them again and again. So moving towards what makes it successful is how they process, how they have quality assurance, how they do training, the whole defined organized way they are doing something is critical and you copy that.

Apala Chavan: Yeah, you know, how they manage cross-cultural interaction of staff and staff going to another culture and consulting the clients in another culture, these sorts of issues have been dealt with by offshore coding outfits and so those kinds of things certainly are you know what we look at and what we pick and choose from those kinds of successful processes.

Eric Schaffer: Recruiting for user sessions is a challenge, especially when the product is aimed at a more general audience. How do you go about recruiting? In many cases, incentive is needed to make sure participants are attracted. Is that the responsibility of the HFI, I assume they have written HFI, or is it something the client covers?

Apala Chavan: Well, it really depends you know, many times, in fact I would say most of the time, HFI really takes the responsibility of recruiting and we have tie-ups with you know, professional recruiting agencies around the world. So we know them, they know us. We work with the client to draw up the screener which allows us to filter the kind of participants that we do not want and we go through a process that is again, very well set up, what is the incentive going to be? Depending on the culture, different kinds of incentives are needed. You've got to know whether you are offering too much money because you will have nobody come in certain cultures.

Eric Schaffer: They're very scared.

Apala Chavan: They're very scared. Why is someone offering so much money for going in for two hours into this place? Some cultures you know, you often – you don't offer money. You offer things like games, you know, small computer games or mobile phones even so these intricacies are something that HFI understands and we work with our recruiting partners to make sure that it goes smoothly. On the other hand, there are some occasions when clients will say, we will take care of the recruiting and so that's fine too, there is no problem.

Eric Schaffer: So we do recruit all over the world. The toughest one we ever did was we had to find men with erectile dysfunction in New Zealand and this is true. I'm not making it up. And we did, we found them.

Usability is maturing in the US. How is it progressing in Asia? What percentage of HFI projects are US based versus other countries?

Apala Chavan: I think at this moment if you talk about HFI projects, probably 70% are US based and 30% is you know other countries.

Eric Schaffer: I'd say about the same.

Apala Chavan: If you're talking about Asia and how usability is progressing in Asia compared to the US, well usability is even more of a baby in Asia than it is in the US. In the US, you know, some strides have been taken and it is, as Eric has been saying for a while, usability is a mainstream opportunity phenomenon now. In India, it isn't but it's much better than five years ago, when we said "usability", people wondered "What the hell does that mean? Are you talking about design? Are you just using a new term for it? What is it?" We don't have that now. We've got large groups of usability professionals who are coming together in all the major cities in India which is amazing. The same for China. China has phenomenal conferences being held each year for usability, just focused on usability. So definite progress, but you know still baby steps.

Eric Schaffer: But you'll see in Asia, I think, the interesting thing in the move towards usability is very much driven at the executive level. So we have companies, very huge companies in China, for example, where the executives are very into usability and the sad thing is often they make mistakes in terms of thinking how to put it together. Remember that Chinese group we ran into and they said "we've got 20 usability people here" and we just look at them and say "okay, any one in affordances?" and they "No." But it's coming fast and that's exciting.

What types of usability activities are offshore or blended teams most suited for that is usability testing user analysis etc.?

I think that we have demonstrated offshore operations working throughout the entire life cycle. If you look at the Schaffer-Weinschenk method everything from product ideation – coming up with new product ideas, expert reviews, usability testing, the whole user-centered design process, the development process, all of that can be done offshore with a little bit of travel. It's just that it's better practice to do it in a blended team because the offshore team may not quite understand the local culture. Just like if you went to India, you wouldn't understand all the words quite right. If you come from India to the US, you might not understand that the word credit card is a common term. So having somebody local to bring that together is very powerful. Currently at HFI, we have half the work done in India and half in the US or Europe.

Which special tools do you use for offshore usability collaboration, remote testing etc.?

I'm not sure we're going to tell you exactly which tools we use because we don't endorse companies easily but clearly there is a set of tools that is evolving and much of it is the general, publicly available, web conferencing kinds of tools. There are other tools more specially organized. The usability central toolset that HFI offers is what we use for our usability specific kinds of activities and that's maybe the most important toolset as it has standards and it has the templates that we use. Beyond that, there are some specialized tools that certainly do help.

How have your clients responded initially when you propose a blended team to work on their projects?

That's interesting. So when we talked about offering these initially five or six years ago, people were very resistant. People within the HFI were resistant and then people outside of HFI picked up on the anxiety. Many people had thought that the CEO had gone mad and disappeared to the sub-continent and would never be seen again. That's changed. People talk, within the HFI, of being addicted to their team in India and there have actually been many arguments about wanting to get the right staff, and get more staff and having more people. That's probably where a lot of your time is spent.

Apala Chavan: Correct.

Eric Schaffer: Struggling with "I need 6 more people", okay. So that's really made a complete change. It's interesting, initially clients were very skeptical of having internal usability groups have an offshore team and yet all of a sudden huge companies are signing up for year-long projects and especially in the last year or so, we've seen it take quite a leap forward. So we have teams that work for the biggest cell phone companies and computers sellers and printers and companies and so on and that's really taken a leap forward. But initially there was skepticism partly because of the experience of some of the coding outfits or partly because of the fear of "it's far away and it's a different culture, it's not going to work."

Apala Chavan: Yeah. The cultural disconnect that they had experienced with coders many, many times; you know that's the first thing they said. "Oh, we find it very difficult to communicate and understand when the team is remote", it's a big challenge.

Eric Schaffer: But the fact that we have been doing it with such a good result for now over 6 years. We've proved it.

Are there any lessons about usability practices that you learned from integrating global practitioners in a single team? If you had to do this again (i.e. set up a global enterprise), would you do it differently?

Well, you know, at least we are re-inventing. So there are all these things that we were doing when we were tuning the process, improving the organizational structure, we had meetings this morning in terms about how we internally organize and how we manage and things like that but the basic foundation of what we've done I think was close, wasn't it?

Apala Chavan: Absolutely. I think we arrived at this sort of really successful formula of the blended team after some experiments. Think of it the moment we started up the offshore, if anything was to be done differently, it would be maybe to go straight for the blended team model instead of all the other things that we tried initially. There are many things that I think we've learned that I think we would have done differently. Getting people together, the team that works together, getting together at the first place, the main people from the team, the team leaders meeting in person makes a lot of difference. Then when they work in different places, they are integrated, you know, there is a bond because they've met. On the other hand, if you try working the team, you know, from day 1 when they haven't met each other, it isn't always that successful. So it's really a myriad of lessons we have learned.

Eric Schaffer: You said that the think-aloud protocol does not work well for usability testing in some parts of the world (e.g. Asia). What alternatives would you suggest?

Apala Chavan: Okay.

Eric Schaffer: I'm just going to sit back for a while. Go ahead and tell them.

Apala Chavan: Well, we certainly have you know, one alternative that we have used quite successfully in India and that's where we try to sort of see what is the main problem with the think-aloud protocol in Asia? Why doesn't it work? The main problem is that the think-aloud protocol assumes that the participant is very comfortable critiquing or expressing critical, dissenting opinion about an interface to a perfect stranger. This assumption is completely untrue, incorrect for India and many other Asian countries. So in India, if you get a participant and you go ahead as the facilitator and you say, "Well now, we are testing you, we want your opinion, do you use this interface? Do you use these stuffs? Can you think-aloud and tell us?" There is a lot of distrust about "Why on earth would they get me here, pay me money and then you know, she as a facilitator looks like she is very sort of well-educated, she seems higher up in the hierarchy that I am as a participant, I think there's something going on. If I say something bad, they are going to do something to me." So the participant is going to say very positive things and will be very scared to you know, even work through any of the tasks. It all looks very difficult and they don't want to say that. At the end of the whole test, you're not going to get any data worth, you know, even the paper you are writing on because it's all positive while you've seen how much the participant struggled with the interface. So what we have come up is to see what is the trigger in India, in that culture which enables people in their daily lives to forget their inhibition of this higher-up hierarchy, about strangers and to put them in a slot and then decide what to say. Are there moments in our daily lives in India when we forget all that? I'm sure there is. One of the main triggers is the films, is Bollywood, a major obsession in India. Whenever we watch films, we watch Bollywood films, it doesn't matter who we are watching it with so you know, I could be sitting with the family in a sort of living room-the grandfather, the really fearsome patriarch and the father and the grandmother and everybody in front of whom, normally I would not express my opinion at all particularly if it was dissenting. When it's a movie, everybody has equal rights. So it's a huge debate, it's heated at times. You know, the grandfather says, "My God, nowadays I don't know why they have only 35 songs in the movie, in my time, they used to have 70 songs. Why did they cut it down?" You can then turn around and say, "Grandpa, things are moving faster now so the 35 is there and look at the quality, each song they are changing their dresses ten times. They're in Switzerland, now they're in Austria." And you can have a real argument. But if he feels bad, your grandfather is not going to say, "How dare you open your mouth and say something?" So we use Bollywood in our think-aloud protocol. What we did was, we changed the task scenarios to make them full of drama and to give you an instance, we had you know when we experimented at first we had participants to come in and first of all the participant would sit in front of the website and we'd say, well the traditional way we do think-aloud protocol and we will say "Okay, this is an interface. This website is the Indian Railways website, we would like you to try and see if there is any seta available for a journey that you want to make from the city of Ahmadabad in western India to the city of Bombay tomorrow morning." Now the website was a nightmare so of course the user struggled and struggled and the user couldn't do anything because the website somehow for some reason expected that every user would know the number of the train that they were going to take. Everybody knows the names of the trains and nobody said there was a train number but that's what the Indian Railways based their entire website on and so nobody could even progress you know, sort of midway with the task but at the end of the test, everybody would say the same thing. They would say, "It's wonderful. What a wonderful website, look at the number of features, so many things on the website. Who designed this? Must have taken a lot of time. If I just get some time, I'll learn it and it's really good. It really works for me." And you know, you say, "Well I just saw for the last 1 ½ hours, this poor soul was so frustrated." The same participant, we got him again a few days later. Now this time, it was the same task but we changed the way we expressed it to him. So we said, "Okay, now you and your friend, both are consultants, both have gone to Ahmadabad to work with a company on assignment. Suddenly in the morning, your friend comes to you and she is in a state. She says "oh God; you know my life is going to be ruined. My parents just called up last night and they said that they have found a match for me. They are going to arrange a marriage for me. This boy has come from the US. He's a green-card holder, he's come to India. He's going to look around for a week, find somebody he wants to marry and take her back and they think that I am that girl. I have to go to Delhi by day after tomorrow and they are going to do it, they are just arranging it. But you know I have a boyfriend in Bombay, how on earth am I going to tell my mother?" At this point it is a typical Bollywood plot that we have all seen for the last 50 years. Every movie has the same plot. So the facilitator, as you tell the participant, the participant is already in the thick of things and is feeling like "oh my God, that's so wrong. " And then you tell the participant, "You can help your friend. So your friend has to take her boyfriend to meet her parents and otherwise her life is doomed. So you tell your friend, "Look don't worry, you have to go to Bombay by hook or by crook you have to go to Bombay, take your boyfriend, go to Delhi, make him meet your parents – your parents will love him. He is, you know, he's got a Masters degree, he earns so much money, he's got an apartment that is this many square feet, he's got a car, you know, that is this model, everything is okay. Just go." But your friend says, "But how can I do that? I have a consulting assignment. I'm going to be running to the client's place now." She dials the Indian, Air India airlines number and of course, as we all know in India, airlines' numbers are always busy. Finally she tries a few times and she gets through and the person who answers the phone at Air India, the point is that she is here inquiring about something and he just says, "You have to go to the main office and see." And he just bangs the phone down which is very common in India. She is just completely distraught so now you tell your friend, "Don't worry, and just go. Go to your assignment. I will get onto the Air India site and I will see if a ticket is available for tomorrow." You have to now get onto the Air India site and you have to see if any tickets are available." The moment you say that the same participant who came and tried the Indian Railways site and very gingerly tried the tasks and said, "Yeah, yeah, it's very good." That participant is a different person. She jumps onto the computer and she is trying hard to get this ticket that will save her friend and she says, "What is this? What is this that I didn't even see? Why is the drop-down like this? Why is this color so horribly low in contrast, I can't see anything? Did you design this website?" She's almost ready to kill me, forget about critiquing the website because you know what; she is the heroine's friend. She is imagining herself to be a part of a Bollywood fantasy and if she can save the heroine, she's part of this major happy ending. So that's just an example, you know, you have to find the trigger, the local trigger that gets past the problem because of which that instrument or that technique does not work. Sorry I took a little bit of time but I get a little carried away.

Eric Schaffer: That you get. Another question, the last question really. Do you recommend setting up an internal usability group before looking at acquiring offshore resources?

I would say that in order to use offshore resources effectively, you need either an internal usability group or a local group. In many cases, HFI just provides local staff as well as the global. That has to be there, you can set them up at the same time and in a sense, that maybe the best. But I would not recommend just saying, "Okay, we're going to do usability, let's have an offshore group. Okay, we've brought the offshore group." And there is nobody local that can help integrate it with the local culture or communicate to move it ahead, to facilitate communication, to provide local insight so I do recommend always having some local staff inside of the company, there really ought to be somebody and there can be somebody in the global team as well.

Okay, so that's the end of our webcast but I'd like to mention that this webcast will be available in two weeks on the HFI website. Thanks for joining us.

Apala Chavan: Thank you.

Top