Suresh K. Bhavnani
Dept. of Architecture
Carnegie Mellon University
Pittsburgh, PA 15213 USA
+1 412 363 8308
suresh@andrew.cmu.edu
Bonnie E. John
Computer Science Department,
Psychology, and
HCI Institute
Carnegie Mellon University
Pittsburgh, PA 15213 USA
+1 412 268 7182
bej+@cs.cmu.edu
The few laboratory and field studies on CAD usage that are available present a dismal picture. Bietz et al. [3] found that mechanical engineering students who had passed a CAD course produce better and more complete drawings with less effort using paper and pencil than on a CAD system. Luczak et al. [13] studied 43 subjects using 11 CAD systems in 11 factories. They found that even when the subjects were highly trained, the high complexity of the commands (due to many input parameters, restrictions, and requirements) led to low performance, reduced creativity, frictions, and frustrations. Finally, Majchrzak [15] found no improvement in the performance of 25 engineers and 60 drafters using CAD systems in comparison to non-CAD users.
In order to understand the problems faced by CAD users, we observed and recorded professional architects using a CAD system in their natural environment [4]. We begin by analyzing an example of suboptimal CAD usage from those real-world data in addition to another example from a study by Lang et al. [12]. These examples will demonstrate that the efficient use of CAD is dependent on the use of strategies that take advantage of CAD capabilities. To understand why experienced users do not have this strategic knowledge, we explore three approaches. First, we review some of the training literature and demonstrate that this strategic knowledge is not defined or taught explicitly. Second, we suggest that in the absence of these strategies, users develop an approach that is a mixture of manual and CAD methods resulting in suboptimal strategies. Third, we attempt to understand why experienced users do not themselves realize and change their suboptimal strategies. This is done by analyzing the relationship between drawing strategies and the quality of drawing produced for manual drafting as well as for CAD. We conclude with an attempt to define explicitly some of the strategies that would improve the use of CAD systems and discuss the ramifications for training and design.
To construct the fire protection enclosures for the first column, B1 had to draw two identically patterned, L-shaped polygons. As shown in Figure 1a, he first drew the top shape (1), and then mirror-copied the shape to create the bottom shape (2). He then poched (patterned) each of the shapes (3). A more efficient way would have been to detail the first shape by drawing and patterning it, and then mirror-copying the patterned shape to create the second shape. This strategy would have saved him the extra operations for patterning Shape-2.
Figure 1: Methods used by B1 to draw the fire protection.
In addition to the above strategy to complete the entire task, B1 also used a suboptimal method to complete the subtask of drawing the L-shaped polygon. To draw a closed shape that could be patterned, he used the method shown in Figure 1b which was very similar to a manual drafting technique. First he drew the top horizontal line (1), and then the left-most vertical line (2). Next he used the Copy Parallel command to make copies of the two lines drawn, (3), and used the Modify to Intersection command to cleanup the intersection of the two lines (4). Finally, he drew one endcap of the shape (5) and used the Copy Parallel command to make a copy of the inner elbow of the shape to create the lower endcap of the shape (6).
The above method to draw a shape with lines had two repercussions. First, as the automatic Pattern command is designed to pattern only closed shapes, he had to pattern each shape by copying individual dots and triangles from a nearby concrete shape. Second, when he decided to mirror-copy the shape, he had to temporarily group the individual line segments together using the Fence command before he could mirror-copy the shape. In the version of the CAD system used in our study, this procedure included several actions requiring the user to select individual pixels. This difficult perceptual/motor task was quite error-prone. Instead, if he had used the command Place Orthogonal Block to create the shape as a closed polygon, he could have used the automatic Pattern command as well as the regular Mirror Copy. These would have avoided the errorful steps of precise line drawing, manually patterning, and creating a fence to achieve the grouping.
It is pertinent to note that B1 had no difficulty in interacting with the commands he used. He rapidly executed commands like Mirror Copy and even more complex commands like Modify to Intersection with only minor motor slips. However, what B1 did not exhibit was a strategy to decompose the task so these very commands are used in a way to avoid unnecessary steps.
However, an experienced user in the experiment (referred to here as L1), executed the task differently from the efficient way described by the authors. L1 skipped step 3 and projected the two dimensional drawing before cleaning it up. Therefore, he had to clean up the drawing in two places. Similar to B1 in the previous example, L1 had little trouble using the commands. However, because the task was not decomposed into the proper subgoals, the resulting command sequence caused him to execute more steps than needed.
Figure 2. Mechanical part drawn by the user described in example 2 (reconstructed from Figure 2 in [12]).
Although both the above users had many years of experience using CAD, they used suboptimal strategies to complete their tasks. To understand why these users did not demonstrate the use of efficient strategies, we investigated the nature of CAD knowledge and instruction provided in CAD manuals and books.
Figure 3. Suboptimal and efficient decompositions of the fire protection task. Only the subtasks of patterning (poche) a shape have been decomposed upto the keystroke level.
Therefore, while B1 was proficient in executing the commands in the lower part of the task decomposition, he did not demonstrate knowledge of an efficient strategy at the higher level. We therefore investigated whether the higher level strategic knowledge was contained in vendor provided manuals and other sources.
We found that the highly competitive CAD industry had spawned an explosion of features in CAD systems resulting in systems with upto 2000 commands and a corresponding increase of instruction material. Manuals provided by the market leaders AutoCAD and Intergraph, focus on providing users with volumes of information about the numerous features available. The MicroStation user's guide, for example, begins with MicroStation Fundamentals which contains numerous exercises centered around commands like Save As and Draw Line. Although these descriptions are well presented, they are limited to the location of commands and the steps to use them. In addition to vendor supplied manuals, commercially available supplementary volumes reveal a similar pattern. They include only details of specific commands without any higher level strategies.
In a library search of CAD books for architects, we found only two books that went beyond the description of commands. One of the books [16] states, "It might be necessary to discard some traditional drafting concepts as you learn the sophistication of AutoCAD" (pg. v). The other book by Crosley [10], describes the importance of "thinking CAD". He states, "It's possible to use computer-aided drawing without really taking advantage of its capabilities. Even some experienced CAD users have simply transferred all their manual-drawing habits over to the computer." (pg. 6). Later he adds "Thus, the advantages of CAD are not free; they come at the expense of having to actually design the drawing" (pg. 11). While describing the Copy command he advises users to "never draw anything twice!" (pg. 41).
While such advice goes a long way in stressing the importance of using commands like Copy, he does not discuss explicit strategies to "design the drawing". Therefore, because commands like Copy are learned without a specific strategy, their power remains unrealized. In a book on computer graphics programming, Mitchell et al. [14] liken the efficient use of CAD systems to programming where "you must think carefully about the structure of the drawing in terms of repetition, conditionals, the hierarchy of parts, and the use of transformations." (pg. 515). However, because this knowledge has never been made explicit in any of the sources described, users are left to infer or develop it during use.
Therefore, it appears, that the manuals and books concentrate on providing knowledge at the lower levels of the task decompositions as shown in Figure 3a. This is also the knowledge that is fairly well understood by experienced users. However, the knowledge that is not demonstrated is higher up in the task decomposition which is exactly the knowledge that is absent in the manuals and books we reviewed. While the experienced users were quite proficient in using complex CAD commands like Modify to Intersection that were not present in manual drafting, it was not clear why the high level strategies like drawing a shape with single lines had such a remarkable resemblance to manual drafting. We therefore reviewed research in the area of knowledge acquisition and conceptual change to see if we could find an explanation.
The theory of mental models and conceptual change can be used to explain what appears to be occurring in the learning and use of CAD systems [Jackobson, M., personal communication]. Based on the way CAD systems are described and designed, users might begin by forming a mental model of the CAD system as merely an electronic drafting tool requiring little change in the way drawing tasks are performed. Introductory descriptions often have statements like "CAD is an expansion of the way you draw" [16] (pg. v), and go on by describing the use of the commands like Draw Line. The model of an electronic drafting tool is further reinforced by having concepts such as "drawings" and "layers" that are directly connected to the real world concepts of drawing sheets and overlay drafting (the use of overlapping tracing sheets). Following such introductions, the user is exposed to the details of using many different computer commands. The knowledge, as discussed earlier, is mainly about the location of these commands in deep hierarchical menus, and the procedure to execute them.
Inundated with tool knowledge but without explicit strategies to decompose tasks to make use of those commands, the users simply adjust their initial drafting model to incorporate the new knowledge of the commands. However, as described in the examples, this superficial adjustment causes problems. On the surface the use of Mirror Copy and Project appear to have been mastered as the users have no problem executing them. Because the underlying conceptual model is still an electronic drafting tool rather than a CAD system requiring different strategies, the overall use of these commands is suboptimal.
Although the above explanation appears plausible, it cannot explain why CAD users do not discover the strategies over time and make a deeper conceptual change to their mental models. These users, performing complex drawing tasks for many years, have many opportunities for serendipitous discoveries of efficient strategies. Research in the stages of skill acquisition in many domains show people who have successfully reached a stage of strategic learning. Strategic learning is described as "the improvement that comes about because people learn the optimal way to organize their problem solving for a particular domain" [1] (pg. 257). Why have the CAD users that we observed not reached this level of learning?
To understand why this change has not occurred, we contrasted the relationship between strategies and the quality of the product in manual as well as in CAD systems. We also explored the nature of the feedback loop for strategic knowledge in CAD systems.
These procedures are designed explicitly to achieve drawing accuracy and quality. In most cases, if these procedures are not followed, it is very hard to produce a quality drawing; a wrong strategy invariably leads to a visibly low quality drawing. Because there is such a strong causal relationship between technique and quality, and because the flaws are publicly visible, drafters tend to be highly motivated to improve their technique. This strong causal relationship between technique and drawing quality is absent in CAD. The drawing produced by B1 in example 1, is accurate and clean. This is easy to achieve as it requires only basic CAD knowledge to place accurate lines that meet at intersections. Therefore, there is no visible indication that the drawing was produced by a suboptimal strategy. As the flaws in the technique are not publicly visible, the users neither notice their suboptimal techniques, nor have motivation to change them.
In cases when drawings are shared and manipulated within a group working on the same project, a poorly constructed CAD drawing can cause irritations and problems to other users. For example, a user might expect to move a shape by grabbing a side and realize it was constructed by single lines. In cases like this the drawing strategy becomes public. However, if all the users in a group share a common mental model of the CAD system, the suboptimal strategy can remain undetected. This is exactly the situation at the office where example 1 occurred.
The nature of the feedback could also explain why CAD users never reach the level of strategic learning. In the study by Lang et al. [12], we find two examples of the value of feedback. When L1, in example 2, attempted to draw a line connecting two arcs, he drew only one arc before he started to draw the connecting line. As the line needed to be connected to the second arc, he did not have a precise location to end it. Having failed to complete the task of drawing the line, he abandoned the line to draw the second arc, after which he redrew the line connecting both the arcs. Therefore, the failure to complete the task provided him feedback to change his strategy. Furthermore, to test if he could perform the task more efficiently, L1 was asked to discuss his strategy with another user who had used the more efficient strategy, and then redo the task. In his second attempt, L1 completed the task with the efficient strategy using many fewer steps. This demonstrated that remediation can help even an experienced user to realize and execute a better strategy.
However, while feedback through failure can occur for some tasks, and remediation might occur through peer contact, in most real-world situations they do not occur. In CAD, using a high level suboptimal strategy typically does not preclude the user from completing the task as there are many brute force ways to complete the task. Additionally, as observed in our site visit, while users frequently discuss design issues, they rarely discuss drawing strategies or look over each other's shoulders during the drawing process. Therefore, as feedback through failure and remediation rarely occur, CAD users may never reach a level of strategic learning even after many years of experience.
One way to identify efficient CAD strategies is to understand where CAD offers advantages over manual drafting. Figure 4 contrasts the task decomposition for manual drawing and CAD. The task is to draw three identical complex shapes consisting of lines and arcs. (We assume that the locations of these shapes have been determined through grids or construction lines). Due to the inaccuracy of the compass in manual drafting, the arc must be drawn first. However, to minimize tool changes, all the arcs must be drawn together (1). Next, the vertical lines can be drawn moving the set square from left to right to avoid smudging the lines (2). Finally, the horizontal lines are drawn for all the shapes (3). Therefore, the efficient task decomposition is determined by the nature of the manual tools.
In CAD, as shown in Figure 4b, the efficient way to decompose the same task is different. Because CAD provides aggregation and manipulation commands, it is better to first draw all the lines of the shape (1 & 2), group them (3), and then make two copies (4). This strategy is what we call the Detail-Aggregate-Manipulate (DAM) strategy. The suboptimal strategies in example 1 and 2 described earlier, occurred because the detailing stage of the DAM strategy was not completed before starting the aggregation stage. B1 mirror-copied the shape before patterning it, and L1 projected the two dimensional drawing before cleaning up the lines.
Figure 4. Comparing manual and CAD strategies
The contrast between the manual and CAD way to decompose the same task, as shown in Figure 4, suggests an important difference in the nature of assistance provide by the two media. While the manual drafting medium assists in the creation of geometry (lines, arcs etc.), it does not assist in exploiting the structure of a drawing (repetition, symmetry, projection, configuration). In contrast, the CAD medium, while supporting geometry creation, also provides assistance to exploit the structure of a drawing. However, this assistance can be beneficial only if a strategy such as DAM is used. The DAM strategy appears to be powerful as it has numerous applications, some of which are shown in Figure 5. Given the description of this strategy, one can begin to imagine other forms of suboptimal behavior. A user, for example, might detail the shape, ignore the aggregation stage, and proceed to copy the shape element by element.
Figure 5. Applications of the DAM strategy. Each row represents a type of structure found in drawings, with a single instance of a command sequence to exploit that structure.
CAD systems also provide powerful modification commands such as Add Vertex and Partial Delete. Such commands in combination with manipulation commands allow a user to exploit the occurrence of compositions that are similar in a drawing, a capability that is not supported in manual drafting. The CAD system can therefore assist the user to access and locate any part of a drawing (Pan, Zoom, Reference Files), aggregate elements that are similar to the task at hand (Fence, Graphic Group, Cells), manipulate the aggregation (Move, Copy, Rotate) and modify them (Scale, Partial Delete, Add Vertex) to create a similar but not identical result. This strategy can therefore be called Locate-Aggregate-Manipulate-Modify (LAMM) and can be applied in many different contexts.
We suspect that there are efficient strategies at every level of CAD that need to be made explicit, from organizing a project to accurate cursor input. While strategies such as DAM and LAMM might appear obvious, the value of stating them explicitly cannot be ignored as even experienced users do not seem to be using them. However, it is yet an empirical question as to whether these strategies can be conveyed and learned by users through well-designed instructional aids. It appears that if these strategies and their applications are taught early during training, users can be encouraged to decompose drawings in terms of concepts like repetition and similarity. Such concepts are not new to designers who use them constantly in their designs. However, for reasons we have offered, many users may not be using the CAD medium to exploit the very concepts that they use in design. In addition, if CAD users are introduced early on how to decompose a drawing task ("thinking CAD"), they might be motivated to search for and learn the commands that allow for actions like aggregation and manipulation. In fact it might be possible to reorganize the commands in an interface based on task goals such as symmetry and similarity. This might encourage users to recognize and exploit the structure in a drawing.
In an attempt to reverse this situation, we showed two examples of how CAD strategies can be abstracted and explicitly stated. Such strategies could be used to design various forms of instruction as well as to redesign the interface. We also briefly described an approach to provide unobtrusive feedback to users if they performed tasks using suboptimal strategies.
One of the most common and favorite explanations for the low productivity in CAD systems is that the "D" in CAD does not stand for Design. Many claim that architects design, not just draw and therefore CAD systems as they stand today should be abandoned and approached differently. We believe this line of argument misses the point. Whatever the original acronym meant, CAD systems were designed to assist in drawing and not in design. While new paradigms for design assistance have to emerge and prove themselves, there are lessons to be learned from the CAD productivity problem. The CAD productivity problem, as we have demonstrated, has to do with deeper mechanisms that can plague the proper use of any new technology or medium. If the CAD productivity phenomena is ignored or explained away by the nature of what CAD systems do, then we are doomed to repeat their mistakes. If, on the other hand, we understand that a new technology often requires reformulating old tasks, then we can spend more time in making that knowledge explicit and minimally disruptive. Bowen [6] studying the productivity puzzle states: "The large payoffs come not from increasing the efficiency with which people perform their old jobs, but from changing the way work is done". However, it appears, that the knowledge to make this change is often not as obvious to users as we might assume and while serendipitous discoveries by users are possible, it is not something we should depend upon.
2. Beakley, G.C., Autore, D.D., and Patterson, T.L. Architectural Drawing and Design. Macmillan Publishing Company, New York, 1984.
3. Bietz, W., Langner, T., Luczak, H., Muller, T. and Springer, J. Evaluation of a compact CAD-course in laboratory experiments, International Journal of Human Computer Interaction, (1990): 2, 2, 111-135.
4. Bhavnani, S.K., Flemming, U., Forsythe, D.E., Garrett, J.H., Shaw, D.S., and Tsai, A. CAD Usage in an Architectural Office: From Observation to Active Assistance, Automation in Construction (in press).
5. Bhavnani, S.K., Garrett, J.H., Flemming, U., and Shaw, D.S. Towards Active Assistance. Bridging the Generations-Proceedings of the International Workshop on the Future Directions of Computer-Aided Engineering, D. Rehak, (Ed.), (1994) 1183-1188. Carnegie Mellon University, Pittsburgh, PA.
6. Bowen W. The Puny Payoff from Office Computers. In T. Forester, (Ed.), Computers in the Human Context, Basil Blackwell, New York, 1989.
7. Card, S.K., Moran, T.P., and Newell, A. The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates, 1983.
8. Ching, F. Architectural Graphics. Van Nostrand Reinhold Company, New York, 1975.
9. Clement, J. A Conceptual Model Discussed by Galileo and Used Intuitively by Physics Students. In Gentner D., and Stevens D (Eds.), Mental Models. Lawrence Erlbaum Associates, Publishers, Hillsdale, 1983.
10. Crosley, L.M. The Architect's Guide to Computer-Aided Design. John Wiley and Sons, New York, 1988.
11. Landauer, T. K. The trouble with computers: Usefulness, usability, and productivity. 1995 Cambridge, Mass: MIT Press.
12. Lang, G.T., Eberts, R. E., Gabel, M. G., and Barash, M.M. Extracting and Using Procedural Knowledge in a CAD Task. IEEE Transactions on Engineering Management, (1991) 38, 257-68.
13. Luczak, H., Beitz, W., Springer, J. and Langner, T. Frictions and frustrations in creative-informatory work with computer aided design-CAD-systems. Proceedings of the Fourth International Conference on Human-Computer Interaction, (1991), 175-9.
14. Mitchell, W.J., Liggett, R.S., and Kevan, T. The Art of Computer Graphics Programming. Van Nostrand Reinhold Company, New York, 1987.
15. Majchrzak, A. Effect of CAD on the jobs of drafters and engineers: a quantitative case study, International Journal of Man-Machine Studies, (1990): 32, 3, 245-62.
16. Obermeyer, T.L. AutoCAD Architectural Lab Manual. McGraw-Hill Book Company, 1987.
17. PSMJ CADD application and user survey (Professional Services Management Journal, 1994).
18. Strassmann, P.A. The Business Value of Computers. The Information Economics Press New Canaan, CT, 1990.
19. Vosniadou, S., and Brewer, W.F. Mental Models of Earth: A Study of Conceptual Change in Childhood. Cognitive Psychology, (1992), 24, 535-538.