Training and development of project managers is challenging for at least two reasons. First, the relevant knowledge base is quite large. As identified in the Project Management Body of Knowledge (PMBOK) [1], project management encompasses a large set of project management-specific knowledge areas. In addition to these knowledge areas, the project manager must have an understanding of the relevant industry, technology, and general management issues likely to be encountered on projects. Second, the discipline of project management is both theory- and practice-based. It is not enough for the project manager to have an abstract, conceptual knowledge of project management methods, tools, and practices. The project manager must also be able to apply this knowledge in complex operating environments. As noted by Boyzatis [2], Parry [3], the International Project Management Association [4], and the Project Management Institute [5], competence in an area presumes both a strong knowledge base and the ability to effectively apply that knowledge base on the job. Kolb [6] and Raelin [7] have noted that individuals learn through a process of conceptualization and experimentation. In conceptualization, the individual uses theory to build mental models of the knowledge domain. This type of learning is typically linked to traditional classroom methodologies such as lectures and the use of textbooks. Experimentation, on the other hand, is the process of testing conceptual knowledge by applying it to specific situations.
Classroom methodologies such as case studies, simulations, and role playing focus more on enhancing learning through experimentation. Ideally, concepts focus the experimentation and experimentation refines the concepts in an iterative process over time, leading to a grounded set of knowledge, skills, and abilities. The challenge, then, is to offer training programs that use conceptualization and experimentation to enhance project management knowledge and the ability to apply that knowledge. This paper attempts to address this issue by examining the use of a project management simulation exercise in an academic classroom setting. The simulation exercise is used after project 4 management concepts have been taught, which allows participants to apply their conceptual knowledge through experimentation. The learning effects of the simulation exercise are then assessed. The following sections describe the details of the research setting, present hypotheses to be tested, and examine the results of statistical tests on the hypotheses. The paper concludes with recommendations for future research.
2. Research Setting The setting for this study is two separate graduate level classes in project management offered at the
These topics are explored using various teaching techniques, which include lectures, case study analyses, in-class student team mini-exercises, report writing, examinations, PMBOK review, and the use of Microsoft Project 2000. At the end of ten weeks, students have a general understanding of the important project management subjects. 5 From weeks eleven to fifteen, all students in the two project management classes participate in the project management simulation exercise that is the focus of this research. The final week of class is used to summarize and wrap up the course. The PC-based simulation exercise is from We, and is the key component of their Managing by Project training course. We is a for-profit organization specializing in project management, leadership, and change management training. Since 1988, they have trained over 30,000 project managers using a network of affiliated training organizations. This training course has a track record of success, as evidenced by post-course feedback from participants, an upward trend in number of clients, and significant repeat business from their existing customer base.
While the We simulation exercise has been successfully offered in corporate client settings, it has not yet been used in an academic setting. We and the
3. Research Questions The use of this software simulation exercise is expected to produce a number of benefits for the participants [8-9]. First, participant levels of project management knowledge are expected to increase as a result of their training experience. The participants all come to the simulation exercise with their own levels of knowledge in various project management knowledge areas. Their pre-exercise knowledge may have come from prior work experience as well as classroom learning. Regardless of the source(s) of this knowledge, participants are asked to assess their knowledge levels by responding to the sixteen items shown in Table 1 prior to beginning the simulation exercise. The sixteen items of Table 1 are prefaced by the following statement: "Assess your current level of knowledge in each of the following areas.” Responses are solicited from each participant using seven point Likert scales, where the anchors for the scales are: 1 = Extremely Low, and 7 = Extremely High. [Table 1 here] The responses to the items in Table 1 allow us to create a pre-exercise knowledge profile for each participant. Upon completion of the exercise, the participants are again asked to respond in the same manner to the questions in Table 1. Comparing the pre- and post-exercise responses will indicate participants' perceived changes in knowledge resulting from participating in the exercise. Our expectation coming into the simulation exercise is that participant project management knowledge will increase as a result of performing the exercise, with learning through experimentation causing improvements in participants' conceptual knowledge. The associated hypothesis to be tested is: H1(a): Participants will assess their level of project management knowledge as greater after completing the exercise than they will prior to performing the exercise. In addition, involvement in the simulation exercise is expected to strengthen the ability of participants to apply their project management knowledge. As noted in the Introduction section of this paper, simulated learning experiences have the potential to build a level of understanding in participants beyond an abstract knowledge of project management methods and tools.
Working through the 7 simulation exercise should improve participants' abilities to apply project management knowledge, by using their knowledge in context to make a large number of project management-related decisions. The same sixteen items shown in Table 1 are asked again of all participants, both pre- and post-exercise, with one change. Instead of prefacing the items by asking participants to assess their current level of knowledge in each of the following areas, participants are asked here to "Assess how confident you are in your ability to effectively apply your knowledge in each of the following areas. " This time the anchors for the 7 point Like scales are: 1 = Extremely Low Level of Confidence, and 7 = Extremely High Level of Confidence. Our expectation is that the ability of participants to apply their project management knowledge will increase as a result of the simulation exercise. Thus, the associated hypothesis is: H1(b): Participants will assess their ability to apply project management knowledge as greater after completing the exercise than they will prior to performing the exercise. The above hypotheses 1(a) and 1(b) are targeted to all participants. That is, all participants in general are expected to improve both their level of knowledge and the ability to apply their knowledge as a result of working through the extended simulation exercise. There are also more specific effects expected as a result of performing the simulation exercise. Given the large variation in participant project work experience, an important concern is whether the benefits of the exercise vary with experience. To address this, participants were asked to sum up, to the nearest tenth of a year, their amount of on-the-job experience as a project manager / director, project coordinator, project team member, and other project-related participant.
The expectation is that participants with more project work experience will enter the exercise with higher levels of knowledge than those participants with less project work experience. We certainly anticipate both high and low experience participants to have increases in knowledge levels after the exercise is complete. However, 8 we expect the magnitude of the improvements in knowledge to be larger for the less experienced participants than for more experienced participants. Accordingly, H2(a): Prior to performing the exercise, the level of project management knowledge will be greater for participants with high amounts of project work experience than for participants with low amounts of project work experience. H2(b): From pre-exercise to post-exercise, the magnitude of improvement in the level of project management knowledge will be greater for participants with low amounts of project work experience than for participants with high amounts of project work experience. We expect to find similar patterns in the ability to apply project management knowledge, depending upon the amount of project work experience. The relevant hypotheses are: H3(a): Prior to performing the exercise, the ability to apply project management knowledge will be greater for participants with high amounts of project work experience than for participants with low amounts of project work experience. H3(b): From pre-exercise to post-exercise, the magnitude of improvement in the ability to apply project management knowledge will be greater for participants with low amounts of project work experience than for participants with high amounts of project work experience. The logic supporting these hypotheses is as follows. Participants with higher amounts of project work experience are more likely to have been in situations similar to those they face in the simulation exercise. The types of decisions required of them in the exercise, and the issues to be considered when making these decisions, are not likely to be completely new to them. So while the exercise is expected to be of value to more highly experienced participants, it should not be radical new learning to them. 9 For less experienced participants, however, the simulation exercise is more likely to be a singularly different experience. It should require them to formulate and make decisions in contexts quite different than what they are used to. As stated in hypotheses 2(a) and 3(a), less experienced participants are expected to see themselves initially as having lower levels of knowledge and abilities to apply that knowledge than will more experienced participants. Because less experienced participants start from a lower base than highly experienced participants, hypotheses 2(b) and 3(b) predict that they will make larger improvements through performing the exercise.
Proportionally, they are expected to learn more than the more experienced participants, since they have little to no experience base upon which to rely. While the more experienced participants are likely to still assess their skills after the exercise as higher than those less experienced, it is expected that the less experienced participants will make progress toward closing the gap. Finally, we expect the nature of the participant team experience itself to have a possible effect on improvements in level of knowledge and ability to apply that knowledge. This is considered in two distinct ways. First, each team will have objective measures on how well it manages its project and brings it to a successful completion. Given that the exercise has a fixed, unambiguous project scope, this type of performance is measured in how well the team does in minimizing the project's time and cost to complete. Teams receive ongoing feedback throughout the project as to their schedule and budgetary performance, and at the end of the exercise they see how well they performed as compared to all other teams. The question to examine is whether participants view their knowledge gains differently depending on how well their team performs. The hypothesis used to test this question is: H4(a): Participants of low performing teams will show smaller improvements in level of knowledge and ability to apply that knowledge than will participants of high performing teams. 10 The other aspect of the team's experience is based on team dynamics. This aspect is measured by seeing how well team members perceive the team working together. Table 2 has a set of questions used to assess the quality of the team process. [Table 2 here] If the team process is seen as constructive, with team members cooperating, sharing the workload, staying motivated, and so on, it should foster a positive learning environment. Conversely, if team dynamics are poor, it should result in a less than satisfactory learning environment and, therefore, less successful learning. To gauge the quality of the team process, the series of items shown in Table 2 is asked of all participants after completing the exercise, again using seven point Likert scales. The associated hypothesis to test the effects of team process is: H4(b): Participants involved in a negative team process will show smaller improvements in level of knowledge and ability to apply that knowledge than will participants involved in a positive team process. These hypotheses will be tested in the next section of the paper, using SPSS statistical software for the data analysis [10].
4. Results As noted in the Research Setting section of this paper, a total of 63 subjects participated in the study. The subjects voluntarily filled out two distinct questionnaires, one prior to beginning the simulation exercise and the second immediately upon completing the exercise. The simulated project exercise itself takes approximately 15 - 18 hours to complete, not including another 1 - 2 hours of out of- class preparatory reading to gain basic familiarity with the context of the exercise. This total of 16 - 20 hours occurs across a five week interval, with approximately 3 - 3.5 hours per week of in-class time devoted to the exercise. During the five week interval, the students' classroom efforts are focused solely on performing the exercise; no other class activities are undertaken. 11 4.1 Overall Changes Hypothesis 1(a) asks whether participants' levels of knowledge of various aspects of project management are enhanced by completing the simulation exercise. To address this fundamental question, each participant's self-reported judgments on pre-exercise and post-exercise knowledge are compared. Table 3 shows the average pre- and post-simulation exercise values in levels of knowledge for the sixteen project management areas. Visual inspection of these averages shows that, in general, participants come into the exercise with levels of knowledge that are neither extremely low or extremely high. On the seven point scale, pre-exercise values lie between three and five, indicating moderate levels of knowledge with room to improve. Also obvious from Table 3 is the fact that there are postexercise increases in the average level of knowledge in all sixteen areas. To assess whether participant increases in level of knowledge are statistically significant, we first calculate sixteen differences for each individual by subtracting the post-exercise score from the pre-exercise score. These calculations yield from 57 to 62 differences for each area. (There are slightly less than 63 paired differences for each area because of missing values or invalid responses by a small number of the participants.) Table 3 shows participants' average differences for level of knowledge in the "Increases" column. Across the board increases in level of knowledge are evident, with all sixteen areas having increases greater than 1.0 unit and some having increases greater than 1.5 units. [Table 3 here] To determine whether the increases are statistically significant, t-tests are performed to see if each of the sixteen differences are greater than zero. The t-tests all yield p-values of less than 0.001, providing strong evidence that participant perceptions of their levels of knowledge post-exercise are greater than pre-exercise for each of the sixteen areas. Hypothesis 1(b) asks whether participants' abilities to apply their knowledge of various aspects of project management is enhanced by completing the exercise. The testing procedure for this hypothesis follows the same logic as for hypothesis 1(a). Table 3 shows the average pre- and post12 exercise values in ability to apply knowledge for the sixteen project management areas. A pattern similar to that of the "level of knowledge" data can be seen here. In general, participants come into the simulation exercise with a perceived ability to apply knowledge that is neither extremely low or extremely high, with all sixteen values ranging from three to five.
Table 3 also reveals post-exercise increases in the average ability to apply knowledge in all sixteen areas. Results of t-tests run on this data again show that, at the p <>4.2 Effects of Project Work Experience The strong support for hypotheses 1(a) and 1(b) indicates that participants as a group do receive educational value from the simulation exercise. The next hypotheses, 2 and 3, take a more nuanced view by looking for differing educational benefits depending on the amount of a participant's project work experience. To test these hypotheses, participants are ranked by total project work experience from lowest to highest. After the ranking, the data is segmented into quartiles of approximately 15 participants each. Participants in the first quartile are classified as having a low level of project work experience, while participants in the fourth quartile are classified as having a high level of project work experience. These two quartiles are then used to examine the questions posed in hypotheses 2 and 3. 13 The average amount of experience for participants as a whole is 3.6 years, with a large amount of variability in the sample. The low experience quartile has 14 of its 15 participants with no project work experience, and the remaining participant with 0.2 years. The top quartile of 15 participants has quite a different profile, with an average of 10.7 years of experience and a range of 6 years to 26.5 years. Table 4 compares the low experience and high experience subgroups. The table shows grand means for level of knowledge and ability to apply knowledge, both pre-exercise and post-exercise. The grand means in this table are calculated by averaging all sixteen items listed in Table 1 for each participant sub-group. To illustrate, the pre-exercise level of knowledge for less experienced participants, 3.42, is determined by first calculating the low experience quartile's average level of knowledge for each of the sixteen project management areas, and then calculating a grand average across all sixteen areas. [Table 4 here] 4.2.1 Pre-Exercise Results Hypotheses 2(a) and 3(a) are concerned with the comparative pre-exercise knowledge assessments for the low experience and high experience sub-groups. Table 4 suggests that more experienced participants have higher pre-exercise scores than less experienced participants. This includes scores for level of knowledge and for the ability to apply knowledge. In fact, a similar pattern holds post-exercise. In other words, more highly experienced participants come into the simulation exercise with greater perceived knowledge and skills, and they still have somewhat greater perceived knowledge and skills after completing the simulation exercise. Although Table 4 presents highly aggregated results, it does appear to lend some support for hypotheses 2(a) and 3(a). To take a more finely grained view of the impact of project work experience, values for level of knowledge for each of the sixteen items of Table 1 are presented graphically in Figure 1. The bars in Figure 1 represent increases in knowledge level, with the left edge of each bar 14 representing a pre-exercise level of knowledge and the right edge representing a post-exercise level of knowledge. Figure 2 graphically displays similar information for the sixteen items on ability to apply that knowledge. Visual inspection of Figures 1 and 2 show that, in virtually all instances, more experienced participants enter into the exercise with higher levels of knowledge and ability to apply that knowledge than less experienced participants. [Figures 1 and 2 here] To see whether these visually observable differences are statistically valid, analyses of variance (ANOVA) are performed on each of the sixteen items. A set of ANOVA tests compare pre-exercise levels of knowledge for the two experience sub-groups, to see whether the more experienced sub-group starts off at a higher level of knowledge than the low experience sub-group.
Using a p-value of 0.10 as a cutoff, nine of the sixteen differences shown in Figure 1 are found to be significant. A similar set of ANOVA tests compare pre-exercise ability to apply knowledge for the two experience sub-groups. Again using a p-value of 0.10 as a cutoff, four of the sixteen differences from Figure 2 are found to be significant. Thus, even with the statistical limitation of having small sub-groups of only 15 participants each, the data does offer a strong degree of support for hypothesis 2(a) and a more moderate degree of support for hypothesis 3(a). In other words, highly experienced participants appear to come into the exercise with higher perceived levels of project management knowledge than do the less experienced participants and, to a less significant degree, greater abilities to apply that knowledge. 4.2.2 Exercise-Driven Improvements Given that more experienced participants have an initial advantage, hypotheses 2(b) and 3(b) examine whether this initial advantage translates into differences in the magnitudes of improvements expected after completing the simulation exercise. The results shown in Table 4 shed some light on this question. Table 4 suggests that, while both sub-groups show improvements in knowledge level and knowledge application from pre- to post-exercise, the magnitudes of the increases do not appear to be 15 equal in all cases. The increase in knowledge level for the less experienced sub-group, 1.46, is considerably greater than the increase of 1.01 for the highly experienced sub-group. However, the magnitudes of changes in the ability to apply knowledge do not follow the same pattern. Table 4's increase of 1.02 for the less experienced sub-group is roughly the same as the increase of 1.06 for the highly experienced sub-group. Thus, the aggregated results of Table 4 indicate that less experienced participants are likely to achieve larger overall gains than more experienced participants when considering levels of knowledge, but equivalent overall gains when considering the ability to apply that knowledge. Figures 1 and 2 provide a more detailed, item-by-item view of magnitude improvements related to these hypotheses. Visual inspection of Figure 1 indicates a general tendency for less experienced participants to obtain greater improvements in levels of knowledge than more experienced participants. Figure 2, however, does not present the same tendency. On an item-by-item basis, the improvements in Figure 2 are nearly equally split between less and more experienced participants. Statistical analysis of the data yields mixed results. A set of ANOVA tests are run to compare improvements in levels of knowledge for the two experience sub-groups. Using a p-value of 0.10 as a cutoff, five of the sixteen improvement differences shown in Figure 1 are found to be significant. Of the five differences, one difference ("earned value") is in the opposite direction than expected. So, there are four items of sixteen that statistically support the hypothesis, and one of sixteen that contradicts it. Overall, the degree of support for hypothesis 2(b) is moderate at best. It should be noted that these results may be affected by the small sub-group sample sizes of 15 participants each. A visual inspection of Figure 1 shows that, for most items, there are nominally larger improvements achieved by the less experienced sub-group. However, the small sample sizes preclude our being able to make this statement with statistical authority. The use of larger samples in future research would be necessary to test this conjecture. With regard to the ability to apply knowledge, an inspection of Figure 2 suggests there are no significant differences between the low and high experience sub-groups. Again, ANOVA tests are used 16 to statistically examine this question. At a p-value ≤ 0.10, only three of the sixteen differences from Figure 2 are found to be significant, with two of them ("earned value" and "performance measures") in the opposite direction than expected. Clearly, the data does not support hypothesis 3(b). To summarize, the data does lend some support to the hypothesis that less experienced participants will make greater gains in levels of knowledge than will the more highly experienced participants. Less experienced participants do tend to close the project management knowledge gap by taking part in the exercise. However, the amount of a participant's prior project work experience does not appear to determine the size of the increase in ability to apply knowledge. Both less experienced and highly experienced participants made equivalent, substantial gains in the ability to apply their project management knowledge. 4.3 Effects of Team Performance and Team Process Hypothesis 4 explores whether the team's degree of success in completing the exercise will have an impact on the educational value of the exercise. As stated in the Research Questions section, a team's success is considered in two distinct ways. Of course there is the objective performance of the team in managing their project and bringing it to a successful completion. Given a fixed scope of activities, this type of performance is determined by how well they do in minimizing the project's time and cost to complete. The other aspect of team performance takes more of a process view. This aspect is determined by seeing how well the team does in working together as a unit. The first aspect, team performance, is measured by collecting and then combining two variables for each team after the exercise is complete. The two variables are the elapsed time to complete the project and the actual project cost incurred. Project time for the exercise has a overall mean of 21.6 weeks, ranging from a low of 20.0 to a high of 24.4 weeks. Project cost has a overall mean of $798,000, with a range from $707,000 to $928,000. To calculate a composite measure of team performance, each team's time and cost variables are first scaled by dividing by the corresponding variable's overall mean value, and then the scaled time and cost variables are averaged. This single measure, representing 17 objective team performance, has a mean of 1.0 and a range from 0.93 to 1.15, with a lower number indicating better performance. In equation form, Team composite performance score = [ (team actual cost / 798,000) + (team actual time-to-complete / 21.6) ] ÷ 2 The composite team performance measure is used to rank teams from lowest to highest, with the bottom and top quartiles used to test hypothesis 4(a). Since a total of sixteen teams participated in the exercise, the low performance sub-group, comprised of the four lowest performing teams, is compared to the high performance sub-group, made up of the four highest performing teams. ANOVA tests are used to compare the improvements in knowledge levels and ability to apply this knowledge between these low and high performance sub-groups. The results of these tests do not lend support to hypothesis 4(a). Of the sixteen items of Table 1, only one item shows a statistically larger improvement in knowledge level for the high performance sub-group. For ability to apply knowledge, none of the improvements in the sixteen items are statistically different between the low performance and high performance sub-groups. Thus, there is no substantial evidence to support the view that objective team performance has an impact of the educational value of the exercise. Hypothesis 4(b) examines the impact of the team process on the exercise's educational value. The nine items of Table 2 are used to measure the quality of the team process. To calculate a composite score for team process, each participant's responses to the nine items are averaged and then sorted from lowest to highest. The composite team process score has a mean of 6.29, ranging from 4.25 to 7.00. Participants in the lowest quartile are designated as having a relatively negative team process, while participants in the highest quartile have a relatively positive team process. It should be noted that participants in the low and high quartiles are not tightly clustered in just a few teams. The negative team process participants are spread across nine of the sixteen total teams, while the positive team process participants are members of ten different teams. This suggests that participants' assessments of the quality of their team process, even within the same team, vary widely. 18 The results of ANOVA tests based on the team process composite scores fail to lend support to hypothesis 4(b). There are no statistically significant differences in any of the knowledge level or knowledge application items between the negative team process sub-group and the positive team process sub-group. As a cautionary note, the team process results are based on data that all lie on the upper end of the scale. Fully three quarters of the participants have team process composite scores over 6.00, and the lowest quartile itself has a mean of 5.01. While this is a good situation from a training perspective -- most participants experienced a quite positive team process -- it leaves as an open question whether an extremely negative team process would affect the educational value of the exercise. To summarize, given the data available in this study, the educational value of the simulation exercise is not significantly affected by either a team's objective project performance or by the quality of the team process.
5. Discussion and Conclusions 5.1 Summary Findings The results presented in this paper lend support to the educational value of the project management training exercise. As a group, participants in the exercise came away with heightened levels of project management knowledge and heightened abilities to apply that knowledge. So at the most basic level, there is an educational value to the training exercise. Further, individuals with varying amounts of project work experience all appear to gain educational benefits from being involved in the training exercise. This indicates a robustness in the training exercise, providing some value to relative project novices and experts alike. However, the specific educational value received is a function of the amount of participant project work experience. Less experienced participants appear to make more substantial gains in knowledge level than do more experienced participants, while all participants improve similarly in their abilities to apply their project management knowledge. Finally, neither a team's time and cost performance nor the quality of its team process seem to affect the educational value delivered to 19 exercise participants. The training exercise appears to provide educational value to individuals in good and not-so-good teams. 5.2 Future Research This paper examines a simple question of great importance to training firms that provide and organizations that purchase project management training. The question, at its most basic level, asks whether a project management training course delivers educational value to its participants. This question is studied in the context of a single training exercise used by a particular set of participants in a specific environment and context. As such, this study only scratches the surface of what needs to be investigated in future research. An important extension to this research would be to further examine the contingent value of project management training. We have examined the effects of prior project work experience, team performance, and team process on perceived educational value of a training exercise. There are many other potentially important individual, team-based, and contextual factors that are worthy of examination. Individual factors that may effect the educational value of a project management training exercise include a participant's past performance on actual projects, educational background and areas of expertise, quantity and quality of project management training previously received, and initial levels of motivation and interest in participating in the training exercise. In addition, demographics and personal characteristics of participating individuals, such as age, gender, personality traits, and so on, may influence the amount of educational value received. At the team level, the individual factors listed above may work separately or in combination to affect the perceived educational value of the training exercise, depending on the mix of participants in a team. Other factors, such as whether team members are from the same industry and/or company, and whether team members know and have worked with each other prior to the training exercise, may all play a role in affecting educational value. Teams of individuals can be organized homogeneously or 20 heterogeneously with respect to all of their individual factors. These team profiles can be systematically varied to see how they influence the degree of educational value received by participants. The context within which the training is delivered may also be critical to learning. Because academic classroom training is often different in nature than corporate training, the participant learning gains may be contingent on the setting. Both the corporate facilitator and the academic facilitator are there to maximize the learning of all participants. However, the academic facilitator often plays the additional role of critical evaluator, since course grades have to be assigned. Does this role of critical evaluator affect how participants work in teams and make decisions? Another important contextual factor is whether the training is delivered in a single block of time, or spread out over a number of days, weeks, or even months. Will participants retain enough detail to make a training exercise effective when it is delivered in discrete, separated blocks of time? If so, then firms will benefit by having more flexibility in the scheduling of their employees for training. Finally, there are measurement issues that may be explored further. This study allows participants to self-report their pre-exercise and post-exercise levels of knowledge and ability to apply that knowledge. While perceptual measures are often reliably used in social science research, there are limitations inherent in using a single type of measure to gauge training effectiveness. Is the inclusion of additional measures warranted? These additional measures can include perceptual evaluations from other team members, perceptual evaluation by the training facilitator, or more objective pre-testing and post-testing of participants. To conclude, the objective of this research is not to examine a particular provider's training exercise. Rather, it is to add to a body of research that takes a critical look at the educational value of project management training. As more research accumulates, it should be possible to categorize training offerings by their more fundamental underlying characteristics, and begin to match the effectiveness of certain types of training experiences to participant characteristics and training context. It should also be possible to learn more about the complementary roles and educational potential of conceptual versus experimental project management training. Over the long term, this will provide a valuable service to 21 both providers and purchasers of project management training. Training providers will gain research based insights into the types of participants most likely to benefit from their specific training offerings, allowing them to tailor their training offerings to maximize participant learning. In turn, the purchasers of training will gain by an improved ability to focus their training expenditures more effectively to match the target audience. Ideally this will result in a steady improvement in the effectiveness of project management training. Acknowledgements The author is grateful to Neil Hymas, co-founder of We, for his and his organization's cooperation in using the Managing by Project simulation exercise at the
