Critical Attributes of ID Project Success

The 51 attributes, or project management practices, listed below are grouped according to the 10-Step ID (Instructional Development) Project Management model from Michael Greer’s text ID Project Management: Tools & Techniques for Instructional Designers and Developers, Educational Technology Publications, 1992.  Greer’s workshop Planning Successful ID Projects helps ID project managers develop the skills needed to implement many of these practices. These critical attributes were published as part of the article “Critical Attributes of ID Project Success: Part II — The Survey Results,” Performance and Instruction, July 1993. This article reported the results of a survey of P&I readers regarding project management “best practices.”  For more detailed information, please refer to the article.

Phase I: Project Planning

Before Beginning the ID Project
1. Front-end analysis (needs analysis, job analysis, task analysis, etc.) was completed.

2. Front-end analysis appeared to be thorough and defensible.
Step 1: Project Scoping
3. Preliminary materials specifications (estimates of specific deliverables of video tape, pages of print materials, numbers of overheads, etc.) were completed.

4. Preliminary materials specifications seemed to be thorough and accurate.

5. A detailed project schedule and/or time estimate was completed.

6. The project schedule and/or time estimate was relatively fine-grained (i.e. describes the number of days required to complete various activities – not simply weeks, months, or phases).

7. A budget and/or cost estimate was completed.

8. The budget and/or cost estimate was relatively detailed (i.e. described costs of each phase, of labor, of outside purchases, etc.).

9. All scoping estimates are based on defensible rules of thumb or direct experience obtained from projects in our organization.

10. All appropriate planners, managers, and sponsors reviewed and approved the scoping estimates.
Step 2: Organizing the Project
11. Materials specifications, schedule, and budget were confirmed or revised based on elapsed time between proposal and eventual project approval.

12. List of project team members, including roles and responsibilities of each, was created.

13. Project diary or similar collection of important project documentation was established.

14. A well-organized kickoff meeting, assigning roles and responsibilities and clarifying important project issues, was executed.

15. A detailed project schedule, taking into consideration vacations, holidays, and other staff requirements, was developed and approved by team members.

16. (If applicable) Vendors or contractors were provided with a formal Request for Proposal (RFP) detailing the project requirements before they presented their bids.

17. (If applicable) Vendors were selected in a fair and impartial manner.

18. (If applicable) Clear mechanisms for handling all necessary vendor paperwork and obtaining vendor payment was established and executed.

Phase II:  Instructional Development

Step 3: Gather Information
(… some of this may have been done during front-end analysis)

19. An information gathering strategy and corresponding set of tools (questionnaires, interview guides, observation guidelines, etc.) were developed and then approved by the project sponsors

20. The target audience for the materials was clearly specified.

21. Adequate details about the audience’s relevant work environment was gathered. 

22. The specific tasks or skills which must be taught were clearly specified.

23. Adequate technical details about the course content were assembled.
Step 4: Develop the Blueprint or Specification
24. A debriefing session was conducted after completing information gathering in order to discuss implications of any new information learned on the preliminary design strategy.

25. A blueprint or set of design specifications was created.

26. The blueprint (design specifications) contained:
  • An adequate overview (big picture description) of the instructional materials and course flow.
  • An adequate description of specific performance objectives.
  • An adequate description of specific instructional strategies to be employed to attain all objectives.
  • A detailed outline of content to be included in support of each objective
  • A summary of media and materials to be created to support all objectives.
27. All appropriate reviewers reviewed the blueprint (design specifications) and provided helpful feedback.

28. Reviewer feedback was obtained quickly enough to allow the schedule to be maintained.
Step 5: Develop Draft Materials
29. The design team obtained formal approval (sign-off) of the blueprint (design specifications) from the course sponsor before going on to create draft materials.

30. Instructional designers and developers were provided with clear specifications (page layout, writing style, etc.) for draft development.

31. Preliminary and revised drafts of all materials were developed.

32. Preliminary and revised drafts of all materials were circulated to the appropriate reviewers (SMEs, planners, marketing policy people, etc.)

33. All appropriate reviewers reviewed the draft materials and provided helpful feedback.

34. Reviewer feedback was obtained quickly enough to allow the schedule to be maintained.
Step 6: Test Draft Materials
35. Formal approval (sign-off) of drafts by the course sponsor was obtained before testing draft materials.

36. A test run of all courseware was thoroughly planned; the test included a formal strategy for gathering evaluation data, summarizing it, and using the data to specify revisions.

37. A test run of all courseware was conductedc

38. Detailed revision specifications were developed and communicated to appropriate team members.

39. Revision specifications were approved (signed off) by the course sponsor.
Step 7: Produce Master Materials
40. A production strategy was presented to producers, including clear statements of expectations and deadlines.

41. High-quality master materials that could be used to create correspondingly high-quality reproductions were created.

42. Formal approval (sign-off) of these masters by the course sponsor was obtained.

Phase III:  Follow Up

Step 8: Reproduce
43. A reproduction strategy was presented to reproduction people, including clear statements of expectations and deadlines.

44. High-quality copies of all course materials, as defined by the design specifications, were created.
Step 9: Distribution
45. A distribution strategy was discussed with appropriate distribution personnel.

46. Copies of materials were assembled and properly stored.

47. An efficient distribution system existed for dissemination of course materials.
Step 10: Follow-up Evaluation
(… after the courseware has been implemented)

48. An evaluation strategy and a corresponding set of tools (questionnaires, interview guides, etc.) was developed and then approved by the course sponsor.

49. The planned evaluation strategy was executed.

50. Reports of trainee skill level after completing the training and recommendations for corresponding revisions to the materials were made.

51. Based on what was learned during this project, recommendations for improving our instructional development process were made.

Project Life Cycles versus Key PM Processes

We know that a project life cycle is made up of a collection of related phases. And each phase in the life cycle is made up of a bunch of related tasks or activities. And the exact nature of all these tasks, activities, and phases is dependent entirely upon the finished products (deliverables) you are trying to create. So, a media producer has a “Scripting” phase made up of many tasks related to drafting and refining the script. And a home builder has a “Blueprint” phase made up of creating and refining the home’s floor plans. And a software developer has a “Design” phase in which clear software specifications are created to guide the programmers.

You get the idea:  Deliverables determine the tasks & activities needed, which in turn determine the project life cycle. On the other hand, there are the Five Key PM Processes that pretty much everyone agrees are universal: Initiate, Plan, Execute, Control, and Close Out. No matter what your project, you can apply these processes to keep things moving. The trouble is that a lot of PM newbies (and organizations who are new to PM) confuse the five generic PM processes with their life cycles. They try to pound the square pegs of their project’s necessarily unique phases into the round holes that are the five generic PM processes.

The result is that I sometimes find myself working with clients who insist that their local PM model has a distinct phase labeled “Plan” or “Execute” or, worse yet, “Control.”  But, I usually ask them, how can you possibly restrict all “execution” chores to a single phase? And aren’t you “controlling” throughout the project?  As you can see, this can all be very confusing for a PM newcomer.

Though it usually takes some time to sink in, here’s my bottom line message to these folks: Any project’s life cycle (that collection of tasks, activities, and phases) are unique and always reflect a specific set of deliverables or “best practices” of an industry. On the other hand, any project’s work processes (i.e., what you do to move from phase to phase) involves the 5 generic PM processes: Initiate, Plan, Execute, Control, and Close-Out.

So no matter what phase of a project you’re in (any project!), you must Initiate that phase, then Plan that phase (or revisit & revise the Plan), Execute the tasks associated with that phase, Control the tasks, and finally Close Out that phase.  When you complete the phase (i.e., when you “close out” the phase), then you start all over again with the next phase and Initiate, Plan (replan), Execute/Control, and Close Out that phase. And so it goes… over and over again.

The image below is from a PDF file of a few simple slides I sometimes share with PM-newbie clients to help them see the distinction (and the relationship) between the Five Key PM Processes and your project’s unique life cycle. Click the image, check them out, provide your own narration, and maybe you can help clarify this confusing topic for a PM newbie you know!

Project “Post Mortem” Review Questions

A Web-Published Article by Michael Greer [Click here to download the PDF.]

Overview

It’s important for project managers and team members to take stock at the end of a project and develop a list of lessons learned so that they don’t repeat their mistakes in the next project. Typically such reviews are called post-project reviews or “post mortems.”  I recommend a two step process for conducting these reviews:
  1. First, prepare and circulate a whole bunch of specific questions about the project and give team members time to think about them and prepare their responses individually.
  2. Next, hold a meeting and discuss the team’s responses to the questions. The result of this discussion is often a list of “Lessons Learned.”
The benefit of the first step, done individually by team members, is that it allows the quieter, more analytical people to develop their responses to the questions without being interrupted by the more outgoing, vocal types who might otherwise dominate in the face-to-face meeting. Also, it allows everyone the time to create more thoughtful responses. So what would be on the list of questions? I’ve provided some of my favorites below.

General Questions

  1. Are you proud of our finished deliverables (project work products)? If yes, what’s so good about them? If no, what’s wrong with them?
  2. What was the single most frustrating part of our project?
  3. How would you do things differently next time to avoid this frustration?
  4. What was the most gratifying or professionally satisfying part of the project?
  5. Which of our methods or processes worked particularly well?
  6. Which of our methods or processes were difficult or frustrating to use?
  7. If you could wave a magic wand and change anything about the project, what would you change?
  8. Did our stakeholders, senior managers, customers, and sponsor(s) participate effectively? If not, how could we improve their participation?
Phase-Specific Questions (These will differ from project to project, depending on the life cycle/phases. The phases identified below are explained in detail in The Project Manager’s Partner: A Step-by-Step Guide to Project Management and The Manager’s Pocket Guide to Project Management.)

Phase I: Determine Need and Feasibility

  1. Did our needs/market analysis or feasibility study identify all the project deliverables that we eventually had to build? If not, what did we miss and how can we be sure our future analyses don’t miss such items?
  2. Did our needs/market analysis or feasibility study identify unnecessary deliverables? If so, how can we be sure our future analyses don’t make this mistake?
  3. How could we have improved our need-feasibility or analysis phase?

Phase II: Create Project Plan

  1. How accurate were our original estimates of the size and effort of our project? What did we over or under estimate? (Consider deliverables, work effort, materials required, etc.)
  2. How could we have improved our estimate of size and effort so that it was more accurate?
  3. Did we have the right people assigned to all project roles? (Consider subject matter expertise, technical contributions, management, review and approval, and other key roles) If no, how can we make sure that we get the right people next time.
  4. Describe any early warning signs of problems that occurred later in the project? How should we have reacted to these signs? How can we be sure to notice these early warning signs next time?
  5. Could we have completed this project without one or more of our vendors/contractors? If so, how?
  6. Were our constraints, limitations, and requirements made clear to all vendors/contractors from the beginning? If not, how could we have improved our RFP or statement of need?
  7. Were there any difficulties negotiating the vendor contract? How could these have been avoided?
  8. Were there any difficulties setting up vendor paperwork (purchase orders, contracts, etc.) or getting the vendor started? How could these have been avoided?
  9. List team members or stakeholders who were missing from the kickoff meeting or who were not involved early enough in our project. How can we avoid these oversights in the future?
  10. Were all team/stakeholder roles and responsibilities clearly delineated and communicated? If not, how could we have improved these?
  11. Were the deliverables specifications, milestones, and specific schedule elements/dates clearly communicated? If not, how could we improve this?

Phase III: Create Specifications for Deliverables

  1. Were you proud of our blueprints or other detailed design specifications? If not, how could we have improved these?
  2. Did all the important project players have creative input into the creation of the design specifications? If not, who were we missing and how can we assure their involvement next time?
  3. Did those who reviewed the design specifications provide timely and meaningful input? If not, how could we have improved their involvement and the quality of their contributions?
  4. How could we have improved our work process for creating deliverables specifications?
[Insert your own, deliverables-specific questions here.]

Phase IV: Create Deliverables

  1. Were you proud of our deliverables? If not, how could we have improved these?
  2. Did all the important project players have creative input into the creation of the deliverables? If not, who were we missing and how can we assure their involvement next time?
  3. Did those who reviewed the deliverables provide timely and meaningful input? If not, how could we have improved their involvement and the quality of their contributions?
  4. How could we have improved our work process for creating deliverables?
[Insert your own, deliverables-specific questions here.]

Phase V: Test and Implement Deliverables

  1. Were the members of our test audience truly representative of our target audience? If not, how could we assure better representation in the future?
  2. Did the test facilities, equipment, materials, and support people help to make the test an accurate representation of how the deliverables will be used in the “real world?” If not, how could we have improved on these items?
  3. Did we get timely, high-quality feedback about how we might improve our deliverables? If not, how could we get better feedabck in the future?
  4. Was our implementation strategy accurate and effective? How could we improve this strategy?
  5. Did our hand-off of deliverables to the user/customer/sponsor represent a smooth and easy transition? If not, how could we have improved this process?
[Insert your own, deliverables-specific questions here.] ________________________________________________________________
 Do you like this article? It’s from my new eBook, The Project Management Minimalist: Just Enough PM to Rock Your Projects! and it’s also in the  latest edition of The Project Manager’s Partner: A Step-by-Step Guide to Project Management contains 57 tools, checklists, and guidelines to help project managers. For more information, click on the link above or phone HRD Press at (800) 822-2801.