Case Studies in Knowledge Management

Learning from Simple Systems: The Case of JPL 101 15

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written

In addition, the quiz database recorded the answers submitted each time someone took
a quiz.
The online survey was used to collect basic organizational demographics (tenure,
organizational unit, job category, and whether a manager or not) and responses to two
questions: “Did you learn anything from the questions?” and “Did you learn anything
from the answers?” Taking the survey was voluntary, as was responding to the
demographic questions. The second anonymous response method was an online
feedback form. Users could submit comments, problems, feedback, and candidate
questions for the system. While most users decided to remain anonymous, some made
the effort to include their names and contact information. Finally, the e-mail based
feedback form was available to contact the development team directly. This was not
anonymous and was the least-used form of feedback.

Results

JPL 101 premiered on January 13, 2003, and ran for 12 weeks ending its initial
deployment on April 6. It remains in operation, although new content is not currently
being developed. Results are presented based on analysis of the data collected during
the initial 12 weeks, and extending through Week 19 of operations relative to the
following: design considerations, usage, motivation for use, learning results, and general
reaction.

Design Considerations

Background usage and database data were analyzed to assess how well the design
considerations were met. Background usage data indicated success in meeting the
participation time goals of the system. The average time spent in the system each
workday ranged from 2:01 minutes to 8:21 minutes, with the mean being 3:53, which are
within the limits recommended by JPL Ethics and Human Resources offices.
A second consideration was that the quizzes needed to be challenging but not too
hard. Figure 7 shows the average quiz scores for the 12 quizzes, based on data from the
entire operational period. With the exceptions of weeks five and eight, the average quiz
scores stayed between 70% and 90%, meeting the goal.

0

20

40

60

80

100

1 2 3 4 5 6 7 8 9 101112

Quiz Number

A

verage Quiz Score

Figure 7. Average quiz score per quiz