We had users throughout the Fed assist us with this pilot test, and we collected survey data online from each learner: 1) what scenario he or she was referring to, and 2) what lessons in the tool he or she might access. We also asked the standard "did you like it, what else would you want to see" questions.

Positive pilot data validated our design and gave us a few ideas for tweaks to complete before the official launch.

After the test, the course was launched—on time. Since the launch, survey data continues to remain positive. And plenty of learners still offer suggestions along the way, allowing us to make subtle improvements to the tool.

Around that time, we started working with another course committee on a series of regulation and policy manuals that examiners read, called BEST modules. This is a series of about 20 PDF files, some as long as 100 pages. When surveyed, the examiners said that the content was great, but that they wanted an easier way to access and reference the great information in those modules. With the success of "Examining Bank Operations" under our belts, we decided to work with the course committee to build another hybrid tool. We had learned some lessons on the past tool, and knew what changes we wanted to make.

As before, we started planning on the next tool, this one called "STaRT" for Supervision Training and Reference Tool, by analyzing the content, design, and development structure. The material was very similar to the "Examining Bank Operations" tool; there was a large amount of initial text, some references, a possibility for case studies, and self-checks.

We thought that even though the design might be similar, the development model could look different. We also had to find a different way to access and use knowledge from the SMEs. We wanted the best and brightest for each module, but we didn’t want a long development cycle. Rather than having a large team of SMEs on multiple modules, we created a small team of SMEs on one module. This reduced their commitment to the project and enabled us to work with them over a shorter period of time for a specific content area. We also rebuilt the storyboard templates to include not only the sections of the tool, but also some editing notes for each section to reduce the amount of times each person had to review each lesson. With our development strategy hammered out, all we had left to focus on was the details of the design.

Although we liked the layout of the first tool, we wanted to add in functionality for a true search. We also needed to consider how we would build all of the pages for the tool. We decided to develop a solution that used ColdFusion to assemble the pages for us, so we literally had one page that acted as a presentation page and assembled the appropriate assets as a user selected a module and a lesson. We still had to create the content pages in HTML, but we applied the Cascading Style Sheet (CSS) to the presentation page, in order to make all of the content pages simply that—content.

We still had the requirement to make the tool downloadable, though. After evaluating several utilities that could build static versions of dynamic content, and allow search capabilities without requiring a backend search index, we settled on some tools and felt our strategy was solid.

Finally, there was the potential that we would develop 20 modules over a two-year period. In the end, we used a phased approach in which every six weeks we would deliver two modules within a single week. This phase would last approximately six months, spanning the end of 2004 through 2005. During the non-delivery periods, we would develop and launch content (See table 2004 Development + Launch Cycle). By the end of 2004, we had 10 modules, totaling more than 100 lessons.

We have started work with the course committee on "Examining Bank Operations" and have decided to move all of its content into "STaRT" model. We’re also thinking about the next version of these tools. We would like to add more interactivity, such as an "Ask the Expert" function or perhaps videos of actual bank staffing action. Imagine a video on fraud with the "tells" of when people are not telling the truth and follow-up questions that senior staff have used before to get to the truth.

There’s nothing holding us back now. The shell is there, and it’s working great. More important, it’s giving people a tool that they can use on the job. But we’re not done—there’s always room for improvement.

The Two Tools Compared