Fostering work-based learning with CAL

Kees Floor, Paper presented at CALMet 01; Recife 8-13 juli 2001.

1. Introduction

The Manual of Synoptic Satellite Meteorology: Conceptual Models and Case Studies (hereafter called: SatRep-manual) (Winkler et al. 2000), is not at all the best looking, nor the most interactive or the easiest to navigate CAL-material that is available for use by experienced operational forecasters at KNMI. Nevertheless research amongst KNMI-forecasters shows that they indicate to learn more from this manual and that it is used more often than any other CAL-module or CAL-course available in-house or on the web. Obviously more factors are important in making CAL an effective learning tool for use by experienced operational forecasters, working on their professional development.

The research that revealed the success of the SatRep manual as a special kind of CAL, i.e. a so-called electronic job aid, in fact was aimed to investigate work-based learning of forecasters at KNMI. This investigation, results and conclusions will be described in section 7-9 of this paper. A number of trends in training and learning in organisations - described in section 4 - and the changing role of forecasters and professionals working in other areas - described in section 5 -, were a starting point of this investigation. One conclusion that will be drawn is that factors of work, work context and organisation can have even more impact on the success of learning with CAL than 'traditional' criteria to judge CAL, as described in section 6. But at first we now describe in section 2 the SatRep manual and the SatRep-approach as a tool for analysing the actual weather situation. The SatRep manual can be considered to act as an electronic job aid; therefore in section 3 job aids are introduced and discussed. Job aids fit in a number of trends in training and learning in organisations, that, - as mentioned before -, are summarised and illustrated in section 4.

The SatRep-manual (version 3.0) was developed by the Austrian Meteorological Institute ZAMG, the Royal Netherlands Meteorological Institute KNMI and the Finnish Meteorological Institute FMI, co-sponsored by the European Meteorological Satellite Organisation EUMETSAT. It is written in HTML; earlier versions of the manual were presented at CALMet '97 (Floor 1997) and CALMet '99 (Zwatz-Meise 1999). The manual consists mainly of descriptions of a still increasing number of conceptual models of synoptic scale and mesoscale weather systems. Conceptual models are an important diagnostic tool, widely used by meteorological services, especially in the field of nowcasting, very short range forecasting, training and professional development. They allow both an interpretation of satellite data and the subsequent use of this data in combination with other data sources as numerical model output and conventional observation data. Many examples can be found in the literature, e.g. Bader (1995), Conway et al. (1996) and obviously the manual itself (Winkler et al. 2000).

In the manual each description of a conceptual model contains paragraphs on cloud structure in satellite imagery, meteorological physical background, key parameters, weather events, and typical appearance in vertical cross section. New in version 3.0 is the availability of short versions of each conceptual model described, suited for quick and easy reference, for instance from the forecaster's desk. A table of contents of the manual is included in Lagouvardos (2001); the complete manual can be accessed on the web (Winkler et al. 2000).

One of the reasons to develop a SatRep-manual is to assist forecasters in producing the so-called Satellite Report (SatRep is a link-up of the two words); therefore it also contains some introductory chapters on the SatRep approach. This method was first employed in the 1980s at ZAMG and is developed by Dr. Veronika Zwatz-Meise. Originally the SatRep was a written report containing information of the infrared METEOSAT image in combination with NWP-output of the ECMWF atmospheric model and with observations in terms of conceptual models. Further development took place with the co-operation of the partners mentioned and included the use of satellite imagery in other channels, NOAA polar orbiter imagery and output of regional models, like Hirlam. ZAMG also is working on an automatic satellite image interpretation ASII (Jann et al. 2001). Nowadays the graphical presentation (e.g. figure 1) is more common than the bulletin format; it is produced three times a day by ZAMG (06UT), KNMI (12UT) and FMI (18UT) and made available on the web (www.knmi.nl/satrep/), together with the 'automatic SatRep' ASII.

At KNMI the produced SatReps are considered to be a powerful tool in operational weather forecasting. SatReps are used to diagnose and understand the actual synoptic situation, to monitor NWP output and to relate dynamical processes to surface weather phenomena. They also are part of the guidance package that is used at KNMI to inform colleagues about the state of the atmosphere and about expected atmospheric developments and to facilitate mutual discussions between forecasters (Floor & Maat 2001). When preparing a SatRep bulletin and a SatRep graphic (figure 1) the forecaster connects features seen on recent satellite imagery with the conceptual model of synoptic and mesoscale weather systems that applies best to the actual weather situation. He or she applies these conceptual models to a range of model fields to verify NWP output.

As the SatRep-manual is available on the web, it can be used as an electronic job aid that can be consulted from the forecaster's desk for use during shift work if web access is available, for instance while producing a satellite report (SatRep). In the next section we will discuss job aids and their use.

A job aid is a repository for information, processes or perspectives that is external to the individual and that supports work and activity by directing, guiding and enlightening performance (Rossett & Gautier-Downes 1991). Other names for job aid are 'job performance aids' or 'performance support systems'. Job aids can be accessed and used in real time during the actual performing of a task; they may be available as a hard copy or electronically retrievable. Well known examples of job aids in meteorology are the WMO clouds wall chart and the tables with plot symbols of weather and clouds for manned (not shown) and automatic observation stations (figure 3). The latter is used quite often now in The Netherlands because of the increasing portion of automatic observations. A meteorological performance support system was described for instance by Verteuil et al. (1993).

Characteristic for a job aid is that it is used at the workplace during the work. In fact it often compensates for the limitations of memory. One hundred ww-code weather pictograms, some of them only occurring a few times a year, are really too much, just as more than forty conceptual models, each with their own characteristics on satellite imagery, model fields and cross sections of the atmosphere! So one of the tasks where job aids can be very helpful is finding the right information needed to do one's job. Job aids can also help when executing procedures or taking decisions; an example is COMET's Decision Aid for Hydrologic Forecasting which can be found at the MetEd-site (www.meted.ucar.edu/hydro/hyd2/hydfopro.htm). Job aids usually cost less to prepare than training, can be customised and can be readily changed when procedures or information needed at the forecaster's desk change. Many forecasters create their own 'job aids', which they have developed from experience, to guide them through forecasting tasks.

The concept of job aids got more and more attention because it easily fits in some trends that can be observed in the field of 'training and learning in organisations'. The rise of information technology, the possibilities to integrate computer based training or computer aided learning in training programs and the increasing role of the computer and ICT as a facilitator of learning is a first but by far not the only development that occurred in the last decade. Its effect on job aids is that they are nowadays usually electronically retrievable, resulting in electronic job aids or even complete electronic performance support systems (EPSSs).

Whether or not ICT is used for training in an organisation, for managers and workers learning often equals course-based training (Poel & Van der Krogt 2001); this also seems to be true for many National Meteorological Services. However the past decades showed several new insights on 'learning beyond the classroom' that give a different perspective on corporate learning; they were a starting point of the investigation described in section 7-9 of this paper.

One of these new insights for instance is the increasing awareness of other solutions than just giving 'regular' training when trying to solve problems in organisations like competence deficiencies of employees, inefficient production processes or lack of quality of products. Some of them are summarised by Rothwell & Kazanas (1992); one of the alternatives to instructional solutions they mention are the job (performance) aids, that so are recognised as a learning tool. An overview of methods of execution of education and training in meteorology and operational hydrology other than 'regular training' was recently given by Van der Beken (2000) and Riddaway (2000) (in the context of continuing education and training).

Following specific courses also is only just one of the popular methods for e-learning, that among others include more open, self directed activities like surfing the web, reading information downloaded from the web and surfing an intranet (Honey 2000). Clearly, multiple ways of organising learning have come into existence over the last decade, although formal training still remains a well-known and well-used practice (Poel 1998).

As a third trend the focus of learning in organisations shifted more and more from education and training to learning (Honey 1996, Bolhuis & Simons 1999, Kwakman 1999, Sambrook 2001, Simons et al. 2000, Simons et al. 2001). The choice made by the organisers of this conference in the early 1990’s for the wording 'Computer Aided Learning (CAL)' instead of possible alternatives like 'computer aided training' or 'computer based training', illustrates this trend. Many keynote addresses, oral presentations and workshops at previous CALMet conferences were aimed to emphasise the learner's perspective. When more recently coining the word e-learning the option e-training was not even considered! (Honey 2000). The shift in focus can be seen as a consequence of a shift in educational paradigm from behaviourism to constructivism, as discussed for instance by the Coimbra-Group (1998); it moved the locus of control of the learning process, and with it the focus of education, from the teacher to the learner. Learning is considered to be the result of an entwined interaction between what goes on in the head and what goes on in the environment. The use of job aids can be seen as an example that fits in this shift from teaching to learning.

A fourth training reform that has taken place is the (re)claiming of the workplace itself, the natural place for learning (Raelin 1999), as a legitimate environment for training and learning (Harris et al. 1998). Training on-the-job, since long a recurring part of the training of observers and forecasters, got new impulses by 'structured on-the-job training' (Jacobs & Jones 1995) and was implemented at for instance KNMI (Kuppens et al. 1995). Combining the trends 'shift from training to learning' and 'workplace as an environment for training and learning' leads to an increasing awareness of the possibilities of learning at the workplace. This learning on-the-job can take place in the context of initial training, but can also be used as an instrument to bring about innovations (Glaudé 1997), as was done e.g. by KNMI when developing new procedures for producing and issuing 'human guidance' (Floor & Maat 2001). In line with these developments, after CAL-modules had become available, it has been suggested, for instance by members of EuroMET, that CAL could be delivered at the forecaster's desk. Adding the trend discussed in section 4.3, revealing the importance of other approaches than just 'conventional training materials', the use of (electronic) job aids can be seen as a more promising example of an electronic tool fostering learning on-the-job; they can be considered to be one of the tools to facilitate individual workplace learning (Baskett 1994).

A fifth new insight was the awareness of the dominant role of informal learning, often defined as the opposite of the commonly known 'formal learning' that is taking place in situations like for instance at schools, at many universities and at training institutions of NMSs. Formal training however is thought to contribute only about 10-20% to what workers need to know to do their job and formal educational programs are only small part of the overall learning and change process of professionals (Baskett & Marsick 1992, Baskett 1993). Informal learning is considered as a residual category to describe any type of learning which does not take place within, or follow from, a formally organised learning programme, event or package executed in the presence of a designated teacher or trainer ¹) (Eraut 2000). Eraut though prefers the term non-formal learning over informal learning, because of its associations with so many other features of a situation, e.g. dress, discourse or diminution of social differences. Non-formal learning occurs in the course of participating in every day work activities; the learning process is not determined by the organisation or its training department (The Teaching Firm 1998). It takes place within the context of naturally occurring challenges (Marsick 2001) as part of an effort to achieve organisational results and is not done for the sole purpose or learning (Watkins & Marsick 1992). The learner in non-formal learning might be conscious of his learning but the learning also can be implicit and unintentional, where the learner lacks awareness of his learning. Workplace learning is difficult to distinguish from 'work' because it is embedded in the work experience rather than divorced or abstracted from the job.

Combining 'non-formal learning' and ' workplace as an environment for learning' leads to work-based learning: learning for work, at work and through work (Harris et al. 1998). Job aids can play a role in this work-based learning.

A sixth trend, that has no direct link with job aids but was also at the basis of the investigation reported in this paper, will be discussed in section 7.

Work-based learning is as important for 'professionals' as for other employees of organisations; nevertheless in the literature the role of learning in the work of the professional gets more attention. Forecasters who are 'meteorologist' according to the WMO classification (WMO 2001) can be considered to be at least para-professionals and the developments in other professional areas might be thought to be relevant to forecasters, too.

The trends in corporate learning mentioned in section 4 also have parallels in the views on professionals and the knowledge and expertise they posses and use. Conventional understanding has it that knowledge comes from books, journals, lectures, CAL-modules etc. and can be stored for future use or 'consumption'. Following this assumption the professional's work is assumed to consist of the application of theoretical knowledge which has often been generated outside the workplace This model however is primarily associated with formal knowledge only.

A more recent view is that professionals' learning occurs in interaction with the environment, e.g. everyday work problems, using a wide number of different sources. (Baskett & Marsick 1992); it is increasingly considered as learning in the workplace (Bolhuis 2001) and in fact a type of non-formal learning. People emphasising the role of experience in forecasting in some way also underpin the importance of work-based learning; compare the research of Roebber & Bosart (1996) on the contributions of education and experience to forecast skill.

The concept of non-formal learning however is not introduced as a substitute for structured training or education. Despite of off-job learning's lack of spatial and temporal proximity to the task and work environment and the attendant learning transfer problems, some jobs, like that of the forecaster, demand solid theoretical foundation that can only be gained through formal study. Nevertheless non-formal learning at work and through work should be given more attention; to reach that more additional research is required. An investigation, carried out at KNMI, will be described in section 7 and following sections. But first we discuss factors influencing CAL at the workplace.

When discussing the quality, the usefulness or the success of CAL at the workplace several points of view are relevant at different levels (cf. Sambrook 2001). At a lowest level CAL is just software that should be judged according to the same standards as software that is used for other purposes. Items that can be checked are for instance userfriendlyness, graphics, text, ease of navigation, interaction, IT skills required, color, pace, feedback, links, hardware specifications, help facilities. However CAL is not only 'just software', it is also learning material; therefore additional items like presentation, structure, content, explanation given, examples used etc. also have to be taken into consideration. At a third level there is learning in general; the CAL has to be designed taking into account recent ideas and theories about learning (e.g. APA 1997). Lots of work in this field is done by COMET and has been discussed in workshops on earlier CALMet Conferences. A fourth level is that of the context in which the learner working on his professional development, is expected to use what is learned, usually on-the-job. Individual characteristics of the individual and his managers, structure and culture of the organisation, support given to learning activities, time, money etc. are influencing factors, too. It is believed that these factors of work, work context and organisation can be even more important than the simple criteria at the lower levels, as might be the case with the use of the SatRep manual at KNMI.

The research that revealed the success of the Manual of Synoptic Satellite Meteorology, was not particularly aimed to investigate the role of the SatRep-manual or more generally the function of electronically retrievable job aids. In fact it was set up to explore what factors help and hinder learning at work and to make clear what kind of actions can be taken to foster individual and collective work-based learning of operational forecasters. The five trends on learning in organisations discussed in section 4 and the changing role of (para)professionals described in section 5, were a starting point, together with a sixth trend: discerning between learning at an individual level, learning in a group of peers and learning of an organisation as a whole (Bolhuis & Simons 1999). The concept of a learning organisation was introduced by Senge (1990) and elaborated and used by many others (e.g. Riddaway 2000). As learning organisations are less prevalent in practice than predicted, the less ambitious phrasing learning oriented organisation is sometimes used (Tjepkema & Scheerens 1998). Bearing this sixth trend in mind, collective learning was included.

Investigating non-formal learning is not straightforward, as people often are not aware they are learning, cannot discern between learning and work or find it easier to name the work process than the learning process. Eraut (2000) summarises the problems that are faced and the approaches that have been used. We follow an approach as used for instance by Fox et al. (1989) for physicians, Eraut (2000) for nurses and midwives and Doornbos & Krak (2001) for police officers. In semi-structured interviews first is asked about the nature of the job, challenging tasks that are part of it and the competencies/expertise required to do an excellent job. Only then is discussed how the competencies and expertise were acquired and what sources and strategies of learning had been used. If the focus was too much on meteorological content, the answers on questions on the nature of the job often could be used to also address other aspects of the forecaster's job, like communicating with colleagues, taking decisions, focussing on customers needs, and being a team member and a 'learning resource' for others. Finally it was asked what new competencies and expertise one wanted to acquire, how this could be done and what could hinder or help to achieve this. As the research was focused on non-formal, partly implicit learning at work and through work, following Kwakman (1999) and Doornbos & Krak (2001) the use of the word 'learning' was avoided as much as possible because of its connotations to learning in formal training programs.

For this investigation eight interviews were conducted with operational forecasters. They all were senior level or mid-level meteorologists according to the WMO classification (WMO 2001), who usually spend 75% or more of their time in operational shifts in the central forecasting office of KNMI in De Bilt with two or three forecasters on duty at a time. The interviews were followed a few weeks later by a written questionnaire suggesting 31 possibilities to learn for work and asking them to indicate which ones were used or had potential to learn.

The results gave a lot of insights on what was learned, how it was learned, what factors were helping and what factors were hindering. The interviews gave spontaneous answers that were not suggested before by the interviewer; the answers in the written questionnaires contained judgements on the usefulness of the learning actions suggested in the form. The results (Floor 2001) will not fully be discussed here, however, as the focus is now on the SatRep manual as an electronic job aid and therefore as a type of on-the-job-CAL.

It became clear that all operational forecasters learned from working with the SatRep approach and in fact producing SatReps as a part of the shift duties. During the interviews several of them explained how the use of the manual had contributed to their learning; no one mentioned the use of other CAL. One forecaster gave a very detailed description of the way he is using the manual. When producing a 12 UTC SatRep he first links the features on satellite imagery with the relevant conceptual model. Next he compares this with the insights of his Austrian colleague who has made the 06 UTC SatRep that is available on the web (www.knmi.nl/satrep). Then, in case of disagreement, he uses the web-based SatRep manual to check cloud structure in satellite imagery, theoretical considerations, key parameters, weather events and typical appearance in vertical cross section in order to make up is mind before producing the final version of the SatRep.

Learning input on other topics than SatRep, e.g. from workshops on ensemble prediction and probability forecasting, was not or only scarcely mentioned because it was considered to be less relevant for the work. This was remarkable, as all the forecasters recently had participated in such a workshop! Forecasters, when asked why they did mention SatRep and did not mention other topics, e.g. probability forecasting, explained that SatRep is new, is more integrated in their work, gave more satisfaction and can be applied more frequently.

Even more respondents (50%) than during the interviews indicated the use of the SatRep-manual during work when answering the written questionnaire. Again no use at all was mentioned of other CAL (outside regular formal courses).

From the responses the forecasters provided, several conclusions can be drawn.

At first the use of CAL-modules by forecasters as a part of self-directed professional development is not obvious, even if these modules easily can be accessed on the web or from the intranet from the workplace; special arrangements have to be made to stimulate their use.

Second if knowledge or theory must be offered to the forecasters then the learning potential of the forecasting tasks done is very important when deciding what content to offer. The production of a satellite report apparently is considered to be a more challenging task then issuing probability forecasts and one is more prone to gain expertise on it.

Third work-based learning is really taking place at the forecaster's desk. The availability of the SatRep manual, acting as an electronic job aid, in combination with its usefulness for one of the job tasks, proves to be very helpful. This type of CAL obviously is contributing to the knowledge and skills of the operational forecasters and is really fostering work-based learning.

Fourth factors of work, work context and organisation can be even more important than the criteria like 'good looking', easy navigation, interactivity, animations and all the other things we like ourselves, when using, developing or promoting CAL.

______________

Note

¹ According to Eraut's (2000) definition - that is used in this paper -, all organised training is formal training. Van der Beken (2000) only includes education within the regular and highly structured school system and vocational and academic educational systems in formal education; in his article all Continuing Education and Training is defined as non-formal, even if classroom education is used as method of execution.