(originally written for Family Planning NSW)

We often train people in groups because that is cheaper than providing one-to-one coaching. We know that group training is typically less effective than individual instruction (Bloom 1984), but this is not hopeless.

Computer-based instruction has been designed to simulate human tutoring but has never been shown to be as effective. We can imagine common-sense explanations for this. VanLehn (2011) argued that two explanations – Feedback and Scaffolding – were particularly plausible. However, these are typically weaker in group training. Is computer-based tutoring a justifiable alternative to group-training?

In a review of controlled-trials of computer-based tutoring, VanLehn found good justification for further development of computer based training.

Main conclusions:

1. actual human tutoring is less than ideal (d=0.79 rather than 2.0)
2. step-based interactive tutoring systems can be almost as effective (d = 0.76)
3. effectiveness plateaus as granularity of interaction decreases; i.e. there isa limit to the complexity of system needed.

Application

VanLehn’s review supports granular, interactive, step-based cognitive training. Use computers to train individuals to apply complex procedures, giving feedback on each reasoning step taken, prompting thought rather than supplying information.

The hypotheses:

1. Detailed diagnostic assessments – “human tutors do not seem to infer an assessment of their tutee that includes misconceptions, bugs, or false beliefs, nor do they seem to be able to use such an assessment when it is given to them”
2. Individualised task selection – “on this argument, computer tutors should be more effective than human tutors”
3. Sophisticated tutorial strategies – ”studies of human tutors in many task domains with many degrees of expertise have indicated that such sophisticated strategies are rarely used”
4. Learner control of dialogues – ”although students take the initiative more than they do in classroom settings, the frequency is still low”
5. Broader domain knowledge – ”although human tutors do have broader and deeper knowledge than computer tutors, they sometimes do not articulate it during tutoring, and when they do, it does not appear to cause significantly larger learning gains”
6. Motivation – ”In short, even though motivational tactics such as praise, the “warm body effect”, or false positive feedback are common in human tutoring, they do not seem to have a direct effect on learning as measured in these studies.”
7. Feedback – “the frequent feedback of human tutoring makes it much easier for students to find flaws in their reasoning and fix their knowledge”.
8. Scaffolding – “Scaffolding is common in human tutoring”.

REFERENCES

Bloom S 1984 The search for methods of instruction as successful as one-to-one tutoring. Educational Leadership, May, pp 4-17.

VanLehn K 2011. The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4):197-221.

“IT pros always and without fail, quietly self-organize around those who make the work easier, while shunning those who make the work harder, independent of the organizational chart.” (Ello 2009)

I liked this article in 2009. It is pretty much echoed by Eric Schmidt (in interview with Mitchell 2010). Today it seems more true than ever, and more widely applicable. In my recent experience:

• Doctors behave somewhat like Ello’s geeks, perhaps because health services require expert problem-solving teamwork, drawing on a body of non-obvious, evidence-based, professional wisdom.
• University-based scientists have both the bureaucratic and meritocratic drivers that Ello described, and it is easy to see Ello’s patterns in university departments and projects.

Ello’s take home message to managers of geeks:

“Favor technical competence and leadership skills. Standard managerial processes are nearly useless in an IT group.“
“If you can identify and cultivate those individuals and processes that earn genuine respect from IT pros, you’ll have a great IT team.”

Reference

Ello J (2009) The unspoken truth about managing geeks. ComputerWorld [online] reproduced on http://techoped.com/2009/09/08/the-unspoken-truth-about-managing-geeks/ and http://www.cio.com/article/print/501697

Mitchell R (2010) How to Manage Geeks. FastCompany. [online] http://www.fastcompany.com/37145/how-manage-geeks