Where does active learning happen?

I’ve been scratching my head about whether my blogging is active learning, using definitions that are in common parlance nowadays. On the pro side, I do produce some new or a fresh perspective on what others are talking about by making associations between disparate ideas that are out there. This part fits in with constructivist notions of learning. But the activity is almost pure introspection, especially in the pre-writing phase. Then the activity becomes a blend of introspection, research qua Web surfing via Google (and some other sources of search), dictation to myself and modest keyboarding of that dictation, one phrase at a time, as well as some immediate editing to see if it reads back as well as it sounded the first time through and to catch obvious typos, all part of the process in what I call the composition phase.

So on the con side, it is a solitary activity without social interaction. Once in a while I will write in response to a post elsewhere by a friend or colleague and then perhaps there will be a response back to that, but more often than not I’m initiating based on what I’d call “external stimuli,” for example a New York Times article or an interview on the Charlie Rose show; and clearly I don’t have a co-author with whom I negotiate the content of a post. The lack of social interaction might disqualify it as active learning in the minds of some. Indeed, when talking about learning these days we don’t seem to talk much about introspection, reflection especially as a means of metacognition about some other learning activity, yes, but introspection as a means to initiate an idea, no. For me, blogging is a lot of that. Is it active learning? I’ll let others be the judge.

A lot of what I do in my job working on relationships with others around campus and developing new initiatives I would term active learning. The principal tools are email for asynchronous communication and then face to face conversation, either in one of our offices or more likely at a coffee place. It has the feel of an ongoing conversation rather than a one shot thing and it may very well be that we have multiple distinct but interrelated threads that we deal with in any single meeting. There are some smaller workgroups that also function this way. Larger committees don’t. In large committees, if I were doing the “whip” function I would treat members as a group of individuals and then have one-on-one interactions with each individual, which is where the active learning would happen. It is very hard to do brainstorming in the larger group absent those one-on-one interactions and the discussion will go nowhere in a hurry unless it has some direction based on prior interactions with individual members.

People do have busy schedules so sometimes a conversation doesn’t conclude within a given session. But the conversation doesn’t stop because somebody has to be somewhere else. We’ll pick it up again face to face in the not too distant future and in between there will be email aimed at furthering the ideas. With people I know reasonably well, we schedule generously to not feel rushed when we do meet and because we like these conversations. So for me, active learning occurs at the coffee place, somebody’s office, while at my computer, and if the introspection part counts too then any place I do that, which is basically anywhere and everywhere when I’m by myself.

With that as background I want to consider active learning by students in the courses they take. I have never been comfortable with that notion because it is something unlike what I described above. Often, it refers to quick hitter interactions that are a break from lecture or that are used to jump start an ensemble discussion. Often the active learning ends when the bell goes off signaling the end of class. And in many of the cases of which I’m aware these active learning activities are done as a one off. In the next class the students are paired differently. I know that most of those in the profession advocate for active learning of this type, but I think there are some pernicious consequences to the approach, mainly that a student can readily come to believe that good ideas are there for the having in short order, which in turn creates a sense of impatience in the students that learning should happen quickly (and that they need not put in a lot of time to learn a lot). Indeed, once an instructor has developed that perception in the students further accommodation of the perception requires the instructor to steer away from hard problems. The pressure to meet the perception helps to make the perception self-fulfilling.

If we want our students to engage in deep learning, it seems inescapable to me that most of the active learning has to happen outside the regularly scheduled class time, in small groups that persist for a while and which interact in a manner similar to the way in which I interact in my work, and if we agree that introspection can be a part of the process then also when the students are by themselves but focusing on the issues at hand. We are now many years into supporting our approach to active learning, both on my campus and in the profession at large, but on this basic point we still haven’t reached agreement that active learning should happen this way, at least when considering active learning in the setting of regularly scheduled courses.

And it seems to me that the reasons why we have no such agreement is that the instructor can readily observe what happens during the scheduled class session but can at best only infer what happens outside of class and then make only very coarse inferences, which are based on the work that students produce and often that work itself is only measured by how students do on exams.

Much current criticism of the lecture is based on considering that in isolation, instead of viewing the lecture in conjunction with the work that students do in study groups, which is where the active learning is apt to occur. It certainly seems at least possible, and I believe my first year in graduate school functioned much this way, that the in-classroom lecture during the day serve as fodder for the introspection and the study group interaction that happens mostly in the evening. When I was in grad school the study groups formed as a grass roots thing because many of the students needed it as a survival mechanism in the classes and the rest of us who could have done ok with just the introspection wanted to hang around with our classmates for social reasons.

Let us all agree that lectures that don’t engender active learning outside of class (and while I’ve not looked at the recent results, the NSSE from a few years back indicated that at many public universities we have quite a few classes of this sort) are quite likely ineffective. But what of those that do? In other words, what if lecture is accompanied by rather serious work for the students to complete out of class, with the lecture at least in part an instrument to help the students complete the work? Does that mechanism provide for effective learning or not? If not, does the study group approach fail because group activities are structured by the students themselves rather than by the instructor and hence the students apply themselves in a manner that is not efficient to promote their own learning?

I’ve not actually seen anyone argue on this point. Rather it seems the gospel that lectures are bad pure and simple and that active learning should happen in the classroom and only in the classroom. So I find myself at odds with much of the profession, because as I said that doesn’t make sense to me.

I could readily believe an argument that said some active learning in the class is necessary because students need to learn to scaffold their arguments and discussions in a way the produces results, not gridlock. But surely we shouldn’t be under the impression that the in class work is sufficient in itself. Look at Chickering and Ehrmann’s Seven Principles. They do make sense to me. But I believe it is possible to implement them all while still having some substantial amount of lecture face to face.

Addendum to Previous Post

I did test everything I wrote about in the previous message, but Norma tried the Calendar and it worked like the Gmail and Google Docs, not how I reported. I'm not sure what's up with that. In any event, there is a post in the Announcements area on how to get the group Calendar into your Google Calendars and then use it as advertised.

What’s in a name? And what’s with Google?

Here are a few things about Google stuff, some good and some not so. Let’s start with the good and if you have a Gmail account you can try this first one yourself. Log into Google and then follow this link, to add Prof Econ’s Group (Professor.Economist is a particular account in Google that I have for this demo) as a tab to your iGoogle page. Even if you don’t want to try that, you can get much of the idea by going to your own iGoogle page, clicking on the down arrow next to the active tab, and then following the Share this tab link.

When you do that, you get a window to invite people to share with. In that window there is a checkbox for each gadget on the tab and then there is another checkbox, initially unchecked, that says Share pre-configured gadget settings. I checked that box, then sent the invite message to another of my Gmail accounts. Here is a screenshot of the added tab for those who opt not to actually add that tab to their own iGoogle page. You can follow along with the rest by looking at the screen shot.

There are 6 gadgets on this tab. Prof Econ’s Place is my name for what others might call “Announcements.” It is created by a feed from this blog, which is from where it takes its name. If you’ve installed the tab, you can actually click through to the blog from the gadget. Likewise, the del.icio.us content is what others might call “Bookmarks” and obviously comes from a del.icio.us feed. Do note that the feeds sometimes take a while to update. Let me skip the next three gadgets and return to them. The Google Video gadget also comes from feeds. In this case you do a search to identify the video content so the content you want must be uniquely identified by the search. I did two different searches, each associated with its own tag.

The above three gadgets work pretty much as in any other feed aggregator, such as Netvibes (though I’d be curious as to whether the Google Video gadget works in Netvibes). The next three work differently and in my view inconsistently apply the checkbox for pre-configured gadgets. The calendar one works as advertised. It appears to show the particular Calendar from Prof Econ’s group, whether the user is subscribed or not. The other two gadgets show the most recent items from the user’s Gmail and Google docs, not the most recent items for these tools from Prof Econ’s Gmail and Google docs. That’s ok if it is understood how it works.

The Google docs gadget is really the most interesting one. For Prof Econ to get a document he has uploaded to appear in that gadget for a user, Prof Econ must share the document with the user. Sharing is managed via Gmail contacts. In the process of sharing the document, Prof Econ sends an email message to the user to which says that the document is being shared. The user doesn’t have to do anything with that email. It serves as a receipt only. One reason to show the Gmail gadget on the page is that for each document that Prof Econ shares there should be a corresponding email receipt.

In order to get this to function smoothly, Prof Econ needs to amass the Gmail addresses of all the users. At present, that is the single biggest headache to get this approach to work – how to amass those accounts. In a group with a few dozen people, this is a manageable problem and I’d encourage the approach. In a class with several hundred students, that problem may make the approach unwieldy. From my play with this, I suggest that if the group gets much larger than 5 or 10, that the person playing the role of Prof Econ actually set up a separate Gmail account just for this purpose, so that the contact list coincides with membership in the group. Then the distribution is fairly easy. If the contact list in your regular Gmail account has many names already, deciding who is in the group or not will be time consuming. Perhaps soon in the future Google docs will allow subfolders of Contacts and you can set up the group that way. But right now a separate account seems to be the easiest approach.

In some respect this post is a sequel to an early one, iGoogle and the LMS. I should point out that the LMS my campus uses, WebCT Vista, doesn’t have a document folder of this sort where the instructor shares documents with students and I know from comments I’ve received from MBA students that they’d really like to have such a feature. Indeed, the entire approach with iGoogle has appeal over an LMS approach in that everything is right there at the top level. For example, with the use of AJAX the user can read the message announcements by clicking the little plus sign next to the subject heading and do that without ever leaving that page. Likewise, the videos can be viewed right in the page, with a click-through possible if the user wants to see the video in a larger size. However, the issue goes beyond LMS in the sense that working groups of all sorts probably want this type of functionality, and for many of these working groups it would not occur to them to use an LMS. Convenience-wise, what is shown here can serve as a reasonable benchmark for alternative solutions. It may be less secure than some other possible solutions, and the content sits on Google servers rather than on servers run by the Campus, but in many instances those concerns are peripheral to users while convenience is central. I think this does quite well on the convenience front.

Let me also point out that if Google docs is really embraced this way, then the flow can go both ways and, in particular, students can share docs such as assignment submissions with Prof Econ. (As with the contact list for users, Prof Econ might want a separate account just for the “drop box” function, so to separate out those documents he has uploaded from those that have been shared with him by his students. But if Prof Econ manages his folders well, this may not be necessary.) The big virtue of using Google docs in this way compared to the more traditional drop box – no download is necessary. This makes the workflow of reading and commenting on the documents much easier and might really encourage electronic communication where now we still have a lot of paper flow.

If I were currently teaching the Campus Honors course I taught last in spring ’06, I’d try this approach.

* * * * *

Now to a more frivolous matter, but something that Google is doing that does bother me and I wish they’d stop. If you do a Google search on a first name, on some of these searches images will appear at the top. On others no image appears. As there is a Google Image search and a Google Video search, I’m not sure why they are allowing images in regular searches. At a minimum, it would be good for the user to control what comes back, whether it should just be Web pages or if images (and later video) should be allowed to come back too.

I know this from searching on my own first name. I get some substantial traffic to my blog from people doing that type of search. It turns out there is a porn star with the same first name and (not surprisingly) her site is the #1 response to the Google query. I’m ok with that. It’s what happens with an open Web. But why does her picture have to appear at the top of the page when doing this search?

I have not done a statistically valid experiment of searching on various first names and seeing whether they return images of well known people with that first name or not. But I’d bet that most of those pages that do return images show pictures of women scantily clad. I’m sure whoever at Google thought including the images this way was trying to cut down on the number of searches that the user makes to find what they are looking for. In general, I’m all for efforts that reduce the time needed to find stuff. But some sensibility should be brought to the endeavor. Folks who are actually looking for my site shouldn’t have to be exposed to these photos if they don’t want to. And the reality is that people are sufficiently imprecise with their search terms, so they very well might do a first name only search since my first name is somewhat rare and it’s easier to remember just one name rather than two. Google should reconsider its approach given this unintended consequence.

Do we plagiarize inadvertently?

This morning I read a blog post from Inside Higher Ed’s University Diaries. This one is called Plagiarism: Yours, Mine, and Ayres’ and concerns the apparent double standard where students receive severe punishment for getting caught plagiarizing but where highly visible faculty and administrators who have been found plagiarizing seemingly get off scot-free. The tone of the post is righteous indignation. And for some reason that bothered me, not about the faculty and administrators named in the piece and the sins they have committed, but rather on the question of how far from the norm they have strayed. If you recall the film, Breaking Away, I feel a bit like Dave Stoller, before he rode in the race with the Italian team where one of them rammed his tire pump into the spokes on Dave’s bike, forcing him to fall hard and drop out of the race, only to proclaim to his dad later that evening something to the effect, “They all cheat. I just didn’t know.”

On this plagiarism question, I just don’t know too. I’m not even sure about my own behavior. So I don’t think it a trivial question to ask. Let me illustrate what the problem is and then make some guesses on how to find the slippery slope.

First, having written a modest number of Economics papers published in refereed Economics Journals, plagiarism and correct citation in this case is a comparatively easy matter. A standard part of the writing is a literature review section and one covers the pieces that preceded the current work there. The incentives are in line to get the citations right – citations are one measure of influence of the work and there is a quid pro quo of sorts in citation, but really one needs to cite broadly to show referees that the author understands the relevant literature and to place the contribution correctly within that work. The only possible issue here is with comparatively new work that may still itself be under review. In any event, I’m not thinking about traditional journal publishing or academic book writing in asking my question.

I’m thinking about blogging, such as this blog. With informal publication of this sort, the set of potential sources is huge, one doesn’t typically try to do anything like a literature review, and there is no referee to please or clear standards to meet regarding citation. But that is not the real problem. The issue is how we get information nowadays. We are bombarded with information from a variety of sources: email, Web sites, radio and TV, information that is pushed at us by friends, colleagues, and business relations; information that we actively seek out on our own; and then all the follow up information after we have learned something and want to learn some more. Further, much if not most of this information is gathered with no intent whatsoever to write about it. Instead, we get the information to “keep up” or perhaps so we can feel on par when we engage in conversation where some of the information may come up in passing. Most of us are in this boat.

When I start to write a blog post there is something that triggers that main idea and I’m aware of that something, so that I cite by linking to the source if at all possible. But then a related idea might occur to me. I say to myself, “where did I see that?” And if I’m lucky I can do a Google search or two and find the source that triggered the related idea. Then I link to it. But sometimes I’m not so lucky and now there is an ethical choice to be made – mention the idea but with some faint disclaimer like, “I know I read this somewhere” or something else to that effect just to show that I don’t mean to plagiarize but I also don’t want to put in the research time needed to find the source, mention the idea without any disclaimer whatsoever with the intent most likely simply to make it cleaner reading since the disclaimer is a sidebar and not the topic of discussion, or choose not to include the idea in the post because one should not make claims that can’t be substantiated.

I believe that I do each one of these on occasion. I don’t have a rule book to follow and simply make some judgment at the time (based on what I’m not sure) as to which is the most appropriate. I have no sense of how frequently this occurs. One doesn’t set out to write something with sources that can’t be identified. And it’s not something I fixate on. There are a variety of other small transgressions I make quite regularly – I jaywalk around campus, indeed I do it deliberately sometimes so I can retain my New York roots while at Illinois – at home I believe we violate a codicil to our community association’s bylaws in that the size of our satellite dish is too large – and a bunch more of this sort of thing that are in the mild if not totally benign category. Life is too short to worry about this stuff. But what stuff should it be that we do worry about?

And that gets us back to plagiarism. I have to conclude that I’ve done it on occasion in this blog, though I don’t know when. Certainly, I was not trying to cheat. Nobody is requiring me to write a blog post. And, at least directly, I make no profit from the activity. But do I always cite my sources? Probably not. Should anything be done about that? I hope not. I’m not sure what could be done that makes sense. My inkling is to put this in the mild category and not worry about it, but seeing stones thrown about plagiarism as in that Inside Higher Ed blog, I’m not entirely comfortable with that answer.

Yesterday in my support role for the course management system used by instructors in my college, I read the citation policy as stated in one particular course. There was very strong language used, akin to a Do Not Trespass notice. If I might get shot at by the owner of the private property, I’ll obey the no trespassing notice, no question about that. But if there are strong rules that are mostly self-policed my adherence to them will be adjusted to my personal sense of reasonable behavior and in my world view a modest level of violation is typically ok, especially if there is no apparent harm.

In general, we don’t educate students with the aim to achieve an acceptable norm. Instead we give them absolute norms and then fret when we witness egregious violations. In the case of plagiarism, in particular, we may not be aware of our own behavior, think only of our scholarly publishing and not at all of our informal communication. Those distinctions may mean less to our students. For them, the rules of citation may seem arbitrary. What larger lessons do they learn when we make them follow the rules this way? I don’t know but I do know how I’d react if I were in their shoes.

Who will win the Nobel Prize in Economics?

It's that time of year. The Nobel Prizes in Medicine have been awarded. Time to guess who will win this years Economics award. A list of past winners can help in weeding out who won't get the award this time through.

Whether you teach economics or some other subject, I wonder whether getting students to predict the winner of some well known prize in the field, giving them some reward for a correct prediction, and thereby encouraging them to do some research on who are likely candidates, is a good way to drum up some interest in the subject. Call it the People Magazine approach to instruction.

For what it is worth, I'm predicting William Baumol, using the improbable criterion that among all economists (other than myself) I believe he is the one most frequently mentioned on this blog. (And Paul Krugman seems still too young to be a candidate.)

Online Presentation Ideas

As there are plans to teach a new course offering, Business 101, in blended format and because we are doing a pilot with Adobe Connect, the eLearning Team in the College has been playing with Adobe Presenter, a complementary application. Here are some observations about online presentations based on that experience.

Self-Tests within the Presentation

My office mate, Norma, liked the look and feel of a Presentation done with Adobe Presenter – it does have a clean look where one can readily see the slide titles and timings fore each slide, along with a search capability as another way to get random access to the content. But I was intrigued to learn that you can intersperse quizzes that are “graded” within the presentation – the viewer gets immediate feedback.

The use of this seems manifold. The WebCT Vista course management system that my campus supports does not have a self-test feature within a quiz that is for credit. It only has a stand along self-test. So to deliver the requisite functionality within a quiz for credit, one could put in “hint links” to self-tests in Adobe Presenter. That should work fine.

A different use would be for mini-lectures delivered online. As a way to chunk the content there could be short quizzes that are really there to let the viewer ask the question: “Am I getting this stuff?” Also, the quiz allows some planned pause time for reflection before going on with the presentation. This post from the Tomorrow’s Professor blog makes it clear that breaking up presentation in this way is quite effective.

Do note that the quiz questions must be text-based, no multimedia, and there is a limited range of question types. But one might go reasonably far with what is there.

Examples: Audio Only or Talking Head Video?

Here are two different versions of a similar presentation. The audio only version has the dual virtues that it is very easy to make – the audio can be recorded directly in Adobe Presenter – and it launches quite quickly even though in this case this is on a regular Web server, not a Flash streaming server. The talking head video version takes longer to make, because the video must be recorded outside of Adobe Presenter and then brought in, slide by slide. For viewing it takes some time to buffer the video content, even with a good network connection. And relative to how the videos looked before they were inserted into Adobe Presenter, which resized them to fit into the presenter box, they are kind of grainy here. But might the viewers prefer this mode anyway because it seems more personal? Perhaps the answer depends on the context. Both versions are available so you can make your own judgment.

Moving away from the PowerPoint Style?

Folks who are used to presenting with PowerPoint likely will be comfortable with Adobe Presenter. But do note there are other ways to go and once you start to think about that, other possibilities will suggest themselves. Here is an archive of all the videos that were made for the previous presentation along with two some additional clips, one by me on hyperinflation of college tuition, the other joint with Norma. I think the joint stuff is much more compelling to the viewer. In any event, the video can be delivered on its own via the archive, or a clip can readily be embedded in a Web page such as a blog. One loses the interactive quiz function this way, but there might be other activities which would be more meaningful for the students than quizzes while still achieving the desired chunking of the presentation. For example, perhaps they should do some quick writing in private reflection, or instead make a post to a discussion board with questions they have. Moreover, if there is interest in getting the content out to others, it is an interesting thought that the above mentioned archive is on Google Video, it is readily discoverable by a search there. (Such a search also searches for video content in YouTube, but not vice versa.)

Screen Capture Video

Talking head video has its place, and the interview style makes for interesting viewing, but in many cases it will just be the instructor doing the presentation. Consequently, the instructor will want to look for other ways to spice things up. Several people on campus are already using Jing. This quick demo looks indistinct from a capture done with Jing’s cousin, Camtasia, but it is made much more quickly, and Jing is free. (One suspects there might be a charge in the future. If not a direct charge, TechSmith, the company that makes Jing, will need to find some other way to generate revenue for the service.) Short captures can add spice and they really are easy to make. Further, file size is typically small so Jing movies can be sent out via email if that is preferred to putting them on a Web site.

One thing Jing can’t do is to make a second pass at the recording. I’ve found making my own screen captures that if I talk and capture at the same time then I tend to ramble and the entire thing becomes too long. One needs to be merciful to viewer of these things – get in, say your piece, and get out, quickly. So I do better by doing the screen capture on first pass and then doing voice over onto the previously recorded screen capture. This makes the process more elaborate but gets a better end result. If you’re making movies of 5 minutes or more, I’d recommend being more elaborate in this way. If your entire movie is under a minute, Jing is fantastic.

Modeling for Student Presentations

Instructors will first think of this as a lecture alternative for themselves, but one of the nice things about all of these approaches is that students can use them too. So an instructor might want to embrace an approach that is not just suitable for herself but which also make sense for her students and then try to envision her own presentations as giving her students tips on how to do their presentations.

Getting Students IntoYour Presentations

If your presentation is mainly response to questions that students pose, it will be more compelling to watch. You might elicit questions from the students in advance so you base your presentation around those questions. Or, better still, you might invite the students to participate in Q&A that you record. Podcasting (audio only recording) is quite easy to do this way. Talking head video is only a little bit harder. Also, you can add slides or screen movies to the recorded audio content afterwards if you think it will benefit the students.

Writing versus Presentation

Once the presentation moves online another consideration is what to deliver as written text, for example in a blog post, and what to deliver as presentation. My own view is to mix and match. Highly technical content probably can be delivered more effectively in presentation style (I’m thinking about economics with a lot of graphs and equations) so the students get both the visual information and the audio explanation. But otherwise, I’m guessing that what is best is what is most comfortable for the instructor – if the instructor likes to write…

Do Some Assessment

Given multiple possibilities in method of delivery, it doesn’t hurt to try a few different approaches and then ask the students which they prefer (and why).

Re-Use

Online presentations can be used again. Realistically their half-life can’t be too long because the technology changes and they begin to look dated and the content itself may also lose its freshness. But certainly some re-use is likely. This should encourage the instructors to think of what they want to do up front not being determined solely by the time it takes to produce the stuff. There will be some time savings down the road.

In the Heat of Mocking a Killing Bird

Used to be things were black and white: ice cream sodas, ideas, and race relations. Last night I caught the tail end of David and Lisa. The TCM channel was doing a Keir Dullea retrospective – 2001: A Space Odyssey came on next. This morning I read some brief reviews about the film online and I was surprised. The reviews characterize the movie as a love story and a study in mental illness. Perhaps I’m jaded, thinking about One Flew Over the Cuckoo’s Nest in the same genre. But to me, David and Lisa is plainly an allegory. David didn’t want to be touched – he thought he could remain safe if everyone kept their distance. This is a perfect metaphor for the upwardly mobile middle class distancing themselves from the rest of the world. David blamed his parents for his predicament. This happened earlier in the film, before I turned it on. It serves as a strong indictment that we don’t prepare ourselves or our children for living in the real world when we take a gated community approach to life. Lisa lived in a sing-song universe (I’ve been known to do likewise). She constructed her own reality, a more pleasant place. The movie ends with both of them leaving their comfort zones – Lisa, having spoken a regular sentence, takes David’s extended hand. The message is that we can confront reality, hard as it may be, if we do it together.

Keir Dullea and Janet Margolin are both beautiful to watch in this film. Dullea displays elegance and tension in a single look. At the end of the film he puts the tension aside. He becomes human, caring, outward looking, with genuine concern for Lisa’s well being. Margolin has wonderful eyes, an empathy in her look, her running away a sign she has been hurt badly. The film is so good because it takes on such tough issues and yet does so with delicacy and touch.

I led off with it, on a banal level because it was shot in black and white, on a more serious plane because it reminds that we need to break through our shells and confront the serious issues of the day. This is a post about one of those serious issues – race and what, if anything, does learning technology have to say about it. It’s not an issue I talk about nor do I write about it, even occasionally. I’m more than a bit out of my comfort zone in this post. So I need a way to warm to the subject. Leading off with the film and book references is my mechanism for doing that.

Race is not an issue that should be depicted with delicacy. The resulting violence certainly conjures up a different set of emotions, a sense of assault, a feeling of horror, and fear, lots of fear. The first movie I recall that gave us all of that was In the Heat of the Night. Where David and Lisa is delicate, In the Heat of the Night is raw. Both speak to a possibility that we might work together. But they come at that possibility so differently, one to heal the hurt they brought on themselves, the other to ward off the demons of Southern White racism.

Harper Lee’s novel, reading de rigueur when I was a young teen, bridges both worlds. Ironically, mental illness has a role to play in her story, not as a window into the craziness of the real world, but rather in the character of Boo Radley, who could love the Finch children from afar, creating the mysterious presence of an adoring stranger, and ultimately serving as their protector against the vengeful violence of Bob Ewell, who was fueled both by racial hatred and a strong sense of inferiority that comes from being “poor white trash.” The film version has iconic status in the minds of many. For years, I wanted to wear tortoise shell eyeglasses, to imitate Gregory Peck as Atticus Finch in the film.

Indeed, the film embedded perhaps the most important message of ‘60s with regard to race – with understanding and respect for our fellow humans we could put race issues behind us, a critical example of the more general lesson, time heals all wounds. A few year later, in college, I “was taught” the mechanism by which the healing takes place. The racists die off. Their children take on the understanding of Atticus Finch rather than the hatred of their own forebears. This is the hopeful approach to both multiculturalism and pluralism, should it only be true. Part of the lie in the lesson is in ourselves, not just in the progeny of the Bob Ewell’s of the world. We don’t have the Atticus Finch commitment to right the wrong. We’re back in the David and Lisa world, trying to keep reality from touching us. I’m ready now to address my subject and I’ll begin with my own experience.

* * * * *

As a learning technologist, I had one episode where race was explicitly an issue. I’m not sure of the exact date. It must have been 2000 or 2001. There was a graduate student with whom I interacted who had been doing a study on digital divide among our students, with the Whites in the have category and the Black and Latino/a students in the have not category. The issue was about whether a student was under a serious disadvantage if he didn’t have a computer in his dorm room or place of residence, but had to rely on a computer lab instead for access. And the related set of issues is whether this created more cultural distance between the students as there are a host of related literacies that develop as a consequence of always on computer access.

So it occurred to me on my own (and there was one African American University administrator pushing this theme) that perhaps we should do a like study among the faculty, to see if there was a digital divide among that population. I consulted with one Black faculty member who had some prior experience as a Campus administrator and whom I knew via a summer workshop I had headed up. He thought it was a reasonable idea to investigate. I can’t recall whether he suggested this or if the idea cam from elsewhere, but soon thereafter I had a meeting with the heads of the African American Studies program, Latina/Latino Studies program, and Asian American Studies program, to talk about doing a study on digital divide among the faculty.

Coincidentally, but ironically, we were all interim in our positions at the time. (I was a little irked by my own status as I got my job via a faculty committee that had recommended we have a Center for Educational Technologies and that I should be the director, but there hadn’t been a formal search pursuant to that recommendation.) I recall our interim status serving as a mild bond among this group. They were all interested in doing the study. So we developed an email survey about use of instructional technology and barriers to usage, sent out by each director to their own constituencies, with the option that the recipients could reply to me directly if they so wanted.

I learned a few things from this exercise. Some of the faculty who were surveyed became irked at me for not understanding that as faculty they were treated quite well and had good technical support – race didn’t matter in this regard at all. One faculty member in the History department was unaware of the support offerings that my Center provided and so she received some consultation as a consequence. I also learned that in some cases where instructional technology was not being used in the classroom the underlying explanation was scheduling, not preference. The instructors actually wanted to teach in a smart classroom but because the mechanism that allocated those classrooms favored other criteria – the prior history of space utilization and class size of the department doing the scheduling – it was difficult to get these courses into a smart classroom. I believe we made some efforts to counteract that effect at the time, though it being several years ago I’m now ignorant of the current circumstance on this issue. And a fact I already knew from other contexts, that Black faculty get a large number of requests to participate in events around Black issues on Campus, even if those issues lie entirely outside the domain of the faculty member’s research, meant that these faculty had even more than the usual time pressure so lack of uptake of instructional technology could be explained entirely by that, rather than by inadequate technology support. But only some had done little with learning technology. Other faculty of color had accomplished much with technology. Overall, there was no digital divide among the faculty that I could discern.

Several years earlier, I was engaged in a different sort of issue about race. The Econ department was recruiting a minority faculty member under a campus program called TOPPS. The candidate was Sandy Darity and his recruitment was something of a big deal. I recall going to a lunch with him at the Union, where many of my colleagues in the department attended. I went to a cocktail party (I can’t recall where) that had faculty from around campus who were interested in this particular hire. I attended Darity’s seminar. And the evening after the talk I went to dinner with him, the department head, and one other faculty member. I had no special role in the recruitment other than being a faculty member. This happened before I gone too far down the path with learning technology. With junior recruitment I sometimes had gotten involved fairly heavily because I felt responsibility as a potential colleague. But in this case Darity would be coming in as a full professor, so my involvement was due to this being a big deal.

Econ has been a divided department and this particular case was controversial. Darity could be classified as a Sociologist by some; I don’t mean that as a derogatory term but I do mean it to contrast with the Chicago School heavy theory emphasis (in the tradition of Lucas and Prescott) in the Econ department here. There was awkwardness around this disparity and, unfortunately, you could not parse it in a straightforward way from awkwardness about race. I found his talk strange – not the subject, which was about North-South trade and how the evidence put the lie to the rising tide theory – but rather the style, which was pure anecdote with no model and little data. Also, it ended abruptly after 50 minutes. Our usual for seminars is an hour and a half. One of my colleagues whom I respected and still do respect a great deal characterized the talk as a polemic rather than a paper. Another colleague, whom I’m more suspicious about, said a few days later that if it were Yaw Nyarko, then fine, no problem with such an offer. (I don’t believe that the Econ department later tried to recruit Nyarko, but I’ve been involved with learning technology matters since and have not tracked all the hiring activities in the department.)

At the dinner, I felt a need to be a human being to Darity; he’d been under a lot of scrutiny during the earlier part of his visit. So I gave him some grief for ordering White Zinfandel – real men drink red wine. It’s funny, you joke about the things that don’t matter because you can’t talk about the other things that do. I also recall engaging him in a semi-serious conversation about something Thomas Sowell had written in the New York Review of Books. (I was still a subscriber at the time.) Unlike the rest of his visit where he was entering our space, in this conversation I was treading on his territory. My recollection is that it was a reasonably pleasant dinner and that we ended with the standard parent question on a recruiting visit – how good are the schools? But overall the visit was quite awkward.

It brought back to mind a different experience I had in graduate school. Glenn Loury was an Assistant Professor at Northwestern while I was a grad student. There was a nighttime seminar series then held at 626 Noyes street, one of the houses where the Econ department had offices before it moved into Anderson Hall. It must have been 1977 or 1978 and I was probably a second year student at the time. Loury gave a theory paper on the dynamics of black-white income inequality and what would be the optimal time path under which the inequality would be eliminated. The underlying question was whether it made sense to give an immediate push to reduce inequality or if laissez-faire was better. This was supposed to be a theoretical way to get at the question of whether affirmative action was good or bad.

The tradition after the seminar was to go out for some beers and talk about the paper some more afterwards. After the talk one of the other students asked me to ask Loury to join us at one of the bars in Chicago. (Evanston was dry at the time.) It was an awkward moment. On the one hand, by then I had a well deserved reputation for drinking beer and engaging in conversation over beer. On that dimension, I was the logical one to ask Loury. But there were several other graduate students at the seminar who were senior to me and on academic grounds one of them should have asked. They obviously felt uncomfortable doing so. I did ask Loury. He turned us down. He had some other obligation, or so he said. The rest were relieved. I don’t think Loury held this incident against me. On several subsequent occasions I had a beer with him. That night never again came up in conversation. But I don’t think those type of situations are easily forgotten.

Each of the previous incidents lie in that gray zone of discomfort where civil discourse remained but where there was a sense of unease. I’ve had one experience outside of the gray zone, squarely in the world of hate speech. This had to with my teaching. When I was teaching intermediate microeconomics in the large section with 180 students, I managed it all via undergrad TAs who graded the homework in addition to providing office hours, face to face in the afternoons and online in the evenings. I recruited those TAs from students who did reasonably well in previous offerings of the course. A couple of those undergrad TAs were African-American students.

The first was kind of strange because I knew this guy from afar before he took my class. In the winter time when it was icky outside, I used to jog on the track at IMPE which was in the main gym above the basketball court, 11 laps to the mile. While I was doing my 9:30 – 10:00 minute miles, this guy would lap me a couple of times. I saw him there quite frequently. Later, when he became my student and them my TA, he reminded me of a different student I had known when I was a TA at Northwestern. That guy’s name was Phil – no clue about his last name. He was always in the Library and every time I saw him he was by himself. There’s nothing wrong being in the Library and by yourself, and he was quite a good student, but I got the sense that it was a coping strategy, that there were no peers who could lift him up only friends who might drag him down. I don’t really know this. It’s only a guess. But I’d bet that was the case. The same was true this for my TA, Harold. He was quite serious and kept to himself. As far as I know he TA’d without incident.

That’s the way I’d like things to be. My own philosophy to collegial interaction; I’m OK, You’re OK; works fine if race is a non-issue. Then we can interact as colleagues should, critiquing each other’s work, trying to lend support and improvement to the ideas we’re floating. It’s a world I’m comfortable in and one I’d like to encourage. For example, after Nyarko visited here (some time before Darity’s visit) I did have some follow up email with him and made suggestions about this paper. But it begs the question, what does one do when I’m OK, You’re OK is not appropriate or simply doesn’t work?

The other Black TA I had was a girl. She endeared herself to me fairly early on. We did nighttime training sessions in a computer lab at the beginning of the semester to get the students in the class up to speed using the software. That year I was mentoring a junior faculty member who was teaching another section of the same course and using my method. This was an apprenticeship for him before he adopted the approach to teach economic statistics on his own. He had a wheelchair bound paraplegic student attend one of my training sessions. Previously I had welcomed him to send any of his students to one of my training sessions – I had many more training session than he did because I had more students. But I was unprepared to meet the needs of this particular student and if I recall correctly she arrived a little late, spoke loudly, and appeared to be disrupting the session.

Because I hadn’t gotten prior warning I was not in a position to immediately accommodate her needs, especially since these training sessions were outside the regular class meeting time and asking my students to attend them meant I had made an implicit promise to focus on them during the session. The Black TA figured out the predicament, spent the rest of the session giving one-on-one support to the handicapped student, and it went ok. Regarding intelligence on human relations, this TA gets high marks.

But her math background was a little weak and to do economics at this level you need the math. My mechanism was that each TA graded one problem from the problem set and for which I had written up the *correct* answer that they could use to help them in grading. Until then I didn’t assign the problems to particular TAs. I let the TAs choose the problems in a first come first serve manner. As luck would have it, the Black TA got the hardest problem to grade on the first problem set. She was slow in evaluating the submissions and slow sending them back to the student teams; my mechanism relied on fairly quick turnover so the teams could resubmit problems based on the feedback they had received. The slow grading was gumming up the works. After that first problem set, I believe I assigned the problems to the TAs for the remainder of the semester.

On the second or third problem set one of the students in the class, a white male, sent an overtly racist message along with the submission of his homework problem, in addition to which he accused the TA of being incompetent on the grading. I didn’t know what to do about this. I may have mumbled something to my TA but otherwise ignored the problem. Sometimes I let students work things out on their own because I think it is best for them to do so, though when they are getting the fundamental economics wrong I’m more apt to interfere. In this case I should have interfered based on the merits but I didn’t know how to handle it. So I mumbled and kept my nose out of it. I’m not proud of that at all.

* * * * *

Finally, let me bring learning technology into the discussion. And for that, let me start off with the final lecture of Randy Pausch. In the introductory remarks by Steve Seabolt, he refers to Randy as “the White-ist guy I know.” The remark gets a laugh from the audience and is probably completely benign; Seabolt is referring to Pausch’s taste for mayo on white bread (in turkey sandwiches) and presumably other like un-hip behavior. I thought it a strange thing to say nonetheless. But I don’t want to linger on that. Instead, I’d like to focus on something Paush himself describes near the tail end of his quite moving presentation, how misdirection, something he learned about on the football field, is one of the essentials of good teaching. You teach students something that is both hard and important by getting them to work on something else that captures their immediate attention. You, the teacher, overly focus on that something else to get the students to buy in. But underneath you understand that you’re mainly after the something else and in the end you judge your own success by whether that something else has been well learned.

We have a problem on campus talking about race and interacting on race issues. It’s not surprising that we’re in this boat. The campus problem reflects the same problem in society at large. We’ve had multiple incidents of hate speech recently. We have a new program on campus- Inclusive Illinois, one campus many voices. This program is aimed as a counterforce. As far as I can tell the approach with this program is straightforward, no misdirection.

There are multiple projects on campus involving Second Life as a teaching tool. One of those has been led by Professor Sharon Tettegah in the College of Education. I got to know Sharon soon after she joined the faculty here and over the years we’ve become friends. I’ve seen her give multiple presentations on Second Life. In the first talk she gave she emphasized the role for students as designers in of virtual worlds and the type of learning we might encourage by giving students such a designer capability. In subsequent presentations she talked about a different type of empowerment that Second Life affords, taking on different personalities via the avatars the person uses navigating within Second Life. It is a great environment for role playing and via the interaction with others a good way to learn how others react to a persona that one takes on in role play.

I have written previously about my not getting it regarding the profession’s fascination with online games. And, truthfully, I didn’t really get it about role playing either. I always put suggestions of this sort through my own set of internal filters with one of the primary questions, would it work for me? Role playing didn’t seem to make it through. After all, I already had a strong voice. I’ve been told that shows up in my blogging. Why would I need role playing?

But sometime over the weekend I came to a different conclusion and in this post I belabored my own experiences with race to show why. I don’t have a strong voice when it comes to race relations. Instead, I feel uncomfortable. I’d like to hide till the discussion turns to something else. And then, in something I read in the last week or two, perhaps something on the espn.com site about race relations fueled by the Michael Vick and O.J. Simpson stories, there was the story how even nowadays Blacks receive almost constant reminders that they are second class citizens who shouldn’t be able to succeed in society. I’m oblivious to the vast majority of those reminders; either my realm of experience avoids them altogether or I’ve been wearing the blinders for so long that I just don’t see what isn’t immediately in front of me. I really don’t know which is the better explanation, but it seems clear to me that both fit in fairly nicely with being uncomfortable actively engaging on race issues.

And now I make a leap, but one I do quite regularly in thinking about learning technology issues, that I can generalize from my own experience and thinking. The generalization is that role playing would be quite good for many of us as a way to heighten our sensitivity on race issues, that technology like Second Life might be quite good for such role playing, indeed something like Second Life might be necessary to pull it off and then, bringing Pausch back in, that the only way to do this and overcome the sense of discomfort is by taking a misdirection approach. We have to be overtly about some other use of role playing with technology or some other use of the technology itself – maybe that’s the design of online environments, maybe it’s to use the technology for virtual tours; on what the misdirection should be I’m not sure.

Let me close with one other point. My Campus at the level of Purchasing and Campus Legal has been extremely conservative about Second Life and giving University sanction to faculty or departments in purchasing “islands” in Second Life. Second Life mimics the real world in many respects and it can be a rough and tumble place. The University, on the other hand, we think of as a cloister, a safe haven. We go to efforts to preserve that. Narrowly speaking, I can understand the Purchasing point of view.

But putting two and two together, and understanding how on this Campus there is a tendency for silos to develop on issues, following the maxim that to keep things tractable make them smaller, I think it is a mistake to deal with race issues over here via the Inclusive Illinois campaign and deal with Second Life issues over there via the approach advocated by Purchasing and Campus Legal, never for the twain to meet. We’re in a position to put learning technology into service in support of a huge social issue on campus, in a way where it could make a real difference. We’ve got the previous 45 years of experience to make us realize that these problems don’t go away by themselves. Shouldn’t we be designing learning experiences for our community so that we’re not in the same boat 45 years hence?

A Vulture of Evidence

Last week Bill Richardson wrote an opinion piece on No Child Left Behind --- scrap it. His view sounds right to me. My kids go to reasonably good schools in Champaign which suffer from none of the problems that Kozol has identified about inner city schools. But even for them, we’ve seen how NCLB has narrowed the curriculum and pushed the teach-to-the-test approach, something my wife and I are not happy about. Yet Richardson’s pronouncement got me agonizing for another reason. The question is whether the bad outcomes associated with NCLB are a consequence of an approach to education that emphasizes measuring student performance or if instead that main issues with NCLB are a consequence of racial prejudice and that an evidentiary approach to student learning can work and work well if there aren’t other issues that dominate and handicap it. I was reminded of discussions I had in graduate school about whether communism was inherently a bad system or if, instead, it was the Soviet Union that was bad but communism might flourish under other circumstances. It’s hard to cling to the theoretical possibility that such a system might work well when there is such a prominent case in point indicating otherwise.

So I began to fret about what NCLB should be telling us in Higher Ed and mostly independent of NCLB and Richardson’s pronouncement, I’ve been trying to find my own comfort zone thinking about accreditation. The College of Business where I work is going through its year of self-study for AACSB accreditation and at the same time my Campus is getting ready for its NCA review. And so I’ve been wondering about whether these efforts will create their own pernicious consequence and so should be resisted. (Or, perhaps, we should comply with the letter but tacitly resist the spirit because overt resistance is too costly). Within the learning technology field, I’ve heard nobody express this concern. There seems to be a lot of buy in to the general idea of an evidentiary approach. So in this post I’m going to try to poke holes. My goal is to show my source of discomfort so to encourage others to ask similar questions. Some of this stuff, I believe, needs to be debated not just taken for granted.

Here I’m further reminded of a book I read while in graduate school, The Foundation of Statistics by Leonard Savage, where in the first chapter the author asserts that sometimes it is the foundations of a discipline which are the most controversial, even for disciplines that are flourishing. The foundational question for those who do advocate for an evidentiary approach is this: what exactly are we measuring? And for those who might want to want to follow Kozol’s lead but focus on the situation in Higher Ed, the question might be: what are the unintended consequences of the measurement efforts we do engage in and how harmful are those consequences? Those are my core questions in my effort when I try to poke holes.

Let me begin with testing and my own personal memories of that, some of which are still strong although I took my last class for credit in 1979 and defended my dissertation in 1981. There are many things I’ve learned subsequently and have since forgotten. But some of these memories of testing remain strong. Why is that? And is it good or bad.

Let me start with the SAT. I took it twice. The first time, I had a broken right arm which was in a cast not quite up to the elbow. I was nervous that it would affect my ability to fill in the bubble sheets. After I got my scores I joked that it only adversely affected my score on the Verbal, not the Math. At the time, there was a lot of emphasis on the word “aptitude” in the SAT name and that the test measured aptitude. I got a 790 on the Math. I assume that means one question wrong, but I don’t really know that. The next memory is blurred between that test, the next time I took the SAT, and GRE, but on at least one of those exams there was a question on Roman Numerals. To this day, I don’t remember which is the Roman Numeral for 500 and which for 1000. I’m ok with Roman Numerals up to 100 but not beyond that. So I likely garbled the M and the D, and got that question wrong. But there is no way that was an indicator of aptitude. After all, you can look these things up. I’ll get back to that in a bit.

My scores that first time were 620 V and 790 M --- definite math nerd. I took it a second time, hoping that the Verbal would go up and got 650 V and 770 M. In the West Wing, my favorite TV series the last few years, the President, who is supposed to be a genius, got 1590 total the first time and took it again! The SAT, in particular, seems to play the same role for high testing students that tattoos play for society at large. Does it leave such a mark on students who don’t test so well? There is now a big industry devoted to getting tattoos removed. Are those who advocate for the evidentiary approach aware of the marks they may be leaving behind on the students from whom the evidence is collected?

Next let me talk about my last semester as an undergrad taking classes at Cornell. Having taken just one intro to macro course in economics, I was advised to take the advanced graduate course in math econ – I had the math credential but no other economics. The course started out with 7 students in total, 4 grads and 3 undergrads. The grads all dropped so for the final there was just the three of us undergrads. We had a take home final and it was brutal. I got a 65 on it (two problems right, partial credit on a third, and the fourth wrong) and with that I was put in my place. Because much of it was over my head I had made some major conceptual errors – trying to prove that a divergent series converges, that sort of thing. I latter learned I was the high scorer among the three of us. One of the other students, who has since become quite a prominent economic theorist, but who was a year or two behind me at the time, got a 20 on the test. I know that during the class I had the feeling that I could do the math (in spite of the serious mistake on the final) but I had no sense at all about why any of the theorems were interesting. Absent that intuition it was hard to be passionate about learning the method of attack for the proofs. So perhaps I didn’t work hard enough for the course. But what else might the exam measure? I know that in many cases we talk of students being able to apply what they learn in a course to other problems they might confront outside the course. Just what other problems would I confront on this stuff? (In grad school I didn’t see this content at all, but when I came to Illinois I taught Undergrad Math Econ and used David Gale’s book, so did ultimately return to some of this stuff and the problem I did confront was how to teach it.)

I took several exams in grad school that I recall but I want to focus on one in particular. Northwestern is on the quarter system and in the first quarter there I took probability from Ted Groves, a tough and rigorous course. The second quarter I took statistics from Ehud Kalai. He is a world class game theorist but his heart wasn’t into teaching statistics. Also, my class got kind of burnt out by the Paper Chase approach to instruction we experienced the first quarter. So the course was less intense than the one Groves taught. Kalai asked us if we wanted a midterm. You can guess the answer. And the in-class final was open book. During that final I recall doing a problem where I read the book to learn the relevant statistics, not as a reference to what I already knew but rather to learn it the first time through; then I did the problem. I got 100 on that exam. And I finished in a little more than an hour, definitely the first one among my classmates to leave the room. Nobody else got 100. But I didn’t know much about statistics. I probably could have scored nearly as well if I were given that test before the course was offered because what I did know was how to solve math problems. I’m an ace at that. Clearly solving math problems is a correlated skill, but it is not the same as knowing statistics. In much else we teach we really want students to develop learning to learn skills so they can solve problems in situ. Knowing how to do math problems is a learning to learn skill on how to do exams with math. Exams with math might want to test that to some degree. But they want to test knowledge of the subject where the math is applied as well. It’s very hard to parse the one from the other, at least with an open book test.

The best type of exams are oral. There is back and forth and the follow up questioning becomes situated in the previous responses. In this manner the examiner can learn quite a lot about the student’s knowledge, how robust it is and how well the student can think through related issues based on what the student is presumed to know. In this sense every real conversation has an element of testing in it. Perhaps that is an observation to suppress. Most students who go through an oral exam, and here I’m thinking of doctoral studies, are extraordinarily nervous beforehand and during and may be quite self-conscious when giving their responses. That type of fear can be an inhibitor and restrict the conversation; to go deeper into the issues the participants in the conversation should be relaxed. In the process the participants learn quite a lot about what each other is thinking. We can surmise that Aristotle certainly understood the competencies of the young Alexander.

Apart from testing and conversation, the other evidence we have of student learning is through the works they create – their writings and increasingly their multimedia creations – and through in class presentations they give. As the latter are typically not recorded (and many courses don’t entail in class student presentation) there has been more focus on the former. And because we now live in a digital world, the works can readily be archived for later review, scrutiny, and reflection. This is the basis for the current fascination with ePortfolios. And at this level, that is for the good.

Originally, I was quite high on ePortfolios as a concept because I thought it was associated with longitudinal assessment, measuring growth and hence measuring learning, looking for increments rather than for snapshots of performance as most testing does. Every eager parent anticipates the weighing and measurement of height of their infants to assure the child is healthy and growing properly. And most parents I know (admittedly a small sample) continue with the pencil marks on the basement wall to track the growth of their kids after those visits to the pediatrician have ended. Measuring child growth in this way is a labor of love. In primary school some of my kids’ teachers kept an archive (paper based) of the kids’ work and used that to show the kids’ progress when we had parent-teacher conferences. Primary school teaching, at least in the school my boys attended, had a labor of love aspect to it for which I’m quite grateful. That changed with middle school as the kids rotate through the subjects with different teachers and more of the work they do is regular homework.

Longitudinal growth becomes harder to measure because the rate at which students grow intellectually slows, because the teachers become less invested in the individual students, and because the bulk of what we do measure is within course learning. At the college level, ePortfolios are not about longitudinal growth. Instead, they are about “measuring competencies” via “genuine assessment.” In other words, ePortfolios are about setting a bar and seeing whether the student has cleared it. I’ve written a prior critique about measurement of this sort, about the interrelationship between where the bar is set and what to do about the imprecision in measurement, particularly as it pertains to the performance of students in the gray zone near the bar. I don’t see those advocating for the evidentiary approach talking about this issue much if at all. They should. Here, I’m going to push on.

There are some disciplines where a portfolio approach has a long tradition, e.g. design and writing studies, and instructors in those fields may be drawn by instinct to ePortfolios and welcome the technology for this purpose. But in many other fields an ePortfolio approach is alien, with traditional testing much more common. Faculty in such fields show little interest in ePortfolios and in the evidentiary approach more broadly. The push for this is coming from elsewhere.

It is coming from the Department of Education, the Spellings Commission Report on Higher Education, the various state and local governments who have become increasingly stingy about funding public higher education (this link requires access to the Chronicle), and from the accrediting agencies, who are feeling the heat from these other sources. As one of my colleagues in the Finance department here put it, we’re driven by a yuppie culture, consumerist to the extreme, where the key to any financial transaction from the buyer view is understanding what you’re paying for. With college tuition so high and still rising at a hyperinflationary rate, the consumerism demands an answer. That is the core driver.

The traditional approach has been a trust model, where at each point in the hierarchy there is some activity that indirectly affects quality assurance of teaching and learning. For example, academic departments put in huge effort on recruiting faculty. Recruitment is a big deal. Promotion and tenure, another big deal, gets reviewed at multiple levels --- department, college, and campus. Those and salary review are the main mechanisms for quality assurance. They are all indirect. As a faculty member, I have an ethical responsibility to teach my courses where the content is correct for the course listing and where the teaching approach is suitable for students at that level. Usually, there is not direct monitoring of that, but only the indirect results from course evaluation, which feeds into the other indirect mechanisms, and the write ups by the instructor to document their teaching at the various junctures where they are evaluated along their career path.

The trust model was working reasonably well when I joined the faculty at Illinois in 1980. I’m not saying it was perfect, but it was reasonably effective. And the confidence in public higher education was high. The confidence is waning, as evidenced by the documentary Declining by Degrees. The question is whether the decline in confidence is purely a matter of perception – the quality is largely the same as it was in 1980 but in an industry clearly subject to Baumol’s Cost Disease, the increased tuition has created this change in perception in and of itself, or if there are cracks forming in the trust model and those cracks are largely the cause of the change in perception. Truthfully, I don’t know the answer to that question and I’m not sure how to produce an answer, though I think it is a good question to ask. But returning to the start of my post, my fear about the evidentiary approach that the Bill Richardson’s editorial triggered is that it will lead to complete breakdown of the trust model and overall worse results, particularly at public higher education institutions such as Illinois, creating a view of Higher Education akin to the view many now have of urban public schools. Let me explain how that might happen.

The best teaching is highly innovative, one experiment after another with the approach under constant modification. The learning of the instructor about how to teach most effectively drives the learning of the students; it serves both as inspiration and model. My friend Barbara Ganley is a great exemplar of this excellence in instruction. Innovation of this type leads to idiosyncrasy in the teaching, an idiosyncrasy we should welcome. This type of innovation can thrive under the trust model. That is not to say that the trust model drives the innovation but rather that creative instructors like Barbara can feel free to experiment.

The evidentiary approach is different. It demands a roll up or aggregation of the evidence. Goals in the course syllabus must somehow align with goals articulated by the department, which in turn must align with goals articulated by the college and so on up the hierarchy. The need for evidence that can be so aggregated might very well act as an inhibitor on instructor innovation, encouraging a more cookbook approach. (Much of Kozol’s criticism of NCLB is indeed that it has engendered a militaristic approach to instruction pushed from above that seemingly hinders the play and creativity of the students.) This would be the unintended but highly pernicious consequence of a total embrace of the evidentiary approach. We’re not there yet, but it seems to be a real possibility worth considering.

Of course it is possible to envision an alternative outcome with the evidentiary approach, one more benign. The goals that get articulated could be sufficiently broad as to not inhibit the good teaching at all (and not inhibit the bad teaching either), indeed that accreditation may have only a minimal impact at a place like Illinois but is nonetheless sufficient to alert the public about disreputable Diploma Mills.

Whether those represent two endpoints of a spectrum I’m not sure. It seems so to me but so many other of my colleagues are advocating for the evidentiary approach that I’m willing to admit I may be missing something. But mostly I fell there is a lot of fuzzy thinking out there about measuring learning and as a consequence not nearly enough attention put on the how the gathering of evidence might affect the teaching and learning practice as well as how it might affect the teachers and learners themselves. Further, there is the related issue that if the measurement is done poorly then should we be doing it at all.

At Illinois we already know what the core learning issues are. These relate to large class size, particularly during the general education part of the curriculum and the concomitant problem that students can become anonymous and consequently disengaged in their own learning. What we need to do is promote conversation, between students and faculty and between students and other students, particularly during the general education phase. We do that with our Living and Learning Communities. It is an excellent approach but it does not scale. We need some other alternative that does for the students who reside elsewhere. But it’s not the families of these other Freshman who are really concerned about the value they are getting. The public that is most upset with us are the families of students who applied to Illinois but were not admitted, especially in the case where the parent did attend yet had a lower ACT score and GPA than the child.

Given that, accreditation is more of a tax on the institution (or college in the case of disciplinary accreditation like AACSB) rather than an opportunity to engage in needed self-reflection. As long as it is perceived as such, like most tax payers we’ll engage in mild tax avoidance and otherwise comply.

But will that approach satisfy the consumerist demand for information? And if not, will that force an outcome like what NCLB has produced? I don’t know, but I think we should talk about it.

Form Factor

Yesterday I finally got around to purchasing Acrobat 8 Pro. Staff on my campus can get this through the Webstore, with purchase through department at a very good price of $40 and purchase by individuals at a price of $137 (this is still much cheaper than the list at the Adobe site). It turns out that the product comes bundled with another Adobe offering, Adobe Lifetime Designer, which is a form builder tool. It makes forms within PDF files. This was immediately intriguing to me. As regular readers will note, I’ve been fascinated with the possibility of instructors collecting data from their students and then repurposing for instructional use. For example, this post on the future of Learning Management Systems has a section specifically on data ingest and data manipulation.

Also, I’ve had some experience building dialogic presentation and self-test quizzes inside an Excel workbook, for example here and a different one here. The students really liked those (they give a good sense of how I think basic economic theory should be taught) so on content fine. But I had always wanted to collect the student responses and have those compiled into a single file. (There is some randomization in each spreadsheet based on the demographic info the students supply, so the answers will differ across students, even if they each got all of them correct.) The last time I taught I tried to do that compilation with Excel. It was a disaster. The linking required was quite unreliable. (And maybe there was fault with the person doing the linking. That would be me.) So I’ve had it in the back of my mind since then to find another way to achieve that data collection end.

I also did some work on the reverse problem. If you have a table of responses from a survey, with each row a response for a particular individual and each column a response to a particular question, can you find a convenient way to display the individual responses back inside the form in which the survey was rendered? It is hard to read the individual data in the table – much easier to read within the form. I built this spreadsheet as an example, not a systematic solution.

Given that background, I was like a kid with a new toy as I played with Acrobat Pro and Lifetime Designer. I started in on the tutorial, but after 5 or 10 minutes of that I lost patience and tried to build my own form from there. It’s always refreshing to consider how one learns to use new software. I had never tried building forms in PDF files before. I was extremely impatient at first because I didn’t know how to get stuff done and where I could make some progress I made tons of mistakes in design – the first looked very ugly. But in the “learning by futzing” manner that I favor to get familiarity with an application (my approach has to be similar if not identical to Ericcson’s effortful study) I go through a variety of creations of my own design and negotiate that with my understanding of the software and the underlying capability of the software (those are not the same but it would be nice if they converge eventually). I can’t learn software without those creations. Further, my personal sense of competence is wrapped up in being able to make something with the software that gets over my personally set bar – not bad, pretty decent, functional if not beautiful.

The first few didn’t make it. This is the fourth one I made, the first that used Tables in the layout. To view it you need version 8 or later of Acrobat or Acrobat Reader, not some other PDF viewer. While some readers of this blog might prefer non-proprietary solutions, my own view is that the functionality I’m after is pretty hard to deliver in a clean way and since Acrobat Reader is freely available, this works fine. Also, I want to make a point of comparing this solution to other possible ways of gathering the data. For that purpose it would help if the reader took a look, so the reader has some understanding for why I’m high on this approach.

Let’s consider the form first from the point of view of someone filling it out. The look is fairly clean (and with some more experience in design it could be even cleaner). The main table is 4 by 3 with a pull down menu in each cell. Those work reasonably well; when I did the analogous thing in Excel sometimes the pull downs would be “sticky.” I probably should have increased the column width since the longest response runs over, but otherwise it looks ok.

I believe these forms have to be one page and no longer. (I need to verify that.) So it is useful to put a lot of response fields in a small amount of space. This also has the benefit of convincing the person completing the form that there isn’t too much work to be done. For this particular survey, the person fills out the table, perhaps adds some comments in the paragraph box, puts in some identity information and that’s it.

The key feature is the Submit by Email button and I’m going to belabor the discussion of it because, as it turns out, using email is not the critical part though it is nice how it can integrate with the user’s email tool, if that is desired. What is critical, however, is that with that button and only Acrobat Reader, not the full Acrobat, the user can save the form with the data the user supplied filled in. This is most obvious if after clicking on the Submit by Email button the user selects the second option (Internet Email) or the third option (Other) and then clicks the button in step 1 that allows the user to save the pdf file. One can then transmit that saved pdf to the collector of the data in any manner conceivable – it doesn’t have to be by email. But for obvious reasons, email is the least common denominator and they do have a nice integration with it.

From the perspective of the survey creator it is really a snap to get those responses into the data file. If they are submitted as email attachments, double clicking on the attachment is all that is needed. But it is also possible to bring in other completed surveys simply by navigating to the folder where those surveys are located and then selecting them. Here is the view of the survey designer who has collected some data. In this case there are three entries. The first is from “Mo Better,” a fictitious character I used when I completed the form myself. The second and the third are identical, actually submitted by my colleague Norma Scagnoli. In this case I deliberately wanted to see if it would accept the ingest from the same file more than once. There is the answer.

With that basic familiarity for how it works, let’s consider the benefits of the approach. The miniscule number of instructors who design their own databases and make Web forms to collect data will find nothing new here. But for the rest, this approach should be empowering. Heretofore these instructors had to rely on programmers to make those Web forms. Now they can dispense with the Web form altogether and build the PDF form themselves.

Learning Management Systems have some ability to collect this sort of data via the survey tool. I could do something analogous in our LMS but then I wouldn’t have the ability to use pull down menus, I’d have to use radio buttons instead, and I couldn’t deliver the survey as a single table. I’d need 12 separate multiple choice questions to gather the data. Apart from the convenience in presentation I mentioned earlier, by delivering it as separate questions the person completing the survey might not see the connections across questions that become transparent in the table view. Further, at least with the WebCT Vista survey tool, which is the one I’m most familiar with, horizontal space is eaten up by the place that records whether questions have been completed and one saves the response on a question by question basis so the form is filled with buttons (if the entire survey is delivered as a single scroll), so the look is clunky not clean.

Further, from the instructors viewpoint looking at the collected data, one can look at the report as a whole and one can view the individual’s response, but that appears as a row of data, it is not visible back in the form that the student completed. The LMS approach is clearly more secure and so I would not use with grade related information. Further, the LMS approach is likely time saving for the instructor in high enrollment courses. But in smaller classes, I’d likely favor using the Adobe tools.

Also, in case it isn’t obvious, by getting away from the LMS as the distribution vehicle, one doesn’t have to restrict the population for a survey to a class roster. It can be quite other things. And, of course, it can be used in many other contexts, among the most obvious being IT training activities where paper surveys are currently administered at the end. Student groups who want to administer surveys might find it a real boon, as might committees, research groups, and perhaps even bloggers who’d like their readers to answer specific questions. (I’m not sure of that last one, but you can dream, can’t you?)

I think this is worth a try and I’d be very interested to hear from anyone who does.