Issues with Podcasting

Today I tried yet another podcast client. This one is a freebie called Doppler. It seems to work reasonably well. I pointed at the feedburner feed for my test blog at http://feeds.feedburner.com/LannysDemo and it brought in all the posts and the files too. So that was nice. But it still is not as good as iPodderX, though it does have a "link to enclosure" in the message. In Doppler it is a generic link, and doesn't say anything about file type or size. But it is manageable. One can use the blog posts as metadata for the enclosure. That I think is important. When I tried jPodder, it only brought in the files, not the messages from the blog.

This leads me to my first issue. Is the Doppler client idiot proof? My answer, no it isn't. It is simple to use, but not idiot proof. No software is and if this sort of thing is used only for instruction and the students use something else for their entertainment content (iTunes would seem to be the number one candidate) then they won't be very familiar with the application and might encounter issues that I can't anticipate here.

So why not use iTunes? That is an application many if not most students already know. My answer to that one is --- if it is only audio content to be distributed, iTunes is fine. But in any technical course, in my opinion, one wants to distribute some visual content - a PowerPoint presentation, a pdf file with images embedded, etc., along with the audio content and so one would like the podcast client to handle any file type as an enclosure. My understanding is that iTunes doesn't do that. Further, and here I'm only guessing, my sense is that this type of higher ed use of Podcasting to have arbitrary formats in in the podcasting may not have a parallel on the reactional side. Apple's business model has to first and foremost address the recreational market. So it seems to me that Apple doesn't have a good business reason at this point to move to supporting general formats for podcasting. I would love it if they did, but I'm guessing it won't happen any time soon.

So if one used iTunes as the podcast feed, then the students would get the other content some other way (most likely as they currently get it from the course management system). That puts us in a world where the students are using two different clients (the second one is a Web browser) to get to the course content. It would be much nicer to have one client for everything. One might argue that for other functions - checking out within course grades, participating in an online discussion, etc. the students are going to have to use the course management system anyway, so what is the big deal. My response to that is many instructors really only want the file distribution function. For that population having the student use two clients is nuts. Further, I'm guessing we're going to see more and more large multimedia files to distribute and for that podcasting is great because it can happen in advance of use. But if that occurs, then one wants the ancillary docs to come in along with the multimedia and that is true even if the instructor used the CMS for other purposes.

Now let me switch to the other side of the coin, the creation side. My campus is creating a tool like Feedburner, primarily intended for department or college Web masters to manage RSS on thier own sites. This tool, almost certainly more powerful than the Feedburner tool, is also more complex to use. I don't expect much uptake of it by instructors. The complexity may be a virtue for the Web masters, but I don't think it is a detriment for instructors. Having made the Feedburner feed, which is a one time thing, managing the podcast is fairly easy. The sole recurrent pain with adding enclosures to my blog is that I have to find and paste in the url for the file. So first I upload the file to a server, then lanch the file in a browser if possible, copy the url from there, and then paste into the blog. If the file doesn't open in a browser, I open the directory in which the file resides, copy the url for that, and then complete the url by typing the file name with the file extension after the dot.

There is nothing particularly hard about doing this, but it is clunky. With our course management system now, you can upload the file to the server and then select the file name and it will be inserted in the appropriate place in the organizer or learning module. Having to bother with urls for the file is dark ages stuff in my view. What I'd really like is to have the file sitting on my desktop, then drag it onto my particular blog post, and from doing that simple dragging, the file should be uploaded to the appropriate server and the url should be placed into the right box in the blog --- automatically. And did I say I'd like to do that for a file in arbitrary format?

We've been a little slower than some campuses going doing down the podcast route. The reward from that, I would hope, is that we do things a little more sensibly when we do role out a such a service. But I'm not yet convinced there is a sensible path to take that will fit the needs I've articulate. Maybe this is a glass half full problem. And maybe I'm just greedy.

My course next semester

Next semester I’m teaching economic principles to 15 campus honors students (these are our best students) and beginning to get set up for the course. The course is aimed at freshmen, but the last time most of them had sophomore standing (and one had junior standing) because they had so much AP credit. I was given some very good advice the last time to conduct the course as a seminar but remember these are undergrads. The last time what I came up with fit that and it was a good course.

So one of the big questions for me is how much I tweak and how much I keep as is. The other big question for me is in my own thinking how much of this is about the style to learn the economics and how much of it is to exploit the technology to improve learning. Some of the bigger innovations I had the last time that I plan to keep are no textbook but rather other readings, an expectation that they will learn about core economic ideas via large projects that focus on particular issues rather than a lot of smaller assignments, and having each project team do in class presentations – book reports on the readings. None of these are fundamentally ideas driven by the technology, but they are all enabled by it.

I’m finding in my own thinking I’m more concerned about being clever on the economics than with the technology, and that is where I’d like to innovate where I can. Part of this is on the assignments and choice of readings. Another part is on numerical simulations. I think I can show students some pretty sophisticated economics in this freshman course, something one wouldn’t do otherwise, by using numerical simulation in Excel, so the students don’t have to “solve the model” which would be over their head but simply see the consequences of their choices in the simulation.

The other part of my thinking is on how the course can make a lasting impression on the students. Two years ago when I taught the course for the first time, I know it had this effect on some of them. But I also know I took advantage of Hawthorne effects big time. Now some of the novelty has been lost and the question is whether the sense of inventiveness can be maintained. One big metric of success, then, is what they read after my course as a consequence of taking the course. Another is whether they think fondly about it in retrospect.

Still another question I’m asking myself - this course will have a lot of in class discussion; I will not lecture much if at all – what online technology works here. Last time the first project had them make content surveys using Respondus. The beauty of that is that those surveys were administered to other students in the class after they were submitted and then based on the responses of the others we’d have discussion about the topic in the survey. I found those interesting as a good way to get some “layering” on the thinking as I would use the surveys and the student responses as a launch point to talk about something a little deeper in the subject.

This worked for me teaching-wise, but I’m not sure the approach will transfer to other instructors. Small class instructors might not otherwise do content surveys at all. They might prefer using a Wiki to get similar interaction (though I don’t understand how they’d achieve the layering effect). Alternatively, large class instructors might find my approach too daunting as administering student created surveys to other students might be unwieldy. So I’m scratching my head on whether I should be going for practices that likely transfer to others or focus on what I think will work best in this setting.

Periodically, I will comment on my teaching in this blog. If you want to take a look see at what I’ve got so far, that is the syllabus and a course blog.

Multimedia editing, writing, and literacy

I should offer up a mea culpa at the outset. While I have done a fair amount of multimedia production, in some definition of the term – voice over in PowerPoint, screencasts, talking head videos, that sort of thing, I’ve never done any substantial editing of multimedia content nor have I ever acquired multimedia media content by doing several takes of the same presentation as one would do in making a movie. Sometimes that limitation in my experience makes me feel under qualified to do my job, because the issue is coming up big time now about developing that sort of skill set in our students as the 21st Century equivalent of teaching students how to write, and truthfully I’m not sure how I feel about it other than that I know it makes me somewhat uncomfortable to voice my opinion on this topic in the absence of experiential knowledge.

Yesterday I had a coffee with my colleague Burks Oakley, he is really into podcasting now, and he told me that using Audacity he does the type of post recording editing that I’ve never done and indeed he does multiple takes on the audio recording in the sense that if he stumbles on something he will do it again to get it right. I was impressed with his diligence. Obviously, Burks feels that the production value will translate into listener satisfaction with the podcasts and he is willing to put in the effort to achieve that end.

Today, at a meeting about our fledgling Learning Commons, the discussion turned to our new course called Writing with Video. Joe Squier, a faculty member in Art and Design and one of the pioneers behind this course, has said we want students to develop a multimedia communication capability and that is what we should aspire to in producing the next generation of our graduates. At one level this sounds right to me and it clearly is an inspirational vision. Many people have gotten quite excited about the Writing with Video project.

But on another level I’m afraid of this. Last night we had parent-teacher conferences at my kids’ middle school, the first one with my older son’s language arts teacher, who is also his home room teacher. We spent a little bit of time talking about the writing of other students in the class. The teacher reported many of them don’t write well at all because their knowledge of things is fuzzy and with fuzzy ideas it is hard to command the language that represents the ideas. On one level, this report is not surprising. My kids go to a public school in Champaign and while that school does well when compared to other schools around the state, all the hullabaloo about our schools not keeping up with the schools in the high performing countries of East Asia and Europe certainly should have prepared me for it. But at a different level, I think of most kids at that age (13 – 14) as bright and enthusiastic and hearing a teacher say that their ideas in the main are fuzzy is quite disturbing.

I have no clue with whether the kids this teacher is referring to, the peers of my son, are the type of kids who will attend the U of I or not. But, of course, not that long ago we were all discussing the Declining by Degrees documentary and if the thesis in that documentary is even approximately right, then it is not too much of a stretch to presume that many of the kids taking eighth grade language arts around the nation have fuzzy ideas and weak knowledge in many areas. One might surmise further that it is this weakness which is the source of the student disengagement in college (and likely disengagement earlier in their schooling).

I have no idea about how to rectify this problem with young adolescents. I can’t help but wonder, however, that if the problem is not addressed early on then we will be unable to teach communication in multimedia when these same students are in college. And if we insist on trying nonetheless, we’ll be seen as masking the lack of literacy rather than perceived as giving students the right education for the 21st century.

To this I want to add another dismal dimension to the issue. No Child Left Behind, in a school such as the one my kid attends, almost certainly is having the effect of dumbing down the curriculum and providing less enrichment than there was previously. (Tight school budgets are also a contributor.) So if the retort is that general literacy has always been an issue and that the kids who start attending the U of I in 5 years, when my kid should be a college freshmen, will be part of an elite that has bypassed the literacy issues, my response would be that if these kids are the products of public school education then in many cases we need to question their literacy, no matter how much innate smarts they have.

Is it fair of me to lay all this at the feet of Writing with Video? Surely not. But to make me feel comfortable about the notion of getting students able to communicate well with multimedia I’d like that to be built on top of a more traditional notion of how to communicate well in writing and speaking rather than as a substitute for it.

Écoutez

It is not uncommon in teaching for the instructor to deliver a lecture for the bulk of the period and then ask at the end, “are there any questions?” Frequently the response is an awkward silence and then after a bit the instructor dismisses the class and laments, “they just don’t talk up.” Ten years ago when I was getting started with learning technology, getting them to talk up was the prime reason for going to asynchronous conferencing, and the mantra for some was “students who are quiet in class will express themselves online.” There may be some truth to that. But I don’t want to overplay the case.

Instead I want to ask whether the instructor, perhaps unwittingly, is really communicating with body language and teaching approach that she doesn’t want to have students ask questions at all. I believe that the method of inquiry the instructor adopts matters in who speaks up. Giving clear and repeated affirmations that an open dialog is desired is necessary to get students to talk in class.

The last time I taught, without much planning at all but armed with the advice to conduct the class as a seminar, I tried to hold the class session much as we do committee meetings within the IT organization. There were only 15 students and me but we were in a room that accommodated 90, with the seats arranged in columns and rows. I made a point of rearranging the seating so we sat in a semi-circle near the front of the room. The idea was first that everyone in the class should be able to see everyone else’s face, no heads behind other heads, and second that if we needed to see the blackboard or the projector screen, that was in front of everyone so nobody had to turn around. Setting this up at the beginning of the semester was a little uncomfortable at first, but soon the kids figured it out and helped with the seating configuration themselves. There is no doubt in my mind that making a thing about the seating is one dimension of conveying to the students that the instructor wants them to speak up.

Another important concern is the tone in which the class is conducted. Truthfully, I’ve spent more time in meetings over the last few years than I have in the classroom and so I led the class in a manner that I was used to from the meetings. I tried to treat the students as colleagues and then give them opportunities to speak up throughout the session. Directed questions during the session were helpful. Showing the instructor has weaknesses (I’m not talking about demonstrating incompetence, I mean the instructor showing a human side) also mattered.

In the case of this class it didn’t really happen the first day, or even the first couple of weeks. They had to find their comfort zone. I had hostile looks from some of them, because it wasn’t a style they were used to. So there has to be persistence in a good natured way and, I think this part is critical, when they do talk up there has to be a way to react that shows the instructor is listening to them. Too often in other class settings the student questions and comments are dismissed too early – they are not framed right or the questions seem trivial to the instructor. That may be true, but the ultimate goal is not to answer the particular question. It is to get the students to feel comfortable and open up. And that requires both listening and reacting in a way that the students want to say more.

In a small class setting this is possible. I believe not enough instructors when given the chance to teach in that setting make the effort to encourage the students to speak up, mostly out of fear of losing control of the agenda, also out of ignorance in knowing how to proceed. That is a shame, especially with experienced instructors. But for junior faculty this is not a surprising outcome at all. Teaching can be frightening because the instructor’s weaknesses are readily exposed. Listening, really listening, requires overcoming that fear. It isn’t easy.

In a large class setting, it may be possible, but I’m less sure about it. Technology, particularly “clickers,” may be helpful in allowing the instructor to be more responsive to the students. Active learning exercises give students the opportunity to speak up with peers and to engage in problem solving activity. But those activities must be meaningful in the context of the class and from my point of view the approach too often encourages “quick hitter” solutions rather than more penetrating investigation because of the time constraints. In settings where the quick hitter is appropriate, I do believe the instructor can demonstrate she wants real dialog in the large class setting. Otherwise, practical necessity will dictate a mostly presentation approach in the classroom with perhaps some dialog online.

I want to take these observations and apply them to another setting. My university is in the midst of a major strategic planning process initiated by our still in his honeymoon phase new president, B. Joseph White. We started with a structure that he provided and some overview documents from that. There are now draft plans from each of the campuses. These are available at the President’s site. Still to come, there will be plans at the college and department levels.

There has been a lot of work within the administration to gather information for the current drafts and put the contributions into a passable form. But because we’re not used to strategic planning this way, the process seems hurried and clunky. The plans at the various levels are supposed to be pictures taken at various elevations with the same fundamental view, the President’s documents at the 100,000 foot level, and the department plans near to the ground. But our process has been to start from outer space and move to the ground, not vice versa. I hope before the process comes to a conclusion there is some of the reverse. But there have been discussions on the sidelines that have said if somehow the campus level plan doesn’t have a place holder for some department’s item, then that item won’t be included when the department plans emerge.

This puts many of the faculty who have yet to participate in the process in the same position as the students who having sat through the bulk of the class period listening to the instructor lecture are then asked for questions at the last moment. Certainly, there has been faculty cynicism about the process. That is not news. The issue is whether well after the fact the process continues to be perceived as too top down and hence too removed from the concerns of the faculty.

In spite of these concerns, I want to applaud President White for taking this approach and for encouraging others to put in the energy to make us think hard about where we are and where we want to be going. It is critical that we do this. And, certainly, there is no perfect process to encourage a serious discussion on these issues. Further, I do believe that President White, because of who he is and that the strategic planning is his first big activity, still has a substantial amount of goodwill that can readily overcome the current issues with the strategic planning process.

But this will require some change. Though I'm sure there is the urge to get these plans out to the Board of Trustees and to potential donors, I hope President White is wise enough to see that a wooden approach to the planning process will bring about the enmity of the entire university community and ultimately will cause the outcome none of us want to see – the University of Illinois as a mediocre institution, one that didn’t adjust to the changes and pressures that all of public higher education is now facing.

The lesson to learn is simple to describe but hard to deliver………. Écoutez.

Maslow

This week’s Magazine in the Sunday New York Times has some interesting pieces. The featured article, A Doctor for the Future, is excellent as a read, though I couldn’t make a connection from it to learning technology. It is the story, primarily, of genetic diseases among children of the Amish (and Mennonites) of Pennsylvania who are more prone to these problems because of inbreeding, how modern medicine – post the mapping of the human genome – can and cannot help them, and about a particular doctor, Holmes Morton, who has been tending to these people and in this practice building a case for tying diagnosis of genetic deficiency to the treatment of patients, though struggling to disseminate these ideas because treating patients competes for Morton’s time with publishing journal articles about the approach. It is truly a moving story and fascinating because of the various juxtapositions as well as the power and determination of Morton to provide decent medical care to these deserving but uneasy to serve people.

There is another piece in that same Magazine, this one called The Literary Darwinists, that I am going to try to tie to learning technology, even if that is a stretch. The article is about an emerging school of literary criticism, school may be too strong a term but I’m not sure what else to call it, that as a tonic to the more standard deconstructionist approach (that is standard from within English departments, not for the rest of us who read the occasional fiction for fun and personal expansion), which looks at literature from the point of view of a species preserving activity, more or less in the same way that school socials might be considered species promoting, in that they encourage dating and hence mating.

While most of us who are not English professors (myself included) may shy away from current literary criticism because we don’t have enough background to participate in the discussion, we nonetheless do have some feeling for Darwin’s precepts, especially as they are practiced through the lens of evolutionary psychology. Within the scope of their work the literary Darwinists get to opine about what species preserving function reading has, especially reading of what would be considered great literature. Clearly, the direct effect is to take the reader away from more obviously productive and consequently species preserving activities – providing food or shelter, making the environment otherwise safer, or engaging in or encouraging procreation. But such reading, when it is effective, tends to reach us deeply touching our inner being. This may provide hope, inspiration, and a spirit of accomplishment that may help us in deeds that more obviously fit into the Darwinian mold.

Having recently completed Tom Friedman’s The World is Flat, I’m struck by how his entire vision can readily be cast in Darwinian terms and his concluding chapter focused on the need for imagination, not the imagination of Osama Bin Laden, but rather the imagination of those who brought down the Berlin Wall. If this sort of creativity is critical for species survival and literature and the reading of such acts as a spur for such creativity, perhaps the evolutionary interpretation is not such a stretch.

Then, to bring this back down to learning technology, the question/concern is whether there is something unique about language in its written form as a means of communication to spur the imagination or if other forms, those that are more graphical and otherwise engage the visual and auditory senses, can also promote the imagination and thereby supplant this role that literature has played for the last several hundred years.

Let me get back to that. Although the discussion was about evolutionary psychology, I could not help but think of Abraham Maslow and his notion of self-actualization. Maslow may have been too descriptive in his methods for studying self-actualization and his theory may itself be self contradictory as a prediction of behavior to get him mentioned in the Times article (the theory asserts that if more basic needs are not satisfied the individual can’t go on to satisfy higher needs, although perhaps Maslow’s most impressive self-actualizer, Lincoln, endured all sorts of hardships while simultaneously self-actualizing) yet Maslow’s view of self actualization is uplifting and should be considered by educators in its own right.

With a very simple Google search I found this delightful Web site on Maslow that gives a brief biography and some of basics of his theory. The bulleted list of needs that the self-actualizer feels seems to me a good set of aspirations that we should want for our students when they graduate. It seems to me a fair question to ask whether our curricula help the students become self-actualizers and if so to be explicit on how that is accomplished. According to the article, Maslow himself thought only a limited elite comprising no more than 2% of the population engaged in self-actualization on a regular basis. The piece does not say whether in Maslow’s view this represents inherited differences among the population, or if it was because most of the others are somehow engaged in more basic life struggle. If the latter, one might imagine that as real income rises that fraction of the population that self-actualizes would also rise. And one might think that education, particularly higher education, would be uniquely suited to encourage this behavior.

I want to bring in one more observation that is relevant and then wrap this up. Over the weekend I saw an interview with Ron Howard the director (and former star of Happy Days and Andy of Mayberry). He was talking about lessons he learned from John Ford, the great director as, in particular, applied to The Missing, where the characters lived a very hard life and had to go through some excruciating circumstances. Ford’s advice was to show only part of what the characters were going through on film and to let the audience’s imagination fill in the other parts for themselves. This seems to me to be correct and explains why Hitchcock films, while dated in their sets and other environs, nonetheless still come off as interesting stories to watch, and why more modern “special effects movies” lack the ability to offer other than mindless entertainment.

So I for one don’t think it is the medium, written text on the one hand versus highly visual content on the other, that is fundamental to inspiring creativity in others. It is that the artistry with which these works are created that matters and that artistry can happen across media. But with writing or painting or photography, there is the self-actualization of the artist at play and that is primary in determining the work which is produced. With motion pictures and video games, there clearly is team production and managing that would seem to me possibly in conflict with artistic creation – certainly the management of the team is itself a complex activity and therefore could be a distraction.

It does seem to me that emerging technologies have this potential to inspire potentially self-actualizing students if coupled with the creative use of the instructor and that early adopter faculty, in particular, have an intuitive sense of that possibility. Perhaps we need to shift the framework of the conversation from good instruction to inspirational teaching. I for one would like to see Maslow discussed more frequently in learning technology circles.

On killing off apps to be supported and doing something new

Sorry about not posting for a week. I started to write a longer essay last weekend and it didn't turn out to my liking, so I got a case of writer's block. I might get back to that piece in a while, but for now will stick with somewhat more modest themes.

*******

I want to do a little stroll down memory lane with the hope of providing a lesson for here and now.

I started to run the organization called SCALE (Sloan Center for Asynchronous Learning Environments) in September 1996. Within the next 6 months or so, not recalling this all too well, we killed off support of an application called PacerForum. This was a very popular Mac-based conferencing system. But it relied on AppleTalk protocol rather than TCP-IP and it was unable to make the transition. The company that made PacerForum went belly up. We continued to support it for a while after they went bankrupt (a bad idea) but that was untenable so we dropped it. Our users knew nothing of this. They just knew they liked the software and we stopped supporting it. So we were the bad guys.

That same year we started to support a Web based conferencing program called WebNotes from the company Sypglass (which got the Mosaic license from the University). At the time our mainstay conferencing system was FirstClass, but that required a dedicated client at the time. FirstClass was pretty popular, but some people wanted a Web system, even in 1996 when I believe the browser was Netscape 2.0. There was some uptake of WebNotes (I used FirstClass in my own teaching) but the product was a dog and after a year we dropped it for an alternative, WebBoard.

We continue to support WebBoard to this day. And we continue to support Mallard. Those were the mainstay apps I used in teaching my intermediate microeconomics course. I switched from FirstClass to WebBoard perhaps in 1998, primarily because there were some support issues with the students installing the FirstClass client. Many other of our users did likewise.

FirstClass did come out with a Web client but it was not sufficiently differentiated to justify supporting it. So we did end the FirstClass service in December 1999. There were only a few dedicated users at the end, but I think we lost those people permanently to other environments. I don't believe they came over to WebBoard or the CMS products we had started to support.

What is the lesson? It is that if we start to support something we really can't stop supporting it unless the product is a dog or unless the provider no longer supports the product. There is an implied commitment to ongoing service. One might think that we can go to something else that is "better" but beauty is in the eye of the beholder and loyal constituencies form for any environment that has been useful to the instructors.

We are now in a situation where new apps are appearing in abundance and there are many alternatives from which to choose. Moreover, it is apparent that things are changing at a rapid pace. Is this the time to "lock in" to a particular path, say to hosting Drupal or Movable Type?

Now let me do a tiny bit more history. In fall of 2003, Educause Review published a review/critique of a piece that appeared earlier that year in Harvard Business Review called "IT Doesn't Matter," which is reprinted in that same volume of Educause Review. That article depressed me quite a bit, because I bought much of the argument. The essence is that if the IT organization is providing utility services that are really "commoditized" then IT doesn't matter. So a second question is whether the new collaboration technologies are commodities, which the market should provide, or value added applications, which campus IT should provide. Personally, I lean toward the commodity side in answering that question.

When talking with college level administrators who support IT, they might very well conclude likewise. When talking with early adopter faculty, these lessons are extremely frustrating. They are immersed in the new collaboration technologies and want to see the campus right along with them. My final question, one I don't know the answer to, is whether that is the nature of the beast or if there is some other way to frame the issue so the early adopter folks feel more comfortable with what the campus is doing.

Smart Use

Economic theory shares some common methodology with engineering in that both formalize design problems as optimization subject to constraints. The framework helps in considering practical problems like the following. An instructor who has used a different course management system in the past is now using the system her campus currently supports. How should she best configure her course site?

In a world where the campus trains the instructor on the software but leaves it to the instructor to make the decisions about course site configuration, the likely solution is to as much as possible recreate the prior course site from the old CMS and where that is not possible kludge the rest. The implicit problem being solved by this approach is to minimize the instructor’s time taken to configure the site while producing something that is tolerably usable by the students.

As with the purchase of new hardware or software that carries a big ticket price, we are often taken with that “sticker” price rather than with the total cost of ownership, in part because we can readily see the former and at best only infer the latter. But clearly, a kludged in site now will create use issues for the students later and usually that means more work for the instructor down the road. Therefore, the initial design of the site really should take into account that “total cost of ownership” approach, to the extent that is possible.

However, the typical instructor, even one who teaches engineering or economics, may be ill equipped to solve that problem, in part because she is unaware of the capabilities and limitations of the technology and in part because she hasn’t solved many like problems and thus lacks familiarity with the requisite type of thinking. We who support these instructors, in contrast, have the experience to do the needed problem solving. The issue then is whether our normal ways of communicating with instructors enable us to communicate the best practice in situ so either the instructor can implement herself or we implement it on her behalf.

Since on my campus we have many large classes that use our course management system and since most of them make heavy reliance on the grade book, let’s focus on that particular area within the CMS. Many of these instructors had previously used either Mallard, our home grown quiz system with some very nice features for doing automated student assessment, or Campus Gradebook, another home grown system, this one a derivative of Plato. Both of these are flat file systems under the hood and for grade book functionality, I believe that can convey an advantage over the database CMS that represent current state of the art, such as the WebCT Vista system that we are deploying. I will elaborate on why below.

Beyond the database versus flat file issue, there is the issue of data representation. Most of the commercial CMS envision a single worksheet view. In the Vista product one can run queries on column characteristics to get a more refined view but that view only endures through the current Vista session. In contrast, one can hide or unhide columns and that action endures till the next time the instructor makes further changes hiding or unhiding columns. Our Campus Gradebook, which we have now retired, allowed a more hierarchical approach. Column data was put into categories – homework, exams, projects, etc. – and row data was put into sections. One could view individual section or all sections pooled, but on the categories one could either view individual categories or aggregates of each category, but one could not view all categories pooled. In other words, there was an asymmetric treatment between columns and rows. Visually, one can scroll up and down easily enough but scrolling across is a pain and hence aggregation is better than keeping track of lots of individual columns.

In an ideal world we’d like a hierarchical grade book as described above, especially in the database driven approach, because each cell in the grade book is its own independent object that must load and hence having a grading with tens of thousands of cells is something to be avoided if at all possible. This is not a concern in small classes. A grade book with 30 students and 30 columns of data is peanuts. But it is big concern in large classes. Furthermore, it is the large classes that are likely to do a lot of quizzing, because it is an efficient mechanism for students assessment in that setting. But then it is possible to get, for example, a grade book with 600 students and 100 columns. Unfortunately, grade books of that scale become almost impossible to manipulate.

We’ve had many large class instructors belly ache about the CMS grade book, but to my knowledge we done little about giving them coping strategies to render the data. At this juncture let me note that how to cope depends critically on whether the instructor is comfortable with Excel or not. If comfortable, the best and most sensible coping strategy is to envision that the grade book is kept by the instructor as a spreadsheet on the instructor’s desktop. Raw data (from online quizzes, for example) can be downloaded into that spreadsheet. All manipulations of the data are done in Excel, and then the instructor uploads into the CMS what the instructor expects the students to view. This requires a sensible approach to backing up the downloaded raw data and the updated spreadsheets, but this is not hard – simply save with the date appended to the file name and then make sure there are duplicates on some other computer.

But for those who don’t feel comfortable keeping their grade book in Excel and therefore want to keep the data and the calculations in the CMS, there is that constraint to deal with that too many cells in the grade book make it perform poorly. What should be done in this case?

If there are TAs and say they have 3 sections under their control with perhaps 90 students in total, then by having one class site per TA perhaps the grade book can have up to 40 columns without becoming unusable. How does one keep from going over that 40 column limit? If there are lots of quizzes, instructors should consider combining them. There is no doubt some pedagogic benefit from giving more shorter assessments that more readily imply the student will get some rapid feedback on his work. But that benefit can be offset by a grade book that becomes unwieldy given too many columns. Thus having fewer, longer quizzes might be a sensible accommodation. A partial alternative is for the instructor to become disciplined and learn to view only a handful of columns at any one time. Most of the instructors I have talked with who have huge grade books insist on viewing all the data in one screen. Other than for self-flagellation, I don’t understand why. Still another alternative, though this requires at least some manipulation with Excel, is to report data in columns like:

In class quiz #8, Sum of in-class quizzes #1 - #7.

That is, give out the most recent info but report prior info only as aggregates. This means the instructor will on a regular basis take down columns and replace with different ones that are more current. And if those quizzes were done online rather than in class it means that after a period the instructor makes those quizzes unavailable to students and then hides those columns in the grade book.

What we who support instruction should not do is allow the instructor with the 600 students and 100 columns of grade book data to wail about how poorly the grade book performs. Lets note that within CMS grades are communicated through the tool – quiz, assignment, etc. – as well as through the grade book. Give them an alternative with which they can manage the constraint and then let’s get on with it. Of course the world and the software we use could be better. But let’s show we can manage in the world we actually live in.

U Phoenix, Online Programs at the U of I, and Extension

Yesterday afternoon and this morning I'm at an online learning retreat for our campus. At dinner last night, I sat next to some colleagues from the Computer Science department. It is one of the top rated programs in its field nationally. They have an online Masters program with about 100 students per year. A question is whether they want to/are able to grow. I was told that a big deal to them is quality of the applicants. They must have good and relevant work experience and have good GPAs as undergrads. This limits their pool of applicants. But it does allow CS to treat its online program as equivalent to the on ground program and to preserve the integrity of the degree.

This is an aggregator Web site of Computer science programs available online. The program at Illinois is not listed, but University of Phoenix is. Indeed, the suggestion that only qualified applicants need apply is nowhere to be found on this site. It does say that programs are either accredited or state licensed. That is the quality assuring mechanism.

After dinner last night President White made some remarks about online education and he asked why Land Grant Universities haven't done more in this area, since it would seem to be a natural continuation of the mission. One simple observation is that the Land Grant mission is manifest mostly via extension programs, which are available to all citizens of the state but are non-degree, while the market that U Phoenix operates in is in the main a degree program market.

But one might make a different distinction, one that President White might harp on. Extension is still fundamentally an agricultural program and run through that college. The Continuing Education possibilities that online learning offers apply mostly to the other 95% of the citizenry, who may very well go through job displacement as a consequence of global flattening. So, I would argue, we should consider whether our historic structures actually block our ability to do current and new ventures in online learning that would open opportunities to people we haven't reached.

This could be our own alumni, or presumably the alumni of peer institutions. Some of them, one might imagine, are facing job pressure from global flattening and one would like to think that we'd think our own alumni would pass the litmus test regarding student quality.

But I don't see us beating down the doors to start a venture of this sort. Is it, perhaps, that we have less to offer people who've already gotten a bachelors degree than one might imagine?

Digitial Repositories versus Virtual "Junk in the Basement"

I am a faculty member and my department, Economics, used to be in the then College of Business Administration. (Now Econ is in Liberal Arts and Sciences and the the former home is called the College of Business.) As a matter of course, CBA gave out Web space to college faculty members and adopted a no quota policy. You can see some of my old stuff still there: the class homepage for my Intermediate microeconomics class from 5 years ago and my personal home page that is from even further back. Most of this was published with FrontPage, but I believe I also used WS_FTP to get content onto the site. I want to make no claims for this stuff in terms of layout or good use of the technology other than that after a period of time, it's still there (though not all the links work).

However, and I realize that those who read this post are looking for ed tech ideas not economics, I'd like to consider my paper "Wage Policies" which is available in pdf format on my personal home page. At the time of the authoring, I thought it was an interesting paper, with a clever idea behind it. I tried submitting it to some of the better Econ Journals, but no luck. Since at that point in my life I was getting involved with ed tech administration, eventually I stopped shopping it around. I note now that neither Google Scholar nor the regular Google search engine can find that paper. So if it is to be archived, the burden is on me (or the College of Business).

Last Thursday night at Educasue, I went to dinner with some of my Frye Institute classmates. One of them is Susan Gibbons from the University of Rochester. We talked about Institutional Repositories. Susan reported that theirs was now going quite well. They had tough going at first and hired an anthropologist to study the research and deposit habits of faculty. One of their "findings" was to make the repository all about "me" from the point of view of the faculty member. In other words, Rochester is using the D-Space application in a way not unlike my homepage on the College of Business site, without the jokes and with more references to scholarly output and with file formats to accomodate the discipline (e.g., audio formats for those in linguistics) . And Susan reported that their "download counter" which gave faculty info about how often their research had been accessed has been very popular.

Susan also reported that many faculty at Rochester had inadequate home pages provided by their department or college and so there was a latent demand for the IR to satisfy this need. Those faculty that did have good support would continue to use that rather than to rely on the IR. My sense is that there is no vetting process for the IR, but perhaps some workflow so that items that are contributed may appear only with a lag. There are still a lot of details about the process that I would like to understand, particularly on what faculty must do with respect to metadata.

If the Rochester experience is replicated elsewhere for the time being we'll end up with a bifurcated approach where some faculty use their department as the host and put up "virtual junk" along with their research on those pages, while other faculty will use the IR. This sounds like a social scientist's dreamworld, to have such a division that invites comparison. Just what is it that we want to know about these spaces? For starters, where do all those paper downloads come from? Are they getting the info from Google or the IR search engine?

DRM

I don't know why but we in educational technology regularly seem to over constrain ourselves in terms of desiderata for services, when something that functions simply and seems to fill a need would seem to nicely. Consider audio reserves. In the old days students students went to the Music Library to sit in a cublicle and listentto recordings that were on some fixed medium. Then some time in the late 90s, someone got a brainstorm and said, wouldn't it be much more convenient for students to stream this content behind some form of authentication to handle both the copyright and the licensing issues, on the one hand, while enhancing student access on the other.

And voila, we have audio eReserves. Now we're seeing students with portable devices that can store audio that is downloaded, and hence further mobility can be given to student, but straight download doesn't pass muster in terms of copyright - the students can redistributed the content very easily. DRM (Digital Rights Management) software applied to this content "can" effectively limite the redistribution. Does this work in practice as well as in theory? I don't know. I'm especially interested in the related support issues if a campus goes this route. If anyone is trying the approach for audio eReserves, please let me know.

Where is Tom Lehrer when you need him?

I spent a good part of the afternoon today talking with folks from various WebCT campuses and all seem quite antsy about the proposed merger. Me, I’ve got other headaches. I get back to my hotel room after have Dinner and a few drinks with buddies from Frye, and there is a message waiting for me from my wife, all stressed out, because she can’t do my kid’s math homework. He is in 6th grade and they are doing something new.

The question at hand: under what conditions can you take four lengths and make them into a quadrilateral? Hmmm. I had an idea and tried to explain it to my wife on the phone. That was a total disaster. So I wrote up the following

https://netfiles.uiuc.edu/l-arvan/Capture1.jpg

But I’m scratching my head asking, can 6th graders articulate this? I couldn’t explain it to my wife, how could my kid explain it to his teacher? I’m all for giving conceptual problems to kids at an early age but if they are really to come up with a general solution to this in a pre-algebra class --- that is asking a lot.

I do think using a Tablet for writing up stuff like this is really cool. If they were cheaper I’d buy one for my kids for Christmas.

More on Deferred Gratification

I’m now in Orlando and finished with my first pre-conference meeting at Educause. It was an interesting session but all these meeting rooms at the convention center are without windows and though spacious they make it hard to concentrate for the entire (3+) hours. How can I complain about that after they gave us an excellent lunch and we did have interesting discussion? Let me get back to that one in a bit.

I didn’t sleep well the night before so last night after the ballgame ended (Congratulations White Sox! Ozzie Guillen has my vote as manager of the decade.) I went right to sleep and slept in till almost eight. Having noted there was a Starbucks across the street from my hotel, I put on some shorts and my tennis shoes and headed over there for a vente coffee. I couldn’t imagine starting the day without it. Last night I had some decaf from the room during the game --- it didn’t quite hit the spot.

I know I’m not alone in this craving for Starbucks (back in Champaign I go to Espresso Royale, which has a near monopoly on upscale coffee places in town). Yesterday evening I drove up from Boca to Orlando and made two stop on the Florida Turnpike. As it turns out Starbucks has one of the concessions at their rest top areas. At each place, the line was enormous, and this was after 6 PM. I think we as a culture have the Starbucks habit. What was once an indulgence has now become a necessity.

By the standards of my parents (I’ve written before about them. They both lied through the Great Depression and their views about work and spending were formed by that.) I am spoiled, no doubt about it. The singular periods where I totally ignored that aspect of my persona was when I went to visit them before my Dad passed away six years ago. On those trips I suspended my needs entirely to accommodate their wants. That way we got along and I understood that. Sure I believe in giving people their own “space” in my interactions with them. But I almost never will deny my own needs in the process. Indeed, I’ll spend some time identifying some of mine so they feel comfortable doing likewise.

What’s the point of this discussion about personal self-indulgence? Here is the question. Suppose for a second we buy the argument that the biggest thing we can teach our kids, whether as instructors in a class or as parents, is to have some patience, do things for the long term reward rather than the immediate joy and more broadly push ourselves in ways that we’d expect their to be benefit down the road, but not immediately. This includes things like studying hard, taking up pastimes that are nourishment for the mind, and being more serious and ambitious about the future. Then we should ask: are we in a position to teach those things to our kids? Have we shown the sacrifice that we want them to develop? Can we teach this if we ourselves are so pampered? And if we don’t pay the price for something, how can we breed anything but the nihilism that seems present in so many of our students?

Those questions seem straightforward enough but perhaps they are too simple. In fact I do put in a lot of hours at work and I think about work fairly regularly in my “down time,” for example during that drive up from Boca. I typically get into the office by 9 AM and am often there till 7 PM or later. So the way I’d refine the above question(s) is as follow: If I’m a confirmed workaholic in some dimensions but I’m self-indulgent and spoiled in other dimensions, can I teach the kids to defer gratification so as to invest in themselves (and society) but tone down the latter? On the parenting side this is very tricky, because the TV, the video games, and the computer games, are all immediate gratification baby sitters that we give to the kids so we can do our own work (or have a few minutes of peace and quiet.) We might have to work less at our jobs so we can work more with our children to teach them this lesson.

But let me take this up in the setting of college instruction. What about in that setting? In days of yore (up to and including spring 2002) I used to teach a large section of intermediate microeconomics and I used a lot of technology in that. I had lectures online (PowerPoint with voice over), quiz questions done in Mallard for students to learn the basics, and problem sets done in a team framework and submitted through WebBoard, with TAs online during the evening to help students with the work. There were a lot of College of Business students in this class and they were quite critical of the course – too much work, more than for courses in their major, stuff was dull or too dry, they had to learn it on their own, I didn’t teach it to them. If anyone thinks that teaching students nowadays to be more serious about their own futures and to work harder to get there will be something easy to achieve, they have another thing coming.

And if individual instructors try this on their own while their colleagues continue with business as usual, well, the chance for this to backfire is enormous and it might take quite a while to establish a reputation for the instructional equivalent of tough love, before any of it will work. So I believe that if this is to work for individual instructors, they have to defer some gratification themselves in their own teaching and possibly how they are perceived as teachers via the course evaluation questionnaires.

This doesn’t bode well. While there might be an equilibrium where all of us instructors are very demanding on our students, the students in turn are serious and put in a lot of effort to do the work, and all are relatively content with the outcome, how do we get there from here? My sense is that large public universities will be the followers in resolving this question (I wish it weren’t true but the economics of instruction suggests that will be the case). Who will be the leaders? Small private universities have the aura of coddling their students. That doesn’t quite mesh with promoting a deferred gratification view.

This is a topic that more of us in learning technology should be talking about. The Net Generation, if anything, is more immediate gratification oriented than its predecessors. (Recall that I don’t usually like to make generational learning type arguments. But there is no doubt that in terms of hours per week exposed to various media that can provide instant gratification, those numbers are higher for my kids than they were for me growing up, and I believe that is typical). So a reasonable argument can be made, I believe, that says to irritate them by working them to the bones rather than accommodating them in their every expressed need. That debate is not happening now. It should be front and center.

On writing, blogging, and podcasting

I’m in Boca Raton now on a stop to visit my mom for a day before heading up to Orlando for Educause. I rented a car in Orlando and drove down here yesterday afternoon. I’ll drive back tonight. On the flight to Orlando I had planned to nap because I was going to do more driving that afternoon but instead I read from my paperback. Normally I read turn pager fiction on flights like that, squished into my seat, uncomfortable and in need of distraction. But I found I had an entire row to myself so I could stretch out and get comfortable. And I had brought the complete short stories of Herman Melville, because I had already started it over the summer and because I had been a little disappointed with the last few page turners. I read a story called Cock-A-Doodle-Doo, one I had never heard of, but one that even in my drowsy state clearly was a masterly piece with glorious sentence construction, a compelling story, a parable with power and emotion, and a social criticism all bound up into one short work.

I’ve wanted to be a writer for some time now, but if Melville is the standard, there is no way; so I’ll content myself with being a blogger. The writer is a craftsman, an artist in the case of Melville. Everywhere in the writing there is evidence of the mastery - the word choice, the imagery that is conveyed, the feeling of uplift that is created in the reader. Bloggers don’t show such mastery. They are in the main reporters. They record what they see, they tie strands of other Web content together via hyperlinks, and they voice their opinion as commentary. To do those functions well, there need not be as much craft in the writing itself. Timeliness of the post matters a lot. Some novelty in perspective is also helpful.

When blogging is truly news reporting it amounts to taking something witnessed over here, perhaps capturing that with photos, audio recording, or video recording and then transferring it back over there to the blog in a way that others can digest. In my blog there is much less news and much more analysis and commentary. For me, generating the prose is not so much a matter of transfer as it is of refining the argument and getting the pieces to fit. It is a good thing for me that sentences have periods and come to an end. The conclusion of a sentence is a chance to do a little check. Am I on track? Can I keep going or do I have to go back and make changes? Further, oftentimes the ideas I have in advance, while certainly necessary to get started are far from complete. The writing forces me to see the argument through to completion. I learn something as a result. Occasionally I’m surprised with the conclusion. I believe the common term is “discovery” – writing to learn means writing is a process that aids the thinking and conversely thinking aids the writing.

Coming into writing this post I had thought the distinction might be that writing is formal while blogging is informal. But that is not it. Melville wrote his short stories for magazines to be read by subscribers during their leisure time. He did not write them to be amassed in some collection, to be studies by scholars a hundred years laters. Melville is definitely a writer but his stories are also informal, disarmingly so. Yet I very much doubt that he dashed one of his stories out in a couple of hours of passionate writing. I suspect he had to layer in his ideas and the writing process he followed somehow had to capture that layering to create an effect that is similar to oil paint on canvas, which can look thick with the layers of different colors and different strokes of the brush. For me, blogging is more of a blurting, a distribution of a first draft that doesn’t intend to get refined. It’s one coat of paint and that’s it.

Now let me skip to something else and then I’ll return to the blogging in a bit. Last night I watched a little bit of the Angels/White Sox game. The third man in the both was Lou Pinella, until recently the manager of the Tampa Bay Devil Rays. When he gave his introductory remarks it was obvious that he was reading from a teleprompter. And it came across poorly because it was so stilted and unnatural. He was a little better during the game and made a few interesting points. But I think he was uncomfortable in that role.

On the same subject but from a different perspective, the last several years we’d have Norm Combs, emeritus professor of History from the Rochester Institute of Technology,
who speaks on the accessibility of online content and who himself is blind. Norm has told me he listens to a lot of books on tape. In his presentations he shows the audience how he uses Jaws, the screen reading program, to hear documents and to navigate around the screen. He has the Jaws program set at a pace that most of us would find incomprehensible. His point is that we read faster than we speak and he has Jaws set to read for him not to speak to him. In his last presentation for us, I was curious to hear Norm say that if people are doing voice recording for the computer they should just talk. They shouldn’t read. I suppose too many of us are like Lou Pinella. We’re not trained to read aloud in the mellifluous voice of the TV commentator (Joe Buck, the play by play guy, never sounds like he is reading from a teleprompter, even if he is.) The listener wants to hear something that sounds natural.

I’ve done a fair amount of voice recording sitting at my own office computer and narrating into the microphone. Mostly I’ve done lectures in microeconomics this way. I’ve found that quite difficult to do. It’s hard to be conscious of the speaking and be reflective at the same time, so the talking is mostly from memory. Unlike with writing there may be an interval of two or three minutes of talking – that is many sentences, perhaps a paragraph or two, where there is no time check whether the argument is on track or not. In the old days when I did this narration to accompany PowerPoint slides I had made, there would be one .wav file for each slide. One could catch a breath and recollect between slides. But during a slide this was almost pure blurting.

So it would seem there are two possibilities with podcasting of lectures and discussions. Either this represents a reading of what was initially a document as text. Or it represents an off the top of the head discussion without much reflection in it. Those who do podcasting as interviews with a dialog and Q&A may be able to get the best of both. One can have a sense of learning when in a dialog that one does not have when delivering a monolog. And yet the speech can sound natural and unlike reading aloud. So podcasting dialog seems like an interesting thing to me.

But I think we’ll be seeing mostly podcasting monolog – either recording of live lecture or recording in the office in front of the computer screen. What do we hope to achieve by that?

I understand the argument for mobility and that giving students access to content on their iPods can be empowering for them. In today’s local paper there is an article about the renewal of inflation and it disproportionately affecting the younger workers. Students on commuting campuses, in particular, may be switching from driving to taking public transportation, as a way to economize on fuel costs. What do those students do while commuting? Giving these students academic content for their iPods would seem to be a real boon. But note that either they have to listent to that content over the din of the bus or training rumbling, or they wear sound cancellation headphones that may be ideal in such an environment but make the students too unaware of their surroundings when riding a bike or walking across a busy street. And let’s not lose sight of the thinking and craft in construction that one will find in such content. Finally, let’s remember that good teaching is mostly not about content delivery.

Drop Date

Today is the last day in our fall semester where students can drop a course without getting permission (from a dean in the college in which the student is enrolled). As one instructor pointed out to me recently, this means the week or so before becomes a period of reckoning for students who perceive themselves on the margin. (And this doesn’t have to be on the pass-fail margin. Student’s who carry a high GPA may want to protect that and drop a course that will have give them a C.) In large classes, in particular, this might create certain communication between instructor and student that could have both a beneficial and a pernicious aspect. On the plus side both likely become more aware of the students standing and progress to date. One the minus side this may result in some haggling about grades that most instructors find unwelcome.

I believe that most people don’t consider campus regulations of this sort and treat them as part of the institutional firmament. But I’m an economist, prone to be anti regulation by nature (really from four years of graduate school nurture and twenty plus years in the profession), and want to consider this along with other campus regulations as to their rationale and their consequence. Also, let me not here that by rationale I don’t mean that campus has various reporting requirements to the state and that some of the campus regulations emerge to fulfill these requirements. I mean, instead, what’s in it for the students and the faculty? With the drop date, for example, one might consider two extremes as alternatives. First, students can never drop a class without getting permission from a dean. Second, students can drop a class even after final exams have been given and the student learns his score on that.

I know that some private colleges do the latter. I don’t think that makes sense here because for students who are not doing well, there is too much of an opportunity for them to spin their wheels before they get some assistance. But I am intrigued by the former. In itself it would substantially reduce the churn that we see at the beginning of the semester. If the “cost” of dropping courses rose students wouldn’t so freely add them earlier. The churn, which is huge and clearly has been facilitated by online registration systems, has no obvious social benefit to me. And from the instructors point of view it is quite pernicious. When does the semester really start, on day one or after day ten?

I know for sure that we in IT, on the both the administrative and academic side of the house, pay a big cost for enabling that churn. In supporting the course management system we have to continually update rosters and, indeed, because there is so much churn the administrative student system has to have enough bandwidth and crunching power to accommodate it.

Does the institution understand the extent of these costs? Who out there now really argues for all the adds and drops. During that early part of the semester they are almost certainly not driven by student performance in the course. What if we took did in fact move the drop date to day one and then took the savings that would produce and put it into advising? Might not that produce a better outcome?

Now let me turn to a related regulation, the grading scheme used by the university. When I start at the U of I in 1980 we had letter grade only. Somewhere in the mid 90s they moved to plus/minus grades, but in an either/or way. Some classes had them, others didn’t. This is insane, especially at a big institution. It creates big cost to sustain. Where is the offsetting benefit? I have a mild preference for letter grade only schemes (less haggling about grades with students) but I’d certainly rather see one scheme applied uniformly than to have instructor choice.

We are in an era where efficiency is paramount and yet I don’t think anyone is looking at all these regulations and asking whether they are serving or hurting the institution. Why not?

More on Agency Issues in Teaching and Learning

Before I get started with my topic let me toot my own horn for a second and note that while everyone is abuzz about the Blackboard and WebCT “merger” I wrote what now seems to be a prescient piece on the CMS industry structure back in May. Maybe that means the rest of my posts aren’t all blather.

* * * * * *

I want to return to the agency framework from yesterday because once one starts to think that way it opens up insights into a bunch of related issues and gives a different way to consider them. Also, I want to make it clear that I’m not picking on the students. There are agency issues with the faculty and agency issues with campus administration, as well.

Let’s begin with course evaluation. It’s part of the institutional culture here and I suspect at many other campuses, but really, it makes no sense whatsoever as it currently is practiced. It relies on students giving their opinions at or near the end of the semester in a manner from which they derive no obvious benefit. In the paper administration their is an element of coercion (the students can’t leave the room until they submit the form to the collector) but in online administration the coercion is absent. The low response rates that almost everyone reports for online administration is an acknowledgement that students don’t want to complete course evaluations. So, first and foremost, the current approach suffers from the problem that student buy in to the process has not been elicited. Yet their opinion is valued nonetheless.

At my campus there are two questions on the evaluation that count in the faculty member promotion and tenure decision (and possibly in their salary decision): (1) Rate the instructor and (2) Rate the course. Outliers on either extreme are singled out, low performers either get reassigned to other courses or if they continue to perform poorly then they are denied tenure or promotion, while outliers on the other extreme become candidates for teaching awards. And perhaps there is some information content in these outlier evaluations, the elicitation of student opinion problem notwithstanding.

But for the rest of us more ordinary souls, there really is scant information in these numbers. Further, it is known that these results are manipulable via the instructor grading scheme and, indeed, grade inflation may be a direct consequence of that. Conversely, grade inflation reduces the information content of these evaluations.

One could evaluate the instruction by looking at student performance, in down the road courses or in the course being taught via artifacts the students produce or via some survey done after the course has been complete where the institution makes clear to the students that their opinion is being sought for the good of the institution and not just to prop up a P&T process that otherwise doesn’t know how to consider teaching. My campus does a senior exit survey of this sort and it is an emblem of this after-the-fact type of evaluation.

If we wanted to continue with course evaluation but in a way that accounted for the student’s preference, we’d deliver two different type of surveys. One would be during the middle of the course (where the instructor could react to it and base changes in course delivery on it). This would ask students about what is working for them and what is not working for them in the course. Then at the end of the course the survey would ask whether the instructor made changes in the course according to the needs identified by the prior evaluation. This process, still not perfect to be sure, does provide an obvious reason for providing the evaluation – to affect the quality of the teaching. (The summative assessment offered at the end of the course acts as an enforcement mechanism for instructors to take seriously the formative assessment offered in mid semester. Students, understanding that, now have a sensible reason for doing the summative assessment at the end.) Some instructors, of course, do try mid semester formative assessment on their own, and that is to their credit. But the institution hasn’t given its endorsement to the practice, so most instructors don’t bother.

Another issue that exemplifies agency issues is grading, particularly the grading of final exams or term papers that are turned in at the end of the semester. In particular, consider the comments the instructors provide in that instance. Do they facilitate learning? Or do they merely rationalize the grade the instructor is giving? My guess is the latter and that one would predict the instructor writes more when giving a poor grade than when giving a good grade. This means, of course, that the instructor views grading as a nasty business rather than as an opportunity to teach and indeed when it is the case that all students have written on the same subject, it is quite dull and trying to continue to pay attention to the writing and find the relative merit of one student’s work over another. I know that many faculty hold machine score-able exams in low esteem, but for finals in particular they have the merit that there is none of this agency issue present (and that the results will be returned to the students in a timely fashion).

For those who don’t want to go that route, consider instead having the instructor evaluation of the work occur before the end of the semester (which means the work must be delivered earlier as well) and then having other students in the class review the work in some manner. There are a variety of ways to do this but in almost all of them the role of instructor changes from one of pronouncing judgment on the students to one where they help the students to learn.

Thus, as with the post yesterday, what is considered good teaching practice should be derivable at some level by a consideration of the various agency problems that crop up in teaching and further that by adoption this agency view one can develop a more realistic way to consider modifications in approach based on evaluative evidence gathered while teaching.

The Principal-Agent Model and Out of Class Work

For the last few years that I did Economic Theory research (in the main this ended as I took up ed tech administration) my work was on various applications of the heavily studied principal-agent model and knowledge of that informed my approach to teaching.

The canonical example of principal and agent is “share cropping” and designing the optimal contract in that environment. The basics of the model are something like this. The share cropper is assumed to be risk averse while the land owner is risk neutral (or risk averse but less so than the share cropper). That implies, all else equal, that in a contract between the parties the land owner should absorb the bulk of the risk and in the process insure the share cropper. The output of the farm depends on two factors. First is the share cropper’s effort. More effort produces greater output but more effort also creates more disutility for the share cropper. Effort needs to be interpreted abstractly and what it means depends on the context. (I’ll say more about that below.) The other factor is a random component, “the weather” or the aggregate of other external factors, that can’t be controlled by the share cropper or anyone else.

If the land owner (and an independent third party who enforces the contract) can observe effort directly, then the optimal contract specifies the effort level where the marginal product of effort equals the marginal disutility (these are the standard conditions that determine the efficient output level) and then the sharecropper receives a flat payment, independent of the actual output, so the land owner gets the entire farm output less that payment to the share cropper, and thereby absorbs the risk. Because economists love jargon, this solution is referred to as “the first best” which really means an ideal solution we won’t see in reality because the information requirements are too high (everyone knows the effort).

The model predicts “the second best” solution where the contract can depend on the farm output, which is observable, but not on effort. In this case, under a flat payment scheme the share cropper would choose zero effort (because effort causes a disutility while the payment does not depend on effort) while under a scheme where the share cropper pays a fixed rent to the landowner and gets to keep the entire farm output less this rent, the effort level would be what it is under the first best solution. But the latter means the share cropper absorbs all the risk, and that is not efficient. The optimal share provides the correct tradeoff between inducing share cropper effort, on the one hand, and having the land owner insure the share cropper against the environmental risk, on the other.

We see elements of this type of solution to the agency problem in sales, where commissions are very important, and in a variety of other cases where one hires an agent, such in the buying or selling of a home. There are some people who like to work on commission, but most people I know would prefer to be salaried and would prefer to have their good effort acknowledged through raises and in non-financial ways that fall under the category “recognition.” In a lot of cases, this happens although the output can be measured, meaning piece rates are possible. The conclusion is that there are other, superior ways to provide incentives and those are compatible with a salary approach.

Most of those ways can be characterized by the idea that the employee has a stake in the work. In the simplest of approaches, the stake is in the form of deferred compensation. A more sophisticated view takes the work itself as potentially rewarding and part of that is how much discretion and initiative the employee is delegated, as well as how much of the work is “play.”

Thus, the employer can create a stake in the job by making the work particularly rewarding. Typically this means the employer must take a longer term view because the employee “doing his own thing” will not pay off immediately, if at all. But the engaged employee is likely to produce more interesting things if given sufficient opportunity to do so.

How do these insights translate into good instructional practice?

First, consider textbooks. How does an instructor and potential adopter of the textbook evaluate one? I know that for intermediate microeconomics books, I look first at the end of the chapter questions? Sadly, in most cases these are dull, unimaginative, an apparent after thought for the author(s). Similarly, many of the ancillary tools for the book are developed not by that authors themselves, who have the biggest stake in the success of the book, but by others. If you ask economists how students learn economics (there are reasons to be suspicious about how economists teach) they’ll invariably tell you by working problems. So in this agency area we seem to be do less well than we might and raising awareness of the issue might encourage the market to produce a better solution.

Next, consider instructors who spend a lot of time preparing their in class presentation. In analogy to the textbook situation, they might find it more effective to instead spend time on the in class “active learning” exercises and out of class assignments that the students do. Further, open ended assignments are intriguing to assign because they have the potential to seriously engage the students. But there is also the problem that students may go astray. So if the instructor does opt for more open ended inquiry, the instructor should be helping students work through those open ended assignments rather than wait for them to be turned in. This work is time consuming. How does the instructor find time for doing it? The answer is by spending less time on presentation and preparing that.

Third, instructors might reconsider what they mandate as required readings for the course. Are the readings interesting on their own account? Will the students retain any of what they’ve read after the course has concluded? Will the readings get the students to consider majoring in that area of study?

All of these ideas fall under the category of a learner centric approach to teaching and none of them are particularly novel when set under that umbrella. So why consider the agency issue at all. Why not simply preach learner centric methods.

The answer is two-fold. One is that instructors, like students, have a basic need to understand “why” one should adopt a particular methodology. What issues does the methodology address? What issues does it create? The agency framework allows the instructor who is contemplating a modification in her teaching to consider the likely consequences.

The second reason, I believe, is the more important one. In general, the first best solution and the second best solution are quite different. An instructor thinking idealistically about instruction may contemplate the first best solution, try to implement a reasonable approximation thereof and then be rudely disappointed by the outcome. The agency framework can help temper the instructor idealism and provide a more realistic way to view the issues at hand.

What about closed content?

This will be a real quickie tonight. I've had some conversations with Librarians here about doing eReserves via Podcasting, but that means folks outside the campus community have to be blocked from accessing the content, for intellectual property reasons, and perhaps some Digital Rights Management is applied to the content so it can not be redistributed, to make the Fair Use case stronger or to appeal to the TEACH Act. The question I want to ask here is whether the end user can be using the same podcatching software in this case. They'd obviously prefer that. Is there any chance that it can become a reality?

More on Open Content

Yesterday I took Merlot.org to task, mildly to be sure, but there is no doubt I did it. Merlot has two ways that contributed content gets reviewed: Users reviews a la Amazon.com book reviews and for some of the content there are professional reviews by hired faculty very much in the mode that peer-reviewed journal articles are refereed. The latter mechanism emerged because some of the founders of the Merlot wanted the creation of learning objects to count for promotion and tenure in the same way that the writing of journal articles counts. Their process was driven by that consideration.

One might have come up with quite a different process if one took as the primary need identifying content that down stream users would put into their courses. Merlot still doesn’t do a good enough job for them. In fields where there is a lot of content to choose from, how do I as a potential user of the content choose? The Amazon.com approach to this offers some information but is it sufficient? Don’t I as a user want to know more about the recommender before I give credence to the review? There are a lot of wackos out there. Why should I rely on their judgment? And, in truth, doesn’t that same criticism apply for the peer reviewed content? Just who is that reviewer and why should I trust that person?

Now let’s step away from the particular issue and ask how people find Web content now. The answer, to me, seems to be that perhaps one starts with a Google search, or one goes to a known “trusted source” meaning a place where interesting and valued content has been found previously. Those are launch places. Then from the trusted sources there are links out to something and one might follow the link from a link, etc.

I want to think a bit more about non-Google trusted sources. And for the moment let’s focus on blogs. So go to Bloglines.com, do a search on your favorite topic (I chose edtech) and then for some of the blogs that come up, click on “Subscribers” and you should see the list of those subscribers who made their subscription public. Then click on a few of them and see what they have as their feeds in the topic you searched. Then a couple of more iterations on the same. I think you’ll find that its seems like there are a few core blogs that many people read and then a bund more out there. Those core blogs end up being the trusted sources I mentioned above. And the commentary that those blogs provide, in my opinion, is similar to the type of commentary that movie reviews provide, expect that the topic need not be movies. The movie review metaphor, however, is helpful here, in my opinion because many people choose what movie to see based on what Roger Ebert says, or in a bygone era what Pauline Kael said.

So how about developing learning object content critics a la movie reviewers? Here’s another area to compare with Merlot. Merlot had reviewers who were subject matter experts (though in some of the science disciplines I believe how expert some of the reviewers were was an issue.) Ebert and Kael are experts in film, not experts in the subject matter of those films. Couldn’t we have generalists who are expert in learning objects that review content across disciplines? We couldn’t if this content creation is to count for promotion and tenure. But in terms of what works for the learner, I think it is more than possible. And, especially, if the bulk of the learning objects are to focus on the first and second year college experience, then this type of review might be much more valuable than disciplinary review. (Of course if the content was fundamentally wrong but was pleasing pedagogically that would be a concern. So in reviewing any single piece of content, it would have to be done in comparison with other content that is already trusted.)

Does anybody do content reviews now? I think that it comes up occasionally en passant in various blogs but I’m not aware of folks who do that as a regular avocation and go out of their way to find new content to review. So the question I want to pose here is whether we in higher ed who want to promote open content should provide incentives for reviews to take place? And if we did, could we make it other than cheer leading and more like film criticism. Typically authors don’t know their peer reviewers in the refereeing process. But everyone knows Roger Ebert. For the film review process to work the reviewer needs the intellectual freedom to say what he thinks.

Consequently, I’m unsure whether a review process of this sort can work. But it seems to me we should explore this approach more before we build yet another repository for online content. In my opinion we need to spend more time considering the social dimensions of the user’s choice and work to improve that. Repositories don’t do anything in that dimension.

Open Content –

The podcasting craze has created a renewed interest in open content that supports instruction. While we in the ed tech business probably think about content too much and so spend too much time on how to make good content (my guys have a nice tutorial in making Web based video content that just came out) it is nonetheless intriguing to talk about how content might be shared. Here I’d really like to separate out “readings” that may be from Journal articles or elsewhere from other types of “learning objects” like screencasts and online quizzes.

One question that arises is for faculty who keep their stuff on their own course Web site, without any password protection rather than in the CMS. That content is already open in some sense. But is it findable? I’m guessing that we have a significant amount of content in this category. For example, consider this site for an introductory course on plant pathology. I did a general Google search for the site and incorrectly used the word “pathology” in the search rather than the word “pathogens,” which is in the course title. It didn’t show up on the first Google page. I’m about 50 – 50 on whether I click through to further pages on the Google search. It depends how hard I’m looking and what shows up on the first page. But I would say that as a student searching, this content could easily be missed.

Of course on the campus Google, it comes up immediately as the first item. So if the instructors only care about making the content known to the local community, they are doing fine, but if they would like their stuff to be more broadly used, one might ask whether they could do some things that would aid the Google placement of their site. More broadly, one wonders whether a campus could do things to make content open content of their instructors more findable.

Here’s another example. Purdue is now offering a service to Podcast lectures and knowing that, I can do a Google search on Boilercast and voila the first link is to a site with lots of content. But if I do a more general search on Podcast lectures, that Purdue page doesn’t come up and instead there a several links to Chronicle pages as well as to Publisher sites. Purdue as an institution has more of an incentive to make that content site well known to the outside world than any individual instructor has. Given the enormous effort they’ve already put in to make this service a go, are there incremental things they can do to make the content more visible?

We know MIT got a lot of mileage in that regard with all the pre-announcements of the Open Courseware Initiative and certainly if an institution can get that type of publicity it will attract a lot of eyeballs to the site. In this case, people will know to search on OCW and then find the site straight away. From there on in, these people will browse, not search, and I think that is helpful because it assists in being able to compare materials and also getting a sense of how much is there at the outset. With the pure search approach, one doesn’t get a good sense of how relevant the links are without going through them all.

So to me, this suggests that a registry would be useful for open content. Merlot plays something of this role but for whatever reasons it hasn’t taken off here and so I’m suspicious about whether a registry where instructors need to enter some info for their content to be listed can really work.

Hence I’m thinking about a hybrid of some sort, where first categories are input, say from OCW or Merlot, and then Google searches are returned on each of those categories. The category labels would then be something of a restricted language and the content creators who really cared about getting their content known would compete by trying to achieve high placement of their stuff in those categories. The other creators would simply be involved en passant.

I don’t know if this cans work, but it is intriguing to me and something I’ll be looking into more.

Radii

One thing that both good managers and good teachers encourage is to reframe questions to encourage a different way of thinking about an issue. When problem solving, the reframing often yields a different path with which to find a solution. So reframing is a good thing to do when stuck on a problem. When doing strategic planning, the reframing offers a different way to consider whether proposed strategies are best given the articulated priorities. In other words, the reframing is a good way to accommodate a diversity of views, especially when those views appear to be in conflict.

I often use the expression “draw a circle” or “draw a bigger circle” as a simple way to reframe because often with IT provision issues a big part of the question is the size of the audience and there is a tendency to draw the circle too small. In teaching, we use the expressions “learner centric” or “teacher centric” but I think we’ve reached the point where those labels are less useful. It is more helpful, I believe, to ask about the size of the audience – the top quartile, the median and above, or the entire class and after having identified the audience then talk about how they are being addressed. Regardless of the situation – teaching, IT administration, thinking about a topic for this blog, you name it, drawing the circle differently can have important implications for the course of action we take.

Let’s consider some examples and let me begin with the following observation. People outside the IT organization have no understanding at all about what IT costs. They look at what’s happening in the home computer market and they abstract, in an imperfect way, from that. Consider storage and backup, for example. A lot of what we pay for in supporting our big systems are costs of redundancy – power supply, hardware failover, data integrity. In other words, self-insurance is a big cost. As I said, the folks outside of IT don’t understand that. But the folks in IT don’t seem to ask, is there a better way to self-insure, at lower cost and with greater reliability?

Here I’m talking outside my area of expertise, but certainly Katrina has brought out the fact that having geographic distribution of the data is probably necessary if you are to do an adequate job of data redundancy. In my IT organization we do a reasonable job of insuring against fire in a building by taking tape backups off site, but against a regional disaster we have no self-insurance.

So why don’t we engage in a grid with like schools to do data archiving elsewhere and in turn we do our share of archiving for our sister institutions? Might we get better insurance that way and do it at lower cost, because then we wouldn’t have to take all the other extreme measures we take to protect the information? I know that in the digital library world the multiple copies approach has a lot of enthusiasts. Why not do that for backing up email, Web publishing, and learning objects in the Course Management System?

Two obvious barriers to me are first privacy and security and second the ability to retrieve the data from elsewhere in case a failure does occur. Those are important issues, to be sure, but are they insurmountable? I’m guessing an important third barrier is that we are egotistical, so we want to be in control. A shared approach of this sort implies less control at the organization level. Control, to the extent it exists, gets passed along to the grid or the consortium that runs the grid.

Ask yourself this question on a personal level. Do you have important data on your home computer that you want to protect? How do you protect those data? Have you considered along with some of your friends providing a way to share your data so you each protect the other? And why doesn’t somebody come up with some software to enable a scheme like this so we all can have better data protection? Why when thinking about this issue do we consider backup an individual task, rather than a social task?

It’s usually easier for me to ask questions about making the circle bigger than to pose analogous quations about making the circle smaller and, in particular, one of the values I bring to the campus IT organization is that I have perspective from outside, given my role as a faculty member. Arguing to make the circle smaller means disregarding people who probably should be valued and so the question becomes how can you in good conscience disregard these folks? And yet, of course, in economics we disregard information all the time so we can come up with a simple enough model to analyze and to make some relevant conclusions. So at least in principle, I’m not antagonistic to the idea to shrink the circle to make the task at hand more manageable. It’s coming up with a good example that is the challenge.

I think I’ve got one – writing. Consider the student writer who does much of the writing to fulfill requirements in the various courses he takes. This writer has many masters to please and to the extent that they are not of all one voice and don’t make overt what in the writing will please them, they create a very uncomfortable situation for the student. Writing in that situation can be unpleasant and writer’s block is a natural consequence, because the writer is stuck on what to do to please the instructor and really there is no way to reframe the task as it is posed to get the student unstuck.

Suppose these instructors went out of their way to get this student off the hook by each of them saying to the student, “Write to please yourself. Don’t worry about how I’ll react to writing. But do be concerned about your own reaction. Are you happy with the writing? Does it say what you want it to say? That’s the goal you are trying to achieve.”

I don’t know too many instructors who actually do this, but let’s say it became the practice. What is the consequence on the writing? One possibility is that students become smug and self-satisfied with their work, so we move away from the writer’s block problem into a world of mediocrity of product that the students expect to be labeled as excellent.

But this is really a delightful turn of events, because now we as teachers have something on which to coach the students and we’ve moved away from the apparent student shirking. So our job becomes one of instilling taste in the students and creating expectations in themselves that they can meet that sense of taste in their own work. We concentrate on taste and we do this by showing what pleases us and why. We model for the students what we want them to do. And then we trust them to respect that and therefore to not be self-satisfied when their work doesn’t reach that level.

Will this type of shrinking of the circle actually work? I don’t know. But I can say that anyone who continues to write after college and does so on a regular basis will almost certainly do it to please themselves. And if others are to appreciate that writing it has to be because that author has a sense of taste that the readers value. So why can’t we get students to think they are writers early on and give them a sense of empowerment in the process?