We understand that asking the right questions is at least as important (arguably more so) than finding the right answers. What problems are we trying to solve, and how should we go about framing a question that is solvable? The way the questions are asked will guide the answers that we accept and will tell us much about the assumptions that we make about higher education. Many people agree that higher education needs to be fixed, but we can’t agree on what it is that is broken.

In preparing for the retreat, invited participants were asked to think about the following questions. One of our stellar graduate students (Art Zavala and Kathy Hayes) phoned each participant and tape-recorded their answers.

What are the findings or theories from your area of expertise that we could apply to higher education? (Feel free to define your area of expertise any way you want. Some examples of what we are thinking about include research on the transfer of training, understanding how people work in groups, encouraging creative or analytic thinking, cultural or motivational factors, learning with multimedia, and maintenance of attention.)
What are the (most important) unsolved problems? What should be included in an agenda for research?
What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?
What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?
What additional questions should we be asking?
What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Here are summaries of the responses that they provided and they questions they thought we should be asking. Additional responses will be posted to this site as they become available.

Ralph Wolff

Carol Tomlinson-Keasey

Sharon Riedel

Anne Petersen

Kaiping Peng

Vimla L. Patel

John Newman

Keith J. Holyoak

Frank Dempster

Elizabeth L. Bjork

Robert A. Bjork

John R. Anderson

Franca Agnoli

Phillip L. Ackerman

Ralph Wolff

Area of Expertise

Leading assessment of student learning for all WASC accrediting schools
Action Research or educational effectiveness
Executive director of an accreditation association

What are the (most important) unsolved problems? What should be included in an agenda for research?

We do not have a common definition of learning. There are implicit definitions of learning. Our definition of learning grows out of the industrial model and the Newtonian physics model, in which learning is about mastery of facts. I have read research and I have found, consistent with what my observations are, that we confuse learning with mastery of fact. Faculty are also divided about what learning is and about whether learning can be measured or evaluated effectively. So it is necessary that we continue and enrich the conversation about learning and that our discussions are grounded on research. Students should also not be left out of this. We need to have faculty and students in learning centered conversations and discuss what is learning and what are the methods by which we acquire learning

The change in the demography of American higher education has forced us to really reassess what are the purposes and what are the indicators of effectiveness in our institutions. Higher education is no longer a privilege; it is perceived to be an absolute economic necessity by policy makers for continued economic growth and by students and parents to be the only ticket to success in the American culture. As a consequence the role of higher education has taken a huge significance and it has taken a vital role to the future success of the American dream. It has become an economic one, but nonetheless we now have a diversification of the educational system (e.g., typical students work 20 hours a week, the average age of the students in the CSU system is 28 instead of 21, typical student can be married with children). So we need new indicators of success and effectiveness. The only ones we are relying upon now are through put statistics—how many graduate in six years. But we do not have good qualitative measures. Most people would agree that the SAT and ACT are only good predictors of the first year of success and are really not intelligence tests, although they are viewed that way. So we need indicators of academic quality because we can no longer rely on the SAT and entry level statistics (GPA). There is an increasing need to know whether graduates will be able to do or know

The second point is that we are also at a critical time with the information revolution and the explosion of what there is to know and thus there is a desperate need to build connections to redefine what learning is. The model that most faculty were raised in, myself included, is that there are a set of facts that need to be mastered and each course is separate from one another. What students need is a different model of learning and knowing. A much different model that is generative, contextual, that is interrelated, and that truly understands the changing character of knowing. The foundations of every major discipline are rapidly changing, although there are some basics that are constant. Students need to be prepared for this change and this notion of fact mastery has got to give way. So we really need to rethink what learning and knowing is and whether curricula are actually structured for student learning.

We are also trained to teach and to tell, but not to induce learning. Faculty were trained to profess, but not to think about learning. Grant Wiggins tells this story that Michael Jordan didn’t become a great basketball player because he scored an A on it. Wiggins terms this feedback-based learning. True learning or mastery occurs through constant practices. A pianist does not become a concert pianist or a person does not get a black belt by reading about it, it is practiced and there is ton of feedback that is provided to them. We have a post-hoc assessment model of evaluating learning and teaching, but not a feedback and confidence-building model.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

On major barrier is that there is not agreement that higher education needs to be redesigned. A lot of people you have in higher education are quite satisfied with it. We are the victims of our own success. We believe that we are the best system of higher education in the world. We are the richest, but I’m not sure we are the best anymore. I have been in other countries and met with enough other people to actually know that they don’t hold that view. We have the richest system, which means that for graduate study the Chinese and others want to send people here because of our research facilities. But at the undergraduate level that’s questionable.

Second, I think there is a lack of a willingness to experiment, particularly in the western part of the United States. There is an established paradigm that the only knowing is scientifically validated knowing. The only true work is that which is scientifically verifiable and within that only research universities can produce good science and that only quality comes from that or the elite institutions. There is just such a higherarchy and little experimentation.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

There is a tremendous amount of work in teaching writing and it reflects feedback-based learning. In music and in art it is more of a feedback-based system, in which the criteria are as openly discussed as the product. But traditionally universities as a whole do not adopt a feedback-based learning system in terms of curriculum design and evaluation. Sure there are prototypes, there is Evergreen State and Fairhaven College of Western Washington, so there are a lot of those who take education in a more holistic way. I just saw that Washington state is trying to develop a whole community college based on integrative learning.

What additional questions should we be asking and what do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Why would we even care about learning? Do we care? And if we do, then are we satisfied that what we are doing is sufficient. If the answer is we don’t care or that it is not our job to care and that the faculty are doing a good job and things are working well, then I don’t think what we produce at this meeting will create change. So the real question is are we satisfied with the level of learning? I would have to say that I do not feel satisfied with the level of learning or engagement, nor do I feel that faculty consistently employ methodology in their relationships with one another, let alone with the students, that are geared to stimulate the kinds of student learning and mastery that we are trying to accomplish in large enough numbers. We do produce a lot of wonderful graduates, but there are a lot of people who leave college feeling burnt out and devastated. The second issue is do we have a common basis to even talk about what even learning is. Thirdly, is there a protocol by which we can discuss and evaluate whether learning is effective? My biggest fear is that the only protocol we have is the post-hoc model called the test. Another issue is how do we make learning as interesting as cancer research. I think it is. How do we enlist faculty to make learning as energizing, interesting, and researchable rather then the byproduct of professing?

(Back to top)

Carol Tomlinson-Keasey

Some of the Challenges Facing Higher Education

Increased need for higher education. More people are attending college than in the past. In most of the 20^th century people could survive without a college education, but as we enter a new century there is an increased need for people to attend college.
Changing Demographics. The increase in population combined with the first point that more people will need to attend college only exacerbates the problem.
Increasing Costs. In California we have been lucky that our public institutions have not faced the same increases as other institutions across the nation. Still approximately 15% of a family’s income goes towards education for a child that is of college age, whereas just a few years ago it was only 9%.
Lifelong Learners. We need to be educating people much longer and throughout their lives.

When these factors are added together it becomes apparent that higher education is not going to meet these challenges using its current model. This is where the use of technology and looking at different ways to approach the problem become incredibly relevant.

What are the (most important) unsolved problems? What should be included in an agenda for research?

On an academic scale our institutions are divided into departments and colleges that do not easily morph into new departments or colleges to reflect the way that the fields are growing. All of the funding and resources go to the departments and colleges not to the new interdisciplinary areas that people are excited about.
We need to know more about the way that students learn and especially how they learn via the new technology.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

Rensalears' model for studio courses.
Also, interested in looking at open university options.

What are the major problems with or barriers to redesigning higher education?

We don’t change quickly.
Our institutions are designed to support a model that is time and place bound.

Do you have any ideas for overcoming them?

I think it was Sir John Daniel that said, “Technology is the answer. What is the question?”
There are many ways that technology will be helpful and that it can be used to help us overcome some of the barriers that we are not able to overcome as institutions.
This is not to oversimplify the problems and to say that technology will be able to solve all of the problems, but it will definitely be a solid tool with which to begin.

What additional questions should we be asking?

There are many levels on which to address each of the questions that have been asked.
Each question can be answered globally, for the U.S., for California, or for individual campuses.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Outline the problem so that people are drawn away from their day-to-day character and begin to see that what they do really does impact education over the next 5 to 10 years.
Show people reasonable solutions

(Back to top)

Sharon Riedel

(Back to top)

Anne Petersen

What are the findings or theories from your area of expertise that we could apply to higher education?

I have done research on cognitive functioning especially gender differences in spatial ability. I have a couple of books and probably a couple dozen research articles on the topic. I've especially been interested in how biological and social factors influence cognitive functioning over the first 25 years of life. This relates to higher education in understanding different learning styles of men and women (among other groups) in higher education. For example, there is evidence that experiences such as through sports affect cognitive performance and probably brain development.

From a totally different part of my professional experience that in university administration, I've learned a lot about what affects student learning. While at Penn State University, I was a member of a group examining undergraduate education what practices enhance learning and what dampen it. For example, research demonstrates the minimal effectiveness (in terms of retention of information) of traditional lectures whereas more interactional teaching approaches have better learning outcomes. Another memorable finding in this review of the learning research was that young women who drop out of science and technology majors do not do so because they are doing poorly in classes; rather, they drop out because they find the classes boring (repetitious, requiring memorization and rote learning).

While at the National Science Foundation, we worked especially on efforts to emphasize hands on learning of science and technology at all educational levels. This approach has been effective at engaging young people in the science and math and motivating them to continue their studies in this area. Higher Ed was the most difficult level at which to implement change.

Currently, I chair a Board at the National Academy of Sciences on Behavioral, Cognitive, and Sensory Sciences. One of our committees examined the research on The Aging Mind (also the title of the resulting volume). Among the fascinating conclusions of this inquiry (whose focus was to propose a research agenda for the National Institute of Aging in the area of cognition) was the effectiveness of a compensatory strategy to maintain performance competence. For example, people could continue to perform cognitive tasks by using various ways to enhance memory, or continuing physical performance by reducing the number of things being done at once. Compensatory strategies would seem useful to consider at various ages to deal with weaknesses in skills. Similarly, a group examining learning at the other end of life concluded that activity based learning was much more likely to be retained.

Another inference arising from many professional experiences is the high salience of interdisciplinary learning for application in the real world. Higher education has yet to implement interdisciplinary learning beyond various exploratory efforts; most are not integrated into usual practice despite their appeal to students and relevance for most kinds of work.

What are the (most important) unsolved problems? What should be included in an agenda for research?

(1) stereotyping of learners and its effects on performance over the short and long terms

We know (from the research of Steele and others) that stereotyping exists, and that it has powerful effects in the near term. We don't know how pervasive these practices are, and we don't know the long term effects.

(2) effective ways of changing teaching practice to increase learning of students

We know about practices that increase learning. Achieving effective implementation of these is less well known.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

Many prototypes exist of most any important practice. Challenge is to get these used more systematically.

What are the major problems with or barriers to redesigning higher education?

Entropy.

I suspect that change will require both top down and bottom up approaches. It will require great patience and tremendous persistence. Everyone needs to get on board with desired change and persist until it is fully integrated into practice. It may take a generation.

What additional questions should we be asking?

I would be interested in knowing what faculty feel about the need to change. I know that most administrators have been there for about a decade know. Initially many felt that the challenge was a hopeless one. More recently, things appear to be moving some.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Both university leadership AND faculty need to want change and be supportive of the use of more effective approaches.

(Back to top)

Kaiping Peng

Area of Expertise

Cultural Psychologist.
Culture and Cognition

Study how culture-specific etiologies and theories and representations affect people’s thinking, reasoning, decision making, and emotion

What are the findings or theories from your area of expertise that we could apply to higher education?

Culture matters. There are cultural-specific etiologies and theories that affect people’s behavior, including the behaviors of learning something (as in an education setting). These issues have not been fully studied in the field of psychology. Sometimes they have been overlooked, not just internationally, but also in the US. We all know that many of the subjects in our studies, for many years, have traditionally been white male sophomores. We have not studied other ethnic groups to same extent, and we have not explored fully the impact of socioeconomic, racial, and other ethnic experiences and the ways in which these experiences may affect learning and education.

A lot of what we study in cultural psychology primarily focuses on definitions of culture and the formulation of cultural-specific theories., We have some of understanding of these issues, but we have not applied our understanding of cultural effects to higher education. We should be able to design better ways of teaching based on our knowledge of cultural influences on how people learn.

What are the (most important) unsolved problems? What should be included in an agenda for research?

Two kinds of problems:

1) The problem of the independent variable. What is culture? Is it a purely cognitive, affective, or behavior? We do not know much about culture from a psychological perspective, even though we talk about it all the time. So the issue of what is culture is still not well understood.

2) There are issues of the dependent variable as well. How broad are the cultural effects? How psychologically deep (in terms of mechanisms or processes) are these effects? We also talk a lot about multiculturalism, but mostly from a political perspective instead of a psychological one. We need to focus more on what are the cognitive benefits of multiculturalism.

What prototypes can you point us toward where principles from the science of learning are already being applied?

We know that there are differences in school performance by students from different ethnic groups. One example we all know is that Asian students perform better on math tests than any other ethnic group. There are a lot of theories about why they do so well in math. One possibility that has not been talked about much is that math is taught differently in Asian countries, which may be key to their success. Professor Howard Stevens at the University of Michigan provided one example of how teaching styles may affect performance. Dr. Stevens looked at an inner city school in Chicago with a predominantly Black student population. He found that the school did very well in a standard math test. He was very surprised to learn that the principal and the school in general emphasized what they called an “Asian way of teaching math.” So what is the Asian way of teaching math? One important component is that whenever you teach something, you do it in groups. New information is never taught solely with student-teacher interactions. An important part of the “Asian model” is an emphasis on students learning from each other. So they have a lot of group activities, and the activities are not simply presenting the answer as a group, but rather have the groups solve specific problems together. Also, when they solve the problems they do them in public. That is, they ask kids to solve the problem in front of the whole class, which can be very intimidating and humiliating, because if you do not know how to do it everyone will know. No American school or public school teacher would do this in the US because they believe that it would hurt the child’s self esteem. However, in Asian societies, teaching is always done this way. The public group method is based on the theory or folk belief that if you get learners excited or aroused, even by some humiliation, then they will be motivated to try and do their best. And even if learners don’t solve the problem, they remember the solution to the unsolved problem for quite sometime because they were aroused. When they applied the Asian way of teaching in that inner city Black school, the kids did very well. What I’m saying is that there are different practices from the ones we use in the US that might be helpful. We can look to other countries and other ethnic groups to understand alternative practices that we could be using.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

Different beliefs about education may be a problem. People in the US have their own folk beliefs about education and about the intellectual development, which I sometimes call the Socrates view. The nature of the Socrates view about intellectual development can be categorized in this way:

1) We emphasize questioning. That is, you have to question what people tell you.

2) We also emphasize evaluation. That is, that you have to evaluate judgments about other peoples stories, theories, and results.

3) We also emphasize self-generated knowledge.

4) We believe that there is a single truth

These are ideas that are promoted and emphasized in the US. These ideas are not bad, but they are cultural-specific because people in other cultures do not share them. Asians views about education are more influenced by Confucius, which emphasize:

1) Effort—effort for learning--you have to spend some time to learn something

2) Respect—everybody has some knowledge that you don’t know.

3) Dialectical—thinking from two different perspectives, contemplating the contradictions. For Westerners, the contradictions are error or mistake. For Asians, much of the intellectual and human life is about contradicting statements and they do not seem to feel the need to resolve contradictions

So I think those are the cultural specific views about intellectual development and cultural specific views about education. I think that the barriers to redesigning higher education are, in fact, cultural biases and cultural different beliefs.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Reach out to the general public by way of the media. Inform them of our goals, and then present them with the major findings of the retreat. Publish articles that summarize the findings as well

(Back to top)

Vimla L. Patel

(Back to top)

John Newman

(Back to top)

Nora Newcombe

Area of Expertise

Development of spatial skills in children and adults, and how to facilitate skill development. Sex differences and other individual differences in higher level cognitive skills. Teaching cognitive science theories to educators.

What are the findings or theories from your area of expertise that we could apply to higher education?

Major concern is that there is a body of knowledge on such topics as analogical transfer, reading, and other issues concerning how people learn, that needs to be communicated to prospective teachers. Education majors need to have courses in cognitive science in the same way that engineering majors need physics and premed majors need biology.

What are the (most important) unsolved problems? What should be included in an agenda for research?

The major unsolved problem is a political one. There is a real need to convince colleges of education and also school districts that the knowledge provided by cognitive science is important for educators to have and to continually be trained in. How do we sell the achievements of cognitive science to those that could benefit from it, namely educators?

In looking at an agenda for research, we would want to look at the leverage points for achieving the transfer and wide scale acceptance of knowledge about cognitive science to educators. There is a need to fund demonstration projects that can exploit these leverage points and to evaluate their effectiveness. It should be a funding priority to have people explore technology transfer in cognitive science.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

John Bransford and Marilyn Adams are good examples of people that have worked very effectively in developing cognitive science applications and actively interfacing with school districts. However, the work that they are doing is specific to types of problems and to age groups. So I would say that there are not any prototypes for the broad intertwining of cognitive science with colleges of education and schools in the way that I am suggesting.

What are the major problems with or barriers to redesigning higher education?

There are institutional barriers that are created by the current structuring of academic departments. Colleges of education do not traditionally employ cognitive scientists. The people who would need to teach the courses that I am envisioning would need to come from different departments and this creates turf issues

Do you have any ideas for overcoming them?

Nothing is as powerful in overcoming barriers as financial incentives. That is why there is a real need for funding demonstration projects that can illustrate the effectiveness of this type of training. Financial incentives to get colleges of education to work with cognitive psychology departments would be very helpful.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

It is typically the case that people will have personally rewarding and illuminating experiences at the conference or retreat and come up with interesting recommendations. However, as the outcomes are turned into a report written to the funding agency there is often no further monitoring or follow up. I think that it is important to see what could be done with Spencer Foundation and other funding agencies in terms of further monitoring and exchange after the writing of the report. It would be helpful to set up a format in which there was a meeting with Diane and others along with the funding agencies to create an exchange about the ideas that were generated. The follow up with funding agencies is key and it would be very beneficial to continue to develop funding contacts with other agencies so that the ideas generated by the retreat could be pursued.

(Back to top)

Jose Mestre

Area of Expertise

* Learning in the sciences (particularly physics)

* Study the following:

o Problem solving

o How to develop expertise

o How do people organize knowledge in memory so that it is useful for retrieval and solving problems

* How to apply research in the above areas to teach better, or more efficiently, or teach in ways that makes students learn the material at a deeper level

What are the (most important) unsolved problems? What should be included in an agenda for research?

1) Lecturing to students in large classroom settings is not the most effective method of instruction. Findings that come from studies about how people learn indicate that whenever students are actively engaged in their learning, they learn better and faster. So, we need to find better ways of getting students to learn material than just lecturing in large classroom settings:

-- In a small class you can address students' needs more effectively

-- In large classes (50+ more students), which are typical of many introductory classes (in science in particular), lecturing is not effective. Other methods should be employed

Classroom Communications Systems. It is a version of the audience-polling device of “Who Wants to be a Millionaire.” Students are given questions to work on in groups and then submit answers that get displayed onto a histogram. The instructor then uses the histogram to direct the direction of the discussion in the class. The instructor is viewed more as a learning coach rather than a repeater of what is in the textbook. It has its benefits in the sense that the instructor can shape the instruction to meet the needs of the students…if students get it, then the instructor can move on, and if they don't then he/she needs to back track and figure out what is wrong and deal with it. Students seem to like it and they seem to get more engaged.

2) Implementation—How do you get instructional techniques and implications of research into practice? Getting a charismatic instructor is not the answer. Charisma only goes so far; research evidence in the sciences suggests that students learn about the same amount regardless of who teaches them. How do you take things that work and scale them up so that other instructors are willing to adopt them—that’s a big problem.

Two problems to implementation:

-- To teach at the university level, you never have to take a course on teaching, pedagogy, or cognition (the only exception may be school of education professors). You become a teacher by getting a Ph.D. in some subject and it is assumed that content expertise is the only thing needed to teach that subject, and nowhere in between are you taught about instruction. In general, you are never taught about how to apply findings from research and learning to instruction in your discipline. We are essentially reproducing a failed system of instruction, and not because we don’t turn out good Ph.D.s, but we because we fail at teaching the general mass of students.

-- People who have been teaching for a number of years think that they “have it down,” and feel that if they say it in the right way or make lectures a little more clear then students should “get it,” when, in fact, the issue is students have to be engaged in learning. Lecturing is really a boring way to go for students.

Telling people what to do to improve their teaching does not really help, even if they are willing to change. You may have to have an apprenticeship model where teachers observe or co-teach a subject or class with somebody else. Teachers might be able to adopt a new technique so that it meets their style and their students' needs with this learning model.

So, how do you train perspective Ph.D.s in all fields so that they know something about teaching and learning, and how do you get at the people who are teaching already to adopt new styles? These are big challenges.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

1) Books: How People Learn—has a number of chapters on teaching math, science, and history.

2) Physics education research is becoming more accepted in physics departments. In our department and 10-12 other research departments around the nation, there are physics education research groups with doctoral students who do their dissertation on some aspect of physics education research. So you have some degree programs that are geared to help people with both the content and the teaching and learning of that content.

What are the major problems with or barriers to redesigning higher education? Do you have any ideas for overcoming them?

1) Higher education has been very compartmentalized into departments, groups, and schools. And often times, those groups look down their noses at the other groups. For example, it is typical for scientists to look down their noses at schools of education. We need to break down barriers to solve the complex problems of education.

-- One example where barriers need to be broken is in the training of teachers. Every science department plays some role in the training of prospective teachers (at the high school level). So, if you are majoring in biology and you want to become a biology teacher, you take several biology classes. But, biologists teach those courses. You would have to go to the school of education to get the pedagogical methods course. However, based on what we know from the NRC's How People Learn report, the pedagogy and the content need to be taught together to develop "pedagogical content knowledge". Put a different way, just because I can teach physics well does not mean I could teach biology well.

So, although it is unlikely, there needs to be cross-disciplinary work

2) Telling people that they are doing something wrong is really not going to work.

-- One way the physics community was able to get faculty interested in finding effective ways of teaching large lecture courses was by developing a conceptual test in physics and encouraging faculty to administer it at the end of their courses. After students scored very poorly on the test, a test that most physics instructors considered "trivial" and thereby thought that their students would perform very well in it, the professors started to question what they were actually accomplishing by teaching in large lectures courses with the lecture method. What was clear was that students were able to solve "standard" problems, but they did not really understand the concepts. This opened up dialogue across many departments and faculty began to ask “How do I go about teaching in different ways.”

-- Once you have the attention of the faculty, it is important that they are given enough support, so that they are able to implement new techniques.

What additional questions should we be asking?

What are the goals and what are we (at this conference) trying to accomplish, and are they modest enough so that you can make some headway? Trying to reform higher education is a big job.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Need administrators to be involved, either at the conference or at some point.

(Back to top)

Richard E. Mayer

Area of Expertise

Cognitive Psychologist, Cognition and Instruction

What are the findings or theories from your area of expertise that we could apply to higher education?

At the most general level I would say that we need a theory of learning, an understanding of how people learn if we want to design instruction that will help people learn. We need findings and theories that tell us how people learn. Essentially what we need is an educationally relevant theory of learning.

What are the (most important) unsolved problems? What should be included in an agenda for research?

There are very few solved problems, which leaves just about everything else as an unsolved problem. The question that I am most interested in is how can you teach in ways that promote transfer. What kinds of learning experiences allow people to take what they have learned and apply it in situations? Transfer is a classic issue in psychology and in education, but I think that it is really the fundamental unsolved problem. We really want to know how to teach for transfer. I think that we know how to teach facts and procedures, what we don’t know is how to teach higher-level concepts and strategies.

An agenda for research needs to include an educationally relevant theory of learning.

What prototypes can you point us toward where principles from the science of learning are already being applied?

We can all point to our own work. At this point in our research in learning we have decided that it does not make sense to have a general theory of learning and what we have are much more domain-specific models of learning. The theories that we are going to develop are going to have to be domain specific to a certain extent.

What are the major problems with or barriers to redesigning higher education?

One of the barriers is that we don’t have a good concept of what “good instruction” is and we don’t have good theories of what “good instruction” is.

Another barrier involves the use of technology in education. I have been interested in educational technology. I think that sometimes people confuse media and method. This is kind of a traditional issue in instructional technology. At the level of higher education people sometimes think that innovation means using technology. We know from many, many years of research on educational technology that technology does not create learning; it is instructional methods that cause learning. Just inserting technology into higher education is not really a viable solution. We need to find the instructional methods that help people learn and then see how technology can be used to serve that purpose. Many people take a technology-centered approach rather than a learning-centered approach.

Do you have any ideas for overcoming them?

To overcome these barriers we need more research. Research on what constitutes “good instruction” and research on how to use technology to improve learning.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

Until we know what constitutes “good teaching” I am not sure that we are ready for change. I am interested in what do we know about how people learn that would let us know what the change should be. Do we have a theory of learning that is relevant and what are its implications? I get nervous about people talking about change before we have really looked at what should be changed.

(Back to top)

Marsha Lovett

Area of Expertise

Cognitive Psychology; acquisition and transfer of problem solving skills

What are the findings or theories from your area of expertise that we could apply to higher education?

There are two empirical studies that I point to whose results can be used to improve instruction. In particular, these exemplify some of the simple things that can be done to enhance transfer and generalization of knowledge.

The first is by Cummins (1992: JEP:LMC) in the domain of algebra. She found that when subjects were given simple instructions to compare and contrast analogous problems, they were much more likely to induce a generalized problem schema (& transfer the problem solving skills). This finding helps to illustrate that some of the things we can do are very simple: participants benefited from the comparison task without even solving. It is often just a matter of asking questions in a way that prompts students to provide their own self-explanation or will allow some other processing that will allow students to generalize their results (see recent work by Chi that extends her work on the self-explanation effect).

The second study is by Catrambone (1998: JEP:G; 1996: JEP:LMC; 1994: Mem&Cog) regarding procedural skills and schemas. He was able to demonstrate that if you highlight the structure of problem solving steps in examples presented to subjects, the subjects are more likely to generalize the procedure to other problems. He found that this was even effective when subgoals were highlighted by being formatted into groups with arbitrary labels. His explanation for this was that highlighting the steps to students helps to lead to self-explanation.

Another important contribution to this is Singly and Anderson’s theory of transfer of skill (Singley & Anderson, 1989).

What are the (most important) unsolved problems?

The first one involves the research on epistemology and students’ beliefs about learning. There has been a considerable amount of work classifying students’ beliefs about knowledge or describing the impact of metacognitive beliefs on learning. Much less work has been done on understanding how students change their views on learning. Often this change has been thought to occur naturally as a result of maturation, but an unanswer question is: can we create interventions that will help to effect this change.

For example, students’ typical belief that learning is quick and problem solving is immediate is an obstacle to their learning: they think that if they can’t do something right away it can’t be done. The belief that authority on knowledge comes from “the teacher” or is “in the book” can also impede deeper learning. Again, the unsolved problem is: How can we help to change these beliefs?

The second issue involves the link between motivation and cognition. People tend to study either cognitive issues or motivational issues; an important problem is understanding (and dealin with) the interactive link between the two.

A final issue is how students should be scaffolded in their instruction (both computer-aided and other forms of instruction) and how and when this scaffolding should be removed. There is empirical work on this topic but not much theoretical development in this area. I believe that framing the empirical results in a theoretical structure (and extending them) would help to guide instructors in how to apply these results in new situations.

What should be included in an agenda for research?

In developmental psychology, we have gone from emphasizing the description of developmental states to actually describing the changes between states and the mechanisms for accomplishing such changes. In a similar way we could have people trying to understand the changes that occur in students’ epistemology (see first unsolved problem above), not just describing a student’s state of mind but describing the change process. Additionally, it is important to get people collaborating on different pieces of learning from a variety of perspectives, ranging from the cognitive to the motivational.

What prototypes can you point us toward where principles from the science of learning are already being applied (e.g., activities, courses, fields of study, degree programs, or entire systems)?

We are in the process of refining instructional design in an introductory statistics class at Carnegie Mellon. At each step of the redesign, we try to bring more of the principles of cognitive psychology and learning research into practice (cf. Lovett & Genovese, 1999 from The American Statistician; Lovett, in press: A collaborative convergence on studying reasoning processes: A case study in statistics. To appear in S. Carver & D. Klahr Cognition and Instruction: 25 Years of Progress. Mahway, NJ: Erlbaum). We put theory into practice when we select the materials for the course and design activities that engage students in learning. The last step of the process will be to incorporate a cognitive tutor to help students solve statistical problems based on their own level of understanding. This work is based on cognitive tutors that have been very successful at the high school level in algebra and geometry classes. The work of Ann Brown (Communities of Learning, I believe a relevant ref is from 1992 Journal of the Learning Sciences) is also a good example of interdisciplinary and collaborative work.

What are the major problems with or barriers to redesigning higher education?

Everyone is inventing their own wheel for solving instructional problems. People are not building enough on the work that has already been done and they fail to fully explore what is already out there, both in terms of what works and what does not work. One part of this problem is that the information is not readily available or easily accessible. The other piece of the problem is more of a cultural impediment in that people get invested in “name brand work.” This produces resistance for people to build on other people’s work.

Do you have any ideas for overcoming them?

The lack of information problem could be addressed with searchable databases or archives that could provide a place to reference work that is being done in applying cognitive and learning principles to higher education, and to document what works in different areas and what does not.

What do we need to do so that one outcome of the retreat is to effect change (in ways that we want)?

After looking at some of the different perspectives on the most important problems, it would be helpful to find the commonalities in these problems and then to set out to have people attack these problems using shared resources. One success would be to get people communicating and looking for ways to get people together for longer-term collaborative projects. Herb Simon gave a speech at Carnegie Mellon on instruction and learning with a title something like, “Why isn’t teaching a team sport?” This title gets at the cultural issue of the solo aspect to instruction and instructional design. The statistics project we are working on is a very collaborative effort, with cognitive psychologists, instructional designers, statisticians, and software designers all working together to tackle this large problem. This requires extra effort (from developing a common language for communicating to figuring out how to meet with 5 other busy researchers for whom this is not a major project), but it would be great to get people to work on and maintain long-term collaborations.

(Back to top)

Joel R. Levin

Interested in improving the quality of evidence and the credibility of claims in educational research.
Focusing on the quality of research in education and psychology, especially, as it applies to policy implications.

Suggestions for Improving the Quality of Educational Research

The first suggestion would be to improve the design of educational research and to move toward more scientifically based models. It is important that we are not faddish in research questions and that we do not simply jump from one hot topic to another. There is a need to employ more systematic long-term research methodology. We need to be scientific in our method of inquiry—begin with a research hypothesis, test the hypothesis in small laboratory settings, and then move on to large scale, but very controlled settings (as in medicine, for example). We also, must not rely merely on demonstrations and uncontrolled studies, but instead, we should be guided by an approach where we would use multi-site testing in our research and use analyses that match the design of our research.

It is important that we begin to get away from testimony and opinion in education. Too many programs are put into place for political reasons and subjective opinions. New ideas hit the press and school districts implement these programs without really evaluating the effectiveness of these programs.

We need teams of people working together—interdisciplinary teams of people with strong scientific research backgrounds, along with people who are good in applied settings and in implementing programs. We need to entice people who know how to do scientifically sound research to work on field research and program evaluation, and we need to get the funding to do applied research. To answer questions about the effectiveness and outcomes of teaching strategies in large-scale classroom settings, as opposed to controlled laboratory studies of individual learners or short-term implementations of programs, there needs to be much bigger and better controlled studies than have previously been done. There is a need for classroom-trials studies (Levin & O’Donnell, 1999), research in classrooms that is modeled after clinical- trials research.

Additional Questions

What do we need to do to get people who are in positions to evaluate the evidence (funding agencies, policy makers, implementers of special programs) to be able to discriminate between differences in the quality of evidence? How do we build the understanding in people that not all evidence is equally credible?
How do you get both the people who evaluate programs and people outside of the education arenas who are not necessarily educated in scientific thinking to change the way that they think?

Programs for Students Who are Academically at Risk

Are there programs that can be used to help these students succeed?
Should we expect different educational outcomes for different students?
How do we address these concerns, and how do help such students succeed?
We know that with adequate resources and proper motivation and interest from faculty at-risk students will do better.
What are the critical components of successful retention programs for at-risk students?

Thoughts on Students’ Thinking and Learning

What is the relationship between memory and critical thinking?
Must information be memorized and readily accessible before it can be used in the service of thinking critically?
If so, would it be beneficial to help students learn underlying factual information in a shorter amount of time, so that they could spend more time on higher-level tasks?

(Back to top)

Alan M. Lesgold

What are the findings or theories from your area of expertise that we could apply to higher education?

Most of my work is in the area of developing technology so that students can “learn by doing.” As more opportunities for computer simulations and instructional design become available, we can provide more instruction that involves “learning by doing” as opposed to the traditional model of “learning by telling.” There are many areas where in the past we were only able to provide instruction through lecture or by having students memorize materials. In the past, when it was important, we would use