Review of Academically adrift: Limited learning on college campuses, by Richard Arum and Josipa Roksa. 2011, Chicago and London: University of Chicago Press.
Whatever subject or subjects students elect to study at university, it is generally expected that education at this level will foster an ability to think critically. Critical thinking covers a range of abilities, such as the evaluation of evidence in relation to theories, assessment of the internal coherence of arguments, the generation of methods to test ideas, and so on. Employers in the US and the UK typically list critical thinking as one of the attributes they expect to see in graduate applicants.
But to what extent does the critical thinking ability of students actually improve during the course of their college education? On average, not much, according to Arum and Roksa. They examined the performance of 2322 students, at various low- to high-ranking American universities, on a measure called the Collegiate Learning Assessment (CLA). The CLA consists of three open-ended writing tasks, one of which is performance-based ("making an argument") and two of which are analytical ("breaking an argument"). All respondents were enrolled on four-year courses, and the authors looked at CLA performance in the students' first semester (Fall 2005) and at the end of their sophomore year (Spring 2007).
The most notable finding is that 45% of students showed no significant improvement from the first to the second test. The retention rate from the first to second test was just under 50%, but on the basis that it is the stronger students who are likely to be retained the study is likely to have overestimated the extent of learning, rather than underestimating it. Most of the analyses in the book are about identifying the factors that distinguish students who do show clear improvements from those who show little or no improvement.
At the point of entry to higher education there were already inequalities among students: lower average performance on the 2005 CLA was observed among those whose parents had lower educational attainment and among ethnic minority groups, especially African-Americans. Such students also had lower SAT scores and were less likely to have taken Academic Preparation classes. In general, inequalities were exacerbated over the two year period of the study: students from more advantaged backgrounds showed greater CLA improvement than did their less advantaged peers. However, statistical analyses showed that part (but not all) of this differential improvement could be accounted for by the degree of academic preparation. The implication of this is that better academic preparation for less advantaged students could go some way to improving their learning experience in higher education. As has been noted elsewhere, "stereotype threat" may also be part of the reason why ethnic minorities tend to perform less well on tests (e.g. the mere presence of a demographic question, such as one about race, may create anxiety in stereotyped groups and hinder performance).
Further analyses examine the effects on learning of faculty expectations and students' study habits. More stringent faculty expectations about reading and written coursework were associated with better CLA scores in 2007 (after controlling for other individual and institutional factors). However, the amount of time that students spent studying did not have a straightforward relationship with CLA performance. More time spent studying alone was associated with increased CLA performance whereas studying in groups was associated with a decline in CLA performance. This is contrary to much contemporary thinking about pedagogy, which promotes groupwork, including study groups, as an effective learning mechanism. Of course, because this is not an experimental study it could be that stronger and weaker students spontaneously choose one study method rather than another, hence the study method is not the cause of students' performance per se. However, the relationship between study type and CLA growth remained even after controlling for individual factors (e.g. academic preparation) and institutional ones.
Putting their research into a broader context, and drawing upon other research, the authors note that students' overall study time has been declining over recent decades, and other performance measures suggest that the rate of learning may have been slowing across this period of time. Arum and Roksa suggest that one of the factors causing this pattern is the increasing perception of higher education as a gateway to a better job, rather than as a means of self-development. Students take out loans to finance their education, and expect to obtain the qualification that will lead to well-paid interesting employment. However, intrinsic interest in education is often lacking and students do the least that is required to obtain a degree, often with the complicity of faculty ("we won't push you too hard, as long as you do enough to get through"). Even when students have a clear aim as regards post-university occupation, they often fail to make course (in the UK: module) choices that are geared towards these aims, choosing courses that are more likely to be fun or not too demanding rather than those that are linked to their goals. In this regard, students are said to be "academically adrift", the title of the book.
It is also perhaps of some interest that the proportion of a students' education that was covered by grants or scholarships was positively associated with growth in CLA performance. By contrast, the size of loans held by students did not show any relationship with CLA performance.
Of course, it is also the case that students may need to work in order to help finance their education, and this takes time that could be spent doing other things. However, the evidence indicates that up to a point work does not interfere with learning. Students engaged in on-campus employment showed improvement in their CLA scores, whereas students employed off-campus did not. By contrast, engagement in campus social life does not appear to be reflected in better performance: more time spent in fraternities or sororities was associated with lower CLA performance.
It is not just students who are taking a more instrumental view of higher education, but often the institutions themselves. As an example of this, Arum and Roksa point to the changing face of campus accomodation. College residences were once seen as an integral part of students' social and moral development, a way of molding collegiate spirit in a way that could not happen if students lived away from campus. Now such residences are more likely to be seen as "revenue and cost centers", and constructed in such a way as to maximise the privacy of residents (thus reducing the opportunity for social interaction). Competition between institutions, combined with the expectations of parents and students, is leading to increasingly upscale facilities, but facilities which have significantly changed the nature of the college atmosphere.
Survey research cited by the authors indicates that some students, at least, are not frightened by the accrual of debt. Whilst students frequently need to take out loans in order to finance their education, they often in fact borrow money to finance their social lives as well. For some students it is only once they leave university for the world of full-time work, meaning their loans must be repaid, does the reality of their indebtedness strike home, leading to regrets about their past choices.
One of the ironies associated with the expansion of higher education is that, on the one hand, it is motivated by the desire to improve economic competitiveness via an increasingly educated workforce; on the other hand, the perception of higher education as a mere passport to gainful employment may militate against students putting in the effort to learn. Nonetheless, there appear to be multiple causes for students' underperformance, hence Arum and Roksa suggest that reforms must likewise be multifaceted. They propose that there needs to be a greater focus on students' preparation for higher education, one of the things that was associated with students' ability on entry to university and with their subsequent intellectual development. They argue that institutions need to exercise leadership in developing an appropriate culture among the student body, helping them develop "a shared sense of mission and collegiate identity". There needs to be a greater focus on curriculum and instruction, with sufficiently challenging activities presented to students that require reading, writing, and the exercise of higher-order thinking skills. The issue of collaborative versus individual study strategies needs to be revisited. The authors allow that the much-vaunted collaborative methods may have some merit, but that not enough faculty members are sufficiently skilled to structure such activities well, and that left to their own devices students may not always appreciate which methods of study are most effective.
In addition, more prestige needs to be given to teaching activities. Promotions are heavily biased towards research activities, such that the most successful researchers are rewarded with reduced teaching (often replaced by graduate assistants). There is thus less motivation for academics to develop their pedagogy.
The authors also point towards issues of institutional transparency and accountability, in terms of the setting of goals and measurement of how well those goals are being achieved. This proposal may be of less relevance to the UK context, given our system of external examining and oversight by HEFCE. However, as the cost of HE in the UK is increasingly transferred to the student, and as demand for HE places shows little sign of abating, there are some clear warning signs and lessons to be drawn from Academically Adrift. Indeed, many of the concerns about the level of students' engagement, not to mention progression and retention, are already with us.
Whatever subject or subjects students elect to study at university, it is generally expected that education at this level will foster an ability to think critically. Critical thinking covers a range of abilities, such as the evaluation of evidence in relation to theories, assessment of the internal coherence of arguments, the generation of methods to test ideas, and so on. Employers in the US and the UK typically list critical thinking as one of the attributes they expect to see in graduate applicants.
But to what extent does the critical thinking ability of students actually improve during the course of their college education? On average, not much, according to Arum and Roksa. They examined the performance of 2322 students, at various low- to high-ranking American universities, on a measure called the Collegiate Learning Assessment (CLA). The CLA consists of three open-ended writing tasks, one of which is performance-based ("making an argument") and two of which are analytical ("breaking an argument"). All respondents were enrolled on four-year courses, and the authors looked at CLA performance in the students' first semester (Fall 2005) and at the end of their sophomore year (Spring 2007).
The most notable finding is that 45% of students showed no significant improvement from the first to the second test. The retention rate from the first to second test was just under 50%, but on the basis that it is the stronger students who are likely to be retained the study is likely to have overestimated the extent of learning, rather than underestimating it. Most of the analyses in the book are about identifying the factors that distinguish students who do show clear improvements from those who show little or no improvement.
At the point of entry to higher education there were already inequalities among students: lower average performance on the 2005 CLA was observed among those whose parents had lower educational attainment and among ethnic minority groups, especially African-Americans. Such students also had lower SAT scores and were less likely to have taken Academic Preparation classes. In general, inequalities were exacerbated over the two year period of the study: students from more advantaged backgrounds showed greater CLA improvement than did their less advantaged peers. However, statistical analyses showed that part (but not all) of this differential improvement could be accounted for by the degree of academic preparation. The implication of this is that better academic preparation for less advantaged students could go some way to improving their learning experience in higher education. As has been noted elsewhere, "stereotype threat" may also be part of the reason why ethnic minorities tend to perform less well on tests (e.g. the mere presence of a demographic question, such as one about race, may create anxiety in stereotyped groups and hinder performance).
Further analyses examine the effects on learning of faculty expectations and students' study habits. More stringent faculty expectations about reading and written coursework were associated with better CLA scores in 2007 (after controlling for other individual and institutional factors). However, the amount of time that students spent studying did not have a straightforward relationship with CLA performance. More time spent studying alone was associated with increased CLA performance whereas studying in groups was associated with a decline in CLA performance. This is contrary to much contemporary thinking about pedagogy, which promotes groupwork, including study groups, as an effective learning mechanism. Of course, because this is not an experimental study it could be that stronger and weaker students spontaneously choose one study method rather than another, hence the study method is not the cause of students' performance per se. However, the relationship between study type and CLA growth remained even after controlling for individual factors (e.g. academic preparation) and institutional ones.
Putting their research into a broader context, and drawing upon other research, the authors note that students' overall study time has been declining over recent decades, and other performance measures suggest that the rate of learning may have been slowing across this period of time. Arum and Roksa suggest that one of the factors causing this pattern is the increasing perception of higher education as a gateway to a better job, rather than as a means of self-development. Students take out loans to finance their education, and expect to obtain the qualification that will lead to well-paid interesting employment. However, intrinsic interest in education is often lacking and students do the least that is required to obtain a degree, often with the complicity of faculty ("we won't push you too hard, as long as you do enough to get through"). Even when students have a clear aim as regards post-university occupation, they often fail to make course (in the UK: module) choices that are geared towards these aims, choosing courses that are more likely to be fun or not too demanding rather than those that are linked to their goals. In this regard, students are said to be "academically adrift", the title of the book.
It is also perhaps of some interest that the proportion of a students' education that was covered by grants or scholarships was positively associated with growth in CLA performance. By contrast, the size of loans held by students did not show any relationship with CLA performance.
Of course, it is also the case that students may need to work in order to help finance their education, and this takes time that could be spent doing other things. However, the evidence indicates that up to a point work does not interfere with learning. Students engaged in on-campus employment showed improvement in their CLA scores, whereas students employed off-campus did not. By contrast, engagement in campus social life does not appear to be reflected in better performance: more time spent in fraternities or sororities was associated with lower CLA performance.
It is not just students who are taking a more instrumental view of higher education, but often the institutions themselves. As an example of this, Arum and Roksa point to the changing face of campus accomodation. College residences were once seen as an integral part of students' social and moral development, a way of molding collegiate spirit in a way that could not happen if students lived away from campus. Now such residences are more likely to be seen as "revenue and cost centers", and constructed in such a way as to maximise the privacy of residents (thus reducing the opportunity for social interaction). Competition between institutions, combined with the expectations of parents and students, is leading to increasingly upscale facilities, but facilities which have significantly changed the nature of the college atmosphere.
Survey research cited by the authors indicates that some students, at least, are not frightened by the accrual of debt. Whilst students frequently need to take out loans in order to finance their education, they often in fact borrow money to finance their social lives as well. For some students it is only once they leave university for the world of full-time work, meaning their loans must be repaid, does the reality of their indebtedness strike home, leading to regrets about their past choices.
One of the ironies associated with the expansion of higher education is that, on the one hand, it is motivated by the desire to improve economic competitiveness via an increasingly educated workforce; on the other hand, the perception of higher education as a mere passport to gainful employment may militate against students putting in the effort to learn. Nonetheless, there appear to be multiple causes for students' underperformance, hence Arum and Roksa suggest that reforms must likewise be multifaceted. They propose that there needs to be a greater focus on students' preparation for higher education, one of the things that was associated with students' ability on entry to university and with their subsequent intellectual development. They argue that institutions need to exercise leadership in developing an appropriate culture among the student body, helping them develop "a shared sense of mission and collegiate identity". There needs to be a greater focus on curriculum and instruction, with sufficiently challenging activities presented to students that require reading, writing, and the exercise of higher-order thinking skills. The issue of collaborative versus individual study strategies needs to be revisited. The authors allow that the much-vaunted collaborative methods may have some merit, but that not enough faculty members are sufficiently skilled to structure such activities well, and that left to their own devices students may not always appreciate which methods of study are most effective.
In addition, more prestige needs to be given to teaching activities. Promotions are heavily biased towards research activities, such that the most successful researchers are rewarded with reduced teaching (often replaced by graduate assistants). There is thus less motivation for academics to develop their pedagogy.
The authors also point towards issues of institutional transparency and accountability, in terms of the setting of goals and measurement of how well those goals are being achieved. This proposal may be of less relevance to the UK context, given our system of external examining and oversight by HEFCE. However, as the cost of HE in the UK is increasingly transferred to the student, and as demand for HE places shows little sign of abating, there are some clear warning signs and lessons to be drawn from Academically Adrift. Indeed, many of the concerns about the level of students' engagement, not to mention progression and retention, are already with us.
Posts
