Research Methodologies

Tuesday, March 29, 2005

A slight change in approach

I'm starting to think of methodologies that will work with my research on students and self-efficacy. I looked up several sources on self-efficacy since not much has been done on GSP by itself. Self-efficacy is interesting but I'm having trouble redesigning my research question to include my thoughts on it. Right now I have several ideas, in no particular order in my head that I'm struggling to define and/or organize for my proposal.

Will students with low self-efficacy in writing place themselves in English 120?
Will students with high self-efficacy in writing place themselves in English 121?
What are the implications if the answers are yes to both questions?

How will I determine what is considered low and high self-efficacy? I think this is something I will determine by modifying what previous researchers have done to measure high and low self-efficacy. Most seem to resemble a questionaire/survey format. Once I determine who has high writing self-efficacy and who has low writing self-efficacy I can maybe interview a couple students from each group to find out more on why they enrolled in English 120 or English 121. This would give me a better idea of where they are coming from and whether we should compensate for self-efficacy in addition to placement decisions based on the GSP questionaire. Something tells me the questionaire is not enough by itself. Do we need a small form of checks and balances, or does that undermind our students' rights to place themselves in composition courses? I feel like I'm running around in circles...am I thinking too big for this sort of project?

Tuesday, March 22, 2005

More research...

Bashford, Joanne. "Placement Decisions for First-Time-in-College Students Using the Computerized Placement Test." Office of Instructional Research, Miami-Dade Community College (1998).

This research report on the scores incoming freshmen received in computerized placement tests. These test scores are compared with the percentage of students who then passed the courses in which they placed themselves. Bashford found that when they adjusted the score ranges to more effectively match the skill levels for the courses, more students passed the courses they were enrolled in. Though just a list of tables, this information is important to consider when deciding what type of placement method is to be used for fycomp. Like other research on this issue, it seems that this method of placement is still not as accurate or useful as portfolio assessment or GSP.

Armstrong, William B. "Pre-Enrollment Placement Testing and Curricular Content: Correspondence or Misalignment?" (2001) ERIC Document Reproduction Service ED 454 895.

Armstrong looks at the relationship between placement tests scores and academic achievement. To measure academic achievement, this researcher administered placement tests before and after students took their first semesters of freshman comp. He found, by no surprise, that there was no significant change in test scores for students. It makes sense, because the tests only predict how well a student will do on that test, not necessarily the course itself. Armstrong suggests that in order for there to be a significant change in scores, curriculum needs to be more closely related to material on the placement tests. This is important to note, but do we really want to measure students' successes on placement tests? Wouldn't this be the same as giving secondary students the MEAP test and expecting that to determine how well they will do in fycomp courses? Sounds like the same thing to me.

Royer, Daniel J. and Roger Gilles. Directed self-placement: Principles and practices. Cresskill, NJ: Hampton Press 2003.

This text seems to explain the standards for self-placement standards in writing programs. Since these authors can be considered the "fathers" of directed self-placement, they manage to perfectly outline the methods for using it in any writing program. So far, I've breifly scanned through the text. However, I do think that it will offer some valuable insight on why schools who use GSP tend to stay true to what Royer and Gilles recommend. EMU has wandered a little down the beaten path with its methods, which may be good or bad; we don't know for sure yet. I will have to further consult this text to form a more clear opinion about the issue.

Armstrong, William B. "Validating Placement Tests in the Community College: The Role of Test Scores, Biographic Data, and Grade Variation." Annual Forum of the Association for Institutional Research Boston, MA 28-31 May 1995.

This study looks at the validity of placement tests and how the scores of these tests relate to student grades. The main thing that Armstrong found was that these placement tests did not predict grades significantly because there are too many outside variables that determine final grades for English courses. Some of these variables were age, gender, previous knowledge, committment to learning the material, instructor variations, etc. This study makes me wonder about how the GSP questionaires would predict how well students do in the course. Judging by this study, it seems superfluous to look at this for my own study. I think it further convinces me that I need to modify some aspects of my study to include writing self-efficacy in comparison to placement strategies.

Luna, Andrea. "A Voice in the Decision: Self-Evaluation in the Freshman English Placement Process." Reading and Writing Quarterly 19 (2003): 377-392.

At Lyndon State College writing program administrators tried placing students into freshman courses by letting them choose the course right for them AND then writing an essay defending that choice. I think this is a great idea, but one that probably works best in a small school like LSC. I'm sure it may be more difficult for something of this nature to be implemented at EMU, but still something to think about as we move toward the GSP-type procedure for fycomp. What researchers noticed were that some students would sell themselves short when deciding which course to enroll in. They may have said in their essay that they lack organization skills, but the essay itself was clearly organized. In this case, advisors compensated and placed these students in course that they thought would be better. Luna feels that this "lack of confidence" stems from the fact that students have little experience as self-evaluators of their writing. I agree. This study further confirms my questions about how well students will place themselves in the GSP method. Will our students sell themselves short as well? Maybe this is something to look at in-depth for my study. Do students' writing self-efficacy predict the course they choose to enroll in?

Borrowman, Shane. "The Trinity of Portfolio Placement: Validity, Reliability, and Curriculum Reform." Writing Program Administration 22 (1999) 7-25.

Borrowman discusses the importance of portfolio assessment for fycomp placement and how it is conducted at the University of Arizona. This program of study reitterates similar thoughts on portfolio placement that I have found in my research. What is interesting is that this program somehow has collaborated with high school teachers where both secondary and post-secondary writing instructors score the portfolios for placement into composition courses. I'm not sure if this is the best idea, unless secondary teachers are familiar with the program used at UA. I think the most important aspect of this placement procedure is that is relates to the way in which comp studies is taught at the university. This placement strategy is one of few that seem to correlate so well with what will be used in the classroom. Doing it this way is reason for making accurate placement decisions for students.

Sunday, March 20, 2005

Getting there...

Wachholz, Patricia B. and Carol Etheridge. "Speaking for Themselves: Writing Self-Efficacy Beliefs of High- and Low-Apprehensive Writers." ERIC Document Reproduction Service, ED 403 563.

These researchers examined the differences in writing self-efficacy in freshman students with either high or low apprehensive feelings about writing. Like many other studies on self-efficacy and writing, performance is greatly related to a student's sense of self-efficacy. They used the Daly-Miller Writing Apprehension Test with follow-up interviews of 5 high-apprehensive students and 5 low-apprehensive students. The study suggest that it is important for instructors to be aware of these students and their senses of self-efficacy in order to help all of their students achieve their writing goals. I wonder if it would be helpful to administer the Daly-Miller WAT to separate students who feel apprehensive into classrooms with peers who feel similar. The same goes for students who are not as apprehensive. The problem is that these groups are still relatively small in comparison to the vast majority of freshman students. Is the GSP questionaire indicative of apprehensive feelings of incoming freshmen? There is a connection between self-efficacy and performance, but does this guarantee that all students will make the right decision with respect to their placement in freshman composition courses? I'm not sure.

Harrington, Susanmarie et al. "The Influence of Word Processing on English Placement Test Results." Computers and Composition 17 (2000): 197-210.

This article focuses on a study that explores the impact of technology on placement testing for freshman composition. The researchers wondered if students would do better on placement tests if they were word processed, and if raters would rate tests higher if word processed. While not directly related to GSP, this study raises some interesting issues about the placement procedure and the need for making decisions based on more than impromtu essays. The researchers found that there was no difference in score between tests that were typed and tests that were hand-written. In both cases, students were placed in courses regardless of the format they completed there tests in. This study relates to GSP because it shows that placement cannot be accurately determined on impromptu essays alone. Researchers even recommend that "placement tests must match up not only with the curricula into which they place students, but also with the prior writing practices of the students who will take them" (205). Perhaps, if administered in a time-efficient way, placement would work best based on combination of both GSP and impromptu essays.

Zimmerman, Barry and Albert Bandura. "Impact of Self-Regulatory Influences on Writing Course Attainment." American Educational Research Journal 31 (1994): 845-862.

This article focuses on self-efficacy levels in freshman comp students. The researchers conducted a semi-field study where they did not control for certain variables. They administered a questionaire to determine students' efficacy levels about writing, which may be useful if I decide to create my own questionaire for my study. They found that students had high self-efficacy when it came to using sources in their papers, but low efficacy when it came to getting work done with distrations. The first result seems a little unbelievable when I compare it to my first year students, but the latter seems to fit well with what I've seen so far. Zimmerman and Bandura recommend that first year instructors evaluate self-efficacy at the beginning of each semester in order to meet their students at the appropriate level. Maybe I need to see if the GSP questionaire is a measure of self-efficacy and therefore omits the need for testing self-efficacy levels of students who have already been placed in English 120 or 121. Maybe this is how I can modify my study....

More annotations...

Blakesley, David. "Directed Self-Placement in the University." Writing Program Administration 25 (2002): 9-39.

This article is one of few sources that directly relate to my research topic. However, it still seems a little general for what I was looking for. The article outlines what we should consider during the process of implementing a directed self-placement procedure for incoming freshmen. Concerns about the reactions and pressure from administrators, stakeholders, and academic advisors are presented throughout this work. Much of what Blakesley describes seems to relate to what the first-year writing committee has had to face this year. One problem that Blakesley describes is the ability to prove that this form of placement works or works better than what we have available now. Theoretically it works, but that isn't enough to convince those at the top or even those around us. He mentions looking at grade distribution and following particular remedial students as they move through college. But this will take time and funding that is not readily available. We want an answer now and a concrete answer so that we can make adjustments to this placement procedure if need be. Unfortunately, this article did not give me the answers I was hoping to find on this topic.

Willard-Traub, Margaret et al. "The Development of Large-Scale Portfolio Placement Assessment at The University of Michigan: 1992-1998." Assessing Writing 6 (1999): 41-84.

This article takes a look at the introduction of portfolio assessment for incoming freshmen at U of M. During the 6 years this placement procedure took place, writing program administrators came across great advancements and pitfalls in the system. I originally thought that high school students would have difficulty meeting the requirements for these portfolios, but according to the article 88% successfully submitted portfolios for placement. I can see how this form of placement would work at U of M because this school tends to recruit students from public schools that have strong academic programs. Eastern, on the other hand, does not attract the same type of student from the same school districts. Granted there are exceptions, but if Eastern were to use portfolio assessment for placement, I feel that students would have difficulty meeting the basic requirements. One thing I am unsure of is why U of M does not use this assessment for placement today. I can guess that time constraints, funding, and the lack of resources in other areas contributed to dropping this format of placing students in writing courses. The thing that I like about this method is that it, like GSP puts the student in more control of her academic future. Allowing for choices, whether it be through GSP or portfolio material seems to be essential in order for self-efficacy to be strong throughout the student body.

Lavelle, Ellen and Nancy Zuercher. 1999. "University Students' Beliefs about Writing and Writing Approaches." ERIC Document Reproduction Service, ED 434 541.

This study looks at the approaches students take toward writing and catagorizes them. Students were either elaborative, low self-efficacy, reflective-revisionist, spontaneous-impulsive, or procedural. Through a format of surveys and semi-formal interviews researchers explored the nature of students in freshman composition courses. This study was interesting because it gave me an idea of the sort of questions I may ask students in my research (if I do a survey, and I'm still unsure about that). These researchers asked questions like:
1) Who are you as a writer?
2) What types of writing tasks do you prefer? Why?
3) Describe your experience of writing. Does your thinking change in writing? Your interpretation of the task?
4) Are you concerned about how much time your writing task takes?
I wonder if student answers to these questions would correlate to their choices to enroll in English 120 or 121? And if asked these questions at the beginning and end of their first semester of composition, would the answers change much? This seems to be somewhat important to GSP because it emphasizes the importance of asking students to reflect on themselves as writers. The study does not talk about grades and whether students of one category of writer tend to succeed more than other students in a different category. I think I'm getting a little too off-track from my original research question here, though.

Campillo, Magda and Suzanne Pool. "Improving Writing Proficiency through Self-Efficacy Training." Annual Meeting of the American Educational Research Association, 19-23 Apr. 1999 Montreal, Quebec.

This report shows the results of a pilot program that involved teaching self-efficacy to a group of "at-risk" students who failed the writing proficiency placement exam. They found that students in this test group, when asked to retake the placement exam at the end of the summer did better than those who weren't in the test group. They didn't find a cause-effect relationship, but they are interesting results. The problem I have with this study is that students were asked to reflect on homework assignments and practice placement tests to learn how to be more accurate in their writing self-concepts. This study seems to imply that students are not necessarily better writers at the end, but better judges of their abilities when it comes to practice tests. I wonder how many students would benefit from self-efficacy training at Eastern. Do we have more "at-risk" students than other schools? If so, will incoming students not previously trained in self-efficacy inaccurately place themselves in composition courses? This study does say that these students tend to think they are doing better than they actually are. Is it possible that a majority of our students (if "at-risk") will be the same way?

Pajares, M. Frank and Margaret J. Johnson. "Confidence and Competence in Writing: The Role of Self-Efficacy, Outcome Expectancy, and Apprehension." Annual Meeting of the American Educational Research Association, 12-16 Apr. 1993 Atlanta, GA.

Pajares and Johnson study a group of undergraduate students who are on track to be elementary English instructors and how their writing self-efficacy impacts writing performance, among other things. Though they found correlations as they expected, they said that writing skills self-efficacy did not change throughout the one semester that these students were followed. They feel that because the course being taught was not a composition course per se where writing skills were focused on, it is no wonder that there were no changes in self-efficacy in this department. It would seem that self-efficacy can only be improved if instruction actively addresses the issue. If not asked to actively reflect on one's self-efficacy in writing, students have difficulty expressing their thoughts or accurately placing themselves at a level of self-efficacy that is representative of what they actually accomplish. This research further restates my concern that some students (who have not been actively instructed to think of themselves as writers) will have difficulty determining the composition course that is best for their skill level with GSP.

Monday, March 14, 2005

And the beat goes on...

Matzen, Richard N. and Jeff E. Hoyt. "Basic writing placement with holistically scored essays: Research evidence." Journal of Developmental Education 28 (2004): 1, 2-4, 6, 8, 10, 12, 34.

This study focuses on using time-based essays to place students into first year composition courses. It compares the accuracy of placement between time-based and the formerly used multiple choice test format. They found that while the time-based essay format was better, it could not replace the multiple choice format and would be best used in addition to the original format. My problem with this study is that it was conducted with a small number of students (far from the size of a new freshman class). Even though the researchers mention this, they do not discuss the implications of cost or time with respect to using the time-based essay placement strategy. Sure it works with 400 or so students, but what about a number many times the size of that? Should only small universities benefit from effective placement strategies? Ideally if there was more communication between university and secondary ed. instructors/administrators the problem could be aleviated. Unfortunately, it is easier said than done.


Holmsten, Vicki. "Report from the Invisible: A Teacher-Research Project in Evaluation in a Community College Basic Writing Classroom." Annual Meeting of the National Council of Teachers of English Conference on College Composition and Communication. Atlanta, GA 24-27 Mar. 1999.

This report on a teacher-research project looks at possible ways to judge when a student is ready to move on to a regular college writing class. Though her research focuses mainly on basic/remedial college writers, the most interesting part is that her students did have a sense of their own writing skills early on. You would think that it would be difficult for remedial writers to accurately assess themselves about their level of writing because they tend to struggle with writing itself. Yet, Holmsten does say that there was a correlation between her thoughts of where her students should go next and thoughts her students had about where they should go. Not only did the students accurately determine their readiness for the next level, but a majority passed the next writing course with grades of Cs or better. The only thing I wonder is whether the fact that she mentioned her study at the beginning of the semester had an effect on her results. If students knew that she was thinking in this direction, they may have been more conditioned to be self-reflective, which led to a good sense of direction by the end of the semester. What happens if students are asked to place themselves and they have never been asked or taught to be self-reflective about their writing? I see this being a problem for students who are coming from secondary schools where it is still sometimes like Friere's "banking concept" of instruction.