Friday, May 05, 2006

Does the sophomore economics tutorial a student takes influence his academic and/or career interests?


Generally, college students are not randomly placed into their courses; rather, they have some freedom to choose which courses they want to take. Consequently, it can be difficult for researchers to determine whether taking a particular course alters a student’s academic or career preferences or whether it is a student’s preferences that cause him to choose that course.

For the economics sophomore tutorial at Harvard, students rank their preferences and are then placed into a course through a lottery system. Because some students are placed in their first choice course while others are not, we can form an experimental group and a control group accordingly. For instance, there are a number of students who ranked “Everybody’s Doin’ it: Social Interactions and Economics” as their first choice. Among these students, those who got into Bryce’s class would serve as the experimental group, while those who were placed in a different tutorial would serve as the control group.

The idea is to determine whether there are any consistent differences between the academic and career preferences of the experimental group versus the control group. If so, because there is no reason to suspect bias, then we can attribute these differences to how the given tutorial is affecting the students who take it. My tentative hypothesis is as follows: “Students who are placed in their first choice tutorial will have different academic and career interests than students with the same first choice who were placed in a different tutorial.”

I will be gathering data using a survey I have created. Depending on how many responses I get, I may need to divide the tutorials into various categories, such as Behavioral, Legal, Finance, Healthcare, etc. If that is the case then my hypothesis will argue that students who are placed in a different type of tutorial than their first choice will have different academic and career interests than students who were placed in the same type of tutorial as their first choice.

Data/Empirical Strategy

The questions on the survey include: gender, house affiliation, race/ethnicity, occupation of mother and father, likeliness of attending graduate school or professional school, summer internship information, and likeliness to take certain courses while at Harvard. Most importantly, the survey asks the students to list their tutorial rankings as they did for the official lottery system and to state which tutorial they actually took. Accordingly, I will have many variables to work with, and I should be able to control for various factors when taking the regressions.

What my regression equations will look like depends on what kind of data I obtain from the surveys. At this point, it is likely that one of the equations will regress getting into one’s first choice tutorial on wanting to go to graduate school. Another equation would regress getting into one’s first choice tutorial against the likelihood of taking a particular class. After I get the data, the regression process will be much more concrete.

Concerns and Limitations

The biggest limitations have to do with the data collection process. First, because of the limited amount of time I will have to collect data, it is likely that the sample size will be relatively small. Secondly, I am collecting information about students’ tutorial choices based on their own memories of their rankings. While most students will probably remember their first and second choice, some students (particularly the seniors who took the course two years ago) will perhaps not remember their lower choices correctly.

As for the experimental design, a potential problem I foresee is that students who do not get into their first choice tutorial may be affected by the tutorial in which they are placed. If there is no bias in terms of the type of tutorial in which such students are placed, then the data will be virtually unaffected. If, for some reason, students with the same first choice often had similar types of tutorials for their lower choices, then this could present a problem.

Despite these limitations, this study should be able to tell us a great deal about how students are affected by the sophomore economics tutorials they take.

Interesting topic, I wonder if (ideally) one would need to control for other classes a student is taking. After all, if every student is taking 3-4 other classes in addition to tutorial, it seems possible that these classes could also affect a student's academic/career interests. Also, what if students who want to get into Bryce's class but don't change their other classes as a result? For instance, say someone was planning on taking the social interactions tutorial, a development economics class, a micro theory class, and a core. If they don't get into social interactions but rather, say, a poverty and economic development tutorial, they might choose not to take the development economics class they were going to take and instead enroll in something similar to social interactions and economics. If such behavior is common, it might be hard to find desired effect. Either way, I'm curious to see what you find.

This all looks good. You are correct in that your main problem is going to be with sample size. The small number of observations you will obtain (from an already small sample) is likely to make observing statistically significant results difficult. That is ok for this project though.
One more thing, I think the regression equations should look like this:

Y = a + b(in tutorial X) + c*(ranking of tutorial X) +[controls]'*d + e
I agree with Patrick that you may need to consider controlling for other classes. If you do not it is impossible to, with certainty, determine whether career choice was fully the treatment effect of tutorial. Other classes have lotteries as well and so students aren't always placed in their first choice classes for other subjects as well. You also may want to consider the classes students are forced to take, like the core curriculum which exposes one to material they would never consider taking otherwise and may then cause that student to change their career and interest path. These are just things to consider though I do not know how necessary they are. Otherwise I think it is an interesting idea. If your hypothesis proves to be corrext, it will be interesting to see how random assignment may completely alter the lives of certain students.
I think that another thing to watch out for is the reputation of the tutorial. Some tutors have former students that can relay information about the class. However, a lot of the tutorial picking is arbitrary. In that hour and a half meeting in Emerson, all of us see and judge each tutor based on the two minutes they are up on the podium. Some people may choose a tutorial because of the relative ease of the class and not necessarily the content that they wish to have. Or, they may pick a tutorial that one of their friends picked as their first choice. These things are really impossible to measure but should be considered.
I know that your data is limited, but I think the most valuable data would come from those students who were not placed in their first choice tutorial. This data would represent the most direct effect of tutorial on future career paths and other academic interests. Another variable you might want to take into account is location of the tutorial--even if it is a river=0, quad=1 kind of variable. I know that many people take classes close to their house (especially quad) even if the class itself is not their first choice.
In order to determine the true effect of the tutorial on academic/career interest, don't you have to examine prior interests? If you were to design an ideal experiment, I think you would need 2 surveys (one before and one after) asking students their career/academic interest level in the "Behavioral, Legal, Finance, and Healthcare" fields. And, again, I think it's very important to include the courses taken concurrently and in the past.
I have just added a Reference List to my economics blog with economic data series, history, bibliographies etc. for students & researchers.
Post a Comment

Subscribe to Post Comments [Atom]

<< Home

This page is powered by Blogger. Isn't yours?

Subscribe to Posts [Atom]