Transcript of the talk, from the video's captions. Auto-generated: speaker names in particular are unreliable. ======================================================================== # Will The University Endowment Tax Slow Scientific Progress? Evidence from Elite Economics PhD Programs Authors: Joshua Angrist, Marc Diederichs, Glenn Ellison Discussant: None Video: https://www.youtube.com/watch?v=7LDwpGoPNFc&t=2761s ## Talk (00:46:01 – 01:25:59) [00:46:01] Delighted to be speaking here. I should say this is a joint work with, uh, Josh Angest and Mark Dederix, who are both here, Josh in the back, Mark up here in the front. Um, so anyway, the paper's title is, will university endowment tax slow scientific progress? Evidence from [00:46:15] elite economics PhD programs. Uh I think of this as having both sort of a particular policy related motivation and a more general motivation related to this conference. You know the policy motivation is that the one big beautiful bill act sharply increased the private [00:46:30] university endowment income tax. Uh five schools are it's it's really a tax mostly on five schools who will pay almost 80% of the taxes that get paid in in the long run. Uh the tax bills these universities in steady state could exceed 100% of their tuition revenues. [00:46:44] So it's a very large tax. In addition to the to the endowment tax, top universities are facing a number of other budget challenges. There have been big cutbacks to NSF and NIH funding. Uh a number of the universities uh like [00:46:57] University of California, Northwestern have also been sued for anti-semitism uh and face large financial penalties. [00:47:04] These create challenges for top universities. In response, we know that Harvard and MIT have cut back on their PhD admissions. MIT announced it was a 20% uh PhD cut back univers on average universitywide. Uh Harvard we know cut [00:47:19] back admissions by 50% in some departments. Uh the question is you know will admissions cuts uh at elite universities have long run impacts on scientific progress? We would like to answer that question more generally. This paper is going to be [00:47:33] about economics because that's what our uh data is about. So as an economics simple question is you know do elite economics programs have causal value added such that if you reduce their admissions you would reduce longrun production of economic research. It's [00:47:47] also sort of you know more generally this related to the economics of talent. [00:47:50] You know it's an idea that's been come up in several places that academia star researchers can have a very large impact. Our data is going to provide us with an opportunity to look at these questions. You know how concentrated is the production of top economics research? How predictable is success? [00:48:05] Are some schools better than others at developing talent? Um but so the you know focal question of the paper uh is a causal question. Do elite economics PhD programs have causal value added? Um we're going to focus on eight schools [00:48:19] five schools that pay the high highest endowment tax rate which is Harvard, MIT, Princeton, Stanford, Yale uh plus Chicago, Berkeley Northwestern. Why the focus on these eight schools? There was a prior paper by Amir and Canal from [00:48:33] 2008 that said that 75% of the faculty at the top 10 US uh economics departments got their PhDs from one of these eight schools. Um reinforcing that we're also going to find that sort of graduates of these [00:48:47] eight schools publish more in top journals than do students from other schools. Um, you know, the fact that you have sort of just eight schools doing a lot of the PhD training of top faculty then says that, you know, it could be that cuts to enrollment at those schools [00:49:01] could have big long big uh substantial uh effects. Uh, but we would say, you know, not necessarily. Why not necessarily? Well, one is, you know, the previous literature has emphasized it's really hard to predict who's going to [00:49:13] succeed in a PhD program. Um, but you know, it could be that there's more predictability than people were aware of and that these programs do well not because the programs have value added, but because they they attract very [00:49:25] talented students. Um, you know, if it if it's just they're attracting talented students, you could send those talented students who don't get into Harvard anymore to Michigan and everything may be fine. It's also the case that it really matters why the schools are doing well, even if they have causal value added. They have [00:49:40] causal value add only because you go to MIT and you get Niga and Ahmed as your as your classmates and having talented classmates helps you be productive. Then again if you sort of cut back and just those students go to Michigan and then [00:49:54] they have colleagues at Michigan who do well again you need to know what the mechanisms are to understand whether there would be sort of consequences from forcing these top schools to cut back on their PhD programs. [00:50:04] Um so preview of the study design and identification. Uh we're going to have a novel sample of 2582 highly rated MIT PhD applicants uh who applied in some point between 1996 in [00:50:17] 2019. Uh it's going to be sort of a Dale Krueger style um uh identification strategy where we're going to have both controls for how students were rated and how they looked at the time of their application and the schools they applied [00:50:31] to as a sort of a symbol of their ambition. Um you know I guess Josh gets to make up terms like this. They're bad controls. They're good controls. They're awesome controls. Um Josh can tell you why we have awesome controls in this paper, but when you have awesome [00:50:45] controls, they make all the other controls irrelevant. Uh and and we're going to argue that sort of the MIT admissions rankings uh serve as that in this paper. Uh and then okay, so related studies, I've got a limited period of [00:50:59] time. So, you know, we're related to several different literatures, but I guess the most simplest thing to think about what we're doing that's different from what people have done before is as far as we know, we're the first paper to use PhD applicant data to estimate school value [00:51:11] added and how value added versus research success of the students who enter PhD programs. [00:51:18] Uh, okay. So uh data the analysis sample is going to combine information on about 150 to 200 applicants per year uh from several application cohorts 1996 1997 [00:51:31] 2005 to 2010 and 2012 to 2019. The year restrictions are simply because of older data had gotten thrown away corrupted whatever these were the years in which we could uh historically find the data. [00:51:43] Uh what we get is one source is going to be MIT admissions records. The MIT admissions records are going to tell us how the faculty who read the folder rated it at the time, whether the student was admitted. We have some demographic data. Uh we're then matching [00:51:58] these MIT application records to um try to sort of track the subsequent careers of these students. So we figure out where they went to school, when they graduated. Um tremendous amount of work. [00:52:12] A lot of this builds off work that uh Josh and Mark did for a different project, but we're matching onto that data set. Uh we then also matched the students publication records and we matched them to ECON lit uh and we figure out what what they published [00:52:25] through their entire career. Uh we also matched the data on NSF graduate fellowship awards and we also for the published papers we're going to track the any paper that gets published, we know how often we know how much it was cited. [00:52:39] Yes, >> think of those as measures of productivity that are that are not confounded by which school you graduate from. Like there's no signaling value in the publication processing value of graduating from [00:52:54] >> Yeah. Well, so we are going to do some things to try to look at whether we think that there's signaling value to get to what PH where you got your PhD in terms of publications, but we're mostly going to be thinking about the publications as a measure of your research output. And we're going to think of papers in top five journals as [00:53:08] top five quality papers and we're going to mostly be using the publication data as if you know but we will sort of talk about there there'll be a section where we talk about journal favoritism and other factors. Um okay so anyway the you know two main variables I want to talk [00:53:21] about one is this application rank where this application rank data comes from is at MIT promising files are read by at least two faculty members. uh if the first two faculty members who read a file judge it to be particularly [00:53:36] promising, then it gets passed on to the full committee and three or four or five or six people read it. Um so what we're going to be looking at is the students who got to this finalist stage. So it's 150 to 200 students who were judged [00:53:49] promising enough that every all the uh readers read the folder. Um we let me say and you know 15200 just for scale MIT gets more than 200 applicants a year who have a perfect quantitative GRE [00:54:03] score. So these are very strong applicants already and we're taking the top 150 to 200 of them. Uh where average ratings across the readers construct a within cohort rank. These ranks are not what admissions decisions were based on. [00:54:16] These ranks were at a preliminary stage where everyone had read the folder and rated them. Who actually gets in? You know, MIT at the time only had funding for about half the students. So, they were often going down to rank 100, 150, 200 to get students who brought their [00:54:30] own funding with them instead of needing university funding. Uh there's also sort of uh balancing across fields. You want to have some number of development economists, some number of labor economists, some number of trade economists, you balance across fields and and other things. Um so anyway, but [00:54:45] these rankings are they're from a preliminary stage. And then as far as research outcomes, the main thing we're going to do is um you know in a previous paper you know we constructed this set of journals paper had deep impact in the [00:54:58] title. So uh publications in 137 journals. For each of these 137 journals we have a influence weight for the journal which is based on how often top five journals site articles in that uh [00:55:11] in that journal. Um let me see. I'm a little skip scared of clicking back and forth. Oops. No, that didn't work. Um, okay. So, um, anyway, you know, what do the weights look like? The top five journals get a weight of one on average, although it's different across journals. [00:55:26] Review of economic studies is lower than the QJ and econometrica. Um, but you know, good fe good good second tier journals like AEJ policy maybe a 0.25, review of economics and statistics [00:55:40] would be a 0.2. Uh, you go to field journals like the journal of public economics that would be 0.15. So it's a waiting scheme that reflects how often every journal is cited by the top five journals and that's how we wait and we [00:55:51] wait 137. We also divide output by n. So if you have a five authored paper you only get one credit for that uh relative if you have a sole authored paper in one of these journals. Um however we do have several alternative publication [00:56:05] measures. So we'll also be presenting p tables that have counts of your papers in top five journals unweed counts of all publications in these 137 journals. [00:56:15] counts of publications in all journals. Um, and then we'll also have some citation data. [00:56:22] Uh, is my thing going forward or backwards? [00:56:26] Okay, now it's going forwards. Okay, so uh where do our uh top applicants end up going to school? Uh, where they end up going depends on the rank. Um, but the main thing we want to get out of this table is that there's a lot of overlap [00:56:39] in the schools that students at the same rank end up going to. So if you look at the uh left bar there, the left bar is students who are ranked first through fifth. Students who are ranked first through fifth essentially all go to MIT or Harvard or Stanford. You see a little [00:56:53] bit of other stuff there at the top, but it's pretty much those those schools. As you move down to 6 to 25, you see a lot of students going to MIT, which is the bottom bar there. A number more students going to Harvard and Princeton, Stanford, who are all in the the lighter [00:57:07] pink bar. Uh by the time you get to 25 to 50, you're getting lots of overlap. [00:57:12] You get lots of students in the 25 26 through 50 bar who are going to Princeton students who are going to Yale and then the gray there is where the gray bar that comes in and that those are the Berkeley Northwestern Chicago students. The orange bar that comes in [00:57:26] are the Colombia Penn NYU Michigan students. But you know if you think about at at the top you get a lot of evidence of Harvard versus MIT versus Stanford of similar students. And then as you get through the lower bars, you're getting lots of actually Harvard [00:57:40] and Stanford and Princeton and Yale and Colombia and Northwestern of students with very similar ranks. Okay. Also noteworthy is by the time you get down to 150 to 200, these are very talented [00:57:54] students and yet a number of them seem not to get go to PhD programs in the end. Um okay. uh publication concentration um publications in economics are [00:58:08] incredibly concentrated you know so we're looking at about 200 applicants those 200 applicants you can think about as those are 5% of the people who start a PhD by the time you take all the US and European students you if you count [00:58:21] China it's it's much smaller than that um but despite being only two 5% of the the applicants they produce eventually produce about half of the research in top journals and then if you look within in the top 200. Here's a Loren's curve [00:58:34] for how much publication gets done by those top 200. And you know, the top 20 out of the 200 produce probably 65% of the 200's uh output in top journals. So, it's a you know, you got 4,000 people [00:58:48] starting PhDs, but these 20 are producing a third of all of the output of that 4,000 cohort. Um so, it's a very concentrated uh output. uh if you look at it at the school level um there these [00:59:01] are you know I mentioned Amir and Canal have had the big differences in probabilities of ending up in a top 10 job these are publications uh as a function of which graduate school you attended uh so MIT is normalized to 100 uh students at Harvard [00:59:15] uh publish on average 72% as much as MIT students students at Princeton it's 60% students at Stanford it's 40% uh when you get to other US schools the other US schools are going to be the Colombia, [00:59:28] NYU, Penn, whatever tier that's 28%, the foreign schools, it's 19%. And and these are all remember these are all restricted to these top 200 applicants. [00:59:39] And so there are these big publication differences across schools. Um I've got one here that these differences are in the intensive and extensive margin. Um also sort of I guess you know striking evidence that we have not seen before. [00:59:53] There are tremendous differences in publications across publication rank. Uh if you look at students who have a rank from one to five in the admissions uh rankings, uh they on average have uh two [01:00:06] WDI publications and again two you know like if you have a four author uh QJ paper that's only giving you 0.25. So getting two is a you've done a lot of publishing to get a WDI measure of two. [01:00:19] Um, students who have ranks one through five end up publishing almost three times as much as students who have ranks six through 25. Students with ranks 6 through 25 publish twice as much as students with ranks 51 to 100. Students [01:00:33] with ranks 51 to 100 are like 70% of students 151 to 200. So, you know, top to bottom there, we're getting a more than 9:1 publication difference uh as a function of your admissions rank. Uh if you look [01:00:47] at probabilities of extreme success, you also get these ex big big big differences. Students in one through five have a 40% chance of being in the expost top 10 in their cohort. Students 151 to 200 have a 2% chance. Um so what [01:01:01] we're get saying is there's a tremendous amount of predictability in success in economics. Uh which then makes it more plausible that maybe those difference in publications across schools are differences in x anti- studentent characteristics not differences in value [01:01:15] added. Um other thing say though is you know people there are a lot of papers talk about predictability and papers that talk about how hard it is to predict things. Um you also see the non-predictability is here too because there are many more 151 to 200 students [01:01:27] than there are 1 to5 students. So if you look at who is the expost top 10 in an application's cohort. It's it's like evenly spread out. It's two students who were 1 to5 on the way in. Two students were 6 to 25. Two students were 26 to [01:01:42] 50. Two students were 50 to 100. two students were 101 to 200. So the expost top 10 is very spread out because there are so many more 101 to 200 students than there are one to five students. [01:01:56] Um and you know the publications show up there more in the uh WDI publications than in the all econ publications but you still see you still see the gaps in the all econ publications. [01:02:10] Oh, I guess the to give you a scale, right? The the top five count, you know, there we're not dividing by co-authorship uh or whatever and and we're just counting this as one. So that's saying the typical one through five student ends up with three and a half top five [01:02:24] publications in their career. Yeah. >> Given that being in the one to five category is a very strong predictor of success, >> it means that you guys were very good at ranking those students in particular. [01:02:36] >> Yeah. what made them so special like this like math Olympian winners etc. [01:02:43] >> Yeah. you know so >> interview before >> so you know so obviously there right there are some people who had their undergraduate thesis came out in econometrica and it it hadn't come out in econometrica yet but we had seen that [01:02:55] paper um but you know it is a mix of is people with you know it is a bunch of people who have taken the PhD courses at Harvard and gotten A+es in them or people who have written papers that are [01:03:08] observably very good or there's something unusual there that makes you think this person is going to be really outstanding. Yes. Question have some signal of doing research but [01:03:23] also kind of maybe have some signal just knowing what the top people are doing this recursive way. Is there a way to sort of break free of that using like an judge approach? Should we write down what is your decision over a right research or important research in a way [01:03:36] that doesn't look like published in a top five but maybe look like doing something that humanity or it changes the way that economist view or something like that and then have the LLM go through and see whether people research look like that not necessarily [01:03:50] >> yeah I don't know it would be interesting if one could try to sort of like you know obviously there's an awful lot of effort that goes into journals selecting papers and you know what what we'll also show you that like it's also true it's not just people are publishing in top journals If anything, their papers they're publishing in the top journals get cited more than the other [01:04:05] papers that are in the top journals. So there there's it's definitely evidence that our profession is valuing what these people what these people are producing. Uh but you know it would be interesting to see if we can you know I'll show you some things about bias. Um [01:04:19] we think there definitely is sort of some sort of insider bias here. Uh but I will it would be interesting if one could do this without inter incorporating the sort of journals. [01:04:29] when it's hard to talk about bias, you know, defin what has been researched. I haven't seen that. [01:04:37] >> Yeah. Okay. Let me see. And just uh again, here's here's just some sort of histograms of descriptive statistics by admissions rank. You'll see that there are differences both in the intensive and extensive margin. You look at the top five people. They're almost sure to [01:04:50] publish something. And then, you know, the the gray bars are people who who never publish anything in a top 137 journal. Uh but then the other big thing that shows up there is those sort of you know the the median person in rank one through five ends up with at least three [01:05:05] top five publications. So this is sort of lots of people who are achieving very high levels of success. Um okay anyway but you know the publication is sufficiently concentrated that means three things can simultaneously be true. [01:05:18] There can be large predictable components of success. There can be large differences in value added across schools. There can be a large unpredictable component. All three of these things can be true simultaneously. [01:05:28] The summary statistics tell us that either one or two has to be there. [01:05:32] Seeing the successes of the top students and the students at the top schools tells you either the top some students are really talented, they go to the top schools, they publish a lot or there's no talent but the students at the top go [01:05:46] to the top schools. The top schools have great value added. But one of the two has to be true. Uh but it's obviously which to what extent it's true is really important for thinking about how cutbacks top schools will affect um the future. Okay. Uh so what's our value [01:06:00] added framework? Uh this is you know standard way that Josh has thought about value added uh for other things before. [01:06:06] So we're going to think of the number of uh you know students we measure research outcome K which you can think of as for instance publications in top journals. [01:06:14] Um we're going to assume that your students output is uh e to the beta not k which is what they would produce if they go to an outside good school. Um and then plus an epsilon i ep epsilon i not k which is sort of the randomness [01:06:29] due to that student being more or less productive than the average student those characteristics and then we have a causal effect uh e to the beta j of attending school j and so we have the sort of standard value added model. Um, [01:06:42] if you look at D as an indicator of what school students go to, you get the observed outcome, which is just the sum of the indicator variables for which school you go to times what your potential outcome would be at that school. Um, you know, this is going to [01:06:55] have a selection bias problem in it and that the expectation of new IK conditional on the school you attend is non zero because the schools have some observable uh about the student. Um, what's our identifying assumption? And [01:07:09] our identifying assumption is that the expectation of the student specific productivity conditional on the student attributes and on the decision they make is independent of the decision they make. That is the decision they make is [01:07:23] not informative and that the expected output as a function as a function of the student characteristics is described by just e to the x time gamma where x are the observables characteristics we [01:07:37] have. So, we're basically assuming that the MIT admissions rank and the NSF uh fellowship award and the other many other right-hand side variables sufficiently define the student quality so that there's now no longer uh an [01:07:51] unobserved student effect that's sort of based on their decision. Uh we're going to do this in grouped models where rather than trying to estimate the value added of going to MIT, what we're going to do is evaluate the value added of T1 going to a top five going to a high tax [01:08:05] school. Uh that's the Harvard, MIT, Princeton, Stanford, Yale that are being targeted by the one big beautiful bill act. T2 is the other elite eight schools. So that's Berkeley, a dummy for going to Berkeley, Northwestern or [01:08:19] Chicago. Uh and then F is a dummy for going to a foreign school. The omitted group is all of the quote other US schools. So the other US schools are um you know again Colombia, NYU, Michigan, [01:08:33] UCLA, UC San Diego, all the schools that places students go to if they're not going to one of the eight eight elite schools. Uh it's going to be estimated by plus on pseudo MLE robust standard errors. Okay. [01:08:48] >> Yes. you have set you have the full set of schools that they applied to that you can condition on or >> um so in in some of the years we have the full set of schools that students applied to and some of the years we don't. So we're going to run the main analysis without that and then we're [01:09:02] going to have sort of a second chart we show here's this tools where we have this student the application sets uh and I said these things already okay so anyway controls all models include as controls demographics like whether a US [01:09:17] citizen or permanent resident uh gender dummy application year dummies dummies for not finishing a PhD uh the awesome controls are the MIT admissions rank uh including the rank interlinearly quadratically as [01:09:32] a reciprocal a dummy for being admitted to MIT a dummy for being awarded in NSF dummies for all the schools to which you applied uh and it turns out that actually just one control matters which is the reciprocal of the rank so just [01:09:45] take the rank take one over that that's what really matters and what really predicts output um uh I'm going to show one table that has all the awesome controls and you'll see it's really just the reciprocal rank that matters and then most the other tables to shorten the tables and make them easier to read. [01:10:00] Uh I've cut down to limiting that. So here's here's a table with um uh you know, so each of these columns is a different measure of publication. Um I've got the first three rows are dummies for whether you attended a high [01:10:14] tax school, an other elite 8 school, a foreign school, and then the other rows are the um uh controls. And just over here, the uh you know, you'll see the blue rank has many stars on it. it's [01:10:28] always got an enormous t statistic. Uh once you put it in all the other things, the rank, the rank squared, the NSF winner, the attended MIT, they all have relatively small effects and they're all always insignificant in all the regressions. Um so anyway, here's a [01:10:41] chart where I just show you the results blown up a little bit so you can read them. Uh and so this first result on value added is that you know we think that high tax schools are boosting student publications. Um if you didn't [01:10:56] include the rank controls, you would have thought that high tax schools um boost publications by 140%, e to the point.88 is like 2.4. Um with the admissions controls, uh we think that [01:11:08] about half of the 140% is is the sort of student selection. And so high tax schools boost student publications by about 75%. [01:11:17] Um but we think it is 75 we think it is 75% as a value added. uh in the second row when you look at these sort of students attending the other elite schools if you leave out the controls and say what's the value added of going [01:11:30] to Chicago Berkeley Northwestern you'd get E to the point49 we put all the controls in and it's e to the point47 and the argument is that the students going to Berkeley Chicago Northwestern don't look very different from the [01:11:43] students who are going uh to the other US schools um students going to the foreign schools end up publishing less in the WDI journals okay Um what where are these effects located? If you look [01:11:56] at students publications in all every publication in everything on econ lit uh we find smaller effects we get their effects are about the the high tax schools the other elite schools are boosting publications by about 20%. So [01:12:09] the 56 points is coming say 22 from boosting the number of publications and then 34 from shifting your publications from lower ranked journals to higher ranked journals and giving you whatever tools you need to publish in the [01:12:23] journals that get more widely cited. Uh if you look at the top five publications, those show up with an even bigger coefficient. And then as you know just goes down as you look at sort of the publications in any of the 137 journals that's smaller publications in [01:12:38] anything on econ 137 that's smaller. If you look at a dummy variable for do you ever publish anything anywhere um just one or zero there we're finding no effect of the high tax schools. That's not what [01:12:50] they're doing. Okay. Um okay. Um, you know, Dale and Krueger have a, you know, well-known paper arguing that, uh, self-revelation controls can be a very valuable way to [01:13:04] Oh, yes. Question in the back. >> I was wondering the previous coefficient. [01:13:10] I would like to know that it's $1 million, you know, then million dollar less or like money. [01:13:22] >> Yeah. So I I um uh I don't know yet. So there are papers that have done sort of like you know from this on the student's private benefit what is your salary as a function of your publications and what are sort of students lifetime wages. I [01:13:36] haven't done that back of the envelope calculation to sort of say what's the student benefit there and then obviously you could do the you know social welfare calculation of this research that doesn't get created how much government tax revenues are the is the government [01:13:51] taking in from these schools and do a tax benefit versus cost tradeoff. Uh we haven't done that calculation either but it probably would be something good for us to add to conclusion. Yes. [01:14:11] Um, so you know, we do have some supporting tables in the back that look at uh things like, you know, is it different for US citizen permanent residents versus non- US citizens? Is it [01:14:24] different for men versus women? Uh, most those tables have a whole lot of just insignificant stuff and we're not finding anything there. Um do we have one with um >> with the rank interaction? [01:14:43] Yeah, it's it's it's I think that the the sort of multiplicative model of just you know rank effect time, you know, like your rank effect times your school effect. We're not finding anything more significant than that. Yes. [01:14:55] >> Well, I had problem to underra and department. [01:15:06] >> No, no. So, yeah, unfortunately, we we we don't have anything, you know, all we have is these admissions ranks. We don't we don't have access to the student like, you know, we don't still have access to the student admissions files and could send some machine learning thing to read through them and create [01:15:20] attributes for why was this student ranked highly. All we know is that they were ranked highly by a committee uh you know, 20 plus, you know, some some of these are 30 years ago. Um, yes, Josh. [01:15:47] salaries. >> Yeah. So, so if if you see here the like the high tax schools are.56 the other leader of 47. So without the controls the 888 and 049 look very different with [01:16:01] the controls the.56 and 47 there we're not able to statistically say those are different from each other. Okay. Okay. [01:16:09] Yep. Uh okay. Anyway, so Dale Krueger style controls, you know, Dale and Krueger argued that sort of where students apply to school is highly informative. You know, you have students applying to Harvard may tell you this student is very ambitious. A student applying to Northwestern may tell you that this student likes technical work [01:16:24] or they're very committed or they're riskaverse or whatever. Um, so that for a subset of the data, we do have application lists and we can put in dummy variables for what schools you apply to. Um, when we do that, if anything, it actually makes the value [01:16:38] added estimates larger. Um so the on this subset you get a 61 value added for high tax schools without the the Dale Krueger style controls you put them in you get a 684 on the other elite 0.54. So where we [01:16:53] have that data it seems like putting in the dummy variables for where the students applied if anything makes the value addeds look bigger uh than they did before. [01:17:03] Um okay mechanisms. So you know two main mechanisms we want to talk about because if either of these mechanisms is what's behind the results then you get a very different conclusion about what would be the aggregate impact of cutting back on admissions to top schools. One is is it [01:17:18] pure effects and the other is is it home journal favoritism. Uh the pure effects channel would be that you know you go to Harvard and MIT and you benefit from having very strong classmates at Harvard and MIT. Therefore, if Harvard just took fewer though, you'd have good classmates [01:17:32] wherever else you went and things might not change. [01:17:39] >> Um, so, um, Josh and Mark have a separate paper where they talk about advisers themselves. I don't know Josh, do you want to say like 30 seconds on advisors or >> it it it's [01:17:53] >> Yes. So um here um anyway so you know here we have sort of there is idiosyncratic variation in the peer effects from in the peers from year to year. So for instance MIT in 2010 only enrolled two of its rank 1 to 10 [01:18:08] applicants. In 2016 it enrolled 10 of its rank one through 10 applicants. [01:18:12] Harvard in 1996 has four rank one through 25 applicants. In 1997 a year later it had 10. Uh so there is this idiosyncratic variation in the number of students who choose to go to each school. Uh we're going to try to use [01:18:26] that to look at peer effects and see what can we say about the peer effects. [01:18:30] Um here's the full table that has three different measures. Let me like this cut down table just the top half of the table was doing how many of the top 10 students went to your school. the bottom [01:18:43] uh table panel is doing of the top 200 weighted by their inverse ranks what fraction went to your school in your year. Uh and a basic bottom line here is that putting in these peer controls the [01:18:56] value added estimates basically don't change. You know the high tax value added goes from 0.59 to 0.56 when we put in the pure effect. The other elite one goes from 48 to 48. [01:19:08] >> Try the top five. Um, so we don't have the top five. We we could do the top five peers. We have the top 10 peers and then we have the top 200 peers weighted by inverse rank. So number one is getting a rank of one. Number two is getting five. Number three is getting [01:19:22] 33. So that's tipping it a lot towards a top five or 10 peers. Um, you know there's a top 25 one also, but basically you put in the peer effects and it looks like it doesn't make that much [01:19:34] difference. Um it is true that like if you didn't put in the school effects and you just put in peer effects, you would think peers really matter a lot. Uh but once you put in the the school effects, the peer effects just get very very [01:19:47] noisy. Um okay, other mechanism, home journal favoritism. There's some articles that say that there's tremendous favoritism in our publication process. The QJ is published at Harvard. [01:19:57] If you're a Harvard student, you get all kinds of QJ publications. um to do these regressions. Let me say I'm going to run these at the school like these are sort of school level biases. So we're going to run these regressions at the school level instead of at the sort of tax [01:20:12] group level. A quick preview of what school level results look like. Uh before I get to that question, you know, MIT and Harvard students as I as you saw in that very first chart are much much more productive. You put in the rank [01:20:25] controls and you see that Harvard and MIT's advantage over uh Princeton for instance largely disappears. That is much of what makes Harvard and MIT special does appear to be the rank of the students they apply. [01:20:38] Once you put those controls in the school effects, you know, get hard to distinguish, but obviously the high-tech schools there you can see still have very high coefficients. Okay. Um so anyway, here's my am I going to get [01:20:50] there? um Harvard MIT grad is doing well at the QJ. Um Harvard's coefficient on the QJ is 1.62. [01:20:58] Uh what that would say is that if you take students who look at Xanti identical, send one to Harvard, one to Stanford, the Harvard student publishes five times as many QJ papers. So yes, the QJ does publish many many papers by [01:21:12] Harvard graduate students. Also publishes almost as many papers by MIT graduate students. Uh if you look at the University of Chicago, do their students publish more in the JPE enormous effect of going to Chicago on your future JPE [01:21:26] publications? Aea papers and proceedings. Again, Aea papers and proceedings is typically organized by someone who's at a top university. Do their former students get A.E.A. papers and proceedings issues? It [01:21:40] looks like that's true. At least you get bigger coefficients on the P&P than you get on the reg on this regular WDI publications. You might ask, you know, are these things what's driving the elite school effect? Um, well, we just [01:21:54] take them out by saying, let's set the weights of the QJ, the JPE, and the AEA papers and proceedings to zero. Those journals now count for nothing. or or in the right column say all we care about is your publications in econometrica [01:22:08] um econometrica AER or review of economic studies and what you find is yes the sort of value addeds of these top schools go down somewhat but we're still finding tremendous value added for the top schools uh once even once you [01:22:23] take that out um uh another thing we ask is are these students papers in these top journals getting less cited than papers other schools. It may be that the, you know, that the QJ editor puts his former [01:22:37] students paper in there and it's not really a very good paper, no one pays attention to it. Uh, what we do is put this is your citations on a paper as a function of the journal in which it was published in the year in which it was published. Then put in were you did you [01:22:51] attend a high tax school or another elite school and what we find is that the papers by students from the high tax and other elite schools are cited more than the other papers in that same journal volume year. Uh so again speaking to this does not look like [01:23:04] favoritism unless it's favoritism coming through the your friends site you um channel. So discussions a simple back of the envelope calculation just about out of time but so you know conservative [01:23:15] back the envelope calculation would say >> we so we we can only do that of the you know the the our top 200 people we know where they all went to school. the other [01:23:30] people out there in the profession. We don't know where everyone went to school. Um so back to the MMO calculation, what happens if the elite 8 schools were forced to reduce enrollment by 20%. Uh this calculation says back the calculation is fraction of top five [01:23:44] quality papers it gets published by the profession goes down by about 5%. Um, so you know, I I guess that's the sort of trade-off the social planner wants to use is I cut back I make cutbacks that force these schools to cut back 20% of [01:23:57] their enrollment. Production in the long run of top five quality research goes down by 5%. What do we think of that as the tradeoff? Um, people have asked is this economics unusual? We know that economics is very highly concentrated [01:24:10] compared to many other disciplines. 75% of the faculty, the top 10 schools went to just eight graduate programs. But you look at many other disciplines, it's not 75% but it is 50 or 60 or something [01:24:23] percent. So whatever you know that precondition for attacks on a small number of schools to have a big aggregated impact appears to be there also in aerospace engineering, mechanical engineering, computer science, chemical engineering, uh many [01:24:38] other fields. Okay. So uh summary and conclusions um you know elite 8 grads produce like 60 to 70% more impact adjusted publications than comparable non- elite grads within the high tax group differences really moderate once [01:24:52] we control for rank and we can't really say that all the elites are equal. Um some of the Harvard MIT advantages home journal success but it doesn't seem like either pure composition or home journal connections explain the elite premium. [01:25:05] Uh we therefore think that the elite advantage likely reflects institutional faculty characteristics other things that enhance uh research training. Um anyway uh thank you very much inclusion goes 20 minutes. So we hold [01:25:28] questions till till the end if leaves uh you know it's up to you how much time you want to read for questions 15 minutes then >> okay I will try to do my best [01:25:42] >> yeah this clarification questions are Okay. [01:25:53] >> Okay. Perfect. Okay.