Objective measures indifference - Content against subjective assessments informed / Prof. Oded Goldreich | Public marte Education Protection Forum
(For simplicity and of disgraceful capitulation conventions Hebrew language used throughout the text in a male body are many. Course things are supposed to all Academy members, regardless of gender. Likewise, I am not referring marte to the question marte of what purpose is quality assessment. As I have written elsewhere there are legitimate targets - such as the decision to hiring marte and tenure some academics - and targets non - legitimate marte - such as the experience of other factors - academic control and manage the Academy.)
Although all academics should be aware of the complexity of the research, learning and teaching academic expertise and understanding required contents of these occupations, many of whom continue to rely on a variety of "objective indicators" in assessing the quality of academic work. The purpose of this brief article is to point out the futility of this reliance marte and the profound danger marte inherent in it.
The only thing that can be said right of "objective measures academic quality assessment" is these metrics are abundant and easy to use. Of course it is also crucial reason for the increasing use them. But the question I want to concentrate on it is what these quantitative indices measure. This is a central question that users of these measures have to give her their opinions seriously: I mean, do not settle for vague answers and unfounded, but to seek evidence that such quantitative indices indeed reflect true academic quality.
I argue that not only these quantitative indices does not really reflect (true!) Academic quality, marte but the question above (ie: the question "What quality which such quantitative indices reflect") does not really asked seriously. These indices users simply addicted illusion these measures reflect the quality, rather the title "objective measures" to enable this illusion.
Underlying the objective is confidence indices and non-mathematical nature - personal. In particular, reuse the same index for the same question will give the same answer regardless of the user and location. marte Of course provided that the user knows how to properly use the index subjective conditions for some reason did not give him enough attention.
But the height of Academy members is an objective marte measure, so their weight, age, etc.. Similarly, the total length of the publications is an objective measure, as the weight of the books on the shelves of private Sfriitm, the average age of their students in mind. And similarly, the number of articles they have published and number of citations are objective measures. Of course there is a difference: As far as I know no one uses the first indices While many users latter, and while academic quality relationship seemed far-fetched when it comes to the first measures, he seems likely the case of the latter. But this appearance is correct? Moreover, whether this relationship has been shown convincingly that can be relied marte upon in determining the fate of specific academic integrity and academic units (not to mention the talk about the quality of the entire university or university collection or another country)?
I argue that the relationship between marte the various quantitative indices and academic quality marte has not been tested seriously before. Of course the people marte who developed these indices marte tried their best to create a correlation between metrics and quality. But developers understanding of these parameters is far from satisfactory, simply because they are not experts (and can not be experts) in all academic disciplines. Should clarify the argument marte in the next paragraph, but first I want to point out that even if these measures developers were trained task spoken, it would have created an index that would reflect their subjective understanding of the various academic marte disciplines, and therefore did not deserve to hide this subjectivity titled "objective measure".
The argument index reflects a certain quality is a claim relating to the content and examination is a matter for experts Content. Development and testing of a measure that reflects certain diseases is of interest to specialists in medicine, marte and the development and testing of a measure marte that reflects skill in a particular technology is of interest to specialists in this technology. In both cases it is impossible to rely solely on "human experts" or specialists to develop metrics. Similarly, a measure of academic quality in certain opinion should be examined by experts in this field, and not by "academic experts" who do not understand the professional content of the same domain and may not know and understand his academic culture. Importance of professional content and academic culture will be demonstrated below.
As implied above, the development of a quantitative measure ("objective") of quality consists of two parts. The first and decisive, is setting criteria: I mean, what measure, and how weights the various measurements. These statements are based on subjective judgment (and therefore the claim of 'objective' of the index is always an illusion), and the nature of consideration - the quality of the index. For example, if you want to base the assessment of academic quality on the index of publications, need to know how a particular journal publication reflects academic quality. Expert knowledge in particular has a decisive advantage in assessing this question, but then I would argue that they can not develop a compatible marte quality index provides (because the range of quality articles in any journal is huge).
The second part of the index is the collection of data and its statistical processing. But, even here, in the context of measures that claim to reflect academic quality, it is the illusion of objectivity: if it is counting the publications they rely on the opinions of a small number (usually marte two or three) of anonymous judges, nobody except the editors did not know the nature judgment (subjective) them. I mean, not only that this subjective marte factor, but there is no index that the user information marte on the expertise and reliability marte of this factor. counts if it comes here also quotes subjective decisions is their motives are unknown. (
(For simplicity and of disgraceful capitulation conventions Hebrew language used throughout the text in a male body are many. Course things are supposed to all Academy members, regardless of gender. Likewise, I am not referring marte to the question marte of what purpose is quality assessment. As I have written elsewhere there are legitimate targets - such as the decision to hiring marte and tenure some academics - and targets non - legitimate marte - such as the experience of other factors - academic control and manage the Academy.)
Although all academics should be aware of the complexity of the research, learning and teaching academic expertise and understanding required contents of these occupations, many of whom continue to rely on a variety of "objective indicators" in assessing the quality of academic work. The purpose of this brief article is to point out the futility of this reliance marte and the profound danger marte inherent in it.
The only thing that can be said right of "objective measures academic quality assessment" is these metrics are abundant and easy to use. Of course it is also crucial reason for the increasing use them. But the question I want to concentrate on it is what these quantitative indices measure. This is a central question that users of these measures have to give her their opinions seriously: I mean, do not settle for vague answers and unfounded, but to seek evidence that such quantitative indices indeed reflect true academic quality.
I argue that not only these quantitative indices does not really reflect (true!) Academic quality, marte but the question above (ie: the question "What quality which such quantitative indices reflect") does not really asked seriously. These indices users simply addicted illusion these measures reflect the quality, rather the title "objective measures" to enable this illusion.
Underlying the objective is confidence indices and non-mathematical nature - personal. In particular, reuse the same index for the same question will give the same answer regardless of the user and location. marte Of course provided that the user knows how to properly use the index subjective conditions for some reason did not give him enough attention.
But the height of Academy members is an objective marte measure, so their weight, age, etc.. Similarly, the total length of the publications is an objective measure, as the weight of the books on the shelves of private Sfriitm, the average age of their students in mind. And similarly, the number of articles they have published and number of citations are objective measures. Of course there is a difference: As far as I know no one uses the first indices While many users latter, and while academic quality relationship seemed far-fetched when it comes to the first measures, he seems likely the case of the latter. But this appearance is correct? Moreover, whether this relationship has been shown convincingly that can be relied marte upon in determining the fate of specific academic integrity and academic units (not to mention the talk about the quality of the entire university or university collection or another country)?
I argue that the relationship between marte the various quantitative indices and academic quality marte has not been tested seriously before. Of course the people marte who developed these indices marte tried their best to create a correlation between metrics and quality. But developers understanding of these parameters is far from satisfactory, simply because they are not experts (and can not be experts) in all academic disciplines. Should clarify the argument marte in the next paragraph, but first I want to point out that even if these measures developers were trained task spoken, it would have created an index that would reflect their subjective understanding of the various academic marte disciplines, and therefore did not deserve to hide this subjectivity titled "objective measure".
The argument index reflects a certain quality is a claim relating to the content and examination is a matter for experts Content. Development and testing of a measure that reflects certain diseases is of interest to specialists in medicine, marte and the development and testing of a measure marte that reflects skill in a particular technology is of interest to specialists in this technology. In both cases it is impossible to rely solely on "human experts" or specialists to develop metrics. Similarly, a measure of academic quality in certain opinion should be examined by experts in this field, and not by "academic experts" who do not understand the professional content of the same domain and may not know and understand his academic culture. Importance of professional content and academic culture will be demonstrated below.
As implied above, the development of a quantitative measure ("objective") of quality consists of two parts. The first and decisive, is setting criteria: I mean, what measure, and how weights the various measurements. These statements are based on subjective judgment (and therefore the claim of 'objective' of the index is always an illusion), and the nature of consideration - the quality of the index. For example, if you want to base the assessment of academic quality on the index of publications, need to know how a particular journal publication reflects academic quality. Expert knowledge in particular has a decisive advantage in assessing this question, but then I would argue that they can not develop a compatible marte quality index provides (because the range of quality articles in any journal is huge).
The second part of the index is the collection of data and its statistical processing. But, even here, in the context of measures that claim to reflect academic quality, it is the illusion of objectivity: if it is counting the publications they rely on the opinions of a small number (usually marte two or three) of anonymous judges, nobody except the editors did not know the nature judgment (subjective) them. I mean, not only that this subjective marte factor, but there is no index that the user information marte on the expertise and reliability marte of this factor. counts if it comes here also quotes subjective decisions is their motives are unknown. (
No comments:
Post a Comment