Description

Quick Links

• Toggle DrawerOverviewFor this assessment, you will locate a scholarly review of a specific standardized psychological or educational test you would likely use in your current or future profession and write a 5–6-page critical analysis of the selected test, based on the scholarly review and on support from relevant resources.As a psychologist, social worker, counselor, therapist, researcher, mental health professional, or educator, your profession will utilize some form of tests and measurements. Even if you do not plan on administering tests, you still need to understand the results described in evaluation reports. Even if you will not need to read reports of evaluations, you will need to keep current with the professional literature.SHOW LESSBy successfully completing this assessment, you will demonstrate your proficiency in the following course competencies and assessment criteria:

• Competency 1: Demonstrate comprehension of the terminology, principles, statistical concepts, and psychometric features related to the construction and application of psychological tests.

• Describe the standardization sample.

• Report and interpret validity and reliability coefficients in a standardized test.

• Competency 2: Analyze key psychometric properties related to tests and measurement, with an emphasis on reliability and validity.

• Describe various forms of reliability and validity evidence in a standardized test.

• Competency 3: Evaluate the properties, techniques, and applications used in psychological evaluation.

• Describe the rationale for selecting the standardized test.

• Describe the population, psychological condition, or psychological disorder that corresponds to the test's purpose and future use.

• Construct an argument for or against using the selected standardized test in a current or future occupation.

• Competency 7: Communicate in a manner that is scholarly, professional, and consistent with expectations for members of the psychological profession.

• Communicate in a manner that is scholarly, professional, and consistent with expectations for members of the psychological profession.

• Toggle DrawerContextThink about your career, future career, or area of specialization at Capella University. As a psychologist, social worker, counselor, therapist, researcher, mental health professional, or educator, your profession will utilize some form of tests and measurements. Even if you do not plan on administering tests, you still need to understand the results described in evaluation reports. Even if you will not need to read reports of evaluations, you will need to keep current with the professional literature.SHOW LESSThere really is no way around it: a basic understanding of tests and measurement is needed. Whether we construct, administer, or otherwise use tests, we have the responsibility to do it ethically. That means having the competence to conduct these tasks. Being able to identify psychological and educational tests; knowing the importance of psychometric properties; and understanding how, when, and why tests are administered is a good start. Applying this knowledge is the next step.The Assessment 1 Context document contains important information about the following topics:

• Tests and Measurement.

• Test Validation.

• Test Reliability.

• Test Quality.

• Toggle DrawerQuestions to ConsiderTo deepen your understanding, you are encouraged to consider the questions below and discuss them with a fellow learner, a work associate, an interested friend, or a member of the business community.

• Which categories of tests are you most likely to use in your current (or anticipated) profession?

• What is a standardized test?

• How are test norms developed for a test?

• What are the three types of reliability evidence? How is the evidence collected for each type?

• What are the three types of validity evidence? How is the evidence collected for each type?

• What is the relationship between test reliability and validity?

• How are validity and reliability coefficients interpreted?

• Toggle DrawerResourcesSuggested ResourcesThe following optional resources are provided to support you in completing the assessment or to provide a helpful context. For additional resources, refer to the Research Resources and Supplemental Resources in the left navigation menu of your courseroom.Capella ResourcesClick the links provided to view the following resources:

• Assessment 1 Context.

• PRINTEVALUATION OF PSYCHOLOGICAL TESTING

• Think about your career, future career, or area of specialization at Capella University. As a psychologist, social worker, counselor, therapist, researcher, mental health professional, or educator, your profession will utilize some form of tests and measurements. Even if you do not plan on administering tests, you still need to understand the results described in evaluation reports. Even if you will not need to read reports of evaluations, you will need to keep current with the professional literature.There really is no way around it; a basic understanding of tests and measurement is needed. Whether we construct, administer, or otherwise use tests, we have the responsibility to do it ethically. That means having the competence to conduct these tasks. Being able to identify psychological and educational tests; knowing the importance of psychometric properties; and understanding how, when, and why tests are administered is a good start. Applying this knowledge is the next step.Tests and MeasurementThis is a course on tests and measurement.
•  Although we generally use the termtest, the logic used in choosing a measurement procedure applies also to any measure or source of information such as interviews, ratings, and informal observations. Measurement is the quantification of the constructs of interest.Assigning a number to the results of a psychological test is an example of measurement. It allows us to compare individual and group differences. Psychological measurement requires a basic understanding of the statistics used in the development, selection, use, scoring, and interpretation of tests. These statistics include validity, reliability, measurement of error, factor analysis, and other aspects. Such statistics have many uses, such as describing the relationship between standardization samples.Some categories of tests are more difficult to measure than others, because not everything we want to know can be easily identified. For example, it is relatively simple to develop a test to measure what a person learned in a course. A valid test will have similar qualities as the course content; these qualities can be compared and are tangible. In comparison, an IQ test measures what we define as being intelligence. Since we cannot easily identify intelligence by looking at it, the results of any intelligence test can be questionable, and are prone to interpretation and circular reasoning, in that we use the test to determine a person's intelligence, but we identify what intelligence is based on the test we've created.Being able to effectively choose the most appropriate test starts with having a solid background for making the choice, such as you are receiving in this course. It also requires knowing what to look for and where to find it. This assessment prepares you to address the most crucial questions about test selection and use with a focus on qualities desired in any measurement procedure.Validation and ReliabilityTest ValidationIt is now considered incorrect to use the phrase, "the validity of a test." Why? While a test may be relevant for making one decision, it may have no value for making another decision.Therefore, test users must ask, "How valid is this test for the decision to be made," or, "How valid are the several interpretations of the test scores I am making?" In other words, validation refers to inquiry into the soundness of an interpretation. Indeed, the end-goal of validation is explanation and understanding.It is important to emphasize that validation is the joint responsibility of the test developer and the test user. The test developer is responsible for providing relevant evidence and a rationale in support of test score interpretation for specified users. In addition to the test developer, other researchers may make important contributions to the validity evidence as it accumulates from their reports empirical findings. Note that, as a test user, you are ultimately responsible for evaluating evidence available for the samples and settings for which you use the test.The following three key terms, referring to types of validation inquiry, are a focus of this assessment:

• Content-related evidence.

• Criterion-related evidence.

• Construct-related evidence.

• Please do not jump to the conclusion that content validation is for educational tests, criteria validation is for aptitude tests, and construct validation is for personality tests. With almost any test, it makes sense to join several kinds of inquiry. To emphasize this point, the latest standards speak not ofcontent validity,for example, but ofcontent-oriented evidence of validity. The end-goal of validation being explanation and understanding, construct validation will continue to be of greatest importance. In practice, the validation process never ends; new information always contributes to a better understanding of a given test and the inferences that can be drawn from it.The following table presents different types of validity, when they are used, and how they are computed.Types of Validity Type of Validity What Question It Addresses How It Is Addressed Criterion-related evidence. Does the test assess performance on a current measure (concurrent) or a measure at some later time (predictive)? Compute the correlation coefficient between scores on a test and the performance on a current (concurrent) or future (predictive) criterion variable or test. Construct evidence. Is the test an adequate measure of a specific psychological construct? Compute the correlation coefficient between a test and another test demonstrating adequate measurement of a similar construct. Compute correlation between a test score and scores on a factor (factor analysis). Measure different methods to measure multiple constructs (multi-trait-multiple method approach). Content evidence. Does the content of the test measure what it purports to measure? Experts compare content of test to predetermined content domains and make judgments about the content representativeness of a test. This assessment explores different methods of validation inquiry rather than distinct types of validity. Devote particular attention to what questions are asked and what kind of evidence is required for different uses of tests.Test ReliabilityIn deciding whether to use a given test or not, the second question we raise is, "How reliable is it?" Note that we are not asking what it measures, but how accurately it measures whatever it does measure.

• What is the precision of the test scores that an individual will obtain?

• If the person is tested twice, would the two score reports agree? If yes, how closely?

• Or, if you have two forms of the same test, will the results be comparable?

• If two different observers judge the same behaviors of a group of learners, will we have agreement between the judges?

• Federal government guidelines require that a test be reliable before we can use it to make employment or educational placement decisions.As you know from your experience, scores on repeated testings tend to vary. What the concept of reliability allows you to do is to understand which proportion of variation in test scores is due to actual changes in performance or behavior, and which is due toerror variance. It is reducing error variance that makes a test more reliable (Salkind, 2005, pp. 40–41). In other words, the reliability of a test means freedom from errors of measurement.Standard error of measurement(SEM) refers to the extent to which an observed score for a given individual deviates from her true score. The true score is an estimate of the variation in scores that could be expected if an individual took the test an infinite number of times. A more reliable test yields a lower SEM.The following table presents different types of reliability, when they are used, and how they are computed.Types of Reliability Type of Reliability What Question It Addresses How It Is Addressed Test-retest reliability. Is the test reliable over time? Compute the correlation coefficient between the scores obtained by the same learners at two different times. Parallel forms reliability. Are different forms of a test equivalent? Compute the correlation coefficient between the scores obtained by the same learners on two forms of the same test. Internal consistency reliability. Do the items of a test assess one, and only one, dimension? Compute the correlation between each individual item score and the total score. Inter-rater reliability. Is there agreement between two raters? Examine the percentage of agreement between two raters. At the conclusion of this assessment you should understand four related terms:

• Observed score.

• True score.

• Standard error of measurement (SEM).

• Reliability coefficient.

• Monetary costs such as the expense of materials and tester time, financial burdens borne by the test taker, and the costs of scoring the test.

• Features facilitating test administration.

• Features facilitating interpretation and use of scores.

• Availability of equivalent forms.