Metodologi Penelitian dalam Psikologi Politik

Rizqy Amelia Zein

##article.authors##

Rizqy Amelia Zein Universitas Airlangga

Keywords:

metodologi, psikologi politik

Abstract

Bab ini menguraikan konsep-konsep dasar metodologi dalam penelitian Psikologi Politik. Topik-topik yang dibahas meliputi pemeriksaan kredibilitas temuan penelitian, pengukuran psikologi, desain-desain penelitian yang lazim digunakan dalam Psikologi Politik, isu-isu etika, dan coba-ulang (reproducibilty) dan reka-ulang (replication) dalam penelitian Psikologi.

References

Alderson, P. (2004). Absence of evidence is not evidence of absence. BMJ : British Medical Journal, 328(7438), 476–477.

Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 3–25. https://doi.org/10.1037/amp0000191

Barnett, A., Fraser, H., Parker, T. H., Nakagawa, S., & Fidler, F. (2018). Questionable Research Practices (QRPs) in Ecology and Evolution. PLoS ONE, 13(7), e0200303. https://doi.org/10.1371/journal. pone.0200303

Bethlehem, J. (2010). Selection Bias in Web Surveys: Selection Bias in Web Surveys. International Statistical Review, 78(2), 161–188. https://doi.org/10.1111/j.1751-5823.2010.00112.x

Biemer, P. P. (2010). Total Survey Error: Design, Implementation, and Evaluation. Public Opinion Quarterly, 74(5), 817–848. https://doi.org/10.1093/poq/nfq058

Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Meta-Regression.

Bornstein, R. F. (2011). Toward a process-focused model of test score validity: Improving psychological assessment in science and practice. Psychological Assessment, 23(2), 532–544. https://doi.org/10.1037/a0022402

Camerer, C. F., Dreber, A., Forsell, E., Ho, T.-H., Huber, J., Johannesson, M., … Wu, H. (2016). Evaluating replicability of laboratory experiments in economics. Science, 351(6280), 1433–1436. https://doi.org/10.1126/science.aaf0918

Chan, M.-p. S., Jones, C. R., Hall Jamieson, K., & Albarracín, D. (2017). Debunking: A Meta-Analysis of the Psychological Efficacy of Messages Countering Misinformation. Psychological Science, 28(11), 1531–1546. https://doi.org/10.1177/0956797617714579

Cho, E. (2016). Making Reliability Reliable: A Systematic Approach to Reliability Coefficients. Organizational Research Methods, 19(4), 651–682. https://doi.org/10.1177/1094428116656239

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

Cohen, J. (1992). A Power Primer. Psychological Bulletin, 112(1), 155. https://doi.org/10.1038/141613a0

Cohen, J. (1994). The earth is round (p0.6em.05). American Psychologist, 49(12), 997–1003. https://doi.org/10.1037/0003-066X.50.12.1103

Committee on Reproducibility and Replicability in Science, Board on Behavioral, Cognitive, and Sensory Sciences, Committee on National Statistics, Division of Behavioral and Social Sciences and Education, Nuclear and Radiation Studies Board, Division on Earth and Life Studies, … National Academies of Sciences, Engineering, and Medicine. (2019). Reproducibility and Replicability in Science. Washington, D.C.: National Academies Press. https://doi.org/10.17226/25303

Cooper, M. M. (2018). The Replication Crisis and Chemistry Education Research. Journal of Chemical Education, 95(1), 1–2. https://doi.org/10.1021/acs.jchemed.7b00907

Cronbach, L. J., & Meehl, P. E. (1955). Construct Validity in Psychological Test. Psychological Bulletin, 52(4), 281–302.

Dalton, D. R., Aguinis, H., Dalton, C. M., Bosco, F. A., & Pierce, C. A. (2012). REVISITING THE FILE DRAWER PROBLEM IN META-ANALYSIS: AN ASSESSMENT OF PUBLISHED AND NONPUBLISHED CORRELATION MATRICES: PERSONNEL PSYCHOLOGY. Personnel Psychology, 65(2), 221–249. https://doi.org/10.1111/j.1744-6570.2012.01243.x

Darley, J. M., & Latané, B. (1968). Bystander intervention in emergencies: Diffusion of responsibility. Journal of Personality and Social Psychology, 8(4), 377–383. https://doi.org/10.1037/h0025589

Davis, N. (2017). The Goldwater rule: Why commenting on mental health from a distance is unhelpful. The Guardian.

Davis, R. J. (2014). Reproducibility Project: Cancer Biology. eLife. https://elifesciences.org/collections/9b1e83d1/reproducibility-project-cancer-biology.

Deming, W. E. (1944). On Errors in Surveys. American Sociological Review, 9(4), 359–369.

Dienes, Z. (2008). Understanding psychology as a science. London: Palgrave Macmillan.

Donsbach, W. (2015). Public Opinion Polls. In G. Mazzoleni (Ed.), The International Encyclopedia of Political Communication. New York: Wiley-Blackwell.

Dorey, F. J. (2011). In Brief: Statistics in Brief: Statistical Power: What Is It and When Should It Be Used? Clinical Orthopaedics and Related Research, 469(2), 619–620. https://doi.org/10.1007/s11999-010-1435-0

Duckitt, J. (2001). A dual-process cognitive-motivational theory of ideology and prejudice. In Advances in experimental social psychology (Vol. 33, pp. 41–113). https://doi.org/10.1016/S0065-2601(01)80004-6

Dumenci, L. (2000). Multitrait-Multimethod Analysis. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of Applied Multivariate Statistics and Mathematical Modeling (pp. 583–611). San Diego: Elsevier. https://doi.org/10.1016/B978-0-12-691360-6.X5000-9

Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046

Eri?en, C., Eri?en, E., & Özkeçeci-Taner, B. (2013). Research Methods in Political Psychology. Turkish Studies, 14(1), 13–33. https://doi.org/10.1080/14683849.2013.766979

Errington, T. (2019). Reproducibility Project: Cancer Biology - Barriers to Replicability in the Process of Research. Stanford University.

Fanelli, D. (2012). Negative results are disappearing from most disciplines and countries. Scientometrics, 90(3), 891–904. https://doi.org/10.1007/s11192-011-0494-7

Fanelli, D. (2018). Opinion: Is science really facing a reproducibility crisis, and do we need it to? Proceedings of the National Academy of Sciences, 115(11), 2628–2631. https://doi.org/10.1073/pnas.1708272114

Fealy, G. (2014). Resurgent political Islam, or astute Islamic parties? New Mandala.

Fowler, F. J. (2014). Survey research methods (Fifth edition). Los Angeles: SAGE.

Fried, E. I., & Flake, J. K. (2018). Measurement Matters. APS Observer, 31(3).

Gaines, B. J., Kuklinski, J. H., & Quirk, P. J. (2007). The Logic of the Survey Experiment Reexamined. Political Analysis, 15(1), 1–20. https://doi.org/10.1093/pan/mpl008

Garrison, K. E., Tang, D., & Schmeichel, B. J. (2016). Embodying Power: A Preregistered Replication and Extension of the Power Pose Effect. Social Psychological and Personality Science, 7(7), 623–630. https://doi.org/10.1177/1948550616652209

Gawronski, B., Galdi, S., & Arcuri, L. (2015). What Can Political Psychology Learn from Implicit Measures? Empirical Evidence and New Directions: Implicit Measures in Political Psychology. Political Psychology, 36(1), 1–17. https://doi.org/10.1111/pops.12094

Giorgi, A. (2012). The descriptive phenomenological psychological method. Journal of Phenomenological Psychology, 43(1), 3–12. https://doi.org/10.1163/156916212X632934

Goodman, S. N., Fanelli, D., & Ioannidis, J. P. A. (2016). What does research reproducibility mean? Science Translational Medicine, 8(341), 341ps12–341ps12. https://doi.org/10.1126/scitranslmed.aaf5027

Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337–350. https://doi.org/10.1007/s10654-016-0149-3

Groves, R. M., & Lyberg, L. (2010). Total Survey Error: Past, Present, and Future. Public Opinion Quarterly, 74(5), 849–879. https://doi.org/10.1093/poq/nfq065

Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., … Zwienenberg, M. (2016). A Multilab Preregistered Replication of the Ego-Depletion Effect. Perspectives on Psychological Science, 11(4), 546–573. https://doi.org/10.1177/1745691616652873

Hammack, P. L., & Pilecki, A. (2014). Methodological approaches in political psychology: Discourse and narrative. In The Palgrave Handbook of Global Political Psychology (pp. 72–89). London: Palgrave Macmillan.

Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015). The Extent and Consequences of P-Hacking in Science. PLOS Biology, 13(3), e1002106. https://doi.org/10.1371/journal.pbio.1002106

Hendry, L. B., Mayer, P., & Kloep, M. (2007). Belonging or Opposing? A Grounded Theory Approach to Young People’s Cultural Identity in a Majority/Minority Societal Context. Identity, 7(3), 181–204. https://doi.org/10.1080/15283480709336930

Hidayat, R. (2018). Rawan Kepentingan: Campur Aduk Lembaga Survei & Jasa Konsultan. tirto.id. https://tirto.id/rawan-kepentingan-campur-aduk-lembaga-survei-jasa-konsultan-cJkL.

HIMPSI. (2010). Kode Etik Psikologi Indonesia (Pertama (Kongres XI HIMPSI 2010)). Jakarta: Pengurus Pusat Himpunan Psikologi Indonesia (HIMPSI).

Hollenbeck, J. R., & Wright, P. M. (2017). Harking, Sharking, and Tharking: Making the Case for Post Hoc Analysis of Scientific Data. Journal of Management, 43(1), 5–18. https://doi.org/10.1177/0149206316679487

Hunsley, J., & Meyer, G. J. (2003). The Incremental Validity of Psychological Testing and Assessment: Conceptual, Methodological, and Statistical Issues. Psychological Assessment, 15(4), 446–455. https://doi.org/10.1037/1040-3590.15.4.446

Intan, G. (2019). Dituduh Tukang Bohong, Lembaga Survei Ungkap Data dan Metodologi Quick Count. VOA Indonesia. https://www.voaindonesia.com/a/dituduh-tukang-bohong-lembaga-survei-ungkap-data-dan-metodologi-quick-count/4885243.html.

Ioannidis, J. P. A. (2018). Meta-research: Why research on research matters. PLOS Biology, 16(3), e2005468. https://doi.org/10.1371/journal.pbio.2005468

Isani, M., & Silverman, D. (2016). Foreign Policy Attitudes toward Islamic Actors: An Experimental Approach. Political Research Quarterly, 69(3), 571–582. https://doi.org/10.1177/1065912916654988

Israel, H., & Richter, R. R. (2011). A Guide to Understanding Meta-analysis. Journal of Orthopaedic & Sports Physical Therapy, 41(7), 496–504. https://doi.org/10.2519/jospt.2011.3333

John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling. Psychological Science, 23(5), 524–532. https://doi.org/10.1177/0956797611430953

Jost, J. T. (2019). The IAT Is Dead, Long Live the IAT: Context-Sensitive Measures of Implicit Attitudes Are Indispensable to Social and Political Psychology. Current Directions in Psychological Science, 28(1), 10–19. https://doi.org/10.1177/0963721418797309

Kaarbo, J., & Beasley, R. K. (1999). A Practical Guide to the Comparative Case Study Method in Political Psychology. Political Psychology, 20(2), 369–391. https://doi.org/10.1111/0162-895X.00149

Kelley, K., & Preacher, K. J. (2012). On effect size. Psychological Methods, 17(2), 137–152. https://doi.org/10.1037/a0028086

Khan, K. S. (2003). Five Steps to Conducting a Systematic Review, 96, 4.

Klein, R. A., Cook, C. L., Ebersole, C. R., Vitiello, C. A., Nosek, B. A., Chartier, C. R., … Ratliff, K. A. (2019). Many Labs 4: Failure to Replicate Mortality Salience Effect With and Without Original Author Involvement (Preprint). PsyArXiv. https://doi.org/10.31234/osf.io/vef2c

Lilienfeld, S. O., Miller, J. D., & Lynam, D. R. (2018). The Goldwater Rule: Perspectives From, and Implications for, Psychological Science. Perspectives on Psychological Science, 13(1), 3–27. https://doi.org/10.1177/1745691617727864

Lupia, A., & Elman, C. (2014). Openness in Political Science: Data Access and Research Transparency. PS: Political Science & Politics, 47(01), 19–42. https://doi.org/10.1017/S1049096513001716

Mair, P. (2018). Modern Psychometrics with R. Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-93177-7

Maris, E. (1995). Psychometric latent response models. Psychometrika, 60(4), 523–547.

Mayr, S., Buchner, A., Erdfelder, E., & Faul, F. (2007). A short tutorial of GPower. Tutorials in Quantitative Methods for Psychology, 3(2), 51–59. https://doi.org/10.1037/0096-1523.32.4.932

McAdams, D. P., Albaugh, M., Farber, E., Daniels, J., Logan, R. L., & Olson, B. (2008). Family metaphors and moral intuitions: How conservatives and liberals narrate their lives. Journal of Personality and Social Psychology, 95(4), 978–990. https://doi.org/10.1037/a0012650

McNally, R. J. (2018). Diagnosing at a Distance: Is the Goldwater Rule Still Relevant Today? Perspectives on Psychological Science, 13(1), 28–30. https://doi.org/10.1177/1745691617731636

McNeish, D. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433. https://doi.org/10.1037/met0000144

Messick, S. (1995). Validity of Psychological Assessment. American Psychologist, 9.

Mietzner, M. (2009). Political opinion polling in post-authoritarian Indonesia: Catalyst or obstacle to democratic consolidation? Bijdragen Tot de Taal-, Land- En Volkenkunde / Journal of the Humanities and Social Sciences of Southeast Asia, 165(1), 95–126. https://doi.org/10.1163/22134379-90003644

Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., … PRISMA-P Group. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1), 1. https://doi.org/10.1186/2046-4053-4-1

Morling, B. (2018). Research Methods in Psychology: Evaluating a World of Information (3rd ed.). New York, NY: W. W. Norton & Company, Inc.

Mullinix, K. J., Leeper, T. J., Druckman, J. N., & Freese, J. (2015). The Generalizability of Survey Experiments. Journal of Experimental Political Science, 2(2), 109–138. https://doi.org/10.1017/XPS.2015.19

Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 2017(15), 201708274. https://doi.org/10.1073/pnas.1708274114

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716–aac4716. https://doi.org/10.1126/science.aac4716

Panagiotakos, D. B. (2008). The Value of p-Value in Biomedical Research. The Open Cardiovascular Medicine Journal, 2(1), 97–99. https://doi.org/10.2174/1874192400802010097

Parackal, M., Mather, D., & Holdsworth, D. (2018). Value-based prediction of election results using natural language processing: A case of the New Zealand General Election. International Journal of Market Research, 60(2), 156–168. https://doi.org/10.1177/1470785318762234

Perezgonzalez, J. D. (2015). Fisher, Neyman-Pearson or NHST? A tutorial for teaching data testing. Frontiers in Psychology, 6(MAR), 1–11. https://doi.org/10.3389/fpsyg.2015.00223

Peters, G. J. Y. (2018). The alpha and the omega of scale reliability and validity: Why and how to abandon Cronbach’s alpha and the route towards more comprehensive assessment of scale quality (Preprint). PsyArXiv. https://doi.org/10.31234/osf.io/h47fv

Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2008). Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. Journal of Clinical Epidemiology, 61(10), 991–996. https://doi.org/10.1016/j.jclinepi.2007.11.010

Price, P. C., Jhangiani, R. S., Chiang, I.-C. A., Leighton, D. C., & Cuttler, C. (2017). Research Methods in Psychology (3rd ed.). Pullman: Washington State University.

Prinz, F., Schlange, T., & Asadullah, K. (2011). Believe it or not: How much can we rely on published data on potential drug targets? Nature Reviews Drug Discovery, 10(9), 712–712. https://doi.org/10.1038/nrd3439-c1

Quintana, D. (2019). Synthetic datasets: A non-technical primer for the behavioural sciences to promote reproducibility and hypothesis-generation. https://doi.org/10.31234/osf.io/dmfb3

Revelle, W., & Zinbarg, R. E. (2008). Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145. https://doi.org/10.1007/s11336-008-9102-z

Rheault, L., Beelen, K., Cochrane, C., & Hirst, G. (2016). Measuring Emotion in Parliamentary Debates with Automated Textual Analysis. PLOS ONE, 11(12). https://doi.org/10.1371/journal.pone.0168843

Ritchie, S. J., Wiseman, R., & French, C. C. (2012). Failing the Future: Three Unsuccessful Attempts to Replicate Bem’s “Retroactive Facilitation of Recall” Effect. PLOS ONE, 7(3), e33423. https://doi.org/10.1371/journal.pone.0033423

Rodgers, M. A., & Pustejovsky, J. E. (2019). Evaluating Meta-Analytic Methods to Detect Selective Reporting in the Presence of Dependent Effect Sizes (Preprint). MetaArXiv. https://doi.org/10.31222/osf.io/vqp8u

Rothstein, H., Sutton, A. J., & Borenstein, M. (Eds.). (2005). Publication bias in meta-analysis: Prevention, assessment and adjustments. Chichester, England ; Hoboken, NJ: Wiley.

Rubio, D. M., Berg-Weger, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27(2), 94–104. https://doi.org/10.1093/swr/27.2.94

Ruel, E. E., Wagner, W. E., & Gillespie, B. J. (2016). The practice of survey research: Theory and applications. Thousand Oaks, CA: Sage Publications, Inc.

Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013). Ten Simple Rules for Reproducible Computational Research. PLOS Computational Biology, 9(10), e1003285. https://doi.org/10.1371/journal.pcbi.1003285

Schafer, M. (2000). Issues in Assessing Psychological Characteristics at a Distance: An Introduction to the Symposium. Political Psychology, 21(3), 511–527. https://doi.org/10.1111/0162-895X.00201

Schoonvelde, M., Schumacher, G., & Bakker, B. N. (2019). Friends With Text as Data Benefits: Assessing and Extending the Use of Automated Text Analysis in Political Science and Political Psychology. Journal of Social and Political Psychology, 7(1), 124–143–143. https://doi.org/10.5964/jspp.v7i1.964

Shaughnessy, J. J., Zechmeister, E. B., & Zechmeister, J. S. (2012). Research methods in psychology (9th ed). New York, NY: McGraw-Hill.

Sijtsma, K. (2008). On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha. Psychometrika, 74(1), 107. https://doi.org/10.1007/s11336-008-9101-0

Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., … Nosek, B. A. (2018). Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results: Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245917747646

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632

Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results. Perspectives on Psychological Science, 9(6), 666–681. https://doi.org/10.1177/1745691614553988

Singh, S. (2018). Understanding the Bias-Variance Tradeoff. Medium. https://towardsdatascience.com/understanding-the-bias-variance-tradeoff-165e6942b229.

Skrondal, A., & Rabe-Hesketh, S. (2007). Latent Variable Modelling: A Survey. Scandinavian Journal of Statistics, 34(4), 712–745. https://doi.org/10.1111/j.1467-9469.2007.00573.x

Soderborg, S. (2019). Indonesia: How the polls are performing. New Mandala.

Spellman, B., Gilbert, E. A., & Corker, K. S. (2017). Open Science: What, Why, and How (Preprint). PsyArXiv. https://doi.org/10.31234/osf.io/ak6jr

Stagge, J. H., Rosenberg, D. E., Abdallah, A. M., Akbar, H., Attallah, N. A., & James, R. (2019). Assessing data availability and research reproducibility in hydrology and water resources. Scientific Data, 6(1), 1–12. https://doi.org/10.1038/sdata.2019.30

Stein, M. M., Smith, E. M., & Holmes, N. G. (2018). Confirming what we know: Understanding questionable research practices in intro physics labs. arXiv:1807.04716 [Physics]. Retrieved from http://arxiv.org/abs/1807.04716

Stone, S., Johnson, K. M., Beall, E., Meindl, P., Smith, B., & Graham, J. (2014). Political psychology. Wiley Interdisciplinary Reviews: Cognitive Science, 5(4), 373–385. https://doi.org/10.1002/wcs.1293

Symonds, J. E., & Gorard, S. (2010). Death of mixed methods? Or the rebirth of research as a craft. Evaluation & Research in Education, 23(2), 121–136. https://doi.org/10.1080/09500790.2010.483514

Tijdink, J. K., Verbeke, R., & Smulders, Y. M. (2014). Publication pressure and scientific misconduct in medical scientists. Journal of Empirical Research on Human Research Ethics, 9(5), 64–71. https://doi.org/10.1177/1556264614552421

van Aert, R. C. M. (2019). Meta-analysis and publication bias. Rotterdam, Netherlands.

Vankov, I., Bowers, J., & Munafò, M. R. (2014). On the persistence of low power in psychological science. The Quarterly Journal of Experimental Psychology, 67(5), 1037–1040. https://doi.org/10.1080/17470218.2014.885986

Veldkamp, C. L. S., Bakker, M., van Assen, M. A. L. M., Crompvoets, E. A. V., Ong, H. H., Nosek, B. A., … Wicherts, J. M. (2018). Ensuring the quality and specificity of preregistrations (Preprint). PsyArXiv. https://doi.org/10.31234/osf.io/cdgyh

Wampold, B. E., Ahn, H., & Kim, D. (2000). Meta-analysis in the social sciences: Asia Pacific Education Review, 1(1), 67–74. https://doi.org/10.1007/BF03026147

Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., … Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3, 160018. https://doi.org/10.1038/sdata.2016.18

Wynd, C. A., Schmidt, B., & Schaefer, M. A. (2003). Two Quantitative Approaches for Estimating Content Validity. Western Journal of Nursing Research, 25(5), 508–518. https://doi.org/10.1177/0193945903252998

Yong, E. (2018). Psychology’s Replication Crisis Is Running Out of Excuses. The Atlantic. https://www.theatlantic.com/science/archive/2018/11/psychologys-replication-crisis-real/576223/.

Zhou, S., Page-Gould, E., Aron, A., Moyer, A., & Hewstone, M. (2019). The Extended Contact Hypothesis: A Meta-Analysis on 20 Years of Research. Personality and Social Psychology Review, 23(2), 132–160. https://doi.org/10.1177/1088868318762647

Zlowodzki, M., Poolman, R. W., Kerkhoffs, G. M., Tornetta, P., & Bhandari, M. (2007). How to interpret a meta-analysis and judge its value as a guide for clinical practice. Acta Orthopaedica, 78(5), 598–609. https://doi.org/10.1080/17453670710014284