home-icon
  • checkmark Responsible conduct of research
    • checkmark Design and conduct
    • checkmark Design and methodology
    • checkmark Possible flaws in a study design
    • checkmark Preregistration and registered reports
    • checkmark Reproducibility and replicability of research
    • checkmark Statistics in research
    • checkmark Research funding
    • checkmark Research Data Management (RDM)
    • checkmark FAIR data principles for research data
    • checkmark Data Management Plan
    • checkmark Reporting results
    • checkmark Presenting your data
    • checkmark Image processing
    • checkmark Authorship
    • checkmark Author affiliation
    • checkmark Citation and referencing
    • checkmark Open access to publications
    • checkmark The quality of a journal
    • checkmark Peer review
    • checkmark Preprints
    • checkmark Novelty of your work
    • checkmark The value of negative results
  • checkmark Declaration of conflict of interest
  • checkmark Science communication
  • checkmark Research(er) evaluation and assessment
  • checkmark References for module 3 - Good Academic Practices

Possible flaws in a study design

Slippery Slope
mindthegap

Unreliable research findings due to flawed study design

When a study is designed in a way not aligned with methodological good practice, you may get wrong answers to your research question. This is not an improbable scenario, but a major problem in science today, as shown by Ioannidis in his 2005 article “Why most published research findings are false”.

Unreliable results, attributable to flawed study design, constitute a terrible waste. That your research is useless, is not only a waste of your time and energy, but even worse, it may distort the literature with wrong insights, guiding future research in irrelevant directions.

mindthegap

As such, designing bad research if it could have been prevented, may be seen as an unacceptable research practice – precisely because of its detrimental impact on the literature. Unreliability of findings can be reduced by carefully designing your research, learning good design practices from the literature and double-checking methodology matters with your supervisor and colleagues.

mindthegap

Bias

Inappropriate study design is one of the most common causes of bias in research which results in false positive findings.

According to Ioannidis, the most important drivers of the high rate of false positive findings in clinical medicine and biomedical research, are:

  • Solo, siloed investigator limited to small sample sizes
  • No pre-registration of hypotheses being tested
  • Post-hoc cherry picking of hypotheses with best p-values
  • Only requiring p <0.5
  • No replication
  • No data sharing

mindthegap

Integration of sex and gender analysis as a mark of research excellence

Wherever human beings or animals are involved in research, sex and gender will be an issue and should be considered and addressed in the research design. Addressing the sex and gender dimension of research implies that sex and gender are considered as key analytical and explanatory variables in research. If relevant sex or gender issues are missed or poorly addressed, research results will be partial and potentially biased. Responsible and excellent research practices therefore consider sex and gender aspects in all stages of the research cycle.

mindthegap

Gender biased research

Gender biases originate in the often unintentional and implicit differentiation between men and women by placing one gender in a hierarchical position relative to the other in a certain context, as a result of stereotypical images of masculinity and femininity. An example of gender bias in research focuses on the experience and point of view of either men or women, while presenting the results as universally valid for both sexes. Gender biases influence the participation of men and women in research and the validity of research.

mindthegap

These tools can support and inspire you in making your research sex and gender-sensitive:

  • Gendered Innovations  Practical methods for sex and gender analysis in science, health & medicine, engineering and environment. Case studies provide concrete illustrations of how sex and gender analysis leads to innovation. (Stanford University)
  • Toolkit Gender in EU-funded research Toolkit with accessible introduction and practical tools on the integration of the gender dimension in research. Analysis of case studies drawn from health; food, agriculture and biotechnology; nanosciences, materials and new production technologies; energy; environment; transport; socio-economic sciences and humanities; science in society and international cooperation.
  • Webinars
  • Integration of sex and gender analysis into research: how to integrate sex, gender, and intersectional analysis into research design, and how this can lead to discovery and improved research methodology.
  • Gender in research (proposals and projects)
mindthegap

Gender-blind research

Gender-blind research does not take gender into account when relevant, being based on the often incorrect assumption that possible differences between men and women are not relevant for the research at hand. This is the case when general categories such as ‘people’, ‘patients’ or ‘users’ do not distinguish between men and women. Research based on such categories may well draw partial conclusions based on partial data. This limits the generalizability and applicability of research findings.

mindthegap

Gender-sensitive research is qualitatively better and more valid. When research takes into account the differences between men and women in the research population, the results will be more representative.