How to Evaluate Autism Research
September 21, 2022
By: Emily Alagha
Many autistic people like keeping up with the latest research in the field. Whether you want to self-advocate or gather information for a loved one, you will likely encounter autism research studies as you engage with the autism community. But, the large number of studies published each year can make this space hard to navigate. With news headlines about new research popping up daily, it can be hard to determine how much stock to put in study results. There is a lot of misinformation about autism online, so it helps to read recent studies with a critical eye. Here are some tips for reading autism studies and determining what the results might mean for you. These strategies are what I have found most helpful in my own journey exploring the autism literature as an advocate.
You can search for articles yourself, or you can use trusted research news services to highlight the latest studies for you. My favorite way to search for studies is Pubmed.gov, a medical search engine managed by the National Library of Medicine. The site provides options for narrowing your search results to only the most recent studies. You can also save your searches and get email alerts when new results are published. Many of the articles in Pubmed are free to read online or available through your local library. You can also get email updates directly from journals that focus on autism, such as Autism or Autism Research.
If sorting through a large number of Pubmed search results seems daunting, you can look to trusted organizations to highlight research studies for you. OAR, Spectrum News, and other autism advocacy organizations often summarize new studies. You may also find exciting studies in news headlines, support groups, and conversations with other autistic community members.
Research studies are usually written for other researchers rather than the general public. While navigating medical jargon can sometimes be intimidating, there are strategies you can use to quickly hone in on key findings from a study. Emily Morson has a good overview of how to read an autism study. Two of her recommendations are especially helpful:
Descriptive studies document behaviors, conditions, or other medical phenomena in an individual or group. One example might be a study associating autism with particular risk factors. A paper where autistic individuals share their experiences in interviews, surveys, or focus groups is also descriptive.
Experimental studies usually test a new treatment to see if it affects a particular outcome. Experimental studies might also evaluate the accuracy of a new diagnostic tool. Some examples include papers testing the effect of probiotic supplements on social behaviors in autistic children or the impact of mindfulness training on anxiety in autistic adults.
Review papers summarize findings from several studies to highlight major themes and significant results. They can include both descriptive and experimental studies. I find they often give you the best bang for your buck when trying to understand the current state of research in a given area.
Research papers are usually broken into the same core sections. Start with the Abstract, which provides an overview of why and how the study was done and the results. Reading the Abstract can help you determine if the paper is a good match for your interests and worth diving further into. Next, read the Discussion section to get a plain-language description of the results and their implications. If you want more technical details, you can go back and read the Methods and Results sections.
Once you understand the methods used and key findings of the study, you can ask some questions to help determine if the results are trustworthy and relevant to you.
Generally, studies that include more participants have more reliable results. Additionally, the authors should give demographic details about the participants. First, were the participants human? Studies that test treatments on mice can be essential building blocks for future research. But as the Twitter account @justsaysinmice can tell you, they are not strong enough to base decisions for human treatment on alone. If the participants are human, do they represent the full spectrum of autism, or are they limited to particular ages or levels of support needs? Do they represent various ethnicities, socioeconomic backgrounds, genders, and geographic areas? If not, the results may be hard to generalize to broader groups. Autism is a broad spectrum disorder, so a study that may have implications for some may be less relevant for others. By looking at who was included in the study, you can better know if the results are relevant to you.
In recent years, researchers are making efforts to consult with autistic people during the process of designing studies, rather than just including them as participants. The concept of including the populations you are studying in study design and decision making is called community-based participatory research (CBPR). CBPR can improve the quality of studies and help the results be more relevant to the communities being studied. Similarly, a growing number of autistic scientists are leading research studies. As neurodiverse voices play a greater role in shaping the research landscape, you may notice more of the studies you read including details about how autistic people were included in the study’s design. Hopefully, these trends will increase the accuracy and usefulness of study results in the future.
“Validity” is a term that describes how accurately the results seen among research study participants represent similar people in real life contexts. Many factors influence validity, and there are some questions you can ask to assess it. Did the researchers clearly state their research question and the outcomes they would measure? Did they use research methods that are relevant to the question? For example, in a study examining a depression treatment, participant depression levels should be measured using standard screening tools. Did the researchers talk about other studies that have used methods similar to theirs? You can also look at the participant drop-out rate to examine validity. A high percentage of participants dropping out of the study may indicate that the treatment is too time-consuming or has unpleasant side effects.
In research studies, “significant” is not just another word for “important.” Most experimental studies will include a P-value. This number determines if the study results are significant, meaning we can mostly rule out that the results are due to chance. If the P-value is less than .05, you can have more confidence in the results. This number will usually be reported as (p<.05) in the results section. Significant results indicate that the treatment or diagnostic tool being tested is effective.
You can dive into more statistics to determine just how large the effect is if you want to dive deeper. I recommend exploring additional statistics primers on the Crash Course Youtube Channel if you would like to gain more familiarity with these commonly reported measures.
A well-designed study should discuss the steps researchers took to prevent biased results. For experimental studies, does the methods section include a description of how participants were assigned to different treatment groups? It is usually a good sign if the methods section mentions blinding and randomization.
You can also look at information about the study’s authors and funding sources to find potential sources of bias. Papers should include affiliation information for each author. Do they work for a university, hospital, non-profit, or pharmaceutical company? Do they get paid to promote specific treatments or medicines? Most articles include conflict of interest statements for each author. These statements provide information about funding sources and company ties that might impact each researcher’s ability to evaluate the results without bias. These author details can help you get an idea of any funding sources for the study that might lead to bias. Finally, you can evaluate the trustworthiness of the journal the article is published in. Is the journal clear about who is on the editorial team and funding sources? They should have standards for evaluating studies for potential publication available on their website.
It can be easy to get swept up in headlines about “groundbreaking” new findings, especially when you are eager to find solutions for yourself or a loved one. As exciting as the results of new studies can be, it is important to remember that research advances happen in small steps over time. Even the best designed studies cannot perfectly replicate real life, so it usually takes results from many studies over time to create change in medical treatment. It is best to rely on medical experts to translate study results into customized treatment plans.
With that in mind, I hope these strategies will empower you to follow new trends in autism research. You can use what you read to ask better questions of your healthcare provider and gain new perspectives on the loved ones’ lived experiences. Understanding research studies can help you be a stronger self-advocate in all areas of your health.
Emily Alagha is a medical librarian and autism advocate. She is passionate about health literacy, and she has published research on how people use new technologies to answer health questions. She has a Master’s in Information Studies, and a Master’s in Health and the Public Interest. You can contact her at her website: https://emilyalagha.com/#contact