Avoid Bad Survey Methods: Pre-registration

Research in research must be conducted in an ethical manner. Researchers must agree on basic ethical principles  that ensure the validity of their research. In the past, however, some researchers used dubious and poor research methods. These methods mainly consist of committing small abuses in order to be able to continue the investigation and later publish it.

The goal of modern research dynamics is to publish results. This goal can put negative pressure on researchers. Therefore, it is common for many of them to succumb to poor research methods. There are some really striking examples throughout history, such as the Stanford prison experiment and the Diedrick Stapels case.

The Stanford Prison Experiment is one of the most famous social psychology experiments. However, some recordings later showed the poor examination methods used to perform this experiment. For example, relying on anecdotal evidence, training the guards, making implicit inquiries and interpreting the results incorrectly. Ever since it became public knowledge, people have been debating whether this study should still be included in the textbooks.

Diedrick Stapel was a teacher of social psychology at the University of Tilburg. In 2011, it was discovered that he had falsified information from many of his studies. Instead of using participants in his studies, Stapel had invented information in such a way that he always got the result he expected.

These poor research methods were unknown to his students, who trusted their teacher. This went on until one of his philosophy doctoral students found it strange that he always got the results he expected. From then on, everything fell apart.

Although these cases came to everyone’s attention, they are in the minority. Most bad examination methods are more subtle. here are the most common forms:

• P-hacking: To test the hypothesis of statistical analyzes, researchers use the P-value, which is normally 0.05. However, when using many variables and experimental conditions, this value can be biased. Therefore, the recommendation is to reduce what researchers should do before conducting the study.
• Low power:  A small sample scale may cause the study to have low statistical power. In turn, low power can very likely cause a false positive. In other words, the survey will suggest something that does not actually exist.
• HARK: ing: This means that the researchers change the hypothesis after they discover that the results are not what they expected.

One solution to these bad survey methods is pre-registration. This means publishing the researchers’ goals and methods for the study they are to carry out. Then everyone can clearly see that the researchers have carried out the study ethically, or as they have said they should in advance.

Pre-registration is very simple. These sites can help you, such as the Open Science Network (OSF). In this place you can find different templates for registration. You can also upload study material (databases, questionnaires, supplementary material, etc.) and make it public in a simple way.

You can find which templates for pre-registration on the website we have already mentioned. Some require more information than others. If we take one of the simpler templates called AsPredicted as an example, the fields you need to fill in are:

• Information:  If you have already collected it or if you will collect it after registration.
• Study hypothesis:  Explain the hypothesis you will investigate.
• Variables:  What are the study variables and how to measure them.
• Permits: The  permits you will attribute to the participants (for example, control and experimental).
• Analysis: The  analysis you will do once you have collected the information.
• Observations: The  number of observations you will collect. In other words, the number of participants you need.

By making the  study’s information public, it is visible to everyone. Therefore, the possibility of using bad research methods decreases. Although registration takes longer because you have to think about what to look for, many people do. In addition, it allows science to become more transparent.