HPS104 Foundations of Psychological Science ASSIGNMENT PART A-D WORKED EXAMPLE AND ADVICE The following is an example using a made-up research issue (issue #4) I chose to address. And in italics I will add comments. I will only provide a complete worked example for Parts A and B. And note that these examples are super comprehensive! You could get a great mark with less effort than this! Part A – Formulation Which issue number have you chosen to address? (just provide the issue number – In my example I’ve written down the entire issue (in blue font) only to illustrate the issue that I’ll be using as an example – Issue #4 – a but you don’t need to do this) ISSUE#4. The majority of women in Western cultures have experienced dissatisfaction with the appearance of their body at some point in their lives. Researchers have attempted to evaluate how a woman’s body dissatisfaction changes when she is pregnant. Pregnancy is characterized by a range of physiological changes that push women further from the thin ideal and one might expect increased body dissatisfaction to result as they move further from the ideal. On the other hand, it is possible that women adjust their ideal in recognition of the normal weight gain that occurs during pregnancy (they may even deprioritize physical appearance during pregnancy). So what happens to body image during pregnancy? Previous research has been unable to address this issue conclusively, with the few studies conducted in the area yielding mixed results. QUESTION 1. Formulate a testable research question that addresses your chosen issue. Does body dissatisfaction change during pregnancy? So short and sweet! Note that in my answer I’ve not made mention of my methodology, measures, or implementation. What I’ve written is precise without being detailed. Note that I’ve not emphasized causality here. If I wanted to do this I might have asked: “Does pregnancy cause changes to body dissatisfaction?” But as I’m probably going to settle for an observational design because of feasibility and ethics (see Q3), I’m not going to set up a question that I can’t really answer. QUESTION 2. List some measures you would need to test this research question. Where appropriate, identify these measures as operationalising independent (predictor) or dependent (outcome or criterion) variables. 1. Independent variable (IV): Pregnancy ‘stage’ is the primary independent variable (‘predictor’ variable). I will need a valid and accurate measure that distinguishes between perhaps three stages of pregnancy. Convenient and concrete levels of IV might be: (i) Trimester 1; (ii)Trimester 2; (iii) Trimester 3. Given that pregnancy stage is concrete and usually known to the woman with some precision, I think a self-report measure would suffice. I am not certain at this stage whether I will need a pre-pregnancy information or even how to define ‘pre-pregnancy’ appropriately. 2. Dependent variable (DV): Body dissatisfaction is the dependent variable (the criterion variable I am trying to ‘predict’). I will need a valid and reliable self-report measure of body dissatisfaction. Although self-report measures are limited, body dissatisfaction is a complex psychological construct and there is no better way I can think of to evaluate it than a brief survey. Again, we’re being conceptual here about these measures, not specific. By this I mean that I haven’t even bothered to look up if such measures exist! It really doesn’t matter. All I’ve done is specify what, conceptually, I’d need in order to obtain my IV(s) and DV(s). QUESTION 3. Rank, name and briefly describe the main research designs (and their results) that could logically, feasibly, and ethically by applied to test this research question. OK, so I’ve gone a little overboard here! Rank Design name Describe design and the expected Results 1 Longitudinal The Design. A single cohort of women surveyed at three time points (the three levels of IV) to obtain their level of body dissatisfaction (the DV) throughout their pregnancy. The Results. To demonstrate that pregnancy influences body dissatisfaction, body dissatisfaction scores in the single cohort of women will need to change over the course of their pregnancy. If these changes reflect physical deviations from the thinideal then there should be an increase in reported dissatisfaction in the cohort as they progress through the stages of their pregnancy. 2 Cross-sequential The Design. Two different cohorts of women – pregnant versus not pregnant (the two levels of the cross-sectional IV) – surveyed at three time points (the three levels of the longitudinal IV) to obtain their level of body dissatisfaction (the DV) throughout their pregnancy. The Results. To demonstrate that pregnancy influences body dissatisfaction while taking into account the possibility that body dissatisfaction changed over time for some reason unrelated to stage of pregnancy (e.g., a new fashion trend emerges causing ALL women to worry more about their appearance), body dissatisfaction scores in the pregnant women will need to change over the course of their pregnancy in a way that differs from changes observed in non-pregnant women over the same time period. If changes in body dissatisfaction occur and are due to physical changes during pregnancy, then there should be an increase in reported dissatisfaction in the pregnant women but not in the non-pregnant women. 3 Cross-sectional version 1 The design. Three different cohorts of women at different stages of pregnancy (the three levels of IV) surveyed to obtain their level of body dissatisfaction (the DV). The results. To demonstrate that pregnancy is associated with body dissatisfaction, body dissatisfaction scores for at least one of the three cohorts of women will be different from the others. If these changes reflect physical deviations from the thinideal then there should be more dissatisfaction reported by women in Trimester 3, than women in Trimester 2, with women in Trimester 1 reporting the least dissatisfaction. 4 Cross-sectional version 2 The design. Two different cohorts of women – pregnant versus non-pregnant (the two levels of IV) – surveyed to obtain their level of body dissatisfaction (the DV). The results. To demonstrate that pregnancy is associated with body dissatisfaction, body dissatisfaction scores for pregnant women will differ from that of non-pregnant women. If these differences reflect physical deviations from the thin-ideal then thereshould be more dissatisfaction reported by pregnant women than non-pregnant women. Notice that the language is descriptive, loose, and informal. Notice also that I’m not delving deeply into issues like validity other than mentioning possible confounds. I’m also not delving into formal ethical issues. Throughout, I’ve kept it very conceptual and conversational. The issues can be dealt with in detail in Part B. Note also that my descriptions of the results required to resolve the research question are NOT in the form of a formal set of hypotheses. Again, that will come in subsequent tasks (specifically, Part C). And note that I only came up with four design ‘solutions’. There were other possibilities, but I’m confident that these are the most obvious. And four is plenty, right?Part B – Justification Please remind us which issue you chose to address, the research question you formulated, and your #1 ranked research design. ISSUE#4. Question. Does body dissatisfaction change during pregnancy? Longitudinal Design. A single cohort of women surveyed at three time points (the three levels of IV) to obtain their level of body dissatisfaction (the DV) throughout their pregnancy. If body dissatisfaction changes during pregnancy, body dissatisfaction scores in the single cohort of women will need to change over the course of their pregnancy. If these changes reflect physical deviations from the thin-ideal then there should be an increase in reported dissatisfaction in the cohort as they progress through the stages of their pregnancy. We are asking for this information only to save your tutor the trouble of looking back to your responses to Part A. Note that you CANNOT change your chose research issue number but you can, based on the feedback you received for Part A, change the way you formulate your research question and which design you select. We suggest that you focus here on the formulation and design you intend to follow through with in Parts C and D. QUESTION 1. What are the advantages of your #1 ranked design? Experimental designs that involve random assignment to levels of the IV and control of extraneous factors definitely offer the best internal validity. Depending on how well they are conducted, how well the measures are taken, etc., they can be used to demonstrate co-occurrence, time order, and control for alternative explanations and confounds. However, the ethical problems inherent in attempting to manipulate an IV such as pregnancy status or pregnancy stage preclude use of an experimental design in this research question. Not only is pregnancy something that a researcher cannot ethically force upon a human participant, once a person is pregnant the progress of their pregnancy (Trimester 1 -2 -3) follows a natural temporal course, one that cannot be altered or rearranged. In other words, I determined that pregnancy, pregnancy state, stage of pregnancy, etc., must all be person variables beyond the experimenter’s control. And for these reasons I chose to forgo experimental designs as the theoretically more valid but less ethical and feasible option and opt instead for observational designs. The advantage of choosing a longitudinal version of an observational design is that it provides timeorder information on a single cohort. By this I mean that I am potentially able to observe which changes first, the IV or the DV. In order words, I can determine whether it is the process of pregnancy, and the changes that are associated with this process, that precede changes in body dissatisfaction (and not the other way around!). Potentially, I could observe the trajectory in participants’ body dissatisfaction over the course of their pregnancy and, with a few tweaks to my research design, relate these trajectories to other variables associated with pregnancy (e.g., change in body size, girth, BMI, etc.). I could go further and track body image post-pregnancy too. In terms of internal validity, this ability to explore time-order relationships makes longitudinal designs superior to other observational design types.And because I am utilising a single cohort of women to test the influence of my single IV (stage of pregnancy), I effectively eliminate person differences. That is, any differences in DV between the same group of women as they progress through the levels of the IV (their stage of pregnancy) cannot be attributed to individual differences between participants. And consequently, I do not require a large sample size. For the above reasons, of the observational designs under consideration, I considered the purely longitudinal to be the best compromise of ethics, feasibility, and validity. Yes, this was a very elaborate response, but I wanted to work through it comprehensively so you can see how I’m thinking this through. Remember, we want evidence that you can apply your emerging understanding of science and research designs. So show us – use formal language, consider the main issues of this unit (especially validity and feasibility in which the nature of your variables is front and centre!) QUESTION 2. What are the disadvantages of your #1 ranked design and how might these disadvantages be addressed? The inability to manipulate the primary IV in my longitudinal design severely limits the design’s internal validity and introduces the problem of third variables. In my design third variables take the form of factors that ‘cause’ both pregnancy and changes in body dissatisfaction. A good example that I outlined in my response to question 2 is relationship status. A strong, loving relationship might cause both the desire to become pregnant and a reduction in the importance of physical appearance. This could account for individual differences in body image and body image change over the course of a woman’s pregnancy, and this third variable could create the false appearance of an association between pregnancy and body dissatisfaction which is not causal (in either direction). Without the ability to randomly assign women to pregnancy groups it is impossible to disentangle these effects. These limitations are inherent to observational designs and are intractable. Another limitation is that it is very difficult to obtain pre-pregnancy body dissatisfaction data (i.e., the DV measure pre-Trimester 1). A pre-pregnancy survey of the DV would enable me to compare the same cohort of women immediately prior to discovering that they are pregnant or even prior to becoming pregnant. Although pre-pregnancy measurement is possible it is not feasible. A longitudinal study is also somewhat impractical (but still do-able I think) because it will take several months to complete. This increases the possibility of attrition occurring, reducing my sample size, and thus weakening both the internal and external validity of my findings. More problematic is that this attrition may be heterogeneous, being associated with the IV (e.g., women who experience complications during their pregnancy will be most likely to withdraw from the study), the DV (e.g., women who commence the study with the highest levels of body dissatisfaction will be most likely to experience distress and will be most likely to withdraw from the study), or worse, associated with both the IV and DV (e.g., women who experience the most profound body dissatisfaction because of their pregnancy will be most likely to withdraw from the study), and thus constitute a potential confound as well. This is why I contemplated several versions of cross-sectional design (see question 4 for more detail). QUESTION 3. Briefly consider the lower ranked designs you proposed but then rejected.Briefly discuss the pros and cons of the alternative designs you came up with and justify their lower ranking. Please focus on internal validity and ethics/feasibility of the other options that you identified. What we want here is for you to that you have critically and broadly weighed up the alternatives. I considered a cross-sequential design in part as an attempt to introduce a meaningful control condition (of sorts) in an observational design context. A control condition (particularly a placebo control) is one of the best ways of strengthening internal validity This is usually straightforward in an experimental design, but difficult in a longitudinal design where the IV cannot be manipulated or randomly assigned in any way. I reasoned that although I would be unable to randomise the primary longitudinal IV (pregnancy stage) I could nonetheless strengthen the time-order aspects of my longitudinal study by introducing a secondary cross-sectional IV (pregnancy state – pregnant v non-pregnant). I presented this as my second ranked cross-sequential option. The idea is that the cross-sectional IV would serve as a control of time-related factors that potentially confound with the primary IV. Why is this important? Given that my longitudinal design will take place in real time over the course of nine months (which is a long time!), it is possible that some external event unrelated to pregnancy itself (e.g., a new fashion trend emerges causing all women to worry more about their appearance) might influence women’s body dissatisfaction over the course of the study. To control for this my cross-sectional IV (pregnancy state) would allow me to track body dissatisfaction in pregnant women and determine if their change in dissatisfaction is different from that reported by non-pregnant women over the same nine months. Of course, such a study would require that I test a larger group of women (at least twice the sample size of a purely longitudinal design) and there is no guarantee that non-pregnant women will actually be a valid control group anyway. On these bases I decided to keep things simple and stick with a purely longitudinal option. I also considered two versions of a cross-sectional design where I proposed surveying the body dissatisfaction of three different groups of women according to their stage of pregnancy (Trimester 1, 2, or 3). The IV is thus now a cross-sectional variable. The obvious advantage is convenience – I can conduct a cross-sectional design very quickly with all data being collected at a single point in time. This also means my study will not be affected by attrition. However, I will need a much larger sample size because I need some way of dealing with individual differences between my groups. That is, because I am sampling three different groups of women, I will need a large sample in an attempt to compensate for the fact that my three samples will probably have had different levels of body dissatisfaction in the first place (regardless of their pregnancy stage) and will respond differently in terms of their body dissatisfaction to being pregnant (again, regardless of the Trimester they are in). Finally, of course, the internal validity of cross-sectional designs is poorest of all. I can at best establish co-occurrence between the IV and DV, but not temporal order and I am unable to discount alternative explanations convincingly. On these bases I discounted these crosssectional options. Again, a very elaborate response! If you have a life, you might not want to write this much! Note that from now on I am going to provide less and less detail in my example responses because, as we approach the implementation phase of your design, I do not want to overly influence your common-sense thinking. I don’t want you using my examples as your ‘template’!Part C – Implementation Please remind us which issue you chose to address, the research question you formulated, and your #1 ranked research design described in formal language. ISSUE#4. Question. Does body dissatisfaction change during pregnancy? Formal description. A longitudinal study of self-reported body dissatisfaction in women as a function of their stage of pregnancy (3 levels: Trimester 1, 2, 3). This isn’t the same as your casual description of the design in Parts A and B right? We want you to apply your now-advanced knowledge of research methodology to this question and come up with a description that includes the design, the variables and the data structure. Here is another more elaborate example of what we mean (totally unrelated to pregnancy and body image!) involving a factorial quasi-experiment: “A 3x2 between (or within or mixed)–subjects design conducted as a person (3 levels: young, middle aged, old) by treatment (2 levels: support, no support) quasi experiment in which...etc.”… Clever right? QUESTION 1. Describe how you intend to implement your study. Describe how your research design could be implemented in the form of an actual (relevant, logical, feasible, and ethical) research study. I suggest that you address the following aspects of your study (in this order): Participants. Be brief but be careful! Think about the following issues: Who are your participants and how will they be recruited? What will be the inclusion and exclusion criteria? How many participants do you except to recruit (think about whether this will be sufficient in terms of representativeness but realistic/do-able)? Remember that the nature of your participants and what they can be asked to do is often a critical factor in deciding which research design to employ! Of course, the extent to which you need to talk about the ways participants are assigned to conditions depends on the research question and the design you have chosen. And remember that by now we will have covered sampling – simple random, cluster, and stratified sampling in particular. How you sample (in all kinds of designs, but especially non-experimental designs) might be critical. You should consider this in your description (and use the correct term!). How you recruit participants for your study, who you sample for your survey, etc. are important issues particularly in relation to external validity. Some designs (e.g., longitudinal) are especially vulnerable to attrition. Heterogeneous/homogenous attrition are potentially serious issues that might require you to rethink (or better justify) your sample size and approach to sampling. Materials. You need to explain how you propose to operationalise variables into something measureable. This doesn’t need to be thorough or researched, but it does need to be convincing to the reader. For example, if your variable of interest is ‘body dissatisfaction’, you need to explain that you will measure body dissatisfaction using a practical, reliable, and valid tool (note that you are free to make up the necessary instruments and measures – there’s no need to read-up on this). This is important because body dissatisfaction is a multidimensional, theory- and definition-dependent, construct! I believe that the research questions we’ve given you are pretty straightforward and youshould be able to think of common sense ways of operationalising and measuring the relevant variables. Procedures. Obviously, you want to describe what participants will be required to do. What is the experimental protocol? What is the content of the survey? What is the order of events in your study? Provide enough detail so that someone else could, in theory, replicate your study if they wanted to. And don’t forget to provide a formal description of the structure of the study in the context of the variables (particularly the IVs) and their levels. Where relevant, use terms such as ‘conditions’, particularly when describing experimental-type designs. Of course, in designs where only associations between variables are of interest, it may not even be possible to describe variables using terms like in/dependent or ‘conditions’. But certainly when talking about within-subject designs and order effects, ‘conditions’ will be an important term to use. Speaking of which, yes, you should describe your study, where relevant, using terms such as within-subject designs and between- subject designs. If the former design type applies, then you will need to consider sequence and order effects, the need for counterbalancing, etc. QUESTION 2. Justify your implementation in terms of enhancing the validity of your study. Internal validity. In addressing internal validity you might want to consider how you will assign people to groups (if relevant); how will you decide who gets what, does what, responds how. How will variables be manipulated in your study (e.g., ‘IV1’ is measured and a median split [look it up!] by this variable will be used to divide participants into upper-IV1 and lower-IV1 person groups; members of both groups will then be randomly assigned to either the level 1 of IV2 or level 2 of IV2 treatment conditions…etc.). If you have a control condition, what sort of control will it be (placebo?). Will there be blinding, single or double? During the teaching of this unit we will be introducing you to betweensubject and within-subject designs. If you opt for within-subjects designs, consider order effects and how you’d deal with them. External validity. To what extent is your design going to allow you to draw conclusions that can be generalized to real-world situations? This is going to depend on how you implement the study, operationalize variables, sample participants… Note that the research issues under consideration are questions about reality, not about laboratory situations and scenarios. So doing the question justice will invariably require you to make some efforts to create a study that is realistic. Realism of the study is important for external validity. In the implementation phase, you might want to consider improving the mundane realism (making the study literally more like reality) and/or experimental (or psychological) realism (making the study feel more like reality) of your study. QUESTION 3. State your hypotheses It is hypothesised that self-reported body dissatisfaction in a cohort of pregnant women will increase over the course of their pregnancy, with the highest levels reported in Trimester 3, followed by Trimester 2, with the lowest levels reported in Trimester 1. Now here’s my very long explanation…because hypotheses are IMPORTANT! It is important that your hypotheses are worded in a manner consistent with the design and implementation. Note that in my example it is implicit who the participants are, how they are being measured (longitudinal), and what results I expect to obtain. Basically, if your design makes sense, and your implementation of this design is logical, then you should be able to come up with unambiguous, definitive hypothesesinvolving the operationalised variables you have described previously. So, provide us with hypotheses – what do you predict? Why is all this important? If you can’t come up with clear hypotheses, or can’t see how your research question could be turned into hypotheses, then you won’t get far in developing an internally valid research study. Conversely, if your hypotheses are sort of inevitable – if it is impossible for them not to come true – then your research study has probably failed the test of falsifiability (the research question can’t be disconfirmed, only confirmed), and that’s a serious logical problem (possibly a tautology; that is, a circular argument). So your hypotheses need to be of a form that not only make it clear what your results will/should look like if the research question is confirmed in the affirmative, but also (at least implicitly) what they will/should look like if the research question is disconfirmed. More subtly, and perhaps most importantly, you might like to re-visit the issue of internal and external validity in relation to these results. For example, just because the results turn out the way you predicted (confirmatory of the research question), alternative interpretations of these results might be possible, and that’s a serious problem as well. The worst scenario is when any and all results outcomes are consistent with your research question – it doesn’t matter what the outcome of the study, the results can still be interpreted as supportive of the research question. In this case you need to go back to the drawing board… A control group might do the trick here! Impress us! Don’t just describe using common terms, use fancy terms you’ve learnt in class! Speaking of which, in describing your hypothesized results, consider using formal terminology (main effects, interaction effects) that we will be encountering later on in the trimester. Why not turn up the suction and add these concepts to your assignment as you encounter them…? My pregnancy design example was observational so these things aren’t particularly applicable, but if the terms are relevant to your design, then use them!Part D – Plain Language Statement QUESTION 1. Please attach a plain language statement (PLS) to recruit participants to your study. You now need to invite (and entice!) participants to your study, explaining to them why they should participate and what the study involves. The information you provide to potential participants is ethically and contractually important. And the most important and concrete way you convey this information to participants is in the form of a Plain Language Statement (PLS). The PLS needs to be provided to potential participants before they sign up for your study; it needs to be understandable to the layperson (i.e., in ‘plain language’); it needs to be comprehensive enough to allow them to make an informed decision about whether or not to participate (which means it needs to include the why, what, and how of the study); and it needs to provide information to debrief them, mitigate possible negative consequences of participating, or at least provide them with go-to resources if problems arise later, after participation in the study is complete. This is a tall order, especially when you consider that your need to be honest and comprehensive can and usually does clash with your need not to reveal your expectations and create confounds, demand characteristics, reveal your control/placebo methods, etc. (By the way, be careful about this! Students seem to have a habit of ‘giving too much away’, even in their project title! So be informative… but coy!). Despite these challenges, the PLS is an incredibly valuable tool and the opportunity for you to show us that you can implement ethical decision-making. What do I mean? The PLS represents the operationalization of an ethical risk-benefit analysis, an analysis that considers (i) any unusual risks/hazards/dangers in your research, (ii) the extent to which consent to participate will be obtained, and the extent to which it will be informed consent (remembering that you don’t want to risk invalidating the results of your research by ‘giving away’ the intended goals in advance!), (iii) that your recruitment methods are free from coercion, (iv) that confidentiality will be maintained at the level of participation, security of data, and publication of results, and (v) whether participants will have available to them appropriate support (e.g., links to mental health services) at the completion of the study should they need it. Note that these issues can sometimes be in conflict with each other – an anonymous study is a great way of ensuring confidentiality (and also a great way of reducing the risk of coercion) but how then can you track down participants whose responses in your study reveal them to be at great risk of some mental health condition, and how can you ensure that they seek out and receive the help/counselling they may need? To help you get started, immediately below I include instructions for completing a PLS provided by Deakin’s Office of Research Integrity: http://www.deakin.edu.au/research/researcher-support/integrity-secure/human-ethics/humanethics-forms-and-guidelines Go to the Example Forms section and click on Plain Language Statements and Consent Forms. But note that you won’t have to submit a consent form! This website is particularly useful because it provides examples in the form of templates for how you might word a PLS for low-risk or high-risk research (you decide) involving adults or children, participating in experiments or completing surveys. And remember, the PLS is written for the benefit of a lay person, the sort of person who is not an expert in the area but is eligible to participate in your study). On the next page I provide a template you can use to create your own PLS, along with a list of the sorts of information you should consider supplying.PLAIN LANGUAGE STATEMENT Project Title: Something relevant and information but that does not create demand effects in participants Principal Researcher: Associate Professor Alex Mussap Student Researcher: That’s you! Writing a Plain Language Statement (modified from the Office of Research Integrity site) Your Plain Language Statement (PLS) is the description that you give potential recruits so that they have the information that they need to make an informed decision to participate. The following information MUST be included: • Who are you? This isn’t only a basic courtesy, it is also important to convince prospective participants that you are worth supporting. • Purpose of the study (What are you trying to achieve? Again, without giving away your expectations/hypotheses!) • Methods (What are you going to do?) • Risks and potential benefits of the research (to the participants, others like them or the whole community). Ultimately, in your head you need to conduct a risk-benefits analysis and then summarise the results for prospective participants. But think about how much you reveal here – it’s important to describe thoroughly what participants will do and why they should get involved, but you should avoid influencing participants’ responses by announcing what you expect to find!) The following additional information MAY be necessary to include for your particular study: • Any alternatives to participation. For example, if a school class is being studied, what arrangements will be made for the students who are not involved? • An assurance that participants have the right to withdraw from a project at any stage, along with any implications of withdrawal, and whether it will be possible to withdraw their data once collected. In some cases (e.g., anonymous data) it will not be possible to withdraw data once they have been submitted, or once all data have been aggregated. If this is the case, you should say so. • Any payments, reimbursements or gifts to participants (remember, this should not be so generous as to be considered potentially coercive) • How the research will be monitored • If there are potential problems, what services or support will be available to participants adversely affected by the research? Depending on the nature of the study, it’s not always sufficient to assure participants that you will debrief them after the study and support them if/when necessary. Why should they go through you for support? And what if they need support long after the study is over? What if they have issues and never even complete your study? • How privacy and confidentiality will be protected• How the research findings will be published and how participants can find out the outcomes. The PLS must conclude with the following complaints clause: Complaints If you have any complaints about any aspect of the project, the way it is being conducted or any questions about your rights as a research participant, then you may contact: The Manager Office of Research Integrity Deakin University 221 Burwood Highway, Burwood, Victoria 3125 Phone: 9251 7129; Fax: 9244 6581; Email: [email protected] Please quote project number XXXXXXX.