Introduction This Research Project Toolkit has been prepared to assist you to plan and carry out a research project successfully. The Toolkit brings together the tips, hints, and checklists used throughout the BUS703 Study Guide (Modules 1-12) as a quick reference guide that can be used when undertaking any research projects. The Toolkit also includes tips and hints to assist with skills such as critiquing research that are critical for this course. Please remember that the suggestions and worksheets provided here are just guidelines and can be varied.  Section 1: Your Research Project While in some cases you will be given a specific topic to focus your Research project on the tips below are directed at those students who may wish to develop their own research topic. Some tips about selecting a research problem: ⦁ Choose an area which interests you and builds on previous courses undertaken in your studies. These previous studies will give you a head start in terms of the theoretical underpinning of your chosen topic. ⦁ Be focussed—your research question should not be so large that it would take several people several years to answer, or conversely so focussed that not all required issues are addressed. ⦁ If you are not sure about a possible topic, read through current academic and practitioner journals in your areas of interest for issues that may inspire you. ⦁ If for some reason you do not want to undertake a project in your own workplace or a workplace you are familiar with, consider options like outside organisations you are involved with or family/friend connections. ⦁ Make sure your research question is decision focussed, that is, what are the management decisions/actions that will be taken as a result of your research. If there are no decisions or actions there is little point to the research. Section 2. Problem definition Problem definition is the first stage of the research process and begins with identifying background issues, then identifying a management problem, and a research question or problem. Finally the information requirements of the research problem are refined into specific research objectives or issues.  ⦁ 2.1 Worksheets and tips for problem definition Worksheet 2.1: Summary of Problem Definition Background issues (Begin general and become more focused.) ⦁ ⦁ ⦁ ⦁ Management problem (In question form and focused on decision to be made by management.) Research question (In question form — what information is required to answer the management problem?) Objectives (In objective form, specific and measurable.) ⦁ ⦁ ⦁ ⦁ . Tips for considering the background issues ⦁ Consider the background to the problem. What are the issues that have led to the problem? Why or how did it come to the notice of management? ⦁ Look for frameworks from your knowledge of theory to help you to formulate questions and structure the background section (for example you might use a SWOT analysis or consider the elements of the marketing mix if your problem is related to marketing). ⦁ If you have difficulties starting to think about the possible background to the problem, consider the relevant theory as a starting point eg job satisfaction ⦁ Start with more general information and narrow down to the specific area of interest. ⦁ Be specific wherever possible, especially when writing up this section of your proposal or final report. For example don’t say things like ‘ABC is a small company’, or ‘Sales have been falling’. Quantify these terms wherever possible to reduce ambiguity. A small company could be interpreted as having anywhere from 1 to 200 employees, depending on the definition applied, and sales are falling can imply anything from a very slight decrease to a drastic decrease. ⦁ Tips for defining the management problem ⦁ The management problem should focus on a decision. This means if there is no decision to be made, don’t do any research. ⦁ The focus for the management problem should be in management’s terms — don’t try to put the problem into theoretical terms at this stage. ⦁ State the management problem in question form to ensure that there is a focus on the decision to be made. ⦁ If the answer to the management problem is yes or no (for example ‘Should we stop marketing product X as its sales are decreasing?’) you might have to think again. An answer of yes gives little direction and simply raises a whole series of new questions, whereas if the decision had been stated more broadly as ‘How can we increase the profitability of product X?’ the research would consider not only the alternative of no longer marketing the product but also possible changes to the current marketing mix that might revitalise the product. ⦁ Good definition of the management problem can often take several discussions with management to clarify issues and ensure that the problem is correct; that is, some background research will have to be undertaken even at this stage. This background research could take the form of discussions with decision-makers, interviews with experts and analysis of secondary data, such as sales figures.  Tips for defining the research problem or question ⦁ The research problem should also be stated in a question format and focus on the information required by management to enable them to make the decision specified in the management problem. ⦁ Use your knowledge of the relevant theory to assist in developing your research problem. For example if the problem is related to consumer behaviour, go back and revise the theory in this area. ⦁ A test of whether you have defined your research question correctly is to ask yourself whether, if you had the answer(s) to the research problem, you would be able to answer the management problem. ⦁ If you cannot write one question but end up with four or five questions, you have probably jumped ahead and started developing specific research objectives. Tips for developing research issues or objectives ⦁ Make sure they are in objective format; that is, they should start with terms like ‘to determine’, ‘to explore’, ‘to describe’, ‘to identify’. ⦁ Draw heavily on your previous theoretical and practical knowledge. ⦁ Make sure you use appropriate terminology and not jargon. For example you will not want to ‘find out who buys a certain product’, but you might want to ‘develop a demographic profile of current purchasers of a product’. This makes it easier to link back to the required theory. ⦁ As a check to make sure your objectives are comprehensive, ask yourself, ‘If I knew the answer to these objectives would I be able to answer the research problem?’ ⦁ Any terms used in objectives should be measurable and clearly defined. For example ‘brand loyalty’ can be defined and measured in several ways, so if you have a research objective related to brand loyalty you will need to clarify which definition you are using and justify why. Checklist for Problem Definition ⦁ Does the background start with more general issues and become more focussed? ⦁ Have terms used been as specific and clear as possible? ⦁ After reading the background, could a reader with no knowledge of the research identify the decision management is faced with? ⦁ Is the management problem stated in question form, and does it focus on a decision? ⦁ Is the research problem stated as a question, and does it focus on information to allow management to make a decision? ⦁ Are the research objectives stated in objective format? ⦁ Are all terms clearly defined? ⦁ If the information identified in the research objectives is obtained, will all required information identified in the marketing research problem be obtained? Section 3. Research Designs ⦁ 3.1 Decide on the Research design Once the research problem has been defined, it is possible to decide on the most appropriate research approach or design. The three commonly used designs in research are ⦁ exploratory, ⦁ descriptive and ⦁ causal research. The choice of the most appropriate design is dictated by the nature of the research problem, and many research projects require the use of a combination of research designs. Refer also the Study Guide for Module 4: Research Designs  Checklist for identifying the appropriate research design or combination of designs How much is known about the problem area? How well defined is the problem? Are there any budgetary constraints for the project? What are the time frames for gathering and analysing data? Is secrecy a major issue? What degree of precision does management require? Remember that you might need more than one approach to address each objective. For example an objective could be addressed by reviewing the literature, then further exploring through depth interviews or focus groups and finally addressing it in a questionnaire. However, in some cases the objective might be solely addressed through the literature review. Also remember that this is only a preliminary plan and that it will become more specific as you progress. Tip ⦁ The way you have phrased your objectives will often indicate the appropriate type of research approach. For example if your objective is ‘to explore …’ you will probably be recommending an exploratory approach and a technique like focus groups or depth interviews, whereas if your objective is ‘to describe …’ you will probably be recommending a descriptive technique, such as a questionnaire, or perhaps information from secondary data.  Section 4 Secondary data 4.1 Gathering and evaluating secondary data Secondary data should first be collected from internal sources within an organisation and then from external sources. The external sources consulted should be drawn from both practical and industry-based sources and journals as well as more academic sources. Tips for getting started ⦁ Pin a copy of your problem definition on your noticeboard so that you maintain focus and do not get side-tracked into interesting-but-irrelevant areas. ⦁ What internal sources of information will be relevant to the problem? ⦁ What specific theoretical discipline am I concerned with; for example, human resource management, leadership, marketing? Answering this question will identify the general area you wish to consider as well as starting to identify some key words for searches. ⦁ Am I involved in a particular industry or industry sector? What are the widely-read industry or practitioner-based sources of information in this area? What is the relevant industry code or classification for searching of any government-based census sources? Worksheet 4.1: Collection of secondary data Complete Worksheet 2 for your research project. In column (a) list you’re the objectives of your research. . In column (b) for each objective, identify possible sources of internal secondary data such as human resource records and previous studies. In column (e) identify external sources of secondary data such as relevant possible journals from the USC Library and as well identify relevant practitioner type sources such as government reports, industry associations etc. Worksheet 4.1: Collection of secondary data a b c d e f g Research objectives Identify relevant internal sources Evaluate internal data Gaps? Identify relevant external sources Evaluate external data Gaps requiring primary data 4.2 Critiquing secondary data Checklist for evaluating secondary data ⦁ Who gathered the data? Are they a reliable source, or is there the possibility of bias? ⦁ How was the information gathered? Was the methodology valid and reliable? ⦁ Is the data is accurate enough to help in the decision process? ⦁ The best ways of checking accuracy are: ⦁ find multiple sources of similar data and compare them ⦁ compare with reliable data, such as census data ⦁ conduct field investigations ⦁ How current are the data? How old? Have things changed since the data were gathered? ⦁ How objective is the source? Worksheet 4.2: Research report critique Worksheet 4.2 is designed to assist you with the task of critiquing specifically in relation to your assignment but a modified version may be useful for general evaluation of secondary data. a. This first column reflects the specific issues you are required to use when evaluating your assignment article. Subsequent columns follow a logical sequence of steps as follows: b. This column requires you to identify the relevant theory that will be the basis for your evaluation, for example, what does theory (or your course notes) say about a good background? This establishes the basis on which you will then evaluate the background in your selected article. Note also the example about sampling where the relevant theory column highlights the stages in the sampling process. c. Column c requires you to briefly summarise what was actually done/said in the article re each of the required issues. So for example in terms of the background you might make notes that highlight the topics covered and their order which would allow you to see if the background started general and got specific, was a context identified, was the scope mentioned etc. d. & e. Columns d and e require you to start evaluating ie given you have identified both the theory and what was done, how well do they match? In column d you highlight strengths while in column e you highlight weaknesses. One advantage of using this worksheet is that it forces you to consider all required areas. If you complete this worksheet and use it as the basis for your assessment it will ensure you do not omit any of the required issues and it will also ensure that you address both strengths and weaknesses in each area. Following this worksheet you will find a very brief example of a critique for one issue (sampling) that is based on this worksheet approach. ⦁ Worksheet 4.2: Research report critique a. Criteria b. Relevant theory c. Description d. Strengths of article e. Weaknesses of article Does the report provide the required information to make the decision? Was secondary data used as a framework/context? Was the research design (exploratory, descriptive, causal, or some combination) appropriate? Are the methods used to collect data (eg focus groups, questionnaires) sufficiently described and appropriate? Were respondents selected properly and were sufficient respondents selected? Population Frame Method Size Was the research conducted ethically? Analysis and Interpretation Were the methods used to analyse data sufficiently described and appropriate? Example of a paragraph critiquing sampling A sampling strategy needs to address five specific steps beginning with a clear definition of the population, identification of the sampling frame (and the associated gap with the population), choice of a sampling method, determination of sample size and instructions for drawing of the sample (Aaker & Day 2003). While Article A clearly defines their population as ‘…..’ they then use a sampling frame of university students and use a convenience method of administering a questionnaire in class to draw a sample of 276. They acknowledge the limitations of this approach and justify this strategy on the basis of similarity in demographic profile between students and their population. In summary, the strengths of the sampling strategy are a clear definition of population, an acknowledgment of the issues associated with drawing a convenience sample and an adequate sample size for the proposed analysis methods. The key weakness lies in the convenience selection of the sample which limits the generalisability of results. Key Features highlighted A sampling strategy needs to address five specific steps beginning with a clear definition of the population, identification of the sampling frame (and the associated gap with the population), choice of a sampling method, determination of sample size and instructions for drawing of the sample (Aaker & Day 2003). This first sentence summarise the relevant theory in the area and shows the framework used to analyse the article. While Article A clearly defines their population as ‘…..’ they then use a sampling frame of university students and use a convenience method of administering a questionnaire in class to draw a sample of 276. They acknowledge the limitations of this approach and justify this strategy on the basis of similarity in demographic profile between students and their population. This second sentence describes what was actually done/reported in the article. In summary, the strengths of the sampling strategy are a clear definition of population, an acknowledgment of the issues associated with drawing a convenience sample and an adequate sample size for the proposed analysis methods. This sentence highlights strengths. The key weakness lies in the convenience selection of the sample which limits the generalisability of results. This final sentence highlights weaknesses.  Section 5. Exploratory research ⦁ 5.1 Exploratory research methods This section begins with a worksheet to assist you summarise your research to date, highlighting the findings from the previous section on secondary data and the gaps identified from the evaluation of secondary data. Note that again your research objectives provide the framework for data collection. Following the worksheet are two checklists, one for depth interviews and one for focus groups. Also consider other qualitative techniques you might wish to use or integrate into depth interviews and focus groups. The section finishes with some hints for conducting depth interviews and focus groups. Research objectives Key findings from secondary data Gaps requiring primary data Will gap require exploratory research and if so which technique (depth interview, focus group, etc) Worksheet 5.1: Exploratory Research  Checklist for depth interviews Before the interview Has the interview guide been developed on the basis of the research objectives? Who will be interviewed? How many interviews will be required? How will potential respondents be contacted and encouraged to participate? When will the interview be conducted? How long should each interview take? Where should interview be conducted? How will information be recorded? Checklist for focus groups interviews Before the focus group Develop a moderator’s guide based on the research objectives. Decide on the number of groups required. Decide on the number of participants per group. Consider any issues regarding composition of groups, and develop recruiting protocol. Where will focus groups be held? The location should be convenient and comfortable for respondents. When will focus groups be held? Identify and recruit respondents. What refreshments will be required? Will incentives or payments be required for participants? Organise recording equipment.  Tips for conducting depth interviews and focus groups Depth interviews ⦁ To minimise the time required for depth interviews, consider sending questions to the respondent before the interview or leaving a questionnaire to be completed after the interview. ⦁ Consider methods like telephone or online interviews if geographic limitations exist. ⦁ Make sure there is a benefit for the respondent in participating. Focus groups ⦁ Try to identify existing groups who might be suitable as participants (for example members of a club or association) as the logistics of getting the group together will be easier and participants will feel more comfortable. ⦁ If you have limited experience try to make sure that at least two researchers attend each focus group. One researcher can then look after any recording equipment and administrative issues, leaving the other researcher to concentrate on moderating the group. ⦁ Make sure you recruit one or two extra participants for each group as it is very common for participants not to show up — even with reminders on the day of the group. An example of a moderator’s guide follows. Briefly, the research in this case was being undertaken by a marketing department within a university’s Faculty of Business. Specifically, the department wanted to increase the number of students undertaking a marketing major and to improve the quality of the major. The purpose of the focus groups was to explore student perceptions of the marketing major and the marketing department. A third-year student majoring in marketing and experienced in conducting focus groups was employed to moderate the groups of first- and second-year marketing students. The third-year student was perceived as independent from the academic and administrative staff of the department and was also able to create good rapport with the respondents.  A moderator’s guide (example) Opening ⦁ Thank students for attending. ⦁ Explain purpose of research in broad terms and advise that session is being taped; however, responses are anonymous and confidential. ⦁ Explain ground rules; that is, a focus group is like a tutorial in that everyone is expected to participate. However, there are no right or wrong answers — everyone’s opinion is valued. ⦁ Get respondents to complete brief demographic survey (gender, age, nationality). ⦁ First warm-up question: ‘Why did you choose to study at XYZ university?’ Issues ⦁ Why did you select a marketing major? ⦁ — reasons for choice ⦁ — when did they choose, i.e. before commencing study, during first year ⦁ — what information was used ⦁ — how happy with choice. ⦁ Perceptions of the marketing major ⦁ — What do they think other business students think of a marketing major? ⦁ — What do they think of the marketing major? ⦁ Perceptions of marketing department ⦁ — What do they think other students think of the marketing department? ⦁ — What do they think of the marketing department? ⦁ — What are the differences between marketing and other majors? ⦁ The marketing major ⦁ — strengths/good points/good experiences ⦁ — weaknesses/bad points/bad experiences. ⦁ Describe the ideal marketing major. ⦁ Describe the ideal marketing department. Close ⦁ Thank students for participation and ask for any final questions or comments. ⦁ ⦁ Section 6. Descriptive research 6.1 Planning the survey Worksheet 6.1 is ideal when drafting a questionnaire to ensure the survey will address the required research objectives and provide data suitable for the required analysis. Complete the worksheet for your own research project. Worksheet 6.1 Specifying information required Objective Internal sec data External sec data Exploratory research Descriptive – relevant questions on questionnaire Level of data Data analysis proposed The following checklists and worksheets may be useful when making decisions regarding any proposed survey. Checklist for choosing a communication method The research study Consider the nature of the questions to be asked — are they potentially sensitive or embarrassing? Do any budget or time constraints exist? Target population Consider the reading and writing skills of the target population. Is the population geographically spread or accessible in a confined or central location? Is the population accessible by a mailing list, email list or fax list? Staff and facilities How many properly trained interviewers can be recruited? Are the skills required to develop web-based surveys accessible? Evaluate against survey methods Personally administered — interviewer completed Personally administered — self-completed Telephone interview Mail survey Emailed survey Web-based survey Faxed survey Consider a combination of approaches. Worksheet 6.2: Summary of survey plan Survey stage Decision made — comments Target population for survey Selected survey method or combination of survey methods Potential implications for length of questionnaire Tip ⦁ The way you have phrased your objectives will often indicate the appropriate type of research approach. For example if your objective is ‘to explore …’ you will probably be recommending an exploratory approach and a technique like focus groups or depth interviews, whereas if your objective is ‘to describe …’ you will probably be recommending a descriptive technique, such as a questionnaire, or perhaps information from secondary data.  6.2 Prepare the draft questionnaire Hints for getting started: writing the first question ⦁ Have your research objectives/aims to be addressed through the questionnaire in front of you. ⦁ Has a questionnaire/instrument already been developed (check literature) that you can use/adapt; if not the total questionnaire you may be able to use parts of the questionnaire. ⦁ Make a list of the demographic questions you may want to ask (such as age or gender). If possible, take these questions from other questionnaires or sources such as ABS publications. ⦁ Identify questionnaires in similar areas from the literature or obtain them from other researchers. Then you may adapt and modify the questions to suit your particular purpose. ⦁ Collect copies of questionnaires that you come across for future reference. No matter what the topic, you will often find useful features in these questionnaires. ⦁ Go to scales handbooks and borrow any applicable scales. ⦁ Under each objective make dot points of information required and then turn these into questions. ⦁ When you have finished, check the questions against your objectives to determine whether all required objectives are covered and that extra questions not related to the objectives serve a purpose (such as screening). If not, then delete the unnecessary questions. ⦁ Now start organising and checking questions for wording and layout. Checklist for drafting the questionnaire Keep the table summarising research objectives and information requirements in front of you Note the hints for getting started in coursework. Develop questions to address each objective or information requirement. Check that each question is asked in a way that will obtain the required level of measurement for the analysis required. Arrange questions in a logical order: begin with general questions and become more specific leave sensitive questions towards the end of the questionnaire leave demographic questions until the end of the questionnaire. Consider layout and appearance. Include any instructions required. Include any administrative details required: interview identification location identification time of interview. Pretest, revise and pretest until no further changes are required. Checklist for pretesting and revising the questionnaire Pretest should include three groups of people: typical respondents peers and colleagues potential users of information; that is, people who have commissioned the research. Decide on the approach to pretesting: allow the respondent to complete and make comments on questions that were confusing or ambiguous get respondent to talk aloud as they are completing the questionnaire. If several changes or major changes are made, pretest again. ⦁ 6.3 Administration — mail questionnaire A recommended plan for administration of a mail questionnaire follows. Modify it as required to suit your specific budget and timing constraints. Table W1: Administration procedures for a mail questionnaire Step Timing Procedure Contents 1 Commencement Questionnaire to be sent to whole sample ⦁ Questionnaire ⦁ Cover letter ⦁ Reply-paid envelope 2 One week for step 1 Combined thank you/reminder sent to whole sample Thank you/reminder letter or postcard 3 Two weeks after step 2 Reminder sent to non-respondents ⦁ Questionnaire ⦁ Reminder letter ⦁ Reply-paid envelope 4 Two weeks after step 3 Reminder to non-respondents ⦁ Telephone call urging response Source: adapted from Dillman (1978) and Salant and Dillman (1994). Checklist for writing cover letter Keep the length to one page only, and cover the following issues: Explanation of the survey and purpose of the questionnaire Stress the usefulness of the survey to the respondent Stress the importance of the individual’s response Promise confidentiality If possible, promise a copy of the survey results Offer to answer any questions respondents might have Close with a statement of thanks. Checklist for first follow up Mention that a questionnaire was sent the previous week and explain its purpose. Thank those who have already returned the questionnaire. Ask non-respondents to complete their questionnaires. Stress the importance of the individual’s response. Invite the non-respondents to telephone for a replacement questionnaire if needed. Thank the respondents and non-respondents. Checklist for checking and recording mail questionnaires As the responses to the mail questionnaire come in, it is advisable to carry out the following procedures: Number and date the questionnaires (the order of receipt might become important later if you want to conduct tests for non-response bias). Check the answers to each question for reasonableness and completion. If you know who the respondent is, you might choose to telephone the person to seek clarification. This action can greatly reduce item non-response rates. In some cases you might choose to use your judgement to interpret the response. Read any additional comments (usually on the back page) that respondents have provided. A response might be required, or additional data might need to be noted. 6.4 Personally administered questionnaires Table W2: Administration of personally administered questionnaires Step Timing Procedure 1 Identify location/venue where questionnaire will be administered Initial survey planning ⦁ Get agreement from selected venue if required Cover letter ⦁ Consider logistics/location within venue ⦁ Centralised vs decentralised locations 2 Recruiting and training of field staff: ⦁ Supervisors ⦁ Interviewers Before questionnaire administered ⦁ Role of supervisor ⦁ Role of interviewer ⦁ Recruitment ⦁ Training ⦁ Scheduling ⦁ Budgets 3 Administer questionnaire ⦁ Administrative details on the questionnaire itself ⦁ The interviewer’s (appearance and identification) ⦁ Selection of respondents ⦁ Handling refusals 4 Field editing and checking During administration of questionnaire ⦁ Selection of eligible respondents ⦁ Legibility ⦁ Completeness ⦁  6.5 Telephone questionnaires Many of the administrative issues for telephone questionnaires are similar to those raised for personally administered questionnaires, and so in this section only those issues peculiar to telephone administration will be highlighted. Table W3: Administration of telephone-administered questionnaires Step Timing Procedure 1 Identify location telephone interviewers will operate from Initial survey planning ⦁ Centralised vs decentralised locations 2 Recruiting and training of field staff Before questionnaire administered ⦁ As per personally administered 3 Administer questionnaire ⦁ Administrative details on the questionnaire itself ⦁ Callbacks 4 Field editing and checking During administration ⦁ As per personally administered questionnaires ⦁ 6.6 Online questionnaires Online questionnaires are very similar to mail questionnaires in relation to administration; the key differences being electronic delivery rather than hard copy. Table W4: Administration of online questionnaires Step Timing Procedure Contents 1 Before administration Check to ensure mailing list and questionnaire site are operational (for web-based surveys) ⦁ Email list of respondents the URL for the questionnaire (or attach the questionnaire to the email) 2 Commencement Email sent to whole sample inviting response and linked to questionnaire URL ⦁ Internet questionnaire ⦁ Cover email 3 One week after step 2 Combined thank you/reminder e-mailed to whole sample ⦁ Thank you/reminder e-mail ⦁  Section 7. Causal Research  Checklist for confirming that an experimental design is appropriate Does the management problem require the testing of a cause–effect relationship? If yes: What is/are the independent variable(s)? What is the dependent variable? What form will the manipulation/control take? Checklist for choosing an appropriate experimental design How many independent variables will be manipulated? If only one, a classical design might be appropriate If more than one, a statistical design will be required What budgetary constraints exist? The tighter the budgetary constraints, the fewer groups and manipulations might be possible. What time constraints exist? Laboratory experiments are quicker than field experiments. Simpler designs are quicker to implement. Implementation problems: Will the organisation cooperate? Is it possible to set up control groups (areas)?  Section 8. Sampling ⦁ 8.1 Steps in the selection of a sample Step 1: Define the target population Checklist for defining a target population Does the definition contain information on: sampling elements sampling units geographic area of coverage. Check research objectives to ensure match with target population definition Have alternative definitions been considered? Have any specific exclusions from the population been identified? Can the population be reproduced at a later time? Will the population be convenient to sample from? Step 2: Determine a sampling frame Complete the Worksheet 8.1 to ensure that you consider all alternative frames and their respective advantages and disadvantages. Worksheet 8.1: Considering sampling frames Possible frame Advantages Disadvantages Step 3: Choose the method of selecting the sample units Checklist for choosing a sampling method Choose either probability or non-probability Is generalisability important? Will the sampling frame allow a probability method to be used? Are there budget or time constraints? Choose a specific method Consider research objectives. Does the population need to be split into identifiable subgroups? (Quota or stratify by relevant criteria.) Is a list available for simple random sampling or systematic sampling? Step 4: Determine the size of sample needed Checklist for calculating a sample size based on non-probability sampling Do any budgetary or time constraints exist? Use rules of thumb by considering how many major groups and subgroups exist. What sample sizes have comparable studies used? How variable is the population? What are the proposed data analysis techniques, and what implications do they have for sample size? How important is the decision? Consider expected response rate. Checklist for calculating a sample size based on probability sampling Determine appropriate formula by considering: Will the sample size be calculated using means or proportions? Will allowable error be stated in relative or absolute terms? What sampling method is proposed? Calculate sample size What confidence interval will be used? What level of error will be acceptable? What is the variability in the population; that is, standard deviation? How will standard deviation be calculated? Using a previous study Using a pilot study Estimated subjectively Estimated based on worst-case scenario Consider expected response rate. Step 5: Execute the sampling strategy Checklist for estimating response rates How will the survey be administered? What are average response rates for this method of survey administration? Is the topic of the survey likely to be of interest to the target population such that response rates might be improved? Is any form of incentive being used to increase response rates? How up to date are any lists being used as sampling frames? Given the estimated response rate, how many surveys will need to be administered to achieve the desired sample size? Worksheet 8.2 Complete the following worksheet to summarise your sampling strategy Step Outcome Step 1: Define the target population Step 2: Determine a sampling frame Evaluate the sampling frame error; that is, what is the difference between the target population and the sampling frame selected? Step 3: Choose the method of selecting the sample units ⦁ probability or non-probability ⦁ specific method ⦁ Step 4: Determine the size of sample needed Write instructions for identifying and selecting actual members of sample Section 9. Research proposals ⦁ 9.1 Checklist for preparing a research proposal Is the title short, descriptive and imaginative? (About eight words is ideal.) Does the executive summary actually summarise and not just tell the reader what is coming? Is the executive summary one page or less in length, and does it cover all major sections of the proposal? Does the introduction clearly state the purpose of the proposal and outline its format? Does the background set the context and scope for the study? Does the problem definition meet the criteria for good problem definition? Has the current literature been reviewed to assess the current state of knowledge in the problem area? Have gaps in the literature been identified? Has the research design for this study (for example exploratory, descriptive, causal or some combination of designs) been clearly stated and justified in relation to the problem definition? What specific techniques will you use and why? For example, if you are proposing an exploratory design using case studies as the technique, why are case studies the most appropriate technique for your research? Have you provided details of your selected techniques? For example, if using a descriptive design with a questionnaire, how do you intend to administer the questionnaire? How will you develop the questionnaire? Have you described your sampling strategy? How do you intend to analyse the data gathered above? How long will the research take? What are the estimated costs for the research? Hint: The three most common problems in research proposals are leaving out details, being generic rather than specific and poor presentation. 1 In relation to leaving out details (such as giving information about sample size but not other details about sampling) keep going back to your theory and using it as a framework/checklist of issues to be addressed. In the example just mentioned, if you went to the theory regarding sampling you would note that sampling strategy commonly has a series of at least five steps, beginning with defining a target population, selecting a sample frame, selecting a method and then determining sample size and so. Make sure you address all of these steps in a logical order. 2 The second common mistake in this section is to simply give generic justifications for your decisions. Throughout this section you should be giving specific justifications and examples related specifically to your research. 3 Refer to the section on presenting the results for further hints about presenting your research proposal. Section 10. Presenting the results and writing tips This final section includes a variety of tips about a diverse range of presentation issues including some of the common mistakes made in presenting results, tips about writing and executive summaries as well as examples of good presentation of tables. ⦁ 10.1 Written and oral reports Common mistakes include: ⦁ learning how to create bar charts (or line graphs or pie charts) and then presenting all results in this one form ⦁ analysing each question from a questionnaire separately without grouping into the relevant research objective ⦁ presenting tables poorly. ⦁ 10.2 Written reports Common mistakes include: ⦁ writing executive summaries that introduce rather than summarise ⦁ squashing tables and charts into written presentation without allowing sufficient space for good presentation ⦁ not checking the final report thoroughly for spelling and grammatical errors ⦁ not including page numbers. ⦁ 10.3 Oral reports Common mistakes include: ⦁ not practising the presentation ⦁ reading from notes ⦁ looking at the screen where slides or transparencies are projected rather than at the audience ⦁ trying to present all results in a limited time rather than focusing on key points. ⦁ putting too much information on each slide or transparency  ⦁ 10.4 Some tips about report titles The title of a report is generally the first thing a person reads so it needs to be both interesting and information. Some basic guidelines for good titles are: ⦁ Around 8 words is a good length, too much longer and your reader may forget the beginning of the title before they get to the end, too much shorter and it may not be informative. ⦁ Keep language simple and avoid jargon. ⦁ Try to catch attention and interest. The following example gives alternative titles for one research project. Consider each in terms of the above guidelines. ⦁ Shopping Behaviour ⦁ A study of the factors influencing the shopping patterns of women 40 years old and above in urban Australia using focus groups, observation and self-completed mail questionnaires. ⦁ Understanding the middle aged female shopper. ⦁ Why women shop. ⦁ A conceptualisation of a theoretical exemplar for an analysis of deviant retail therapy behaviour. ⦁ 10.5 Some tips about the use of adjectives As a rule of thumb try to use adjectives as little as possible, especially if they do not add meaning and can be justified. The major reason for this is that they tend to make both academic and business writing sound less objective. There has even been research done that supported this argument (but I can’t find it so I can’t reference it). Basically, the same article was submitted to a journal with many adjectives and rejected – when adjectives were minimised it was accepted. I will illustrate with a few examples. Examples of ‘good’ adjectives ⦁ small business (small actually describes the size of the business) ⦁ scientific research ⦁ inner-city location Examples of adjectives to avoid ⦁ wonderful staff (who says they are wonderful – how can you prove it) ⦁ dramatic increase (sounds dramatic rather than objective – actually give the increases instead and let the reader judge if they are dramatic) ⦁ outstanding results Hopefully, you can see that the adjectives in the first group can be justified and add detail while the adjectives in the second group cannot be easily justified and sound more like an opinion than something that can be proven. ⦁ 10.6 Tips for writing a good Executive Summary The key to writing a good Executive Summary is to keep in mind its purpose, that is, to act as a brief summary of the entire report. For many busy readers (particularly senior managers or executives) the Executive Summary may be the only section of the report they read. In large organisations where several people may need to know the content of a report the Executive Summary may be widely distributed to let people know the content of the report. Hence, the Executive Summary needs to be a stand-alone summary of the report. Following are some guidelines to assist with writing a good Executive Summary. ⦁ Make sure you actually summarise and don’t just give an introduction. For example, saying things like ‘The key finding of the research is discussed in detail.’ is part of an introduction, that is, it tells the reader what is coming. ‘The key finding of the research was the lack of awareness of the product in the key target market of families with school aged children.’ is a summary, that is, the reader will actually know what the key result was. ⦁ The Summary should cover all aspects of your report. A handy check to make sure your summary does this is to write one paragraph for each major section of your report (as outlined in the tale of contents). ⦁ Be specific wherever possible. For example, don’t say ‘focus groups and depth interviews were undertaken…’ rather say ‘four focus groups and six depth interviews were undertaken…’. The addition of two words adds a great deal more depth to the summary. ⦁ 10.7 Paragraphs Many of you are very fond of writing in single sentence paragraphs, that is one or two lines and then moving onto the next paragraph. In these cases I would typically write a comment like "don't write single sentence paragraphs - they are disjointed to read and don't allow you to develop your argument". I would like to explain this a little further as follows. (note the use of as follows... this is called a 'linker' as it lets you know what will be coming in the next paragraph). A paragraph is a chunk of text which should have a single argument or theme. The first sentence should introduce the theme of the paragraph. Subsequent sentences should expand and develop your argument (with supporting evidence and /or examples) and the final sentence should conclude the argument as well as linking to the next argument (ie paragraph). A major problem many students have with paragraphs is length - either having too short a paragraph or too long a paragraph. As a rule, each page should have two to three paragraphs. Too many more than this and your paragraphs are probably too short. Any less than this and you are probably trying to put too much into a single paragraph and should consider splitting the content into smaller sections. ⦁ 10.8 Linkers and flag words Following from the tips about paragraphs, I would like to talk a little about the links between paragraphs and some flag words that are useful as linkers. To refresh re paragraphs, I made the point that the first sentence of a paragraph introduces the key idea or theme of the paragraph and the last sentence should link to the next paragraph. This is important in academic writing to continually remind your reader what is coming next. I would like to illustrate this point with two examples, linkers and flag words. First, linkers are sentences or phrases that indicate the relationships between paragraphs and highlight changes in your argument or discussion. In this example I finished my last paragraph by highlighting that I would use two examples. Hence the reader knows what is coming next. I then reinforced this by starting this paragraph with the ‘flag’ word ‘first’. This word flags to the reader that this is the first of the two examples. Next, I would like to give some additional examples of flag words. I have already given the example of ‘first’ and you will note that for this paragraph I have begun with ‘next’. I could also have used ‘second’. If I had indicated that I would be making 6 points I may have used words like first, second, next, and so on and perhaps finished with ‘finally’. These words are signs to the reader that clearly indicate your structure and that your writing is following a set plan. Other flag words are ‘however’ – this immediately suggest I am going to make an alternative argument or put some limitations on what I have just said. ‘In summary’ clearly indicates I am going to summarise some points, ‘in addition’ and so on. So in summary good academic writing lets the reader know what is coming and guides the reader by linking paragraphs and using flag words to remind the reader where they are at. ⦁ 10.9 The presentation of results in tables Common problems include: ⦁ Lack of a clear and informative heading. ⦁ No scale supplied to make interpretation easier (for example the number of points on a scale and the labelling of each point). ⦁ No consideration of ordering of columns and rows. In many cases researchers simply repeat the order in which questions appeared in a questionnaire rather than considering how to make the table easier for the reader to interpret. For example if a respondent has been asked their level of agreement with a range of statements, reading and interpretation of results is made easier if the table presenting these results lists the statement with the highest agreement first, down to the statement with lowest agreement last. ⦁ No indication of total sample size involved. ⦁ Overusing tables and limited discussion of important features. Consider the following two examples. In table 1a there is no clear heading, there is no apparent order to the listing of items, and the reader is unsure exactly what is being measured. Now consider table 1b. This table is clearly labelled, information about the scale and sample size is included and the factors being rated are listed from highest to lowest (for Australia at least). The second table is more informative and much easier to interpret. Table 1a Factors influencing choice Australia Mean (std dev) USA Mean (std dev) Canada Mean (std dev) UK Mean (std dev) Standard of courses 3.89 (0.62) 3.47 (0.84) 3.39 (0.78) 4.48 (0.61) Lifestyle 3.6 (0.84) 3.25 (1.03) 3.39 (0.85) 3.37 (0.89) Safety 3.29 (0.96) 2.54 (0.92) 3.74 (0.79) 3.72 (0.80) Potential to emigrate 2.11 (1.0) 2.21 (1.07) 2.18 (1.01) 2.37 (1.10) Ease of entry 3.06 (0.90) 3.56 (0.85) 3.22 (0.79) 2.57 (0.85) Availability of information 4.05 (0.65) 3.55 (0.83) 3.22 (0.88) 3.91 (0.75) Legally working part-time 3.39 (0.82) 3.51 (0.83) 3.22 (0.67) 3.08 (0.85) Comparative cost 3.42 (0.88) 2.87 (0.87) 3.23 (0.90) 1.91 (0.89) Presence of other students 4.31 (0.65) 3.80 (0.79) 3.33 (0.84) 4.10 (0.71) Availability of scholarships 3.51 (0.84) 3.45 (0.77) 3.27 (0.69) 3.57 (0.80)  Table 1b Factors influencing choice of destination for international education by Malaysian students (n = 277) Factors influencing choice Australia Mean (std dev) USA Mean (std dev) Canada Mean (std dev) UK Mean (std dev) Presence of other students 4.31 (0.65) 3.80 (0.79) 3.33 (0.84) 4.10 (0.71) Availability of information 4.05 (0.65) 3.55 (0.83) 3.22 (0.88) 3.91 (0.75) Standard of courses 3.89 (0.62) 3.47 (0.84) 3.39 (0.78) 4.48 (0.61) Climate 3.78 (0.83) 3.41 (0.84) 3.05 (1.00) 3.17 (1.00) Lifestyle 3.6 (0.84) 3.25 (1.03) 3.39 (0.85) 3.37 (0.89) Availability of scholarships 3.51 (0.84) 3.45 (0.77) 3.27 (0.69) 3.57 (0.80) Comparative cost 3.42 (0.88) 2.87 (0.87) 3.23 (0.90) 1.91 (0.89) Legally working part-time 3.39 (0.82) 3.51 (0.83) 3.22 (0.67) 3.08 (0.85) Safety 3.29 (0.96) 2.54 (0.92) 3.74 (0.79) 3.72 (0.80) Ease of entry 3.06 (0.90) 3.56 (0.85) 3.22 (0.79) 2.57 (0.85) Potential to emigrate 2.11 (1.0) 2.21 (1.07) 2.18 (1.01) 2.37 (1.10) All data measured on 5-point scales — the higher the rating, the more favourable.