Surveys

---

Planning for an effective survey takes careful thought and consideration.

OIRE's Survey Planning Doc can help you start the process by considering important questions.

Once you've thought through these questions and determined you would like to proceed with a survey, use our Survey Roles, Responsibilities & Timeline worksheet to further outline your plan.

Both of these documents are intended to serve as a general starting point for most surveys. Your survey may have additional unique considerations. OIRE staff are here to help with your institutional surveys. Please contact us early in your planning process to ensure we can best assist you.

All campus community members have access to the Qualtrics survey platform. Information for accessing Qualtrics is available on the OIT website.

Qualtrics offers a free course that covers the basics of Building a Survey Project on their platform.

Below are links to resources to help with common topics that come up in designing, building, and administering a survey.

• Question Types
• Looping Survey Items
• Skip Logic & Display Logic
• Embedding Data
• Workflows
• Distributing Surveys

The consultative process with OIRE can include:

• Evaluation of Existing Datasets: Determining if there are data that already exist that may provide you with what you need
• Have similar surveys been administered?
• Modality Determination: Establishing the best method of collecting the information (e.g., online survey, paper survey, focus group). Please reach out to us or the Office of Information Technology for help with Qualtrics, the university's survey software.
• Instrument Design:  Developing items and response categories, ordering items, using white space to improve readability, etc.
• Population Selection and Generation: Determining the most appropriate population(s) to include, sampling, and generating a distribution list with demographic data to embed in your survey.
• Analysis: Analyzing survey results, breaking down information by demographic sub-groups, and creating visualizations of data.

Anonymous vs Confidential Surveys – What’s the difference?

Making your survey anonymous or confidential is a great way to encourage candid, honest responses and, in some cases, boost response rates. Anonymity and confidentiality are not the same, and it is important to ensure you are using the right language when communicating with potential survey respondents.

 

Anonymous Surveys

In an anonymous survey, a respondent cannot be identified, even by the individual administering the survey. Anonymous surveys do not collect or embed any unique identifiers, including, but not limited to: name, email address, student/employee ID, or phone number. For a survey to be anonymous, you must ensure that the software you are using is set to not record IP addresses. Qualtrics makes anonymizing IP addresses and contact information easy.

Even if you are not collecting unique identifiers, your survey may not be anonymous if you are embedding or collecting demographic data. If any combination of variables in your demographic data could likely identify a respondent, your survey may not be anonymous. For example, if you’ve included gender, race/ethnicity, school/college, and have a respondent who is identified as a Male, American Indian/Alaska Native, in X College, someone may be able to reasonably assume the respondent’s identity and therefore your survey may not be anonymous.

If you’re embedding or collecting demographic data in your survey and aren’t sure if doing so will remove anonymity, please reach out to OIRE to discuss.

 

Confidential Surveys

In a confidential survey, a respondent could be identified, through unique identifiers or a combination of demographic variables, but the survey administrator is promising to not disclose the identifiable information to others. When administering a confidential survey, you must ensure proper data collection and storage so that others cannot access the information.

In order to maintain confidentiality when reporting data from a confidential survey, responses must be de-identified. The most common way to de-identify data is to only report aggregated data, and not report data for groups that, when aggregated, still have a small sample size. OIRE commonly uses 5 or 10 as the minimum sample size for reporting de-identified data.

While we recommend embedding demographic data whenever possible, there may be times when you need to ask demographic items. In these cases, it can be helpful to include a statement about how data will be de-identified that supplements your initial confidentiality statement. For example:

Only de-identified, aggregated data will be shared in any reporting of the survey findings. Demographic data will not be broken out by group unless there are at least 10 representatives in that group.

 

Open-Response Items

For both anonymous and confidential surveys, there are additional challenges to consider when including open-response items. Survey respondents may include identifying details, share information unrelated to the survey, or disclose information that raises concerns. For these reasons, we strongly encourage you to include the following language any open-response items:

Note that your response may be seen by multiple people, including campus leadership. If you choose to share identifiable information, such as names, we cannot guarantee anonymity/confidentiality. Information disclosed may or may not initiate outreach or a formal investigation and does not constitute official reporting to the university.

 

Emailing Surveys

When sending anonymous or confidential surveys, it is best practice to inform participants in the initial email if their response is anonymous or confidential. We recommend including a brief statement explaining anonymity/confidentiality as it relates to your survey.

 

Sample Language for Emailing Anonymous Surveys

Your response to this survey is anonymous. We know that you must have the assurance your responses are anonymous and secure if we are to receive accurate and candid feedback. Survey participants' privacy is of utmost importance to us, and strict internal safeguards are in place to ensure that privacy.

No identifiable information is being collected with your response, and your response cannot be connected to you in any way.

Survey responses will be aggregated for analysis and reporting. Your anonymity is our first duty of care and will be protected in all reports resulting from this survey. We are committed to ensuring that individuals can provide candid feedback with confidence.

 

Sample Language for Emailing Confidential Surveys

Your response to this survey is confidential. We know that you must have the assurance your responses are confidential and secure if we are to receive accurate and candid feedback. Survey participants' privacy is of utmost importance to us, and strict internal safeguards are in place to ensure that privacy.

No one outside of [office] has access to survey respondents’ identities or to files that may connect names or email addresses with answers to survey questions.

Survey responses will be aggregated for analysis and reporting. We will aggregate the data into groups of 10 or more responses in order to maintain the confidentiality of survey participants. If any demographic group or combination of group characteristics has fewer than 10 individuals, those data will not be reported. [office] will not provide any findings that would risk making someone identifiable due to the uniqueness of their demographic characteristics. Your confidentiality is our first duty of care and will be protected in all reports resulting from this survey. We are committed to ensuring that individuals can provide candid feedback with confidence.

During the fall 2021 term, all CU Denver employees and students were invited to participate in the Campus and Workplace Culture survey. The survey asked respondents a range of questions around their sense of belonging and value within the campus community and invited them to speak to specific moments of harm/harassment they may have experienced in their time with CU Denver.

Survey responses were collected during October and November of 2021. Starting in December, the Office of Institutional Research and Effectiveness (OIRE) began the process of cleaning and organizing the data, so that the data could be analyzed and shared with the campus community. While this process was taking place, the Campus and Workplace Culture Steering Committee, led by Antonio Farias (Vice Chancellor for Diversity, Equity, and Inclusion), shifted their focus towards both the immediate reporting needs of the CU Board of Regents and towards longer-term planning for the eventual actions plans that would be based on the results of the survey.

In January, 2022, overall response rates were shared with the Board of Regents digitally. Those response rates then led off a more expansive presentation delivered in-person at the Board of Regents meeting on February 10, 2022. This BOR Presentation focused on the required Regent metrics around incivility and mental health, presented at a very high level. 

February and March were spent cleaning and analyzing the data, and constructing an initial series of dashboards to share data with the campus community. On April 7, 2022, the Board of Regents was presented with a summary of survey results from each surveyed population – faculty, staff, undergraduate students, and graduate students. Following the presentation, the dashboards were published on the OIRE Website.

Late spring and summer 2022 will see continued work to dive further into the details of the results and refinement of the action plans driven by the survey and our strategic initiatives. The Campus and Workplace Culture Steering Committee is committed to allowing for as much data transparency as possible within the data security confines outlined at the surveys launch - particularly that data from response groups of fewer than ten individuals will never be shown and that all results will remain anonymized.

If you have any questions or concerns about this process, please contact the Campus and Workplace Culture Steering Committee at cwcsurvey@ucdenver.edu.

With regard to external surveys, there may be times when university personnel are approached by an external individual, group, or other entity, such as the CU Foundation, for the purpose of soliciting participation in a survey that involves University of Colorado Denver | Anschutz Medical Campus students, faculty, staff, or alumni. University personnel approached by such groups are strongly encouraged to contact OIRE to ensure the external survey does not overlap or otherwise conflict with a university-wide survey.

The Office of Institutional Research and Effectiveness supports the data collection and analysis of campus surveys that are administered to all students on the Denver and/or Anschutz Medical Campuses. This includes the National Survey of Student Engagement (NSSE), which is available to all freshman and senior-level undergraduates and is administered once every three years. It also includes system-wide University of Colorado surveys as they are released. Results from these surveys are made available on the OIRE website. 

Some surveys may be subject to review and regulation by the Colorado Multiple Institutional Review Board (COMIRB), the university’s institutional review board. COMIRB protects the rights and welfare of human research subjects who participate in surveys and other research activities. Please visit the COMIRB website for more information on what research protocols require COMIRB oversight.  

Some surveys may also require approval from the institutional unit (program, department, college, and/or school) enrolling the student population you seek to survey. For example, in the School of Medicine, requests to survey medical students are vetted through a subcommittee of its Curriculum Steering Committee. You may need to check with the appropriate institutional unit to ensure access to your survey population.