Dancing with Data

Statistics are human beings with the tears wiped off. ~Paul Brodeur

The first step to locating any kind of data is planning. Carefully identifying exactly what data you are looking for will help ensure that your search for quality data is successful. With so much information just a Google search away, the less-than-focused information seeker can easily get lost in volumes of poor quality or extraneous data. Use the following questions to help you prepare for an efficient and effective search: 

1.   Why are you looking for data? What is your goal? What is your research question?

• Do you have an issue for which you are trying to identify a solution? Can you define the issue clearly and specifically?  Narrowing your target or sharpening your question will help ensure the results of your search include the answers you are looking for.
• Do you have a message that you want to convey? Are you looking for information on the depth of the issue or the breadth (e.g., “How many people are in medical debt?” or “To what extent are people in medical debt?”) or both?
• Are you looking for information on solutions to a problem, or do you already have solutions in mind that you want to support or strengthen with quantitative data?

Good data can both identify the need for change and justify the solution.

2.   What type of data would best illustrate your issue?

• Prevalence data (number of individuals impacted)
• Intervention-specific data (cost of intervention, impact of an intervention on a child’s quality of life)
• Data that show the relationship between two factors (i.e., income and disability)

3.   Who is your audience?

• What can you assume your audience will know about the issue?
• If you are talking to legislators, do you have information about the cost of your proposed program? What about the cost of competing programs? Most policymakers require some knowledge of cost implications!

4.    What data do you expect?

• What types of data do you expect to find? Based on your research question, are you seeking out cost data? Prevalence data? Programmatic data? Qualitative data such as family feedback on a program? Keep in mind that using a combination of numeric data and more subjective or qualitative data can enhance your argument and help you make a stronger appeal to your audience. 
• Do the data that you find show what you expected it to show? If the data differ from what you anticipate, you may want to consider the factors that may have caused this.   Should you revise your research question?

In order to effectively present data, it is important to think about some of the limitations or biases in the data. Even good data can’t tell the whole story! Data should make sense. If they don’t, try to figure out why. It could be that one of your initial assumptions was wrong, or it could be that there is something about the way the data were collected or presented that make the results different from your expectations.

Once you have answered these four questions, you are ready to get the data! Now it’s time to get on the Internet. A few good starting points for data on CYSHCN are The Data Resource Center and the Catalyst Center State Chartbook.

The Data Resource Center and the National Survey of Children with Special Health Care Needs

The Data Resource Center for Child & Adolescent Health (DRC) is a project of the Child and Adolescent Health Measurement Initiative (CAHMI) at Oregon Health and Science University
and is funded by the Maternal and Child Health Bureau. It conducts the National Survey of Children’s Health and the National Survey of Children with Special Health Care Needs. The Centers for Disease Control and Prevention (CDC) oversees the sampling and administration methods of this project to ensure that the results are the closest possible reflection of the entire population.

On this website you can find information about children in general and CYSHCN with respect to:

• MCHB Core Outcomes for CSHCN
• CSHCN Health and Functional Status
• Health Insurance Coverage and Program Participation
• Health Care Needs and Access to Care
• Care Coordination and Family Centered Care
• Impact on Families

Tips for Using the Data

• When looking at the results, keep in mind that the National Survey of Children with Special Health Care Needs uses the Maternal and Child Health Bureau definition[1] of special health care needs, which is very broad.
• Cite the data using this citation: National Survey of Children with Special Health Care Needs. NS-CSHCN [Survey Year Cited]. Data query from the Child and Adolescent Health Measurement Initiative, Data Resource Center for Child and Adolescent Health website. Retrieved [mm/dd/yy] from www.childhealthdata.org.

The Catalyst Center Chartbook: www.hdwg.org/catalyst/online-chartbook 

Our Chartbook can help you to think about some of the contextual data that you might want to include in your presentation, in addition to data directly about your topic. On the Chartbook’s state pages, you can find data on a variety of factors related to the coverage and financing of care for CYSHCN, including:

• Demographics
• Economics
• Child Health Services
• Factors Influencing Health Insurance Coverage
• Experience with the system of Care for CSHCN
• Title V Program

In addition to the raw data, you can find links to the data sources (if available) and tips for using the data in the Chartbook. If you click on “Show Full Data Sources,” you will find references including the hyperlink to the source under each data section. By visiting the website of the data source, you can find additional information and ensure that you have the most up-to-date information.

If you use information from the Chartbook, cite the data using this citation: The Catalyst Center: Improving Financing of Care for Children and Youth with Special Health Care Needs.  State-at-a-glance Chartbook on Coverage and Financing of Care for Children and Youth with Special Health Care Needs, retrieved (mm/dd/yy) from www.catalystctr.org. The Catalyst Center is a project of the Health and Disability Working Group at the Boston University School of Public Health and is funded by the Maternal and Child Health Bureau of the Health Resources and Services Administration, U.S. Department of Health and Human Services, under cooperative agreement #U41MC13618.

If neither of these two resources provides the information you need, there is always Google!

Using Google as a research tool

The difficulty with Google is that it can be hard to find good data. There is a lot of bad data on the Internet, and weeding out the bad from the good can be challenging. Here are some pointers on using Google (http://www.google.com) as a search tool:

• Identify a list of search terms or synonyms for what you are looking for. Try to be thorough and methodical. Brainstorm as you start your search. Keep your list and add to it. Cross off the search terms you have already used.

o   For example, if you are interested in information about children with autism, you could also search using ‘children with autism spectrum disorders’, ‘children with ASD’ or “children with developmental disabilities.”

• There are several ways to improve the likelihood of finding useful and accurate information through Google.

o   Click the Advanced Search link that is to the right of the search box. Here you will see a list of options. One of the most useful is the ability to “search within a site or domain.”  This allows you to search for you search term in only a certain type of website.

• “.edu” will search only educational sites such as universities. These sites will have primarily academic papers.
• “.gov” will give you government sites.
• “.org” will give you non-profit organization sites.

• Try placing phrases in quotations (e.g., “Katie Beckett”). If you do this, you will only receive pages that have the phrase written in exactly that way (as opposed to a list of names including Katie Smith and Samuel Beckett, for example). If you write the term as plural or hyphenated you will only receive links to pages that use the term in plural or hyphenated form. This may improve your chances of finding what you need, or you may need to adjust the way you phrase your search term so that you do not miss relevant pieces.  
• Don’t include words such as “in” or “the” in your search terms. The search terms do not have to be grammatically correct phrases or sentences. Also, Google searching is not case sensitive, so you do not need to capitalize.
• If you click on the “More” button at the top left of the Google search page, you will find a link to “Scholar.” Google Scholar searches primarily academic sites and scholarly journals.  Many of these sites now have free articles. If not, most at least allow you to see the abstract.

5.   Bonus question: Who did you get the data from? Can you partner with them?

• If they are collecting data about the topic you are researching, they might very well be interested in the same issues that you are.   Don’t hesitate to follow up on the data.

A workshop titled “Merging Data and Policy for Children’s Health: Influencing Change at the State Level using the National Surveys of Children’s Health and CSHCN” took place at the four-day AMCHP (Association of Maternal and Child Health Programs) Conference in Washington DC in February 2011. To view the recorded presentations or to read transcripts of what was presented, visit http://webcast.hrsa.gov/conferences/mchb/amchp2011/data_to_action.htm.

[1] The MCHB defines children with special health care needs as those who have, or are at increased risk for, a chronic physical, developmental, behavioral, or emotional condition and who also require health and related services of a type or amount beyond that required by children generally.

Compelling individual stories move people to act in a way that statistics alone could never inspire.  In one research study, individuals were asked to donate money to a cause after reading a document. Half of the participants read a statistics-based document, and the other half read an individual story. The people who read the individual story gave, on average, twice as much as those who read about the statistics (Heath & Heath, 2007). Family stories are just as important as statistics in illustrating an issue. Here are a few tips for collecting this information from individuals:

1.   Informed Consent

• Regardless of what you are planning to do with your findings, it is important to let your interviewees know that their participation is voluntary and should they refuse to participate, their relationship with you or your agency will not be impacted. The best method of collecting informed consent is in writing. Written documentation may help people to understand the project and provides participants with something that you can refer back to when they want to be reminded about their role in the project. Informed consent also provides your agency with an important record of participants’ voluntary participation.
• Informed consent should include the following:
  • A description of the project and purpose of asking for the information
  • The risks of participating (i.e., the questions might make you uncomfortable or once your personal story is made public you may be contacted by people who want to discuss it)
  • Phone number and email for someone who respondents can contact for any questions they may have after the interview.
  • If you are planning on publishing your findings on the Internet, you may want to include a statement advising participants that once information appears on the Internet, it is very hard to control its dissemination. While an article or piece of information can be taken off of an organization’s website, it cannot be taken off the web as a whole.
  • If you are collecting information on a child who is developmentally older than age twelve, the child should provide assent for his or her story to be shared.  Assent can either be collected verbally or in writing on a separate age-appropriate form.
  • One issue to consider is whether or not your organization or group would be comfortable publishing stories where the real name of the family is not used. For many families, using a pseudonym may assist them in feeling comfortable sharing their story.

2.   Screening

• Before conducting a full interview you may want to screen families to ensure that they have had the type of experience you want to illustrate and that they will be willing and comfortable sharing their information. You should ask for consent even before conducting the screening.

3.   Full Interview

• Write out your questions in advance. This might seem counterintuitive, since you predominantly want to hear their story. However, many people have a hard time figuring out how to start describing their personal story. Beginning with a set of simple questions and then moving on to more open ended questions may improve your interview and make the interviewee more comfortable.
• Try to collect as many specific details as possible.

4.   Confirmation of Accuracy

• It is important that your stories are accurate and make sense in the context of the point you are trying to make. One way to confirm accuracy is to allow families to read the story before you publish it and welcome their feedback.

Storytelling was the focus of one of the plenary sessions at the four-day AMCHP (Association of Maternal and Child Health Programs) Conference in Washington D.C. in February 2011. To view the recorded session, titled “Storytelling: The First Big Thing,” or to read a transcript of what was presented, visit http://webcast.hrsa.gov/conferences/mchb/amchp2011/plenary.htm#iii .

References

Heath, C., & Heath, D. (2007). Made to Stick (1 ed.): Random House.

General

• Present your information so the reader can understand why they should care about the issue.
• How much time will the audience have to either listen to you or to read what you prepare?  People with more limited time might appreciate receiving information visually and in bullets so that they can quickly digest the information.
• What does your audience already know about the issue, and what do you need to explain?

Data

• For percentages, you may want to use a pie chart.
• For change over time, you may want to use a line graph.
• Comparing multiple populations might be represented in a bar graph.

Remember to include the limitations of your data so that information can be placed in context. Often, data collection methods mean that the results may not be entirely accurate or comprehensive, and it is important that the audience understand this. You may want to include in your presentation a section on how the data were collected.

You may want to address such questions as:

• Are the data representative of the population or might the data sample impact the results?
• Who or what agency collected the data? Do they have an investment in having the data appear a certain way?
• What do the data not show (what questions didn’t they ask that might more fully explain the situation)?