US and German IFG participants were recruited using standard methods, namely, from patient/consumer databases of individuals willing to take part in market research. These databases are maintained by market research companies specifically for such purposes. Some additional participants were recruited by asking database referrals to suggest others they know with similar problems (oily skin). In the US, a small number of participants (n = 4) were recruited from prior face-to-face focus groups addressing patients' concerns and experiences with oily skin.

Potential recruits between the ages of 18 and 65 years were screened by telephone using a Recruiting Questionnaire (i.e., the Screener) and those who met the following criteria were invited to participate:

1. All participants were required to:

     • Perceive portions of their face or their scalp to be oily

     • Experience that their oily skin/scalp was bothersome

     • Actively and regularly attempt to control the level of facial/scalp oiliness

2. A proportion of the samples also included individuals who experienced the following:

     • Mild or moderate acne

     • Seen a dermatologist in the past 2 years for their acne

     • An oily scalp and were also balding (males only)

     • Represented Asian, Black, Latino/Hispanic, White/Caucasian peoples

     • Represented various regions of the country (US only)

The current consumer-based POC study used an on-line IFG application called FocusForums™ to explore how individuals with oily skin characterize and evaluate both the symptoms and impact of their condition on their daily lives. This IFG application contains a number of functions to assist with development and refinement of content for the new PRO item pool (see Table 3).

A

Topic Guide

Thematic Coding Schedule

Table 4 presents a truncated example of the frequency counts of the number of unique individuals who made comments in each of the thematic coding categories. Great skill and patience was required of the moderators to read and code the large number responses (over 770 US and 1040 German responses), each response often contained a number of subtly inter-related themes, in such cases multiple codes were applied. The involvement of moderators in this coding task was a significant alteration in their usual qualitative activities.

As indicated by '**' coding categories in Table 4, some thematic codes were applied more frequently in one of the two countries. These differences were discussed during teleconferences between the IFG moderators and the PRO Development team. Moderators, drawing on their first-hand experience within the IFG sessions, lead the discussion about how such differences in thematic endorsement might be explained. Table 5 presents the possible reasons for observed differences in the coding frequencies between the two countries and the questions that need to be addressed in order to evaluate each of these reasons.

Differences in sample characteristics of the focus groups could have lead to differences in how the participants elaborated and explored topical issues. In turn, such differences could have affected how responses were ultimately coded. Although a standardized recruitment screener was used to help assure that the composition of IFG membership was consistent across countries, some sampling differences may have been culturally unavoidable. For example in this study, the samples of US and German IFGs differed on their medical treatment histories. IFG participants in Germany reported more medical consultations for their condition than those in the US. This may have been due to differences in access/use of health service delivery systems in the two countries or differences in the severity of the condition itself.

During cross-cultural harmonization discussions, it was determined that some differences in coding frequency arose from variation in the number and types of probing questions used by the IFG moderators. While the moderators used the same Topic Guide to facilitate the IFGs, they used additional probes to develop a more comprehensive understanding of certain issues and behaviors. The practice of spontaneous probing is wholly consistent with qualitative research methodologies 50. These probing questions were not prearranged, but rather emanated from the unique dynamics and flow of discussion within the particular IFG. In response to supplemental questioning, IFG members likely made additional comments and because these probes were not applied equivalently across groups and countries, the frequencies of certain thematic categories were unequally represented. An example of differential probe use can be seen in the Distress/Interruption sub-section of Table 5, where US and German coding frequencies differed on "preoccupation with appearance". Such differences should not be automatically assumed to represent a true cultural difference.

Other differences in content frequencies may have been due to how moderators decided to code participants' responses. Decisions about how to classify a particular response were not always clear-cut and were based on coder interpretation. In such instances, moderators made independent judgments about which coding categories to assign to responses. Since coding categories were occasionally changed in response to what was observed within the response transcripts, reliance on inter-rater reliability analyses and coder retraining (an often used exploratory research method) was not considered a useful focus in this study. Moreover, the primary purpose of the content coding activity was to highlight areas for discussion, not to focus on the reliability of the coding schedule itself 51. An example occurred when a modification of the German coding schedule was made to account for a distinction between oiliness of the 'side of nose' versus the 'nose', the US moderator on the other hand, used only the 'nose' code to characterize both types of responses. When such distinctions were encountered during harmonization discussions, moderators evaluated the potential reasons for distinctions and typically agreed to collapse categories where differences were not thought to be culturally determined.

A final explanation for the differences in thematic frequency counts relates to the distinctive linguistic, conceptual, and experiential differences which exist between the two cultures. For example, differences in the use of dry blotting versus wet blotting codes lead to a further review of the original transcripts in this area. It was determined that dry blotting was preferred by US females because, unlike wet blotting, this method of facial oil control did not require them to reapply their make-up foundation. On the other hand, German females, who mentioned fewer make-up concerns and a greater reliance on facial powder to control the appearance of oily skin (shine), seemed less concerned by washing; possibly due to the relatively straightforward task of reapplying facial powder. Possibly providing some support for this notion, both US and German males (who did not report using make-up) indicated that they washed the face with soap and water more often than female participants.

Another potential area of cultural difference was the mention of eating behaviors as a way of reducing skin oiliness. The moderators suggested that the German culture may foster a mindset of "avoidance" of things that might be harmful; while those in the US may tend to believe they can prompt favorable outcomes by being proactive and engaging in positive behavior. This working hypothesis arose out of the observation that German participants more frequently indicated they attempted to control excess sebum by avoiding "bad" things such as chocolate and sweets; whereas US participants more frequently indicated that their skin would be less oily if they did "good" things such as eating "healthy foods." Such differences may reflect cultural differences in how individuals understood and approached the daily management of their condition.

Following harmonization discussions to identify potential areas of cultural differences, PRO item pools were developed based on the most commonly occurring coding themes. During item design, the original IFG transcripts were revisited to assure that wording, phraseology and concepts in the new assessments reflected those used by the focus group participants in each country. Once the questions for the new oily skin scales were drafted, the IFG participants were invited back to provide cognitive debriefing feedback and to rate the degree to which the proposed items addressed important aspects of their condition. The item importance ratings provided yet another opportunity to assess cultural differences in the relative importance of item content and how items might perform differently between the two countries in the future. Table 6 provides an example of importance rating results for a new set of "Symptom Bother" rating scales.

The largest difference in importance ratings of these rating scales occurred on the 'self-conscious' item, with German IFG participants indicating the term was much less important than the US participants. This 'relevancy' or 'importance' rating difference suggests that the cross-cultural performance of this item in particular should be subject to closer inspection during later construct validation activities. Interestingly, self-consciousness was also singled-out by a professional PRO translation services as a term that was difficult to translate into German.

In the past, the role of professional moderators has addressed the largely independent mandate to conduct qualitative inquiry within focus groups sessions. Once moderators identified the major focus group themes and issues which seem important, these themes and issues were then summarized in a final focus group report. Typically, the involvement of moderators ended as they passed this report on to the PRO development teams responsible for preparing the draft PRO item pools and construct validation activities. In the current study, moderators were much more active in instrument design activities, particularly the thematic coding and frequency analyses. It is informative to review some of the philosophic and methodological tensions that moderators may encounter as they take on this new role. Tensions which also seem to exist between various schools of thought about research methodologies in the health sciences, social science, and field of applied marketing 52535455.

When qualitative focus groups are used to validate the content of new PRO measures, either explicitly or implicitly, the investigative methods used by two different epistemologies come into contact. These ways of gleaning 'truth' can be characterized as belonging to either a qualitative tradition, based on an inductive and phenomenological approach; or a quantitative tradition, based on a deductive and positivistic approach 5354. By nature, qualitative focus group research is inductive, open-ended and flexible, responding to the flow of each unique session, rather than closed-ended and fixed. Consistent with various qualitative research methods, the focus group inquiry allows the patients the freedom to provide information that does not necessarily fit with any expectation/hypotheses going into the research. It is precisely this openness to new and unexpected information that allows measurement designers to more fully "ground" the content of new Patient Reported Outcomes in the concerns and issues that patients think are relevant 56.

In turn, PRO design specialists use this deeper understanding of patient themes and issues to design pools of questions that measure the relevant content 57 and the performance of new assessment scales are evaluated in subsequent psychometric studies. These later psychometric studies utilize quantitative (statistical) methods to reduce the length and detail of surveys so that they only measure the most important concepts to most respondents. The resulting measurement scales allow for the qualitative assessment of predetermined concepts – an approach which appears to run counter to principles of qualitative inquiry. Supporting the distinction between qualitative and quantitative methods, Brookes suggests that qualitative methods are used to validate conceptual meaning using phenomenological data (an inductive approach) and quantitative validation activities focus on measurement and operational activities associated with the hypothetical deductive approaches of positivistic science 58.

The apparent duality between qualitative and quantitative methods, however, may not be clear cut and some have argued that both inductive and hypothetical-deductive methods of inquiry may compliment each other 596061626364, or at least provide similar results 65. Supporting a blending of traditions, advocates of most qualitative schools of thought acknowledge that any inquiry is influenced to some degree by the interests and understanding of the interviewer, as well as the objectives of their qualitative work. In order to account for such influences, qualitative research methods often include self-reflective activities where the interviewer identifies their own influences on the processes of qualitative exploration and interpretation 66.

Parallels can be drawn between the influence of moderator's personal knowledge on the direction of qualitative inquiry and the influence of a body of knowledge in a particular field on what is explored within a focus group (67, pp. 92–4). Indeed, current PRO development guidelines recommend that instrument design start by defining a clear 'conceptual framework', developed with input from key clinical opinion leaders' who have experience understanding patient perspectives and a good understanding of applied outcomes research 6869. The 'conceptual framework' should not be confused with a 'conceptual' or 'theoretical

model

Early in the current study, moderators expressed concerns that the Topic Guide and coding activities lead to a quantitative reduction and over-simplification of qualitative findings. Questions arose as such as: 'Do we lead the lines of inquiry too much?' 'Does the detailed coding activities focus too much on the detail versus the bigger picture?' and 'Is it really necessary for the moderators to perform the coding functions?'. Such questions reflect initial concerns as exemplified by the follow statement made by one of the moderators:

"I personally have struggled considerably with the coding. I found it an enormous "stretch" as a qualitative researcher, who tries to see the "big picture in the constellation in the stars," versus focusing on small details."

Over time, however, moderators began to see practical value in thematic analysis as they explored the reasons for differences in thematic endorsement between the two countries. Recursive discussion about the various thematic differences resulted in more expansive ways of describing observations and distinguishing cultural differences from other sources of thematic variation – resulting in a deeper understanding of cultural issues and perspectives. The re-review of previously coded transcript materials provided substantive examples of how in-depth analysis of responses was stimulated by analysis of coding frequency results.

Moderators had a number of suggestions about what could have been done differently in the future and their suggestions provide some direction for refinement of the methodology.