Assessing social care-related quality of life: evidence from England using the ASCOT service user measure

By Eirini Saloniki, Research Associate at Centre for Health Services Studies & Personal Social Services Research Unit, University of Kent,
Health preferences research is now a substantial field in itself, known not least for its contribution to assessing different health and care services by measuring the outcomes that people experience in using these services. Well-known and comprehensive indicators are used to measure these outcomes whereby people place different values on the different ways that services (can) impact their quality of life.

Preference studies have traditionally used face-to-face (paper-and-pencil or computer-assisted) interviews to gather such data. Despite their good completion rates and reliability, face-to-face interviews are costly and time-consuming. A shift towards internet surveys has been gaining popularity, allowing for accurate recording of response time and for targeting groups of respondents quickly at lower cost. Nevertheless, internet surveys have also been criticised, with poor data quality and difficulty in achieving sample representativeness being the main concerns. Can the advantages and disadvantages of the different methods of data collection be a source of variation in the results? Yes, not only as a result of how someone responds, but also because of who responds to each survey.

Several studies in environmental and health economics have compared preferences elicited from face-to-face interviews and internet surveys, but the evidence is mixed. The discrete choice experiment (DCE) technique was commonly used amongst the different studies to elicit preferences while the sampling process varied considerably. Another technique – best-worst scaling (BWS) – known for its lower cognitive burden compared to the DCE, has not been used to compare preferences elicited from different data collection methods. Nor have preferences been compared across the different methods in the context of long-term care.

In this paper, for the first time, we compare preferences elicited from face-to-face and internet surveys for the best-worst scaling (BWS) task using the Adult Social Care Outcomes Toolkit (ASCOT) service user measure. ASCOT measures social care-related quality of life (SCRQoL) across eight domains (accommodation cleanliness and comfort, safety, food and drink, personal care, control over daily life, social participation and involvement, dignity, occupation and employment), and has been recommended by NICE for use in the economic evaluation of social care services. This paper is part of a larger study (EXCELC) that aimed to establish relative preferences for care-related outcomes for people using long-term care.

The BWS experiment involving the ASCOT measure was included in a face-to-face (n=500) and an online (n=1001) survey and was completed by a sample targeted to be representative of the general population in England. Each respondent in both surveys was presented with a set of eight hypothetical scenarios. Each scenario contained eight attributes reflecting the eight ASCOT domains. In each scenario, the respondent was firstly asked to select the best (or most preferred) choice, with the selected choice being greyed out. The same process was repeated for the worst (or least preferred) choice, the second best, and second worst choices. Thus, each respondent made in total 32 choices.

We used a multinomial logit framework to analyse the data, ensuring that it is appropriate to pool the two datasets (internet and face-to-face). The initial pooled model assumed that there are no differences in observable characteristics between the two samples (i.e. no taste heterogeneity exists), but it included a parameter to control for scale differences between the two datasets. An additional (pooled) model controlled for differences in the sample composition of the two datasets as well as scale heterogeneity between different groups of respondents across the two datasets. All models took into account the repeated nature of the data.

The results from our first pooled model revealed several but small differences in preferences for SCRQoL across the two methods of data collection, with half of the coefficient differences compared (15 out of 30) found to be statistically significant at the 5% level. The number of significant differences compared reduced substantially to five when further controlling for observable and unobservable characteristics between the two samples. The limited (and small in value) significant differences were mostly at attribute levels indicating higher needs. A number of significant scale effects were also identified – for instance, participants were less certain when making their worst choices than when making their best choices which relates to framing effects reported in the preferences literature.

Overall, we find fairly similar preferences for SCRQoL between the two methods of data collection, suggesting that we can be confident enough in the internet results from a practical point of view. There is scope for future studies to not only explore a different sampling frame, but also consider providing sufficient clarifications to internet respondents in an attempt to minimise the level of uncertainty in the choice process for this group.

For a copy of the full paper please see:

The lives of Austrian home care service users – first interviewer impressions

Authors: Assma Hajji, Judith Kieninger, Ruth Fulterer, Judith Litschauer and Birgit Trukeschitz (WU Vienna University of Economics and Business, Research Institute for Economics of Aging)

Over 400 interviews with Austrian home care service users were originally planned as part of the EXCELC project in 2016/2017. More than 600 have been carried out successfully. Data collection was supported by the Austrian Federal Ministry of Labour, Social Affairs, Health and Consumer Protection, by social care organisations and local authorities.

More than 60 trained interviewers took on the task of visiting people in need of care in their own homes to conduct interviews. About half of them gave us detailed written feedback describing their experiences and their impressions. This blog is based on unstructured accounts of their experiences. Quantitative analyses of the data collected through the standardised personal interviews can be expected in September 2018.

“Sharing a part of their lives”

Respondents freely voiced their hopes and fears relating to their situations. Our interviewers were able to gain insight into the respondents’ lives and their care situations both through the interviews and the conversations that often followed. One interviewer happily noted: “the ladies were very vivid in their descriptions, it felt like they were sharing a part of their lives with me”.

The term “in need of long-term care“, and the multitude of situations to which it can refer

Some long-term care service users are quite affected by cognitive decay, memory problems, or are depressed or afflicted by other mental conditions. Others, however, have an active mind but are trapped in a frail body. They, for example, follow technological innovations and keep up with digital means of communication – such as Mr. S, who likes to download the newest apps onto his smartphone, or Mrs. M, who regularly uses Skype to make phone calls to her grandchildren living abroad. Digital innovations aside, they also get creative in other areas when it comes to dealing with the physical constraints they are facing. Mrs. L, for instance, makes use of a cord and basket to bring groceries and mail up to her first-floor apartment.

The challenge of accepting one’s own need of care

The first important step – and often a challenge in itself – consists of coming to terms with one’s own need for care and in accepting help and assistance. Sometimes, this leads to a reversal of roles – as in Mrs. W’s case: she has spent most of her life caring for her physically disabled daughter but now finds herself in need of assistance because of cognitive impairments. While some people take a long time to come to terms with restrictions they face, others adapt to their new situation rather quickly. In the course of the interviews respondents discussed the process of adapting and adjusting to new circumstances. In many cases, these mechanisms are offered as an explanation as to why respondents are happy and doing well even in difficult situations. Our interviewers were often impressed with the respondent’s positive attitude and coping strategies.

Care at home – not enough assistance?

In many cases, our interviewers got the impression that the care services people received in their homes helped them a great deal – at least at first glance. Home care enables them to stay in their homes and help them overcome the obstacles and challenges of daily life. However, at second glance, it often became evident that the services provided only basic support and that some needs remained unmet.

As our interviewers have observed, it is the social needs of care recipients which are left unattended. Care service recipients feel clean, well-fed and safe, but seem to lack interpersonal relations and meaningful interaction. Visits from care workers are a welcome change but time is short and the required paperwork often ate up the little time left that could be used for some conversation.

The role of care workers and implications of recurring changes

Care workers are trusted and significant people in the lives of long-term care service users. This is especially true for persons living alone who don’t have many outside contacts – for them, visits from care workers can brighten up their entire day. On many occasions, interviewers have heard respondents talking about care workers using terms of endearment, speaking of their “fairy” or “wonderful Maria”.

However, recurring changes in personnel and uncertainties about the times of visit are often causes of discomfort or insecurity for the care service recipients. A great deal of flexibility and patience is needed on their part to adapt to new schedules and new caregivers, requiring them to explain over and over again what’s important and what to watch out for. This creates a lot of insecurity, especially for older people who might not be as able to adapt to changes and have a preference for clearly structured routines.

Care arrangements you can depend on – especially in the case of live-in carers

Some respondents with live-in caregivers have trusting and amicable relationships with them. Others, however, are less satisfied with their arrangements, voicing their despair and anger. Ideally, most care recipients would prefer a reliable, stable care setting. Finding a suitable live-in caregiver is difficult and full of obstacles, making the process difficult and tiresome. Moreover, even those who have found a live-in caregiver might still be faced with uncertainties, such as their care provider finding a more promising job and leaving them with no other option than to move into a nursing home.

Financial distress and its consequences

Financial strain, small pensions and a lack of means for taking up more care services are brought up and discussed. Thus, long-term care allowance (“Pflegegeld”) recipients became frustrated if applications for higher long-term care allowance had not been successful.

For some, lack of information is a legitimate problem…

Our interviewers noted that there tends to be a lack of information – many respondents don’t know about available long-term care services and ways of paying for them. Mr. P is one of those cases – he would like more social interaction in his daily life, but had never heard of visiting services or day care centres before the interview. Or Mrs. D, who was applying for care allowance for the first time and didn’t know who to turn to for assistance. There is substantial information available via the internet – but only a small share of those in need of care services have access to it or are web-savvy enough to use it for this purpose.

This has been a selection of interviewer impressions from interviews with long-term care recipients in Austria. Results from the statistical analyses will be available in September 2018.

EXCELC team at 3rd ATHEA Conference

By: Birgit Trukeschitz & Assma Hajji (WU Vienna University of Economics and Business, Research Institute for Economics of Aging)

The Austrian EXCELC team held two presentations at the 3rd ATHEA (Austrian Health Economics Association) Conference in Vienna, both on preferences for long-term care outcomes in Austria:

  • “Understanding best-worst experiments for calculating quality-of-life preferences in long-term care settings: How they work and what to consider” (“Hinter den Kulissen – Wie funktionieren Best-Worst Experimente für die Berechnung von erwarteten Präferenzen bei Pflegebedürftigkeit und welche Faktoren beeinflussen die Ergebnisse?“)Assma Hajji gave an overview of the method used, the experimental design and methodological issues. The data was collected using a best-worst experiment for calculating preferences for different social care-related quality of life domains using the ASCOT questionnaire.
  • “Food over company or vice-versa? Results from a best-worst-analysis of expected quality-of-life preferences in long-term care settings for Austria” (“Gutes Essen oder nette Gesellschaft oder..? – Eine Best-Worst-Analyse zu den erwarteten Präferenzen bei Pflegebedürftigkeit in Österreich”)Birgit Trukeschitz presented preliminary results of the Austrian preference weights based on a general population sample, showing some clear differences in preference between various domains of social care-related quality of life.

Both presentations were well received and led to some insightful discussions on potential implications and next steps in the process.

Greetings from iHEA Boston and NHESG Helsinki

by Lien Nguyen Researcher at the Centre for Health and Social Economics, National Insitute for Health and Welfare (THL)

Two EXCELC team members presented their works at the iHEA World Congress in Health Economics held in Boston on July 8–11, 2017. Eirini Saloniki presented early findings on differences in preferences elicited from internet and face-to-face surveys, based on data from a best-worst scaling experiment using the English Adult Social Care Outcomes Toolkit service user measure (ASCOT-S). Ismo Linnosmaa gave a talk on our preliminary work exploring whether general good health leads to better social care related quality of life. Both presentations were well received. We are using the feedback from the useful discussions to refine our work.

In addition to the iHEA Congress, the Finnish team submitted two written papers to the Nordic Health Economists’ Study Group Meeting (NHESG) held in Helsinki on August 23–25, 2017. Ismo Linnosmaa submitted the paper on the relationship between general good health and social care related quality of life and Lien Nguyen submitted a paper presenting Finnish population preferences for the ASCOT service user measure. The discussants for these written papers (Professor Jan Aben Olsen, University of Tromsø, and Associate Professor Line Bjørnskov Pedersen, University of Southern Denmark) provided detailed and useful feedback, which has helped to further refine our analysis. Many colleagues showed an interest in our work and are looking forward to us returning to a future study group meeting with our final results.

The iHEA (International Health Economic Association) World Congress in Health Economics is a biennial conference traditionally held in July. The NHESG (Nordic Health Economists’ Study Group) Meeting is an annual conference traditionally held in August, involving researchers mostly from Denmark, Finland, Iceland, Norway and Sweden. The number of researchers, including doctoral students that attend the Nordic conference each year is about 70–100. Written papers must be submitted to the conference organisers beforehand and 40 minutes are allotted to each paper for discussion, led by a discussant. The discussant will first summarise the paper and then give his/her comments on the paper, after which the paper’s authors can reply and the audience can join the discussion. 

EXCELC Team Meeting in Helsinki 2017

The EXCELC team caught up in Helsinki last week (5 – 6 October) to discuss progress to date with the EXCELC project and the next steps for the study.
On Thursday, the EXCELC team discussed progress with the service user and carer fieldwork in Austria and Finland. Birgit Trukeschitz, Assma Hajji, Judith Kieninger and Judith Litschauer reported that they have nearly completed the Austrian fieldwork (with only a few interviews left) and presented descriptive results. Ismo Linnosmaa, Lien Nguyen and Hanna Jokimaki finished their data collection for the Finnish fieldwork and also presented descriptive results. We discussed various options for analysing the data collected from the fieldwork and constructed a workplan with several streams of analysis. After the meeting, the Finnish team led us on a walking tour of Helsinki and gave us the opportunity to try some delicious Finnish cuisine sourced entirely from Finnish suppliers!

On Friday, Laurie Batchelder and Eirini Saloniki presented results from the preference study, which included models for England for the ASCOT carer measure and models exploring the effect of preference elicitation methods on preferences for the ASCOT service user measure. Birgit Trukeschitz and Assma Hajji from the Austrian team and Ismo Linnosmaa, Lien Nguyen and Hanna Jokimaki from the Finnish team presented preference models for their respective countries. The team discussed the strategies and selection criteria used to choose the best fitting models and outlined the robustness checks needed before the results were ready for publication. The meeting ended by outlining proposed plans for each of the workpackages. Here’s to another productive and enjoyable meeting!

Excelc in OPTIMI

A short article by Professor Ismo Linnosmaa and Dr Eirini Saloniki has recently been published in the OPTIMI. OPTIMI is a newsletter in health and social economics concentrating on recent research and findings about productivity, effectiveness, costs, finance and incentives in health and social care in Finland. The newsletter disseminates recent research findings to support decision-making at various levels of health and social care. OPTIMI is published four times a year and it is edited at the Centre for Health and Social Economics, National Institute for Health and Welfare, Helsinki. To read the article please follow this link.

The Erasmus dream

By Eirini Saloniki, Research Associate at PSSRU & CHSS at the University of Kent.
It was nine years ago when I was given the opportunity to study for a year abroad at the University of Kent with the Erasmus programme. A fantastic and mind-opening experience – in fact, so fantastic that I returned to Kent not only to study for a PhD but now also working at the University. Who would have thought that I will live the Erasmus ‘dream’ again, this time as a staff member? Yes, the Erasmus programme is available for staff too…! I was eager to apply but I had to find a hosting institution first. It took a Skype meeting and “I suppose we talk about 2-4 weeks visit?” before I found myself applying to spend two weeks as a Visiting Researcher at the National Institute for Health and Welfare (THL) in Helsinki. The application process was fast and within a couple of weeks I was informed that I was successful. I put my excitement aside and focused on organising everything for the visit with the weather not being a concern as I was told that it will be over 10 degrees in May!

Nervous (as it was my first time in Helsinki) but at the same time ready to face the challenge, I spent my first day at the Institute trying to get used to the new routine and working environment, including having lunch at as early as 11:30am. Everyone was very welcome and it really felt like I was a member of the Centre for Health and Social Economics (within THL). The days to follow involved many meetings with health and social economists and late stays at the office (I must have lost track of time as it was so bright in the evening). I had the opportunity to discuss the projects we work on at the Centre for Health Services Studies and the Personal Social Services Research Unit with senior economists, and share best practice. I was also able to share my knowledge of how to control for taste and scale heterogeneity in preference elicitation for social care-related quality of life, and present some preliminary results from the work we are doing in Kent as part of the EXCELC project in an internal seminar. The seminar was well-attended with the audience showing real interest in the project. Importantly, I got to know all about the Finnish health and social care system, and its associated difficulties when it comes to data handling and interpretation.  This is an important consideration for the cross-national element of the project that I am involved in.
It was a great honour to be asked to co-write (with Professor Ismo Linnosmaa) a short piece about “Measuring effectiveness in social care: the present and future for researchers and policy-makers” which is to be featured in the THL newsletter, distributed across Finland.

Before I even realised, my time in Helsinki was over. This experience was invaluable and once again undoubtedly mind-opening. The interaction with senior economists working under a different research environment and the transfer of knowledge as well as discussions for future collaboration were beyond my expectations.

Would I do it again? ABSOLUTELY. Lots to learn in this visit but above all lots to gain. I wholeheartedly recommend the Erasmus programme to other researchers.

The EXCELC project: the interviewers’ experiences of encountering home care clients in Finland

By the Finnish EXCELC-team (National Institute for Health and Welfare)

During Autumn 2016 and Spring 2017, six EXCELC-project research assistants visited home care clients and their informal caregivers across many cities in Finland.  They interviewed them about their everyday life, use of home care and other themes related to long-term care services. During the fieldwork, the interviewers met very different kinds of people and have had a chance to listen to their life stories. This blog shares the experiences of the Finnish interviewers.

The interviewees considered the study important

In general, we were warmly welcomed by the interviewees in their homes. Sometimes the interview started with coffee that was set ready for the visitor.
Many interviewees told us that they considered the study to be important and therefore wanted to participate in it. They were pleased to hear that the National Institute for Health and Welfare (THL) is conducting the study and were satisfied that their voice would be heard through their participation in the interview. Informal caregivers often made significant arrangements to make the interview happen. For example, a caregiver who was living in another municipality, came to care for an older relative on her day off, assisted with the interview and agreed to be interviewed herself.

Respondents were motivated despite long interviews

The health of the home care clients and their informal caregivers we interviewed varied widely. Some were in very good mental and physical health while others were very ill. The questionnaires used in the interviews are quite long and demanding, and the duration and burdensomeness of the interview was a lot to take in for many interviewees. Nevertheless, only a few interviews had to be finished incomplete, although to complete some of them we needed to come back on several occasions. For the most part interviewees rarely took breaks, except to sip some coffee every now and then.

The questions of the ASCOT quality of life measure were challenging for most interviewees and the questionnaire as a whole was seen as quite burdensome, demanding and long. Even so, most interviewees were motivated to respond to the questionnaire. It was important for home care clients to reply themselves to the questions and to bring out their own thoughts although in some cases the informal caregiver had to act as a proxy respondent because of the health state of the home care client. After many interviews, we feel that people generally had a need to talk about their affairs, which may explain why people were happy to keep going through the questionnaire despite its length.

Loneliness of home care clients touched the interviewers

The interview situations have been important for both interviewers and interviewees. The stories told by the older people we interviewed have been very moving. Stories of interviewees in very poor health, in particular, are etched into our memories.

Hearing of the loneliness suffered from by home care clients and encountering veterans and their spouses has been very moving. Many of us had not spoken before to people who had fought in wars.  Alongside this sadness, however, there was also a lot of joy in people’s lives: for example, where the home care user has a close family that cares for them. In many interviews the conversations were cheerful, with laughter and people expressing hope for the future.

A unique experience and an opportunity to encounter different people

We have often stayed some time after the “formal part” of the interview had finished to talk to the interviewee about issues relating to either home care or other unrelated topics. The interviews and these more general discussions have been useful for both parties. The hundreds of interviews already carried out have given us a valuable insight into the performance of the elderly home care service system.

Many interviewees were grateful for the life they have lived and for the services offered by the welfare state. Although they sometimes criticized the service system, many older people were grateful for the services that are provided to them. For us, we have had a chance to hear various stories about respondents’ lives, regarding such as war time, loneliness, illness and hopefulness. Despite the challenges, it has been a unique experience and an opportunity to meet different people.

This blog contains thoughts and experiences on the interviews conducted in Finland. It has been written by six research assistants who conducted the interviews. Susanna Nevalainen wrote the manuscript after collecting thoughts and ideas, while Tuula Kontio, Miina Nikkanen, Maarit Ojanen, Iris Pykäläinen and Helena Rovamo contributed to it as commentators.
The field work is an important part of the EXCELC project. The project target is that the research assistants will interview almost 700 people in Finland: home care clients and their carers. Interviews are conducted in the cities of Helsinki, Espoo, Vantaa, Kuopio, Joensuu, Mikkeli, Kainuu Social Welfare and Health Care Joint Authority (Kainuun sote), South Karelia Social and Health Care District (Eksote), Tampere, Hämeenlinna, Lahti and Vaasa. The interviewers were given training before they began with their work and regular meetings have been held with them to discuss the fieldwork progress and challenges related to the work. Interviews are usually done at homes of home care clients and their informal caregivers. Interviews with home care clients have varied in length, about 1.5 to 4.5 hours; while those with caregivers have been generally slightly shorter. ASCOT quality of life measures are used in the study.



By Dr Laurie Batchelder, Research Officer at PSSRU Kent
Greetings from America! Just a quick update on how things went at ISPOR this week (International Society for Pharmacoeconomics and Outcomes Research – 22nd Annual International Meeting). I gave a talk on our preliminary work on the generation of preferences for the ASCOT-C in the English data, which was well received. I fielded a few questions about how we plan to compare these preferences for the ASCOT-C across countries, so I highlighted our plans for the cross-country analysis using a pooled model. Many people were also interested in when we plan to publish the ASCOT-C work, which is great news! In addition, my presentation had been selected as an ISPOR 22nd Annual International Meeting Research Presentation Award finalist!

I also presented our preliminary work on the comparison between face-to-face versus internet data for eliciting SCRQoL preferences using the ASCOT-S in the English dataset, which was also well-received.  Similarly many were very interested in this work and asked questions about the presentation of the Best Worst Scaling task & how this differed between the two modes. There was also a focus on looking at heterogeneity in a very detailed way (which we plan on doing). Again many people were very interested in ASCOT and when we plan to publish this work, which is great news!
In general, I found that there was a lot of discussion talking with others (economists, pharmacists, consultants) after the talk, at the poster, and even in some preference sessions that other methods, such as BWS is the “way forward” and the “next step” with eliciting preferences, given it’s less cognitively burdensome and that it provides more detailed information within a profile compared to TTO,  which I think is really promising.

Full poster and presentation references:

Saloniki E1, Batchelder L1, Malley J2, Burge P3, Lu H3, Forder J1
1University of Kent, Canterbury, UK, 2London School of Economics and Political Science, London, UK, 3RAND Europe, Cambridge, UK

CARER SOCIAL CARE-RELATED QUALITY OF LIFE OUTCOMES: ESTABLISHING PREFERENCE WEIGHTS FOR THE ADULT SOCIAL CARE OUTCOMES TOOLKIT FOR CARERS Batchelder L1, Saloniki E1, Malley J2, Burge P3, Lu H3, Linnosmaa I4, Trukeschitz B5, Forder J1 1University of Kent, Canterbury, UK, 2London School of Economics and Political Science, London, UK, 3RAND Europe, Cambridge, UK, 4National Institute for Health and Welfare (THL), Helsinki, Finland, 5Wu – Vienna University of Economics and Business, Vienna, Austria