This article describes the protocol for the development of an innovative smartphone-based dietary assessment application Traqq, including expert evaluations and usability testing.
To collect dietary intake data in a fast and reliable manner, a flexible and innovative smartphone application (app) called Traqq was developed (iOS/Android). This app can be used as a food record and 24-h recall (or shorter recall periods). Different sampling schemes can be created on either prespecified or random days/times within a predetermined period for both methods, with push notifications to urge the participants to register their food intake. In case of non-response, notifications are automatically rescheduled to ensure complete data collection. For use as a food record, respondents can access the app and log their food intake throughout the day. Food records close automatically at the end of the day; recalls close after submission of the consumed items. The recall as well as the food record module provide access to an extensive food list based on the Dutch food composition database (FCDB), which can be accustomed to fit different research purposes. When selecting a food item, respondents are simultaneously prompted to insert portion size, i.e., in household measures (e.g., cups, spoons, glasses), standard portion sizes (e.g., small, medium, large), or weight in grams, and eating occasion/time of consumption. Portion size options can be adjusted, e.g., only entry in grams in case of a weighed food record or time of consumption instead of eating occasion). The app also includes a My Dishes function, which allows the respondent to create their own recipes or product combinations (e.g., a daily breakfast) and only report the total quantity consumed. Subsequently, the app accounts for yield and retention factors. The data are stored on a secure server. If desired, additional questions, i.e., in general or those related to specific food items or eating occasions can be incorporated. This paper describes the development of the system (app and backend), including expert evaluations and usability testing.
Accurate dietary assessment is crucial to ensure the quality of studies on the role of nutrition in health and disease prevention. Currently, such studies generally use established self-report dietary assessment methods, i.e., food frequency questionnaires, 24-h recalls (24hRs), and/or food records1. Despite the fact that these methods are of major importance for nutrition research, they also possess various drawbacks, e.g., memory-related bias, social desirability bias, and are burdensome for the respondent as well as the researcher1,2. Recent technological inventions now offer the opportunity to overcome these drawbacks. During the past years, various research groups seized this opportunity and developed web-based and smartphone-based dietary assessment tools for nutrition research that address some of these known drawbacks (see Eldridge et al.3 for an extensive overview of web- and smartphone-based tools), i.e., reduce causes of error, improve user-friendliness, and decrease the participant's and researcher's burden1.
Nevertheless, the number of fully automated and validated smartphone applications (apps) that are appropriate for nutrition research is still limited. Most of the available dietary assessment apps (i.e., commercially or developed for research) are either not fully automated (i.e., require manual coding of food items) or are not (well) validated3. Moreover, most available validated apps have been developed for one specific research purpose and use in a specific country; due to rather fixed designs, re-using such apps for other research purposes or in other countries seems challenging3,4,5,6,7,8. Finally, despite the availability of food record-based apps, to date, no recall-based apps appear to exist yet. Although food records are prone to reactivity bias, i.e., respondents may alter their food intake due to the awareness that they are being observed2,9, this is not the case for recalls, which emphasizes the need for the development of a validated recall-based app10. An innovative dietary assessment app called Traqq was developed for use in the Netherlands which can be used as a food record as well as a recall, depending on the research question1.
Besides the possibility to alternate between the food record option and recall option, this app also differs from other dietary assessment tools because of its flexible nature. Specifically, regarding the food list, portion size estimates, sampling schemes, and the possibility to incorporate additional questions. The level of flexibility in the system enables tailoring to multiple research purposes that require accurate assessment of dietary behaviors. Currently, the app is in the process of being validated and will be ready to be used in various types of nutrition related research. The app can also be used, and perhaps further improved for use, in nutritional intervention programs to measure and influence dietary behaviors. As the development of reliable dietary assessment tools is challenging, and reports on these processes are scarce, especially with respect to user and expert involvement3,11,12, this paper provides a detailed overview on how different information sources were integrated in the systematic and iterative development of this smartphone-based dietary assessment app. The process incorporates theory, expert consultation, and user engagement.
NOTE: All the procedures including human participants were conducted in a non-invasive manner by means of mostly qualitative research methods. Informed consent was obtained from all participants before the start of the evaluations. This protocol describes the iterative developmental process that can be roughly divided into four stages in which stages 1-3 are intertwined (Figure 1).
Figure 1: Overview of the stages of the iterative development process of the app. The development process consisted of five stages in total. However, the process was iterative which means that stages 1 through 3 were intertwined. Please click here to view a larger version of this figure.
1. Conduct extensive formative research in preparation of the actual developmental process.
2. Design the dietary assessment app
3. Evaluations by researchers
NOTE: Following each upgrade, the app was tested by nutrition scientists and research dieticians with expertise in dietary assessment (in-house testing) to verify whether functionalities improved as anticipated. The following instructions are to be executed by researchers.
4. Using the backend system for app and study management
NOTE: The system has three authorization levels: (1) administrator-this authorization level provides access to all sections of the backend (i.e., creating new users, determining user authorization, and granting users access to one or more projects); (2) project managers-this authorization level allows access to specific projects and the possibility to create new projects; and (3) researchers-this authorization level only provides access to the specific projects that researchers are involved in.
5. Use of the app by the participants during the study
The system (app and backend) was developed using the steps outlined in the above described protocol; the key results of this process are described below, concluding with the final design of the app.
Formative research
In addition to extensive literature review, several web-based tools were inspected (e.g., Compl-eat13, ASA2414, Foodbook2415, MyFood2416) with respect to dietary assessment methodology and implemented features. In addition, the performance of several food tracking apps frequently used in the Netherlands were compared (e.g., MijnEetmeter17, MyFitnessPal18, Virtuagym Food19), focusing on aspects such as dietary assessment methodology, provision of information, reliability, search engine, and the use of additional features (e.g., images, barcode scanner, recipe functions). The results of this inspection led to the decision to develop the app in such a way that it can be used as a food record and a recall. Moreover, it led to the implementation of the My Dishes function, which can be used to create original recipes or frequently consumed product combinations (e.g., a daily breakfast). Within this function, yield and retention factors are automatically taken into account.
To accurately quantify food and nutrient intake, a complete, albeit practical, food list is crucial. Compiling such a food list requires a trade-off between the extensiveness of the food list and the searchability of the food items (i.e., food descriptions need to be clear, understandable, and easy to locate)41,42. As food composition data form the fundamental basis for dietary assessment21,22, it is important to ensure that the developed food list can be linked to accurate food composition data. The food list included in the app is based on the Dutch FCDB (NEVO)14, which was selected for its reliability and rich food composition data. Originally, the NEVO consists of 2,389 food items (version 2016/5.0), which was reduced to a food list of 1,449 items by eliminating "confusing items" (e.g., foods that cannot be consumed raw, foods that cannot be consumed without additions) or items that are not as essential to include (e.g., due to low consumption rates based on the Dutch Food Consumption Survey (DNFCS)43).
Additionally, the NEVO contains similar foods with different brand names; in such a case, only the generic option was included in the food list. To further facilitate usability, some food items were renamed to eliminate needless terminology such as 'prepared', 'frozen', 'average', and 'natural'. This "cleaning protocol" was developed by three well-trained research dietitians and executed by means of a syntax, which can be rerun once NEVO is updated. In addition, to optimize the searchability of food items, 1,019 well-known synonyms of the included foods were added to the food list. Thus, the food list included in the app eventually comprised 2,468 items. An overview of the food list development is displayed in Figure 2. To note, although this extensive food list has been developed for general use, the backend of the app does allow the import of alternative food lists if required.
Figure 2: Structure of the food list developed for the app. The food list is based on the Dutch food composition database (FCDB) and corresponding portion size suggestions and synonyms were added for each item in the final food list. Please click here to view a larger version of this figure.
Another crucial aspect of dietary assessment is the quantification of portion sizes. Although portion size estimation aids (PSEAs), e.g., images, referent objects, and standard portion sizes, support the reporting of the amounts of foods consumed24,26,44, misreport of portion sizes is still a substantial source of bias24,25,45,46, and literature on the effectiveness of the different PSEAs is inconsistent26. Food images, portion size suggestions (i.e., standard sizes and household measures), and free entry of weight in grams are the most used PSEAs in web- and smartphone-based dietary assessment tools34. For example, whereas portion size suggestions (e.g., cups, spoons, small, large) are used in tools such as Compl-eat13 and Oxford WebQ47, images aid portion size estimates in tools such as ASA2414 and Myfood2416. To investigate the most appropriate PSEA for the app, a pilot study was conducted to compare the accuracy of portion size suggestions (e.g., small, medium, large, or cup, spoon), free entry in grams, and portion size images. The results of this study led to the implementation of portion size suggestions as the PSEA in the app along with the option to enter amounts in grams27.
Expert review
The aim of the expert evaluations was to qualitatively evaluate the app in terms of functionality and ease of learning. As many users prefer to learn software by exploration29, a system's level of learnability is important. A total of 10 experts, i.e., 4 (research) dietitians and 6 nutrition and health behavior experts (scientists) participated in the cognitive walkthroughs in which 60% used an Android smartphone. Most importantly, expert evaluations indicated that the first version of the app was not sufficiently intuitive, e.g., menu-structure was judged unclear due to vague buttons/icons, and the search engine generated an illogical order of results. Another critical point arising from the expert reviews related to the fact that selected items could not be modified. Based on these results, the design of the app was considerably upgraded from stage 2 onwards (Figure 1).
Usability evaluation
A total of 22 participants participated in the think-aloud interviews, which formed the basis of the usability evaluation. The initial sample size was set at 20 participants36, after which data saturation was assessed. As data saturation was not reached after 20 interviews, inclusion continued while assessing data saturation after each successive interview. Participants had a mean ± standard deviation age of 48 ± 17 years (range 22-70 years); 36% were male, and the majority of the population was highly educated (55%). In addition, most participants used an Android device (n=14, 64%), and almost all participants had over 1 year of experience with smartphone use (n=21, 96%) (Table 1). All participants completed the tasks without or with minimal instruction.
Total (n=22) | |
Gender | |
Male (%) | 36.4 |
Female (%) | 63.6 |
Mean age (mean, SD) | 48.1 (17.2) |
Educational level | |
Low (%) | 0 |
Medium (%) | 45.5 |
High (%) | 54.5 |
Smartphone type | |
Android (%) | 63.6 |
iOS (%) | 36.4 |
Smartphone experience | |
Shorter than 6 months (%) | 4.5 |
Between 6 months and 1 year (%) | 0 |
Longer than 1 year (%) | 95.5 |
SUS (mean, SD) | 79.4 (15.1) |
Table 1. Characteristics of the study population and results of usability evaluation. Only the results of the system usability scale (SUS) are portrayed in this table along with the participant characteristics.
Whereas some participants (n=13, 59%) indicated difficulties while using the My Dishes functionality; others (n=5, 23%) encountered minor functionality issues such as slow response of the menu button and difficulties using buttons related to insufficient screen size of smaller smartphones). Moreover, 15 (68%) participants indicated their preference for an option to enter consumed portion sizes in grams. Finally, evaluation of the SUS score indicated a rating of 79/100 (range 40-100), wherein only 3 out of the 22 participants rated the app below 68/100 and 13 rated >80/100, which suggests that the app can be considered user-friendly. Thus, overall, the suggested improvements were minor, and usability evaluations were promising. Subsequently, suggestions for improvement were discussed within the research team and, if deemed relevant, incorporated in the stage 4 upgrade to further optimize likability and usability of the app (Figure 1).
Final design
The steps described in the protocol and the results of the evaluation study eventually resulted in a final design for the app and the backend, which aimed for a simple visual design. This app can be used as a food record and a recall. As described previously, the food list is a modified version of the NEVO. Portion size estimation is supported by food-specific portion size suggestions; consumed portions can also be entered in grams. In case of the recall version of the app, the researcher has the possibility to select different timeslots (e.g., 2hR, 8hR, or 24hR). To collect food intake data on different days and times, various sampling schemes can be created within a predetermined period. Push notifications invite respondents to record their food intake. To ensure complete data collection, invitations are automatically rescheduled in case of non-response. Within the recall module, respondents can only report their food intake after receiving an invitation. In case of the food record, respondents can access the app and log their food intake throughout the day.
In contrast to most 24hR tools, the recall module of the app is not based on the Automated Multiple-Pass Method-a five-step method for collecting food intake data for the previous 24 h48-as this method is too elaborate and time-consuming for use in an app. More specifically, to increase usability and enhance the compliance of the food intake recordings11,38,49, navigation was reduced to a minimum by limiting the number of screens that need to be accessed to 4 (Figure 3): 1) an Vue d'ensemble screen showing the reporting window; 2) consumed food items are reported through the Search screen, and once the desired item is selected 3) a dialog box appears probing eating occasion and consumed amount, after which 4) the user returns to the Vue d'ensemble screen now showing the recorded food items. In addition, the user could also use the My Dishes function to create recipes or product combinations, which can be entered via the Menu button.
Figure 3: Schematic overview of the routing in the app. Please click here to view a larger version of this figure.
The data are stored on a secure server. If desired, additional questions-general or related to specific eating occasions or food items-can be incorporated. The app can connect with online survey tools. Therefore, it is possible to conduct a survey unrelated to food intake via the app at prespecified times (e.g., context, behavioral, mood questions). It is also possible to ask specific questions related to reported food items or eating occasions (e.g., when apples are reported, when lunch is reported). The use of online survey tools provides an opportunity to ask many different questions via the app. The collected food intake data can be exported from the server and imported into nutrition calculation software for further analyses. In case of the use of additional questions, these data will be available in the survey tool as usual. The aim was to develop a well-structured and easy-to-use app. Some screenshots of the design can be seen in Figure 4A–E.
Figure 4: Screenshots of the final version of the app. (A) The Start/Overview screen, showing the invitation with the (in this case) 2 h-recall period. The user can press Product toevoegen (i.e., Add item) to report a food item or Niets gegeten of gedronken (i.e., I did not eat or drink anything) in case nothing was consumed during this time window. (B) The Search screen, showing results matching the search term "Jus" from the food list. The desired item can be selected from the search results. (C) A pop-up screen requires input of details on the selected item "Jus d 'orange". In this case, the app asks for the amount consumed and eating occasion. The user can go back to the search result by pressing Annuleren (i.e., cancel) or Opslaan (i.e., save) to go further. (D) The Vue d'ensemble again, this time showing all the reported items. Another item can be added (Product toevoegen) or the input can be sent (Lijst versturen). (E) After selecting Lijst versturen, a pop-up appears asking the user if they are sure that they want to send, and reminds the user that it is not possible to make any more changes after the list has been sent. The user has the option to cancel (Annuleren) or send (Versturen). Please click here to view a larger version of this figure.
This paper presents the iterative developmental process of the smartphone-based dietary assessment app Traqq. Balancing the required level of accuracy and user-friendliness posed the following main challenges in the development of the app related to decisions on 1) data entry (i.e., selecting the most accurate method for food identification and portion size quantification), 2) food composition data (i.e., selecting an accurate database and creating a full-fledged food list), 3) customization options (i.e., flexibility in food list, portion size quantification, and recipes), and 4) validation (i.e., against traditional methods and/or independent measures)3,50. During the literature review, five validated and fully automated, smartphone-based, dietary assessment tools developed for research were identified3, namely My Meal Mate4, Electronic Dietary Intake Assessment (eDIA)7, Easy Diet Diary8, Electronic Carnet Alimentaire (e-CA)5, and Eat and Track (EaT)6.
Owing to the level of automatization of these five dietary assessment apps as well as this app, researcher burden and costs substantially decreases while data completeness increases compared to traditional dietary assessment methods. Additionally, this app, in turn, differs from the five existing dietary assessment tools in terms of flexibility. Specifically, whereas existing apps are all based on the food record method, this app can be used as a food record as well as a recall. Moreover, whereas the design of these apps is fixed, Traqq has the major advantage that it can be modified to fit different research purposes (e.g., dietary assessment method, food list, sampling schemes, additional questions)3,50. Conversely, other existing dietary assessment apps contain valuable features, which are not implemented in the app (yet). To illustrate this point, some apps allow the user to take photographs of their food for food recognition and portion size estimation such as the semi-automated, technology-assisted dietary assessment (TADA) system51,52.
Participants in the usability study also indicated that the use of photographs could be a valuable addition to aid portion size estimation. However, there were still too many challenges to be addressed to implement such a feature at this stage, e.g., specifying and guiding with respect to the photographic angle (i.e., to assess depth), the need for a reference maker (i.e., to correct for sizes and colors), the essential before and after photo (i.e., to assess consumed amounts), and on how to process recipe dishes. Due to these technical challenges, the existing image-based dietary assessment apps are still semi-automated, which means manual image review must be done by the user, the researcher, or both51,52. Technological advances, such as crowdsourcing and machine learning, have the potential to improve the use of food images for dietary assessment53,54. In the future, these options will be explored to further improve the app. The developmental process of the app was characterized by various critical steps. First, a formative research step was completed wherein the scientific concepts underpinning the rationale for app creation facilitated decision-making in setting up the general outline of the app.
During this stage, special attention was paid to the selection of the FCDB and the selection of the PSEA-aspects that both directly influence data accuracy21. Regarding the FCDB, as the app has originally been developed for use in the Netherlands, its food list is based on the Dutch FCDB, NEVO14. In the future, the aim is to further develop the app for international use, which requires more extensive food composition data as many foods are country-specific. Currently, no international FCDB exists yet and if existent, its use might have been limited. More specifically, as the Dutch food list already contains 2,389 food items, the implementation of an international food composition table, e.g., for 5 countries would probably multiply this number of food items by about 5 and negatively affect the searchability of foods and consequently, the app's usability. Therefore, country-specific food lists will probably be most valuable and often also preferred by professionals55.
This is facilitated by the app as it enables the import of alternative food lists and thus linkage to different (international) food composition tables. Regarding the portion sizes, there are multiple options available to support the accuracy of the estimates, e.g., use of image booklets, referent objects, and/or textual portion size suggestions26. In view of user-friendliness, direct implementation of a PSEA in the app is preferred over using a PSEA alongside the app (e.g., image booklet, referent objects). During the development of the app, the decision was made to facilitate portion size quantification by offering the opportunity to enter portion sizes using portion size suggestions and entry in grams. Portion size suggestion are based on the only available Dutch portion size database56. Although Dutch dietary assessment tools such as Compl-eat and Eetmeter also rely on this database13,17, it needs to be noted that this portion size database dates from 2003, and tableware sizes have since increased57. Using this database may therefore underestimate food intake.
Currently, the portion size database is being updated by the Dutch National Institute for Public Health and the Environment (RIVM), the Dutch Nutrition Center, and Wageningen University and Research58, which will eventually be used to update the portion size suggestions in the app. Discrepancies between the old and new portions will be mapped and adjusted where needed. Although the use of portion size images (i.e., a series of images portraying different amounts of a selected food) may be a good alternative for text-based portion size suggestions59, research has shown that the accuracy of portion size estimation is highest when a series of portion size images is presented at once, instead of one image at a time45,60,61. Generally, currently available smartphones have relatively small screens, which limits the presentation of a series of images. Although new technologies facilitate the use of interactive portion size graphics wherein amounts of food on a virtual plate or cup can be increased or decreased by using a slider61, these techniques are relatively new and still need to be thoroughly evaluated to assess their accuracy.
Another critical step in the development of the app included the involvement of experts and intended end-users. Although not often incorporated in the developmental process of tools (or not described)11,12, feedback from experts-as well as intended end-users-is crucial61, allows maximization of usability, and maintains the required level of accuracy. The feedback of the intended end-users was particularly helpful in the final design of the My Dishes function. Overall, the users were satisfied with the possibility to create their own dishes. However, they did struggle with some of the procedures, for instance, although the function would automatically save data, this was not visible to the user. Therefore, many users kept searching for the Save button and got stuck, afraid to go back and lose their input. Based on these kinds of feedback, the function was improved to better fit the expectations of the user.
To conclude, Traqq is an innovative app with many advantages over existing apps and web-based tools. However, there are still various limitations. As the app still relies on self-report, self-report-related measurement errors still exist (e.g., memory bias (i.e., in case of recall), social desirability bias, and food intake modifications (i.e., in case of food records), inaccurate portion size estimations (i.e., in both))1. In the coming years, recently launched novel technologies will be explored to further advance the app, e.g., by exploring the value of implementing features such as barcode scanners, voice recording, chatbots, and images, which could improve food identification and portion size estimation. Possibilities to connect with other apps (e.g., activity trackers, sleep trackers) and devices (e.g., accelerometers, heart rate monitors, chewing sensors) are being explored as well. Finally, the backend is also being subjected to further development e.g., through the expansion of sampling options.
The authors have nothing to disclose.
The authors want to thank Anouk Geelen and Arvind Datadien for their key role in the development of Traqq. Furthermore, the authors would like to thank Romy Willemsen for her assistance in the data collection and the data analysis in the usability study. Finally, the authors want to thank the experts and participants for sharing their experiences and opinions throughout the process. The development was executed by Wageningen University and Research and partly funded by the Ministry of Agriculture, Nature and Food Quality and industry, in the context of TKI Agri&Food PPS – project Smart Food Intake (AF16096).
ASA24 Portion size picture book | American National Cancer Institute | na | The portion size image database as used in the ASA24-tool |
Atlas.ti v8 | ATLAS.ti Scientific Software Development GmbH | na | Qualitative data analysis software for research |
Compl-eat | Wageningen University | na | The portion size suggestions database as used in the Compl-eat 24hR module |
iOS screen record function | Apple Inc. | na | Build-in iOS feature to make screen recordings |
NEVO (version 2016/5.0) | RIVM | na | Dutch Food Composition Database |
Qualtrics | Qualtrics XM | na | Online survey tool that can be used to implement additional questions in Traqq |
Recordable | Invisibility ltd. | na | Android app to make screen recordings |
SPSS version 24.0 | IBM Corporation | na | Statistical software |
System Usability Scale (SUS) | na | na | Validated questionnaire to assess a system's usability |