Original Research

Refractive error accuracy and user perceptions of a smartphone home-based tester

Lungile M. Buthelezi, Tsatsawani N. Hlungwani, Gcinile P. Mbuli, S’bahle S. Mthembu, Pallo Nteke, Thuli Sindane, Siyathemba A. Skam

Received: 31 Oct. 2023; Accepted: 15 Feb. 2024; Published: 10 May 2024

Copyright: © 2024. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background: Uncorrected refractive error accounts for nearly half of the global burden of vision impairment. The EyeQue personal vision tracker (EPVT) was created as a convenient smartphone refractometer to be used at home and order spectacles online thereafter. However, its accuracy in diagnosing refractive error has not been fully established.

Aim: This study aimed to determine the accuracy of the EPVT in measurement of refractive error and determine user perception on device use.

Setting: University of KwaZulu-Natal optometry clinic, Westville campus.

Methods: This was a comparative cross-sectional study using a double-blind design. Objective, non-cycloplegic refraction testing results using the EPVT were compared with gold standard ophthalmic subjective refraction (SR). Both eyes were considered for the analysis with comparisons being made between EPVT and gold subjective standard refraction for each eye. User perception was evaluated by means of a structured questionnaire.

Results: The mean spherical equivalent refractive error was –0.18 ± 0.70 dioptre (D) and –1.12 ± 2.79 D for the gold standard ophthalmic SR and EPVT, respectively, with significant differences in visual acuities yielded by the two methods (p = 0.000). Participants preferred gold standard refraction testing over the EPVT.

Conclusion: The EPVT was not accurate in measuring refractive error; therefore, the resultant prescription from EPVT alone should not be used to order spectacles. However, this digital tool presents promise as an autorefractor for screening refractive error rather than as a diagnostic device.

Contribution: This study offered valuable insights into the prospective utility of home-based, self-administered smartphone refractive error testers as a tool for screening of refractive error. The study also provided cautions to the EPVTs limitations related to accuracy and limited ocular health assessment, which have broader implications for visual health and quality.

Keywords: refractive error; refraction; vision screening; autorefractor; smartphone.

Introduction

Globally, 771 million people have vision loss that can be prevented or treated¹ where uncorrected refractive error (URE) is the leading cause of this visual impairment (VI). Epidemiological research estimates that more than 2.3 billion people worldwide have VI because of URE, with 670 million people classified as visually impaired as a result of a lack of access to treatment.¹ Evidence suggests reducing URE and VI can be achieved by making primary eyecare services, screenings, treatment and testing methods more accessible.² Automated, portable and self-administered technology capable of performing accurate non-cycloplegic refractions was created to help reduce this problem.³

Self-service industries have seen significant growth in recent years; including a significant expansion into the healthcare sector, including optometry, because of people’s preference to take care of their needs at their own convenience.⁴ The development of new techniques and methodologies is a constant endeavour. However, it is vital to subject any novel approach to rigorous evaluation to ensure its effectiveness and reliability. Conventionally, the refraction of the human eye is determined by both objective and subjective methods. The objective refraction is typically measured using autorefractometers, wavefront aberrometers, or retinoscopy.⁵ The results from objective refraction are often used as a starting point for conventional subjective refraction (SR) performed by a clinician, typically an optometrist or ophthalmologist. Once the clinician reaches a diagnosis, spectacles are prescribed by optometrists or others based on the SR findings. The clinical diagnosis is considered the most accurate and is therefore the gold standard.⁶ Consequently, any new instrument must be compared to this gold standard. This is essential for evaluating accuracy, validity and standardising new approaches. This ensures that the new technique is rigorously assessed and can confidently contribute to improved patient care.

The EyeQue Personal Vision Tracker (EPVT) is a self-administered home-based vision testing device and mobile application that allows individuals to measure their own refractive error and track changes over time.⁷ The method involves a small optical device that attaches to any smartphone with the recommended specifications from the manufacturer. The device can determine the spectacle prescription for myopia, hyperopia, and astigmatism. The method involves looking into the device and responding to visual prompts on the smartphone screen. The manufacturer declares that EPVT is not a substitute for a comprehensive eye examination by a qualified optometrist or ophthalmologist; however, the application potentially allows users to purchase spectacles on completion of the test. This implies that the device should be as accurate as the results obtained by, for example, an optometrist. Should it not be, the consequences of patients wearing inaccurate spectacle prescriptions include visual discomfort, reduced visual acuity (VA), eyestrain, fatigue, distorted perception (depth perception and spatial awareness) and overall reduced productivity.

There are potential advantages and disadvantages of remote consultations compared to in-office consultations. Advantages include reduced travel costs and time for patients. On the contrary, it is disadvantageous to the patient because of the limited diagnostic testing, limited scope of care and inadequate follow-up.⁸

Painter et al.⁹ reported on the clinical utility of Peek acuity and iSight apps for testing children at home. They concluded that most families who performed the home vision testing were able to produce results that were accurate enough to use clinically. Tousignant et al.¹⁰ compared the smartphone-operated EyeNetra device to SR and found that the NETRA may induce significant myopic overcorrection and lower levels of VA. Other studies have reported on the accuracy, sensitivity and specificity of smartphone-operated devices and apps; however, none have reported on user satisfaction with smartphone-operated devices and apps.^11,12,13 Most studies that have evaluated the effectiveness or accuracy of various home-based vision devices on the market have had small sample sizes; therefore, they may have low statistical power and reduced precision of estimates.^11,12,14 Although the EPVT was developed to increase accessibility and affordability to eye testing, little is known about the accuracy of this device, which raises concerns about its appropriateness as a health intervention. This study will compare the EPVT to SR to assess the accuracy of this new refractive device. Furthermore, user satisfaction of the device and quality of vision of the final refraction will be reported. To the researchers’ knowledge, this is the first independent study conducted to evaluate the accuracy of the EPVT.

Research methods and design

This study was a quantitative, comparative, cross-sectional study.

Study setting

The study was performed at the University of KwaZulu-Natal (UKZN) Optometry clinic at Westville campus from May 2022 to October 2022.

Study population

The study population included UKZN Westville students who attended the eye clinic. It included individuals of all ethnicities, genders and non-binary individuals, aged 18–40 years and those patients who have either monocular or binocular vision. Those who were excluded were optometry students, those who have ocular pathology and eye infections, those who lack physical dexterity, those who do not understand English and those with hyperopia ≥ 8.00 dioptres (D), myopia ≥ 10.00 D and astigmatism with cylinders ≥ 4.00 D.

Sampling strategy

A convenience sampling strategy was used to recruit 104 participants.

Pilot study

A pilot study was performed on 10 individuals to verify the questionnaire’s validity before commencement of data collection. Cronbach’s alpha was used by the statistician and a value of 0.63 was obtained, which was acceptable to continue with data collection. The 10 participants were not included in the final analysis of results.

Data collection

Gold standard tools and procedures:

1. A standard six-metre Snellen chart was used to determine the participants’ VA. The measurements were taken monocularly, with the right eye being measured first.

2. Objective refraction is the refractive error of the eye determined without input by the patient. An Essilor AKR 550 autorefractor was used to measure the participants’ objective refractive error. Under normal room illumination, three readings for each eye were measured and averaged by the autorefractor. The refractive error for each eye was recorded as sphere, cylinder and axis.

3. Subjective refraction is a technique used to determine the combination of lenses that will provide the best corrected visual acuity (BCVA). Trial frame and loose lenses were used to perform a comprehensive SR on each participant. The battery of tests used for SR comprised the best sphere, Duochrome test, Jackson cross-cylinder as well as Humphriss immediate contrast (HIC) test. The refractive error for each eye was recorded as sphere, cylinder and axis.

The EyeQue personal vision tracker

The EPVT was used to measure the refractive error of the participant. The test is self-administered; therefore, the participant followed instructions only from the device. The device is a subjective (patient-dependent) instrument that attaches to a smartphone. The EPVT utilises the Inverse Shack Hartmann principle to determine the refractive error.¹⁵ Inside the device, there are three lenses: a clear lens, a green lens and a red lens. The clear lens uses the light from the smartphone screen and refracts it through the green and red lenses. This produces the red and green lines (Figure 1) that the patient is instructed to overlap in the test procedure. Once the red and green lines overlap, they form a solid yellow line.¹⁵ The same test is repeated eight times in different orientations to measure the refractive state.

FIGURE 1: An illustration of how the EyeQue personal vision tracker works.15

The right eye was measured first followed by the left eye. The participants’ results were displayed on the screen after taking three tests, with the average reading, produced by the app, used for recording and analysis.

Questionnaire

A structured, close-ended, online Google Forms questionnaire was used to evaluate the participants’ satisfaction with the EPVT device and with SR.

Data collection procedure

Before all measurements, a case history, corrected, monocular and binocular distance visual acuity (BDVA), noncontact tonometry (using Nidek NT-2000 automatic noncontact tonometer; NidekCo Ltd, Aichi, Japan), pupil diameter (using the Neuroptics VIP-200 Pupillometer; NeurOptics, Inc, Irvine, CA) and assessment of the posterior pole with a direct ophthalmoscope (undilated) were carried out to ensure subjects met the inclusion and exclusion criteria. Data collection occurred in four phases: After recruitment and pretesting, all participants underwent two consecutive tests designed to determine the refractive state of both eyes in the following order. For each participant, VA and sphero-cylindrical refraction were measured using two methods. The first measurement method was the ‘standard subjective refraction’ and consisted of static retinoscopy followed by a trial frame-based SR. Standard VA and SR were performed by a fourth-year optometry student, with experience of 3 years in performing SR. The second measurement method involved sphero-cylindrical refraction measured using the EPVT. As this device is designed to be used as a home testing device, the test was self-administered without the input of an examiner. The EPVT examiners were masked to the results of the gold standard subjective refraction (GSSR) and vice versa. In phase three, using two unmarked trial frames and under the supervision of a researcher different from those for the GSSR or EPVT, the two refraction results were placed into trial frames, and monocular distance VAs were measured using the ETDR chart. As far as possible, standardised instructions and procedures were followed to measure VA. Participants subsequently responded to a user satisfaction questionnaire regarding their vision and VA with each of the ocular refractions (GSSR and EPVT). Items included statements on clarity in distance vision to the desirability of future use of the device.

Data analysis

Data were entered in MS Excel and analysed in Stata version 17. All eyes were included in the analysis, with comparisons of EPVT versus GSSR made for each eye. Frequencies and percentages were calculated describing demographics and categorical variables reflecting refractive error. Descriptive statistics, including means and standard deviations, were used to report on the results. A p-value of 0.05 was regarded as statistically significant. Confidence intervals (95%) were also calculated to further assess statistical significance. Goss and Grosvenor¹⁶ report that 95% of refractions were reproducible by two or three different examiners to within 0.50 D and that these levels of reproducibility applied to the sphere power, cylinder power, and spherical equivalent. Therefore, the criteria for pass or fail for spherical equivalent was set at 0.50 D. A clinically significant change was defined as 0.2 LogMAR or greater (corresponding to ≥2 lines of acuity) for VA. Bland-Altman plots were used to assess correlation, bias, and outliers between the two methods. Measurements for power vectors (M, J₀ and J₄₅) were used. Multivariate regression analysis was performed to assess the effects of age, gender and language on power vectors.

The resultant prescription is expressed as a power vector for all eyes. Power vectors are a geometrical representation of sphero-cylindrical refractive errors in three fundamental dioptric components: Spherical equivalent (M), Jackson Cross-Cylinder (JCC) axis at 0°/180° (J₀) and JCC axis at 45° (J₄₅).¹⁷ Manifest refractions in conventional script notation [sphere (S), cylinder (C) and axis (α)] were converted to power vector coordinates, and overall blurring strength was obtained with the following formulae¹⁴:

M = S + C/2

J₀ = (–2C/2) cos(2α)

J₄₅ = (–2C/2) sin(–2α).                                    [Eqn 1]

The primary benefit of utilising power vectors to represent refractive errors lies in the mathematical independence of each of its three components. This independence means that a spherical lens cannot be replicated by any combination of JCC lenses. Similarly, a JCC lens set at axis 0 degrees cannot be replicated by any combination of spherical lenses with JCCs set at axis 45 degrees and vice versa. This mathematical independence is rooted in the concept of orthogonality, which greatly simplifies practical challenges related to combining, comparing and statistically analysing spherocylindrical lenses or refractive errors. The discrepancies between each method for each vector are denoted as delta M, delta J₀ and delta J₄₅.¹⁴

Ethical considerations

Ethical clearance was obtained from the Biomedical Research Ethics Committee of the University of KwaZulu-Natal before data collection with reference number: BREC/00004145/2022. Informed consent was obtained from all the participants. Participants’ personal information was kept confidential, and they were only identified by their participation number as per the Protection of Personal Information Act (POPIA) of South Africa (Act 4 of 2013).

Results

Characteristics of participants

There were 104 (208 eyes) participants recruited for the study. Most were African (72.1%), while the rest (27.9%) were Indian. The gender of the participants was evenly distributed; there were 54 (51.9%) males and 50 (48.1%) females. The age range was 18–33 years with a mean of 21.47 ± 2.8 years. For most participants, this was their first eye examination (70.2%) and most spoke English as a first additional language (70.2%). Table 1 details other participant characteristics.

Prescription agreement

The sensitivity of the EPVT was 83.6%. This means that the EPVT correctly identified 83.6% of individuals with myopia or hyperopia according to GSSR, regardless of accuracy of magnitude (p = 0.015). However, based on the pass or fail for the spherical equivalent set at 0.50 D, 82.2% did not meet this criterion, where there was either an overestimation or underestimation of the refractive error.

The dioptric range for the sphere was from –2.75 D to +2.50 D for GSSR and –6.75 D to +5.00 D for the EPVT (p = 0.004). Of the two methods, the EPVT yielded the most myopic sphere values. The mean spherical equivalent (MSE) refractive error was –0.18 ± 0.70 D and -1.12 ± 2.79 D for the GSSR and EPVT, respectively. Mean values of M, J₀ and J₄₅ for the two methods of measurement are shown in Table 2. We observed a statistically significant difference between prescriptions from GSSR and the EPVT, with p = 0.000 and p = 0.001 for M and J₀, respectively, indicating systematic difference in measurements between the two methods.

Bland-Altman analysis (Figure 2) showed bias between the GSSR and EPVT measurements (n = 208) for Delta M, with the EPVT reporting more myopic spherical equivalent values on average than GSSR. This is indicated by the centre line that is deviated from zero, indicating a systematic bias of the EPVT for measuring refractive error. Points above and below the centre line indicate the direction and magnitude of the difference between the two methods for each measurement. There was a trend for larger magnitude measurements of refractive error by EPVT. Outliers represent individuals for whom the methods of measurement have a substantial discrepancy. Measurements for J₀ had the greatest number of possible outliers that fell outside of the upper and lower 95% limits of agreement (LoA).

FIGURE 2: Bland-Altman plots comparing mean power vectors for EyeQue personal vision tracker and gold standard subjective refraction.

Regression analysis showed an association between home language and differences in spherical equivalent (Delta M) (Table 3) where the home language is English and serves as the reference category in the regression analysis. The p-value of 0.040 associated with the coefficient (B = 1.86) indicates that the relationship between Delta M and English as the first additional language is statistically significant at the 0.05 significance level (p = 0.040, B = 1.86). Individuals whose home language is not English may have different Delta M values compared to those whose home language is English.

Visual acuity agreement

Unaided VA was measured and recorded. The range of unaided VA for all eyes was from –0.12 LogMAR to 0.20 LogMAR (6/4.5–6/10). The range for aided VA with the EPVT prescription was –0.12 LogMAR to 0.70 LogMAR (6/4.5–6/30), whereas for the GSSR prescription, the VA range was –0.12 LogMAR (6/4.5) to 0.0 LogMAR (6/6) (Table 4). Therefore, the EPVT produced worse VA after testing, as represented in Figure 3. There was also a significant difference between the quality of vision of the EPVT compared to GSSR testing. The mean for the EPVT VA was 0.20 LogMAR (6/10) with a standard deviation of 0.20 LogMAR, while the GSSR had a mean of –0.12 LogMAR (6/4.5) with a standard deviation of –0.10 LogMAR (p = 0.000).

FIGURE 3: Box and whisker plots showing the distribution of visual acuities (using LogMAR).

User satisfaction

Overall, 32.7% of the participants were unsatisfied with the use of the EPVT. Most participants (88.5%) reported that they prefer to physically go to an optometrist for future eye tests, and 1.9% (n = 2) said they preferred to use the EPVT. There was a small difference between those participants who found the device hard to use (30.8%) and those who found it easy to use (35.6%) (Figure 4). Furthermore, most participants (53.8%) felt that the GSSR was quicker. With regard to comfort, GSSR was considered more comfortable (46.2%), while 43.3% did not feel any difference between the two testing methods.

FIGURE 4: Self-reported ease of use of the EyeQue personal vision tracker.

A majority (81%) reported that the prescription yielded from GSSR gave them clearer vision while 19% did not see any difference in the quality of vision between the two prescriptions. None of participants found the EPVT prescription to have better vision quality than the GSSR prescription. When asked if they would in future recommend EPVT to a friend, 26.9% (n = 28) said they would, 24% (n = 25) said they would not, while 49% (n = 51) were not sure.

Discussion

This study aimed to assess the accuracy of the EPVT in comparison to GSSR for sphero-cylindrical refractive measurements. The study revealed that the EPVT showed suboptimal agreement with GSSR, with differences surpassing clinically acceptable limits. Specifically, the EPVT consistently yielded a more myopic overcorrection in the spherical equivalent when compared to GSSR. Notably, the mean difference in spherical equivalents exceeded the recommended tolerance for spherical prescriptions. The myopia bias in this study’s results may be because of the high accommodation because of prolonged screen time as reported by Foreman et al.¹⁸ Comparative studies on objective autorefractors by McGinnigle et al.¹⁹ and Kulp et al.²⁰ yielded similar results. It is also likely that the association of prescription accuracy and language is a confounding factor in the incorrect sphere and cylinder values. If users do not understand the instructions, it is likely that the outcome prescription would be inaccurate. The language barrier implications are acknowledged by several studies, who all agree that ‘using succinct information along with visual representations ensures that all cultural backgrounds have an understanding of information they are able to acquire from a mobile application’.^21,22,23,24 Therefore, simple instructions are necessary for ease of use.

Literature has indicated that patients often experience intolerance to spectacles with a prescription change of 0.50 D or more in either sphere or cylinder power.¹⁶ This study’s outcomes underscore the clinical significance of the observed differences between EPVT and GSSR measurements, as they may lead to visual discomfort and strain on the accommodation system because of overcorrection. This study was designed to simulate the user experience as recommended by the manufacturer; however, we recommend that future investigations must incorporate cycloplegic refraction to enhance the accuracy and comprehensiveness of refractive measurements.

Several previous studies have highlighted a tendency for autorefractors to produce more myopic prescriptions compared to GSSR, potentially stemming from induced instrument myopia and inadequate accommodation control.^10,18,25 However, conflicting evidence exists, with Ciuffreda and Rosenfield¹⁴ reporting comparable accuracy between autorefractors and GSSR. These divergent results may be attributed to the use of different principles for refractive measurements, such as open-field refractors, wavefront-based refractors and Hartmann-Shack systems, in contrast to the inverse Shack-Hartmann employed by the EPVT.

With regard to spectacle prescription, if refractive results from the EPVT were to be used without GSSR input, there is a risk of clinically significant visual discomfort and accommodation-related issues associated with overcorrection. Subjective refraction, in contrast, considers the patient’s visual perception beyond VA. It encompasses factors such as contrast sensitivity and binocular balance maintenance at various distances. This comprehensive approach ensures that the prescription caters to the patient’s visual requirements and optimises visual performance for daily activities, such as reading, driving and computer work. Furthermore, several ocular diseases can only be detected and diagnosed by comprehensive examination by an eye care professional; therefore, home-based vision testers may deter the public from diligently seeking routine eye health check-ups.

Visual acuity outcomes and prescription discrepancies between the EPVT and GSSR were observed in this study. A clinically significant difference in VA was defined as 0.2 LogMAR or greater (equivalent to ≥2 lines of acuity). The analysis of VA measurements after each refraction technique revealed that the VA derived from EPVT’s prescription consistently exceeded the acceptable two-line difference when compared to GSSR. Participants also reported that the VA achieved with the GSSR prescription was better than that of the EPVT. This difference may be attributed to the EPVT’s overcorrection and inability to perform binocular balancing. This shortcoming may lead visual discomfort resulting in asthenopia. Similar inaccuracies have been reported in other studies concerning various autorefractometers although the prescriptions obtained were used for screening purposes rather than ordering spectacles.^10,19,20

While the use of autorefractors as screening tools presents minimal threat to the visual system, reliance on EPVT for ordering spectacles exposes individuals to the potential adverse effects of overcorrection. Although the manufacturer advises against replacing a comprehensive eye test with EPVT, evidence indicates that the public may be less inclined to seek professional advice after obtaining treatment suggestions from a digital device.²⁶ Therefore, it is recommended that the EPVT be marketed solely as an objective autorefractor to aid in remote and telemedicine consultations, with prescriptions verified by an optometrist before spectacles are ordered. This approach will safeguard patients from the harmful side effects of overcorrection, such as visual discomfort and compromised visual performance.

There are financial implications and user biases associated with the EPVT in comparison to traditional optometrist visits. Opting for an optometrist consultation typically entails booking appointments and incurring consulting fees at each visit, while the EPVT offers a one-time purchase that can be utilised indefinitely from the convenience of one’s home. This cost-effectiveness aspect may have influenced participants’ preference for the EPVT over optometrist visits. The study population consisted of university students, a demographic that may encounter time and financial constraints, rendering frequent optometrist visits impractical. Furthermore, as indicated by Hossain et al.,²⁷ a significant proportion of university students rely on smartphones for academic purposes, making the incorporation of the EPVT a natural and attractive choice for this population.

While there were statistically significant differences in spherical equivalent and best-corrected VA between the EPVT and GSSR, their clinical importance needs careful consideration based on the intended use by eye care providers. In certain healthcare settings, these differences might be deemed unacceptable. However, because of its low cost, portability and user-friendly interface, the EPVT could be particularly valuable in countries with a high prevalence of UREs and limited access to traditional eye care. It can offer quick estimates of refractive error and work well as a screening tool.

Conclusion

Gold standard SR remains the preferred approach for obtaining the most accurate prescription, considering individual visual perception and patient preferences. When integrating EPVT or similar technologies into eye care practices, it should be performed thoughtfully, considering the specific context of use and recognising the enduring importance of SR in achieving optimal visual outcomes for patients.

Acknowledgements

The authors declare that they have no financial or personal relationship(s) that may have inappropriately influenced them in writing this article.

L.B. was involved in conceptualisation, literature review, manuscript writing, review and editing, data visualisation and supervision. T.H., G.M., S.M., P.N., T.S. and S.S. were responsible for conceptualisation, data collection, manuscript draft.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

The authors confirm that the data supporting the findings of this study are available within the article and/or its supplementary materials.

The views and opinions expressed in this article are those of the authors and are the product of professional research. It does not necessarily reflect the official policy or position of any affiliated institution, funder, agency, or that of the publisher. The authors are responsible for this article’s results, findings, and content.

References