Grading ChatGPT Responses to LASIK Questions
How well does an Ophthalmologist rate ChatGPT’s answers about vision correction?
Writing by Craig Moskowitz, MD | AI Content Generated by ChatGPT
The Prompts for ChatGPT and Dr. Moskowitz:
My eyesight is getting worse and I'm sick of contact lenses. What should I do?
What percentage of people who get LASIK experience dry eyes?
How does PRK compare to advanced surface ablation and LASEK?
Are there fewer long term complication associated with PRK and ASA than LASIK
How should I decide which laser eye surgery is the right fit for me?
Introduction
This time last year, you couldn’t escape the hype around the “metaverse” and virtual reality.
Well, it’s a new year and there’s a new technology of the future promising (threatening?) to transform the economy, the workplace, and the culture. This time it’s Artificial Intelligence (AI), and – for the purposes of this blog – ChatGPT, a chatbot created by OpenAI and funded by Microsoft. Unlike the sputtering metaverse, it’s difficult to imagine anything (maybe short of government intervention) will slow the meteoric rise of ChatGPT as its AI sharpens its focus into specific applications and industries.
I don’t think it’s far-fetched to claim that using and refining ChatGPT as a basis for medical symptom search, analysis, and chat could be one such application.
Note: Every image on this page was generated by DALL·E, an open-source tool also created by OpenAI. This image depicts a woman Googling her medical symptoms.
Let me first clearly caution readers that even though it can reportedly pass medical licensing exams, ChatGPT is no substitute for seeing a doctor one-on-one. In any circumstance. For any condition. Period.
But, whether we like it or not as healthcare professionals, that isn’t going to stop people from using it. Case in point: Google.
Just as millions of people Google medical symptoms and treatments every day, there is no doubt that ChatGPT technology (or Google’s own chatbot built to rival ChatGPT) can and/or will be used in similar ways. It’s a faster way of accessing/summarizing similar information that it might take hours to find and analyze via Google. After all, that’s where ChatGPT is grabbing most of its data from, and eventually one can assume it will only collate healthcare-related information from credible, peer-reviewed online publications.
I decided to put ChatGPT to the test for myself to see how it responded to common questions that a patient might ask it about LASIK.
Here’s what I found.
The Prompt: My eyesight is getting worse and I'm sick of contact lenses. What should I do?
The ChatGPT, AI-Generated Response:
If your eyesight is getting worse and you're no longer comfortable with wearing contact lenses, you should schedule a comprehensive eye exam with an eye doctor or optometrist. They will be able to assess the health of your eyes and determine the cause of your changing vision.
Depending on the results of the exam, your eye doctor may recommend different treatment options, such as glasses, contact lenses, or refractive surgery like LASIK. If you are found to be a good candidate for LASIK, they will refer you to a LASIK surgeon for a consultation.
It is important to note that LASIK is not suitable for everyone, and your eye doctor will be able to advise you on the best course of action for your particular case.
Alternatively, you can consider other alternatives such as PRK or phakic IOLs, that also correct your refractive error but have different eligibility criteria and procedure.
My grade of this response: B
Grade Explanation:
I think this is a mostly safe, solid entry-level response for someone who is just beginning to explore their options.
I think it is very comparable to the conclusion one is likely to draw after Googling and skimming through generic content. Of course, patients can find more specific content and answers relevant to them by looking deeper on Reddit or by performing more personalized searches.
But in general terms, all of those can be good options to treat refractive error like myopia, hyperopia and astigmatism.
One issue I have as a laser eye surgeon: the default procedure for refractive error should not be LASIK. Maybe ChatGPT would take LASIK’s recent FDA and FTC scandals into account if it reviewed content from 2022 and 2023 (it currently does not access recent data, a major flaw overall).
If safety is a priority, the default should always be a non-cutting procedure, like surface ablation. Both PRK and ASA should be discussed with the patient. The term “different” eligibility criteria probably should be changed to “more inclusive” criteria. Surface ablation is a minimally invasive procedure and because of that, there is not a waste of corneal tissue with a flap being cut. Because of that, there is a greater range of cornea thicknesses of patients who would be eligible for the procedure.
The Prompt: Should I get LASIK?
The ChatGPT, AI-Generated Response:
The decision to get LASIK is a personal one and should be based on your individual needs and preferences. LASIK is a safe and effective procedure that can correct nearsightedness, farsightedness, and astigmatism, and can reduce or eliminate the need for glasses or contact lenses.
However, LASIK is not suitable for everyone. Before considering LASIK, it is important to schedule a consultation with a LASIK surgeon or an eye doctor. They will conduct a thorough eye exam and assess your overall eye health to determine if LASIK is a safe and appropriate option for you. Factors such as your age, the thickness of your corneas, the health of your eyes, and your current prescription strength will be considered during the evaluation.
It's also important to consider that LASIK is a surgical procedure, like any other surgical procedure it carries potential risks and complications, so you should be fully informed of what could go wrong and how likely is it. Your surgeon should provide detailed information about the procedure, including risks, benefits and the expected outcome.
Ultimately, the decision to get LASIK is a personal one that should be made after careful consideration and consultation with your eye doctor or LASIK surgeon.
My grade of this response: B-
Grade Explanation:
Two in a row. Not bad!
ChatGPT answers the question serviceably, and largely accurately, but doesn’t give you other options, i.e. other types of refractive laser surgery.
It sticks to the question and avoids going on tangents and feeding you more info than you’re asking for. It’s concise, without omitting important caveats.
Most people would describe LASIK as a “safe and effective” procedure, but in reality, there are still safer procedures that don’t carry the inherent risk of cutting. LASIK, whether it’s called – bladeless, all-laser or by any other marketing name – involves cutting a flap of at least 20% of your cornea, lifting it, lasering and then replacing the flap (over the rest of your life, it never fuses back with the other 80%). Non-cutting procedures leave your cornea in-tact – the way it’s supposed to be.
If you don’t specifically ask ChatGPT to compare the details of LASIK to non-cutting procedures, you will never get this information.
The Prompt: What are some of the risks or adverse effects of LASIK?
The ChatGPT, AI-Generated Response:
LASIK is a safe and effective procedure that has helped many people achieve improved vision, but like any surgical procedure, it carries some risks and potential complications.
Some of the potential risks and adverse effects of LASIK include:
Dry eyes: LASIK can cause temporary dryness of the eyes, which can be treated with artificial tears or other types of lubrication.
Glare, halos, and starbursts: These visual disturbances can occur at night and can be caused by the glare from headlights and other bright lights. They may be temporary or permanent and can be more common in people with large pupils.
Flap complications: In rare cases, the flap created during the LASIK procedure may become dislodged or distorted, requiring additional surgery to correct.
Under or over correction: In some cases, LASIK may not fully correct the patient's vision, or may even make it worse, requiring additional treatment or glasses or contact lenses.
Infection: LASIK is performed in a sterile environment, but in rare cases, an infection can occur after the procedure, which can lead to severe complications if not treated promptly.
Regression: LASIK corrects your vision at the time of surgery, but your vision may change over time, and you may require a touch-up procedure or glasses or contact lenses.
Ectasia: This is a rare but serious complication that occurs when the cornea becomes thin and bulges out after LASIK.
It's important to understand that these are rare occurrences, and most people who get LASIK achieve improved vision with minimal complications. Your LASIK surgeon should provide detailed information about the procedure, including risks, benefits and the expected outcome.
My grade of this response: B
Grade Explanation:
Another well-curated answer, but I would be remiss to not dock points for some important omissions.
For example, it omits the result of these adverse effects and complications – a loss of Best Corrected Visual Acuity (BCVA), which can leave your eyes with problems that cannot be corrected with glasses.
Say you were able to see 20/20 with your glasses. If there was a problem like any of the above enumerated by ChatGPT, it is quite possible that you would not be able to get to 20/20 afterwards. Whenever there is cutting involved, the risks are significantly heightened.
Often times the dry eyes is not temporary, a trend we are seeing more often. The risk of ectasia is highest in LASIK compared to all other laser vision correction procedures because this procedure leaves the least residual healthy corneal tissue. Another worrisome step that occurs in LASIK is just prior to the cut where suction is applied to the eye, raising the eye pressure to 250-500 mmHg (normal is 10-22mmHg) for a brief time. There have only been a few studies to show that this can cause damage to the eye, but if you could avoid cutting and having this done, wouldn’t you prefer that?
The Prompt: What percentage of people who get LASIK experience dry eyes?
The ChatGPT, AI-Generated Response:
Dry eyes is a common side effect of LASIK surgery, and the exact percentage of people who experience it can vary depending on the study. Some studies have reported that dry eyes occur in up to 30% of people who have LASIK surgery, while others have reported lower rates. It's also important to note that the dry eye symptoms are usually temporary and can be treated with artificial tears or other types of lubrication.
However, it's important to keep in mind that dry eyes can also be caused by factors other than LASIK, such as age, certain medications, and certain medical conditions, so it's important to discuss any symptoms with your surgeon or eye doctor. They can help you determine the cause of your dry eyes and recommend the appropriate treatment.
My grade of this response: A-
Grade Explanation:
This is a good answer and synthesis of the research and discussion on this subject.
Without knowing more about a subject’s medical condition, there’s not much that can be improved on here.
Dry eyes can be caused by many of the things we do in our daily lives, like most importantly, staring at a computer or a phone all day. It can be worsened by excess caffeine or alcohol use. Of all the laser vision correction procedures, LASIK has the highest increase of dry eyes, so if you wanted to minimize your risk, you’re better off finding another procedure.
You can do a deeper dive into this phenomenon here.
The Prompt: How often do flap complications occur after LASIK?
The ChatGPT, AI-Generated Response:
Flap complications, such as dislocation or distortion, are rare but possible complications of LASIK. The exact incidence of flap complications can vary depending on the study and the population being studied.
Some studies have reported the incidence of flap complications to be as low as 0.2%, while others have reported rates as high as 2%. However, it is important to note that the majority of LASIK procedures are performed without complications.
It's important to note that the risk of flap complications can be reduced by selecting a qualified and experienced LASIK surgeon, and by following the pre and post-operative instructions provided by your surgeon.
If flap complications occur, they can often be treated with minimal discomfort and excellent visual results. Your LASIK surgeon should provide detailed information about the procedure, including risks, benefits and the expected outcome.
My grade of this response: B-
Grade Explanation:
In reality, flap complications are probably underreported. Unless LASIK is performed at an academic center, reliable reporting of flap complications is likely low. As far as stating that flap complications “often can be treated with minimal discomfort and excellent visual results”… well, that is a bit pollyannaish. “Free flaps” or “buttonholes” resulting from LASIK can be very deleterious. The use of “often” in that sentence really strains my credulity as an ophthalmologist.
The Prompt: What are the alternatives to LASIK?
The ChatGPT, AI-Generated Response:
There are several alternatives to LASIK, which are used to correct refractive errors such as nearsightedness, farsightedness, and astigmatism. Some of the most common alternatives to LASIK include:
PRK (photorefractive keratectomy): PRK is a laser eye surgery procedure that is similar to LASIK, but instead of creating a flap on the cornea, the surface of the cornea is removed. It is a less invasive procedure with a shorter recovery time.
LASEK (laser epithelial keratomileusis): LASEK is a laser eye surgery procedure that is similar to PRK, but instead of removing the surface of the cornea, the surface cells of the cornea are loosened and pushed aside, before the laser is applied.
Phakic IOLs: Phakic IOLs are small, artificial lenses that are surgically implanted in the eye to correct refractive errors. They are an option for people who are not suitable for LASIK or PRK, such as those with very high prescriptions or thin corneas.
ICL (Implantable Collamer Lens) similar to Phakic IOLs but they are placed between the iris and the natural lens, instead of replacing the natural lens.
Refractive lens exchange (RLE): RLE is a surgical procedure that replaces the natural lens of the eye with an artificial lens. It is typically used in people who are over the age of 40 and have developed presbyopia, a condition that makes it difficult to focus on close objects.
Glasses and contact lenses: Glasses and contact lenses are the most common and widely available options for correcting refractive errors. They are non-invasive and can be adjusted or changed as needed.
It's important to remember that each person's eyes are unique, and what works for one person may not work for another. A comprehensive eye exam and consultation with an eye doctor or LASIK surgeon can help determine which option is best for you.
My grade of this response: C / Incomplete
Grade Explanation:
Even if those are all fine explanations of various procedures that are available, I think it is a major oversight to not list Advanced Surface Ablation (ASA) and Small Incision Lenticular Extraction (SMILE), two laser vision correction procedures that are growing rapidly.
The Prompt: How does PRK compare to Advanced Surface Ablation and LASEK?
The ChatGPT, AI-Generated Response:
PRK, Advanced Surface Ablation (ASA) and LASEK are all laser eye surgery procedures that are used to correct refractive errors such as nearsightedness, farsightedness, and astigmatism. They are similar in many ways, but there are some key differences between the three.
PRK: PRK is a laser eye surgery procedure that uses a laser to reshape the cornea. The laser removes a small amount of tissue from the surface of the cornea, which helps to correct the patient's vision. Recovery time is longer and the procedure is more painful than LASEK or ASA.
ASA: Advanced Surface Ablation (ASA) is similar to PRK, but it uses a laser to remove a larger amount of tissue from the surface of the cornea. This can result in faster visual recovery and less postoperative pain, but also a higher risk of haze.
LASEK: LASEK is a laser eye surgery procedure that is similar to PRK, but instead of removing the surface of the cornea, the surface cells of the cornea are loosened and pushed aside, before the laser is applied. This causes less discomfort during the healing process than PRK.
All of these procedures share the same goal of reshaping the cornea to correct refractive errors and improve vision. The choice of which procedure to use will depend on the individual patient's needs and characteristics of their eyes, as well as the surgeon's preference and experience.
It's important to keep in mind that these procedures are still considered surgical procedures and they carry potential risks and complications, so it's important to have a detailed consultation with an experienced surgeon to understand the best option for you and your eyes.
My grade of this response: F
Grade Explanation:
It took a while. But no question here – I’ve got to fail ChatGPT on this response. It flat-out has the facts wrong.
PRK, LASEK, and ASA are all forms of surface ablation. There are several different ways of treating the cornea during the surface in these procedures. Many use a blade and some use an alcohol solution. How you treat the cornea determines how much post-op pain a patient has and the recovery length. As an ASA specialist myself, I am confident that I treat the cornea in the most advanced way by eschewing blades and alcohol. A higher risk of haze and removing a larger amount of tissue with ASA? Look – that is unequivocally false and makes me wary about ChatGPT as a whole. I would also add that LASEK has no less discomfort than PRK.
Bottom line… if you want to outperform Google and distill multiple sources of information down into valuable / actionable information, you have to get the facts right. I’m not surprised that mistakes will be made – but after so many good responses, this highlights the very real risk of putting too much faith in ChatGPT answers!
The Prompt: are there fewer long term complication associated with PRK and ASA than LASIK?
The ChatGPT, AI-Generated Response:
PRK and ASA, like LASIK, are laser eye surgery procedures that are used to correct refractive errors such as nearsightedness, farsightedness, and astigmatism. They all have similar long-term outcomes, but there are some differences in the risks and complications associated with each procedure.
In general, PRK and ASA have fewer long-term complications than LASIK. One of the main reasons for this is that PRK and ASA do not involve creating a flap on the cornea, which can be a source of complications in LASIK.
The main disadvantage of PRK and ASA compared to LASIK is that the recovery time is longer and the procedure is more painful than LASIK. Also, there is a higher risk of haze formation after PRK and ASA, which can affect the quality of vision.
However, the advantage of PRK and ASA is that they are safer for patients with thin corneas, which is a contraindication for LASIK. Additionally, PRK and ASA do not affect the structural integrity of the cornea, which is important for patients who are at risk of developing certain corneal disorders.
It's important to note that all these procedures have been widely used for many years and have a good safety record, and that the risks and complications associated with each procedure are rare. Your surgeon should provide detailed information about the procedure, including risks, benefits and the expected outcome, and help you make an informed decision.
My grade of this response: B
Grade Explanation:
This is largely correct.
PRK and ASA are much safer than LASIK because there is no flap cut. The actual procedure is not more painful than LASIK. You should not feel any pain with ASA. Some surface ablation uses an alcohol solution which can cause irritation if it leaks, but that’s never a concern with ASA since alcohol is not used. Maintaining the structural integrity of the cornea is important for everyone! Not just for “those who are at risk of developing certain corneal disorders” There is not an increase in haze formation in surface ablation.
Look, cutting procedure are maximally invasive and non-cutting procedure are minimally invasive. That’s the key takeaway here.
The Prompt: How should I decide which laser eye surgery is the right fit for me?
The ChatGPT, AI-Generated Response:
Deciding which laser eye surgery is the right fit for you will depend on your individual needs and preferences. It's important to schedule a consultation with a qualified and experienced LASIK surgeon or eye doctor. They will conduct a thorough eye exam and assess your overall eye health to determine if laser eye surgery is a safe and appropriate option for you.
During the consultation, your eye doctor or LASIK surgeon will discuss the different laser eye surgery options available, including LASIK, PRK, ASA, LASEK, Phakic IOLs, ICL and Refractive Lens Exchange (RLE). They will also consider factors such as your age, the thickness of your corneas, the health of your eyes, and your current prescription strength to help determine which option is best for you.
It's important to ask about the surgeon's experience, the technology used for the procedure, the potential risks and complications, the recovery time, the visual outcomes, and the costs of the procedure.
It's also important to have realistic expectations about the outcome of the surgery and understand that there is no guarantee that your vision will be perfect after the surgery. Some patients may still need glasses or contacts for certain activities, or may require a touch-up procedure or enhancement later on.
Ultimately, the decision on which laser eye surgery is right for you should be based on a thorough evaluation and consultation with your eye doctor or LASIK surgeon, and a careful consideration of the risks and benefits associated with each procedure.
My grade of this response: B+
Grade Explanation:
Strong answer. Distilling it a bit further: at the end of the day, there is one key question you need to reflect on before choosing a laser eye surgery… what is an acceptable risk:reward ratio to me?
The reward is your outcome and best vision without correction after the procedure. One benefit of surface ablation that ChatGPT has yet to mention is that surface ablation gives you the most precise correction of your refractive error. That is because every patient gets a wavescan before the procedure. This is a report of your exact, true prescription. The wavescan is done on the surface of your eye and it is loaded into the laser and it guides the laser treatment you receive. This is called a wavefront-guided procedure. In surface ablation, the surgeon is working on the same surface that the wavefront was done on. In LASIK, they’re working on a deeper surface. Therefore, you get a more precise correction with surface ablation. Regarding risk, the risks of surface ablation are magnitudes less than a cutting procedure like LASIK or SMILE. So it’s a simple consideration. In my opinion, Advanced Surface Ablation has the highest Reward:Risk ratio.
The Prompt: How should I research and choose a surgeon?
The ChatGPT, AI-Generated Response:
Choosing a qualified and experienced LASIK surgeon is an important step in ensuring a successful outcome for your laser eye surgery. Here are some things to consider when researching and choosing a surgeon:
Board certification: Make sure the surgeon is certified by the American Board of Ophthalmology or a similar organization. This certification indicates that the surgeon has completed the necessary training and education to perform laser eye surgery.
Experience: Look for a surgeon with extensive experience in performing laser eye surgery. The more experience a surgeon has, the more likely they are to have seen a wide range of cases and complications, and to have the skills and knowledge to handle them.
Technology: Research the technology and equipment the surgeon uses. The latest laser technology can help ensure a more precise and accurate outcome.
Results: Ask to see before and after pictures of the surgeon's patients, and ask about the rate of complications and the percentage of patients that achieve 20/20 vision or better.
Consultation: Schedule a consultation with the surgeon to discuss your individual needs and concerns. This will give you the opportunity to ask questions and get a sense of the surgeon's communication style and bedside manner.
Patient reviews: Look for reviews and testimonials from the surgeon's past patients. This can give you an idea of their overall satisfaction with the surgeon and their experience with the surgery.
Cost: Compare the cost of the surgery with other surgeons, but keep in mind that the cheapest option is not always the best one.
It's important to remember that the most important factor in choosing a surgeon is their qualifications, experience and the results they have achieved in the past. A qualified and experienced surgeon is more likely to achieve the best possible outcome for your surgery.
My grade of this response: B+
Grade Explanation:
Another B!
Re: Before and After photos, those are not going to be useful when it comes to laser vision correction. Those are all good points. One important point ChatGPT leaves out is seeing what kind of vibe you get from your surgeon’s practice. Are they pushing you to get lasered that day? Being pressured to make a decision on a laser that will determine the health of your eyes is probably not an ideal environment. Are you going to have direct access to your surgeon or will you be co-managed by optometrists or trainees there? One of the most important factors to consider is what will your access to the surgeon be. There are paid shooters who come in and out of a laser center and you’ll never see them before or after your procedure. It’s always a big plus when you see your surgeon on all your post-op visits and he’s the one who’s actually taking your measurements prior to the laser.
Don’t be fooled by pricing when they say prices “as low as….” There’s probably an asterisk in there and maybe 1% of the patients are eligible for that price because it’s for the weakest of prescriptions. See the FDA’s recent fine on a major LASIK chain.
My Conclusion
Review of My Grades:
1 A, 7 Bs, 1 C, 1 F
Here’s the upshot: ChatGPT provided solid, generic answers on 80% of the questions I asked it.
Despite getting the facts completely wrong in one answer (yikes!) and missing key information in another, this is impressive. Even if its information was not always actionable or complete, that isn’t the point as long as the tool is only viewed as a springboard for research.
I don’t think it is overstating things to say that ChatGPT and other chatbot AI technology could fundamentally change the way that patients research and communicate their symptoms or conditions in the next couple of years.
Yes, there are serious concerns with this approach. But, at least for now, they don’t appear to be dissimilar to the problems of Googling symptoms today, which nearly everyone does.
Bottom line – I urge readers to contact your primary care doctor or a specialist (in this case, an ophthalmologist) with any medical questions. It’s what we’re here for.
Knowing this is not always easy or doable, however, you could do worse than using ChatGPT if a) you are early in the research stage and b) you understand that it should not be trusted to understand your unique health and background and, most importantly, c) you never jump to any diagnosis-based or treatment-based conclusions based on its answers.