Researchers collected 195 of the exchanges, taking out all the patient identifying information. They provided the AskDocs answers and a separate set of ChatGPT answers to a panel of three health care providers without telling the panel where the answers originated. The panel preferred the ChatGPT responses to the doctor responses almost 80% of the time. The panel members said they were nuanced and accurate and of significantly higher quality. They even found ChatGPT to be more empathic. For example, the chatbot answer to that toothpick question begins, “It’s natural to be concerned if you have ingested a foreign object, but in this case, it is highly unlikely that the toothpick you swallowed will cause you any serious harm.”
