Testing information quality using generative AI

- Experts estimate that 90% of online content may be AI generated by 2026. Now, if you're thinking, how do I know that statement is true? Who are these experts and what information are they using to arrive at this bold statement? Congratulations, you're thinking critically. How do you know whether you can trust the information you rely on to draw conclusions and make decisions? That's exactly what this I part of pick pack information will help you answer, assess information quality by checking for RRV, reliability, relevance, and validity. Reliability asks, is the information trustworthy? Starting with, does it come from a trustworthy source? The source can be a person, organization, tool of measurement, like a scale or GenAI. It's also important to assess the reliability of the sources your source uses. Sometimes the source is unreliable because it doesn't provide verifiable evidence or facts. Sometimes it's because the source doesn't exist. We'll get to that in a minute. Relevance asks, is this information pertinent to my purpose? Does the information relate directly to my question, topic, or issue that I'm examining? To assess validity ask, is this information accurate in that it tells the whole story? Your conclusion might be backed up by reliable, relevant evidence or information, but it might be leaving out a big part of the puzzle. Remember, Sarah and her board of directors? Their purpose included providing emotional support to their at-risk students through dedicated counselors. Board member Malik provided an article from "Horse Sense Magazine" in support of his claim that horse assisted psychotherapy would provide the best emotional support. The board used GenAI to determine whether Malik's claim was RRV, reliable, relevant, and valid. They suspected that horse therapy was probably relevant to their purpose because they knew of a horse facility that had counselors who work with horses to provide emotional support to foster children. The board assessed whether Malik's Source was reliable using GenAI in two ways. First, they asked Gen AI how to evaluate the reliability of Malik's source. And two, they asked GenAI to evaluate the reliability of Malik's source. Not only can GenAI provide your information source, it can also check your sources. But you still need to bring your own critical thinking to the project, because sometimes when GenAI provides a source, it can't prove its own source is reliable. It may not even be able to prove its own source exists. Sometimes GenAI hallucinates and gives you unreliable information that sounds very convincing. And different GenAI enabled with different capabilities access different data sets. The board asked ChatGPT. We're assessing a claim supported by an article from "Horse Sense Magazine", stating that horse-assisted psychotherapy provides the most effective emotional support for at-risk students. Please evaluate the credibility, both the claim and the source. ChatGPT's response. I found several articles related to equine-assisted therapy, but I wasn't able to locate a specific article from "Horse Sense Magazine" in the available sources. ChatGPT then went on to offer articles from sources that discussed equine assisted therapies. When the board asked Bard, Google's AI chatbot, the same question, this was Bard's response. "Horse Sense Magazine" is a reliable source of information on equine therapy for at-risk youth. It's published by the Professional Association of Therapeutic Horsemanship, a well-respected organization that has been promoting equine assisted activities and therapies since 1969. The board wisely investigated further and found that Malik's source "Horse Sense Magazine" was not reliable because it didn't exist. They knew that horse therapy was relevant to their purpose, so they were open to further evaluating Malik's claim, but they needed reliable sources. They asked GenAI, could you please evaluate whether horse-assisted psychotherapy provides the most effective emotional support therapy for at-risk students? Please provide reliable sources that both support and challenge this claim. Now, validating a claim requires seeking reliable, relevant information that confirms the claim and reliable relevant information that disconfirms it. GenAI's answer provided summaries of studies discussing the effectiveness of horse assisted psychotherapy and other therapeutic interventions for at-risk youth. Ultimately, there wasn't enough information to prove or disprove Malik's claim, so the board chose a six month trial of horse therapy with half the students and a six month trial of canine therapy for the other half. Test information using RRV, so you arrive at an accurate conclusion.