The Right Chatbot for the Right Task
Different AI large language model (LLM) chatbots have different strengths. These advantages also change over time as the multi-million dollar investments in software development play out. Lately, for example, Claude has become better at unstructured data analysis and at writing. And Perplexity continues to excel at fact-based queries.
What does this really mean? Today, I got a great case in point. The Jewish Community Security Service (CSS) announced a change of CEOs. I wanted to know the relationship between it at Jewish Federations of North America’s Secure Community Network. So naturally I asked the chabots, “What is the relationship between Jewish Federations of North America's Secure Community Network and the Community Security Service?”
Here’s what I got from ChatGPT:
The Secure Community Network (SCN) and the Community Security Service (CSS) are two organizations dedicated to enhancing the safety and security of Jewish communities in North America. In March 2022, they formalized their collaboration through a memorandum of understanding (MOU). This agreement recognizes CSS as the volunteer security partner of SCN and commits both organizations to promoting each other's efforts.
This partnership allows SCN and CSS to leverage their respective strengths—SCN's national coordination and law enforcement liaison capabilities, and CSS's grassroots volunteer training programs—to provide comprehensive security support to Jewish communities across the United States.
Not bad, but every citation was from Wikipedia.
Here’s what I got from Perplexity:
The Secure Community Network (SCN) and the Community Security Service (CSS) have established a collaborative relationship to enhance the safety and security of the Jewish community in North America. In March 2022, the two organizations entered into a Memorandum of Understanding (MoU) that recognizes CSS as the volunteer security partner of SCN4. This agreement outlines several key aspects of their partnership:
Mutual promotion: Both organizations agreed to promote each other's efforts in securing Jewish communities4.
Coordination: The MoU establishes expectations for coordination in areas such as intelligence and information sharing, training delivery, and engagement with law enforcement partners4.
Complementary roles: SCN, as the official safety and security organization of the Jewish community in North America, works with over 50 national Jewish nonprofit organizations, 146 Federations, and over 300 independent communities1. CSS, on the other hand, focuses on providing security through trained community volunteers, with over 2,000 active volunteers in more than 150 synagogues across the U.S.2
Shared goal: Both organizations aim to address the rising antisemitism and current threat dynamics facing the Jewish community4.
Leveraging strengths: The partnership allows both organizations to combine their respective strengths, with SCN providing expertise and resources, while CSS contributes its volunteer network and community-based approach4.
This collaboration represents a significant step in coordinating efforts to secure Jewish communities across North America, combining SCN's broad mandate and resources with CSS's grassroots volunteer network.
Several things to note here:
Perplexity tells me it looked at 30 sources.
Of these, it used 27.
I got links to the major sources at the top of the answer.
The information about the partnership cites a “Security Info Watch” article for key details instead of getting everything from Wikipedia.
I’m not saying that Perplexity is better. I am saying that you should be careful to choose the right LLM chatbot for the right task. More importantly, that choice may require some experimentation. And it may change over time as the chatbots advance.
Community benefit organizations need to establish a way to begin to share this information across the organization. There’s no reason for everyone to be experimenting with the same things if that learning can be shared.
This sharing could be as simple as a Teams chat or Google chat with a scroll of findings, although I also recommend a specific place to store particularly useful prompts. And prompts should be sorted by “General” and then specific chatbots.