GLOSSARY: Who Killed Galactica?

Conversations on X are heating up, creating a seemingly endless loop of debate. One such exchange led Meta AI chief Yann LeCun to call out AI doomers, namely Gary Marcus, Grady Booch, and Michael J Black, for killing Galactica, a language model designed to assist researchers that was shut down after facing criticism for producing inaccurate information. But don’t worry—your friendly AI Human, Amit Raja Naik, is here to tell you what’s really going on.

https://s1.designmodo.com/postcards/image-959dbc93-1ffe-45db-9f4c-d61a97b4a89a.png

https://s1.designmodo.com/postcards/image-1723880707726.gif

Just weeks after Galactica, ChatGPT was released by OpenAI. “Contrary to their predictions, the scientific publication system was not hurt by the availability of LLMs, let alone destroyed. If anything, it was actually helpful,” said LeCun.

What are these ‘Prophecies of Doom’?

Black, who is the director of Max Planck Institute for Intelligent Systems, had raised questions about the reliability of Galactica and similar products during the launch.

Back in November 2022, Black posted a thread of all the problems with Galactica and said, “I applaud the ambition of this project but caution everyone about the hype surrounding it. This is not a great accelerator for science or even a helpful tool for science writing. It is potentially distorting and dangerous for science.”

Black replied to LeCun saying that his criticism was genuine. “I’m sorry but the truth is that *YOU* killed it, not me,” said Black.

He mentioned that LeCun promoted Galactica as, “Type a text and galactica.ai will generate a paper with relevant references, formulas, and everything,” which according to Black is a big claim to make.

Thus, when Black conducted the test, Galactica fell short of his expectations. “It made up realistic looking references. And, it did so in a way that sounded authentic,” he added, saying that Meta did the right thing by taking it down as it did not live up to the hype around it.

Marcus, a Professor of Psychology and Neural Science at NYU, is a popular critic of deep learning and AGI, also took to X to state that Galactica got his birthday, education, as well as research interests wrong. Nearly 85% of the results presented by Galactica about Marcus were not true.

Similarly, Booch had said, “Galactica is little more than statistical nonsense at scale,” adding adjectives like amusing, dangerous, and unethical.

As for the ‘prophecies of doom’, Black had done a thorough analysis of the problem with LLMs then, and said there should be new ethics rules for scientific publication. He highlighted issues that there can be a flow of fake scientific papers, in an industry which completely relies on peer reviews and public trust.

He said that he also uses LLMs for research and there should be further research on LLMs instead of slowing it down and building safeguards to defend the public trust in science.

LeCun doesn’t agree.

He replied to Black saying that the negative reaction from a scientist led to shutting down of Galactica. “In simple terms, *YOU* killed it,” said LeCun, asking him if the Galactica demo would have made the world a better place.

So, Who Killed Galactica? Black believes that Meta, as an organisation, should have been more careful when releasing Galactica, even as a demo, and LeCun should accept the scrutiny faced for promotion of the product.

“You will likely say that this was just a "demo" and not a "product". Do you think the public really differentiates these when it comes from Meta, has a slick website, and is heavily hyped?” asked Black. Which makes sense. Since Meta released it as a software and not a research publication, it puts Meta under the spotlight, specifically when the claims were not adding up.

Though Black appreciates LeCun and Meta’s efforts towards science and open source AI, he said that Meta is not a university, thus receives a lot of attention from the world, not just academia, which makes it riskier.

Today, the weights of Galactica are still up for academic research, but the product is not available for direct use. “That was a good decision. It shifted the focus away from Galactica being a Meta "product" to it being a research prototype,” said Black.

Maybe that is also why LeCun had invested in Perplexity AI, which often sells itself as a research focused search engine.

Probably, Black is right. With the advancements in LLMs and people getting accustomed to them in the real world, shutting down Galactica back then sounded like a decent step to take. He didn’t kill Galactica, his feedback was strong enough that Meta had to kill it.

However, now seems like a good time to finally bring back Galactica.

Enjoy the full story here.

[Webinar Alert] Transform Your Career: What Could Be More Important Than Technology?

Join our expert-led webinar on how digital transformation is reshaping industries, jobs, and lives, with insights into the SMU Master of IT in Business (MITB) programme—ranked 2nd in Asia—by Dr. Kiruthika Ramanathan. Register now!

Google’s Project Astra Changing Future of AI

At the Pixel 9 launch, Rick Osterloh, SVP of platforms and devices at Google, concluded that among the first places where Project Astra will come to life will be Gemini Live, a chat mode limited to Gemini Advanced across its devices.

With this, Google looks to envision a future where people could have an expert Al assistant by their side, through a phone or a pair of glasses. And some of these capabilities are coming to Google products, like the Gemini app and web experience, later this year.

GLOSSARY

Поиск по этому блогу

Search1

воскресенье, 18 августа 2024 г.

Who Killed Galactica?

Комментариев нет:

Отправить комментарий