arXiv:2606.06546v1 Announce Type: new Abstract: Evaluating large language models (LLMs) for education requires measuring how models teach, not only what they know. Existing benchmarks emphasize domain-general correctness or depend on manually designed rubrics that scale poorly to long-tail pedagogical scenarios. We introduce Elmes*, an end-to-end framework for constructing, refining, and applying fine-grained scenario-specific rubrics. Elmes* combines a declarative multi-agent engine for teacher--student--judge interactions with SceneGen, a self-evolving module that co-optimizes evaluation criteria and test data from expert-defined pedagogical dimensions. Using Elmes*, we build Edu-330, covering 330 scenarios across 11 subjects, 3 grade bands, and 10 task types, with over 1{,}000 second-level indicators. Experiments on Edu-330 and four expert-authored gold-standard scenarios show that educational capability is multidimensional: top-tier LLMs differ mainly in creativity and values integration, knowledge-strong models may fail at Socratic scaffolding, and the education-specialized InnoSpark achieves the best human-evaluated average score. LLM judges preserve human-comparable rankings with much lower scoring variance, but exhibit judge-specific biases such as self-preference. Ablations show that expert-scored few-shot anchoring improves human--LLM alignment, while reasoning enforcement and greedy decoding are model-dependent. Elmes* thus provides scalable diagnostic infrastructure for pedagogically grounded LLM evaluation.

arXiv:2606.06538v1 Announce Type: new Abstract: In real-world applications, models are expected to perform reliably across diverse settings. Yet, many existing multimodal benchmarks expand task types without capturing the visual diversity needed to handle open-ended visual inputs. We present WorldBench, a challenging and visually diverse reasoning benchmark to evaluate Multimodal Large Language Models (MLLMs). We build a taxonomy of thousands of visual concepts across multiple domains (e.g., living things). Guided by this taxonomy, we curate a broad collection of images from search engines and existing datasets to comprehensively represent the visual world. Through structured trial-and-error, we manually design challenging questions that frontier MLLMs fail to answer. On quantitative and human evaluations, WorldBench achieves higher visual diversity than any existing diverse benchmark. Evaluating 15 MLLMs on WorldBench reveals weaknesses in visual understanding: even the strongest model reaches only 64.0% accuracy, while some models perform marginally above chance-level. We hope our work highlights the importance of visual diversity in building multimodal benchmarks.

arXiv:2606.06547v1 Announce Type: new Abstract: Diffusion Large Language Models (dLLMs) refine tokens iteratively but commit them irreversibly, leading to a "stability lag" where early decisions remain fragile even after being written. We reveal that Post-Training Quantization (PTQ) error easily flips these borderline decisions at the write frontier, which are then permanently locked in and amplified. To address this, we propose Frontier-Aware Instability-Reweighted Calibration (FAIR-Calib), a two-stage PTQ framework for dLLMs. Stage I probes a full-precision teacher to estimate a position prior that combines frontier hits and masked-stage reliability. Stage II performs off-policy, layer-wise calibration by minimizing a reweighted hidden-state MSE, effectively prioritizing the protection of fragile frontier states without requiring expensive end-to-end diffusion rollouts. We further theoretically justify our weighted objective as a surrogate for output KL divergence. Empirically, FAIR-Calib consistently outperforms state-of-the-art baselines on LLaDA and Dream (W4A4), significantly reducing frontier decision flips and suppressing post-commit mismatches across diverse benchmarks.

Lucas Shaw / Bloomberg: Sources: Spotify has approached concert promoters about licensing music festival livestreams, as it pivots to video to become a one-stop shop for music fans  —  Spotify is looking to become a hub for live music  —  Good evening from New York.  I just landed here so that I can help smuggle Charles Oakley …

Bloomberg: Sources: Beijing-based Moonshot, which makes the Kimi chatbot, is in talks to raise $1B+ that would value it at $30B, up from a $20B valuation in a recent round  —  Moonshot AI is seeking as much as $2 billion in a new funding round that would value the startup at $30 billion …

The Guardian: Microsoft says it will tighten human rights controls when working with national security agencies after an inquiry found Israel's Unit 8200 violated its ToS  —  Announcement seeks to close a difficult chapter for the company after the Guardian revealed its platform was used in mass surveillance of Palestinians

Mike Butcher / Pathfounders: UK-based Creator Fund closes a $56M fund to back European PhD founders, reaching them in universities before they're pulled into academia, Big Tech, or US labs  —  Creator Fund has closed a $56 million European fund to back PhD founders before they have a pitch deck, let alone a company.

Discussion | Link

Discussion | Link

A vision for the future of AI, focusing on access, safety, and shared prosperity as OpenAI works to ensure AGI benefits everyone.

Tim De Chant / TechCrunch: Fusion energy startup Helion raised a $465M Series G led by Thrive Capital at a $15.5B valuation, nearly tripling the valuation from its January 2025 round  —  Helion, the fusion startup backed by Sam Altman, announced on Thursday that it had raised $465 million in a new funding round that values the company at $15.5 billion.

In this article, we fine-tune the Gemma 4 E2B model for audio transcription and translation. We train the model using Unsloth on a German transcription dataset. The post Fine-Tuning Gemma 4 for Transcription appeared first on DebuggerCafe.

Discussion | Link

Getty Images; Alyssa Powell/BI AI companies are rapidly expanding into each other's markets for growth. For example, labs are entering the vibe-coding and agentic application space. "Super-app talk is mostly noise that's going to get resolved," one VC said. Earlier this year, I had a noble idea. For my job, I have to remember a lot of AI companies, so I thought I'd draw up a competition map for my desk. I'd divvy a sheet of paper into columns and list major companies based on which part of the AI supply chain they're in. There were the labs — OpenAI, Anthropic, Google DeepMind, etc. — that are making models and chatbots. There were the AI coding platforms — Cursor, Cognition, and Replit among them — that build coding assistants powered by those models. Then, there were startups building hyper-specific applications: agents that live in your email, help you execute a marketing strategy, or automate boring tasks like payroll. I realized that the map would get messy, fast, because AI companies are ruthlessly invading each other's turf. Ever-increasing valuations mean companies need to find new sources of revenue. Becoming a full-stack AI company is especially important for revenue when models are commoditizing fast and big-ticket IPOs loom. Here's how the landscape is evolving, way too fast for my paper map: More vibe coders Companies that start with a narrow AI capability — frontier models, AI agents, vibe-coding tools — are quickly expanding into one or more of those other areas. Last year, Anthropic and OpenAI launched Claude Code and Codex, AI coding platforms that rival Cursor and Cognition. Now, per user screenshots on X — Anthropic won't confirm it — the company may be working on an app builder for non-techies, putting it squarely in competition with vibe-coding stars Lovable, Replit, and Emergent. SoftBank and Lightspeed-backed Emergent, for one, says it saw Anthropic coming. "It's not a surprise. We've been anticipating this for a while and sort of internally thinking and preparing about it," CEO Mukund Jha told me in an April interview. Anthropic's entry into the vibe-coding market is not game over for the year-old startup, he said. "Coding is relatively like 20%-30% of the work. The hard work is actually taking the application to the last mile," Jha said. He added that building secure and production-grade apps, especially for non-technical users, is a tough problem, one that might not be solved by companies that may be "spread thin." Claw rush In the AI turf war, OpenAI is making agentic moves. In February, the lab announced it was hiring Peter Steinberger, the creator of OpenClaw, an AI assistant builder that skyrocketed in popularity earlier this year. The move added OpenAI to the list of companies building in the agentic space, such as former Meta tech chief Bret Taylor's Sierra and Salesforce's AI agent platform Agentforce. Now, Codex has evolved from a coding assistant to a virtual AI agent that can parse and reply to emails, manage files, and schedule meetings. Smaller players are excited about this space, too. Last month, Emergent, which started as a vibe-coding platform, expanded into the personal agent space. Other examples of the growing overlap include Anthropic entering the design market and graphic design giant Canva entering the broader generative AI and productivity suite business. 'Google wanted to touch everything' If it sounds like you've heard this story before, you have. But instead of OpenAI, Anthropic, and Lovable, the characters were Google, Amazon, and Microsoft. Michiel Kotting, a partner at European venture firm Northzone, said that there was a time when the FAANG companies were dipping their fingers in all the pies. Kotting, who cofounded e-commerce platform Shopping.com, said that Google used to be a big source of anxiety. "I remember 25 years ago when I was building my first company, Google wanted to touch everything," he told Business Insider in April. "For us at Shopping.com, we had Google launch Froogle, which was exactly what we were doing. And we're like, "Oh, we're dead." He added: "But then it turned out, it was a side project. They made so much money on their core business, so how hard would they go after it? Well, the answer is they didn't." Tom Sheridan, a vice president at early Lovable investor RTP Global, agreed that a so-called "super app" — one app to rule them all — is unlikely. "Super app talk is mostly noise that's going to get resolved by the IPO calendar. Right now we're seeing the foundational model players in the throes of a game of P&L chicken," he said. "Once these companies go public, cash burn stops being free and shipping into categories where you're good-but-not-best stops making sense." The "Google Graveyard," an unofficial online list, tracks 305 projects sunsetted by the search giant over the years. Apple, too, is famous for "Sherlocking" — introducing a new feature that makes a third-party tool irrelevant — but they don't always stick around. In 2023, Apple launched Pay Later, a rival to Klarna and Affirm. It discontinued it in 2024. Kotting said that we may see the same happen with OpenAI and Anthropic, companies feared by founders for the day they might ship an application startups have been working on for months. It may be more worthwhile for Anthropic to continue building better models so it can charge more for its core service, he said. But if Chinese players or other labs become equally good and models turn into a commodity, Anthropic may go harder on these services. Besides getting "Sherlocked," startups face another big risk: dependency. Startups are building billion-dollar businesses on top of APIs controlled by companies that may eventually compete with them. For example, Cursor depends on Anthropic models to power its features, but the two also compete as coding assistant providers. Short-term win for customers More players doling out more freebies is a win for individual builders and small businesses — but only in the short term, said Sheridan. "Foundation model companies can ship a passable version of almost anything, but if the bundled tool isn't as good as the specialist tool I already use, I revert back in a single session," the VC said. "Product sprawl in search of retention bumps risks worsening UX and users know it." Big labs like OpenAI and Google sprawling in every direction also means that companies like Reddit or LinkedIn, which host tons of data, shutting out scrapers. That's bad news for small startups like sales tech tools or meeting summarizers kept from the data they want to build their services. These changes bring an opportunity for founders who know what users want out of their data. "Today's crop of foundational models can see meeting transcripts shared for summarization, but they don't know which folders they should be filed under, what a team actually cares about or the required follow-up actions," Sheridan said. "That's the gap startups can build into." The market is also ripe for consolidation. "I'd expect one of the major consumer AI breakouts to get acquired within the next 24 months, most likely by Google," he said. "Google has the consumer ads business to absorb the cost and is structurally the most desperate for consumer AI talent." Sheridan said that the first company to be bought gets the best price. "You don't want to be the last consumer AI play standing when each major buyer probably only takes one shot," he said. Read the original article on Business Insider

OpenAI launches the Economic Research Exchange to study AI’s impact on jobs, productivity, and the economy. Applications are now open for selected research projects.

<p><strong>Release:</strong> <a href="https://github.com/datasette/datasette-agent-edit/releases/tag/0.1a0">datasette-agent-edit 0.1a0</a></p> <p>I'm planning several plugins for <a href="https://agent.datasette.io/">Datasette Agent</a> which can make edits to existing pieces of text - things like collaborative Markdown editing, updating large SQL queries, and editing SVG files.</p> <p>Agentic editing of text is a little tricky to get right. My favorite published design for this is for the <a href="https://platform.claude.com/docs/en/agents-and-tools/tool-use/text-editor-tool#use-the-text-editor-tool">Claude text editor</a>, which implements the following tools:</p> <ul> <li><code>view</code> - view sections of a file, with line numbers added to every line.</li> <li><code>str_replace</code> - find an exact <code>old_str</code> and replace it with <code>new_str</code> - fail if the original string is not unique</li> <li><code>insert</code> - insert the specified text after the specified line number</li> </ul> <p>Rather than recreate these patterns for every plugin that needs them I decided to create this base plugin, <code>datasette-agent-edit</code>, which implements the core tools in a way that allows them to be adapted for other plugins.</p> <p>Tags: <a href="https://simonwillison.net/tags/ai">ai</a>, <a href="https://simonwillison.net/tags/datasette">datasette</a>, <a href="https://simonwillison.net/tags/generative-ai">generative-ai</a>, <a href="https://simonwillison.net/tags/llms">llms</a>, <a href="https://simonwillison.net/tags/llm-tool-use">llm-tool-use</a>, <a href="https://simonwillison.net/tags/datasette-agent">datasette-agent</a></p>

Marine wildlife samples used to create marine detection algorithms. Samples provided by the Australian Museum. Dr Vanessa Pirotta Shark fins on a plane, seahorses in your bag and sea cucumbers in the post – these are just a few examples of illegal marine wildlife trafficking. This crime can be hard to detect. But in a new study, published in the journal Frontiers in Ocean Sustainability, we show how artificial intelligence (AI) can be harnessed as a complimentary detection tool to help stop marine wildlife trafficking at international airports and mail facilities. A global crime The cross-border trade in live animals, animal parts or products is a global crime, facilitating the flow of billions of illicit dollars each year. It’s known to converge with other criminal activity, including the trafficking in drugs, arms and humans. The United Nations Office on Drugs and Crime identifies five sources of demand for wildlife trafficking: food, medicine, pets and ornamental plants, specialist collection and adornment. In some cases, such as pet prestige, people are motivated both by the desire to have a pet and the perceived status it brings to own an exotic animal. People traffic marine animals too Wildlife trafficking affects around 4,000 species. Many of the more well-known examples involve land-based animals – ivory from elephant tusks, horns from rhinos and scales from pangolins – the world’s most trafficked mammal. Closer to home, we also see native Australian reptiles and birds, sometimes shoved in tins, put in socks and packaged up live to be sent overseas. Marine creatures, unfortunately, are targeted too. This can include live animals such as fish in people’s bags, or dried marine life such as the rise of the seahorse trade and demand for shark fin. We have small pockets of knowledge of this activity. But the reality is we don’t fully understand how widespread it is. AI to detect marine wildlife trade Currently, the best means of detecting illegally trafficked wildlife is humans. And then there are our four-legged friends: biosecurity dogs. Recently, Australia has also been working to develop the use of AI as a potential means of detecting land-based wildlife in illegal wildlife movements – building on existing detection pathways using 3D X-ray machines fitted with algorithms. For our latest study, we built on these efforts by developing world-first marine wildlife algorithms. We taught computers to look for shark fins, seahorses and sea cucumbers. Shark fins scanned under 3D X-ray. Vanessa Pirotta We did this by collecting a total of 68 samples of dead marine animals, which we scanned in a 3D X-ray machine to create a library of images. We then used this image library to develop algorithms to enable computers to search for what we taught it to look for – in this case, shark fins, seahorses and sea cucumbers. Samples were scanned alone and then in more complicated scenarios to reflect how people actually traffic marine life. This means if a bag or mail item is hiding a shark fin, seahorse or sea cucumber, the algorithm will be able to flag this to an operator, prompting them to inspect the item. Out of a total of 298 scans and a training data set derived from these samples, our algorithm had success rates of 95%, 95% and 85% for shark fins, seahorses and sea cucumbers, respectively. Humans and biosecurity dogs still needed alongside AI While technology fitted with computer algorithms may help people inspecting luggage or mail, we still need people to verify what computers see. Sometimes the algorithms get it wrong and may miss items. Despite this, the broader implications of having AI as a second set of eyes searching for trafficked marine life will aid in identifying key trade routes to potentially stop this activity. The next step is relying on implementation of these algorithms at the front lines. Like computer algorithms and AI, the more we learn, the better we get at detecting and potentially stopping this harmful crime. Vanessa Pirotta received funding from Rapiscan Systems for this research. Justine O'Brien receives funding from the San Diego Zoo and Wildlife Alliance; NSW Department of Climate Change, Energy, the Environment and Water; the Australian Research Council; Institute of Museum and Library Services; Great Barrier Reef Foundation; and the Taronga Foundation. Phoebe Meagher receives funding from San Diego Zoo and Wildlife Alliance and the Taronga Foundation. Zara Bending serves as a Resident Expert for the Jane Goodall Institute Global and is a Distinguished Research Fellow at the Macquarie University Environmental Law Research Centre.

Bloomberg: As wealth managers confront an AI reckoning, the tech is, for now, easing their workloads by picking up routine tasks, freeing up more time to advise clients  —  Wealth managers, who can make upwards of $500,000, are confronting a chatbot reckoning.  —  Down the Wealth Walkway on South Beach …

Former "60 Minutes" correspondent Scott Pelley spoke about Bari Weiss on "The Interview." CBS Photo Archive/Francine Orr/Getty Images Scott Pelley said Bari Weiss is putting a "thumb on the scale" for the Trump administration's version of events at CBS News. CBS News fired Pelley after he clashed with a "60 Minutes" executive producer and criticized Weiss. Weiss took over CBS News after Paramount-Skydance acquired her former outlet, The Free Press. Fired "60 Minutes" correspondent Scott Pelley says CBS News' top editor, Bari Weiss, is directing coverage to be more aligned with the Trump administration's views. During an appearance on The New York Times podcast, "The Interview," Pelley elaborated on his June 2 statement, in which he said CBS News' new management had instructed him to "inject falsehoods and bias into a politically sensitive story." "There was a thumb on the scale for the president's version of events that I felt was a level of political influence that I had never seen in 37 years at CBS News," Pelley told "The Interview." Pelley said his "60 Minutes" team had produced a story in February detailing ICE operations in Minnesota, during which agents shot and killed Renee Good, and the subsequent anti-ICE protests. Pelley said Weiss sent an email to his former boss, Tanya Simon, asking if they could make changes to the segment. "Two of the things in the email include: Can we make the protesters look more violent?" Pelley said. "Now, I'm paraphrasing. I don't have the quote, but that's what was communicated to me. And the other thing was Renée Good's car. You need to describe her as driving toward the officer." Pelley said the second request contradicted video evidence. "I went over the video of the Renée Good killing over and over and over again. Stop motion. Slow motion. And realized the event was not as the president said and not the way Bari Weiss remembered it," he said. Pelley said Weiss could have been trying to give the Trump administration a fair shake, but ultimately, believed the requests were motivated by a political agenda. "She could have been trying to be fair to the administration, except I felt that the story was abundantly fair to the administration and to the ICE officers and to the Border Patrol officers who were caught in that moment," Pelley said. "We were being told to write a version of events that conflicted with the video account." A spokesperson for CBS News said in a statement that Weiss' request had "no political motivation." "In an email, Bari made four points in the course of editorial back-and-forth. They had no political motivation and were proposed solely to make the piece as strong, fair, and accurate as possible. As is frequently the case in any newsroom that operates with collaboration, not everything she raised made it into the final piece," the spokesperson said. CBS News fires Pelley Pelley's comments come after CBS News fired him last week following a clash with Nick Bilton, the new "60 Minutes" executive producer Weiss appointed, and critical comments he made about Weiss. Weiss told CBS News staffers that management attempted to engage with Pelley and "find a way back," but "unfortunately, we weren't able to do so, and so we had to part ways. We did not want that to happen, but that's the path that he chose." Pelley, in a statement at the time, said Weiss "knows what she said is not true." Clashes with a new editor The clash between Pelley and CBS News' top brass began after its parent company — Paramount Skydance — acquired Weiss' anti-establishment news site, The Free Press, in October 2025. As part of the deal, Paramount Skydance CEO David Ellison appointed Weiss editor in chief of CBS's newsroom. Her lack of television experience and a perceived political bias concerned some staffers. Weiss began shaking up CBS News in the months that followed, including pulling a December segment about the Trump administration's use of El Salvador's CECOT prison from the air and telling staffers in January that layoffs weren't off the table. "60 Minutes" has undergone a major overhaul. Longtime correspondent Anderson Cooper left the show in May, while CBS News fired Pelley, correspondent Cecilia Vega, executive producer Tanya Simon, and executive editor Draggan Mihailovich. CBS News also didn't renew its contract with Sharyn Alfonsi, the correspondent who covered the Trump administration's use of the Salvadoran prison. The Paramount settlement Although the fallout at "60 Minutes" has dominated recent headlines, the drama began long before. President Donald Trump filed a lawsuit against Paramount — CBS News' parent company — in 2024, saying the outlet used "deceptive editing" in a "60 Minutes" interview with former Vice President Kamala Harris. Paramount initially fought the lawsuit but settled with Trump for $16 million in July 2025. Ellison's company, Skydance Media, merged with Paramount the following month. In his interview on Sunday, Pelley described that settlement as a "bribe." "That lawsuit against 60 Minutes had caused a great deal of concern in '60 Minutes.' Paying the bribe broke our hearts," he said. "No lawyer thought that was necessary." Read the original article on Business Insider