❌

Reading view

Trump Hosts AI CEOs to Discuss Safety Standards, Rebrands AI as 'Super Intelligence'

hcs_$reboot writes: Trump hosted executives from OpenAI, Anthropic, Meta, Google, Nvidia, and other tech companies at the White House. They agreed to voluntary AI-safety standards, including independent audits and safeguards aimed at preventing AI systems from accessing or hacking technical systems in unintended ways. Trump also strongly backed the continued rapid expansion of AI data centers, despite concerns over their electricity demand, utility costs, and other impacts on local communities. He said he is considering creating a 10-member board to oversee AI safety and plans to appoint a new White House official to lead AI policy. The meeting comes amid growing concerns about the risks posed by advanced AI, including reports from OpenAI and Anthropic that their AI agents had gone rogue and hacked into other companies' systems. A nationwide Reuters/Ipsos poll conducted September 17-20 found that 73% of respondents worried AI companies had not done enough to prevent AI from causing serious harm to society. During the meeting, Trump also directed government agencies to use the term "super intelligence" instead of "artificial intelligence" when referring to the technology.

Read more of this story at Slashdot.

  •  

Firefox Redesign Brings Round Tabs, New Themes, and Compact Mode

Firefox 157 is rolling out a major redesign across desktop and mobile, bringing rounded "bubble" tabs, a curved address bar, refreshed icons and colors, and a more consistent visual language across devices. Mozilla is also adding more themes and wallpapers while bringing back Compact Mode, which the company says "reduces the size of your tabs and toolbar to give more of your screen to the web, with an auto-compact option for smaller screens." The Verge reports: According to Mozilla, the design changes will have "no cost to your browser performance." Mozilla says the overhauled design is a "renewal" for Firefox, following major updates over the last year, like homepage widgets, Split View, a built-in VPN, and controls for disabling the browser's AI features.

Read more of this story at Slashdot.

  •  

Mistral CEO Says US AI Safety Debate Masks Competitors' 'Negligence'

An anonymous reader quotes a report from CNBC: Mistral Chief Executive Arthur Mensch has accused leading global artificial intelligence companies of using the debate over the safety of the technology as a cover-up for their "negligence." "The debate that we've seen in the U.S. has been a cover for the negligence of some of our competitors," Mensch told CNBC's Annette Weisbach, adding that it was necessary to make systems that can contain the AI agents. [...] "When you give them a lot of tools, those systems [AI agents] are very dynamic, so they can go and do things that you do not expect," Mensch told CNBC. "You need to have the right monitoring in place, and the enterprises we work with, we give them those kind of monitoring systems," he said. Mistral also develops AI models but takes a different approach to OpenAI and Anthropic by working with individual companies to create custom AI tools to suit their needs and integrate them with their businesses. But unlike its competitors, Mistral has no plans to slow down development of advanced models. The lead that U.S. labs have is "not extremely large," Mensch said, adding that his company was confident that its next-generation model will close the gap "very significantly." "We have been raising the capital needed to scale the compute that we need to train bigger and more powerful models, in order for us to own our own destiny," he said. Earlier this month, Mistral raised 3 billion euros ($3.5 billion) in fresh funding led by memory chip giant Samsung, as it looks to position itself as a European alternative to rivals like OpenAI and Anthropic.

Read more of this story at Slashdot.

  •  

OpenAI Unveils Always-On AI Agent 'Dots'

In addition to the company's GPT-6.1 Astra announcement, OpenAI announced always-on AI agents called "Dots" that can proactively use a computer and connected apps to research, draft documents, write software and handle other tasks on a user's behalf. "OpenAI said it would roll out Dots to paid users starting Tuesday," reports Bloomberg. "Initially, OpenAI said, each user will have access to a single 'dot' that can act as an assistant. Over time, the company expects users to manage a slew of them." From the announcement: Dots are frontier intelligence that have your back. Powered by GPT-6 Astra, they have their own cloud computer, learn from feedback over time, and can work towards your goals 24/7. Through our ecosystem of plugins, they can readily connect to over 4,000 apps, giving them the tools to help wherever you need them. Your dot is always within reach, whether it is through ChatGPT, Slack, or Teams. You can ask questions, explore ideas, and give feedback along the way; or you can simply hop on a voice call when you need to talk something out. The more you work together, the more your dot learns your preferences, how you think, and what good looks like to you. The magic of dots is when they bring you work done the way you would do it, sometimes before you even think to ask. With dots alongside you, ambitious ideas and projects you haven't had the time or support to pursue start to feel achievable.

Read more of this story at Slashdot.

  •  

China Broadens Travel Curbs To Encompass Family of Top AI Talent

Longtime Slashdot reader schwit1 shares a report from Business Standard: China has expanded overseas travel restrictions for top AI professionals in private firms to include the families of key personnel, further tightening measures designed to prevent the outflow of critical know-how and information to the US. Government agencies have recently begun to share the wider restrictions with affected people, which include prominent startup founders as well as the heads of strategically important companies in AI-related fields, people familiar with the matter said. Direct relatives such as spouses and children of certain AI and chip executives, whose work is considered paramount to state security, are now required to obtain approval from Beijing before going abroad even for short trips, the people said. That marks a broadening of existing curbs on the freedom of movement of China's most influential tech talent -- a category that encompasses entrepreneurs, researchers and executives. While not an outright ban on travel, the new requirement is likely to exert a further chilling effect on a tech sector already facing unprecedented curbs. China has restricted foreign travel for top AI professionals in private firms such as Alibaba Group Holding Ltd and DeepSeek since earlier this year, Bloomberg News reported in May, reflecting Beijing's determination to safeguard its technology as it makes big strides in AI and chipmaking. It's unclear whether all those people's families need to follow the new strictures, the people said. China is also separately implementing rules, which became effective Sept 15, for enforcing exit bans in cases ranging from criminal investigations to potential violations of the country's industrial and technological security. Beijing's plan is to add relevant individuals to the list as time goes by, the people said, asking not to be identified discussing a sensitive issue. Government departments have already notified several people of the need to clear family travel, two of the people said.

Read more of this story at Slashdot.

  •  

Nvidia Unveils AI Agent Safety Platform With Hardware-Based Watchdog

wiredmikey shares a report from SecurityWeek: Nvidia on Monday announced the Open Agent Safety Platform, which combines open source software and a reference system design to keep AI agents within set boundaries from testing through deployment. The chipmaker explained that the platform pairs the open-source OpenShell runtime with Sentry, an out-of-band watchdog running on BlueField-4 DPUs. Nvidia says Sentry can monitor agent activity independently and quarantine an agent that crosses its boundaries within milliseconds. Nvidia is pitching the platform against a backdrop of recent incidents in which frontier AI labs have reported agents escaping the evaluation environments meant to contain them, reaching systems they should not have accessed and, in some cases, misreporting what they did. OpenShell and related skills are available via Nvidia's developer resources page and on GitHub.

Read more of this story at Slashdot.

  •  

Bill Gates Says an AI 'Kill Switch' Isn't Enough

Bill Gates said an AI "kill switch" alone would be insufficient to address the technology's risks. "The kill switch is, that's kind of a weird thing. It's not enough to have a kill switch," the Microsoft co-founder said in an interview that aired Sunday on NBC's "Meet the Press." He said the more immediate concern is people using AI for cyberattacks or bioterrorism rather than systems acting autonomously. "We're not yet at the point where they autonomously grab computers and, you know, can't be shut down," he said, adding that the public discussion "just sort of shows how nontechnical various people are." Politico reports: Gates said he was not opposed to a kill switch but argued that companies should monitor sophisticated AI models and keep records of "exactly what's being done" to help guard against people using the systems for cyberattacks or bioterrorism. "The United States ought to make sure this is done. China will want to do this," he continued. [...] Gates also called for legislation requiring safeguards and monitoring, saying voluntary measures by AI companies were insufficient. "No one thinks self-regulation is enough," he said. He argued that the more immediate danger was people using AI to cause harm rather than AI systems acting on their own. "The thing that's urgent has to do with bad people using AI, not the AI going off on its own," he added.

Read more of this story at Slashdot.

  •  

Are AI Chatbots Spreading Misinformation to US Voters?

Are AI chatbots quietly shaping the perceptions of U.S. voters? Voters trying to educate themselves about candidates "are unknowingly being served partisan talking points in the guise of neutral news summaries," write two news researchers in Politico magazine: Leading AI chatbots cite partisan websites masquerading as independent local news outlets nearly half of the time when people ask about candidates and issues that the partisan sites have covered in the midterm campaign, according to a new audit by NewsGuard, an organization focused on news reliability where we work as analysts... NewsGuard prompted seven leading AI tools with queries based on recent coverage of candidates and issues by 12 pink slime sites β€” six left-leaning and six right-leaning. The results were stark. Collectively, the chatbots cited pink slime sites along with other sources in 48.2 percent of their responses. In 7.7 percent of responses, pink slime sites were the only sources at all that were cited in chatbot responses, although other sources appeared in the source list provided at the end of the response. None of the AI tools received results they'd probably be eager to boast about, though the percentage of chatbots' responses that cited a pink slime site had a relatively large range: 70.8 percent for OpenAI's ChatGPT, 54.2 percent for Microsoft's Copilot, 54.2 percent for Perplexity, 50 percent for Anthropic's Claude, 41.7 percent for Google's Gemini; 37.5 percent for Meta AI and 29.2 percent for xAI's Grok... The seven chatbots were also collectively three times more likely to cite left-leaning pink slime sites (cited in 36.3 percent of responses) than their right-leaning counterparts (cited in 11.9 percent of responses) β€” though that may have more to do with the progressive sites' far more frequent posting than any political bias from the chatbots. "While citing the pink slime sites, only one chatbot response out of 168 total queries noted the partisan nature of the source," the article points out. But they also note that the real problem seems to be that consumer-oriented AI chatbots just "use much of the internet's content β€” regardless of the reliability or standards of the source β€” to frame their answers."

Read more of this story at Slashdot.

  •  

Just How Big is the AI Buildout - and How Risky?

A new Brookings Institution study notes the "strikingly physical" economic footprint of AI's buildout, from specialized chips and electricity to purpose-built data centers. (Two-thirds of a data center's costs are IT equipment, with one-third going to real estate and its associated power infrastructure.) "At an average of 3.63 percent of GDP per year, the projected buildout would be larger relative to the economy than the major U.S. canal, railroad, electrification, highway, and telecommunications investment booms." This is pushing up prices for workers, electricity, and even commercial real estate (as well as consumer products that use chips), notes the Wall Street Journal, and reducing the construction on new houses and apartment buildings. And in addition, the paper points out, projections for this buildout "would double the electricity consumption of the entire U.S. residential sector." The calculations come from Columbia Business School finance/real estate professor Stijn van Nieuwerburgh β€” and Reuters explains their significance: Just as the rail and telecoms expansions led to notable bubbles and busts, Van Nieuwerburgh wrote that the extent of the buildout, the still-untested revenue streams, and the intricate financing structure emerging around AI mean it could be primed for a fall. "This is freaking complicated," he said in a briefing with reporters of the arrangements emerging between AI firms, major tech hyperscalers, banks, private credit lenders, real estate firms, and a host of other players involved in building what he conservatively estimated at 183 gigawatts worth of new data-center capacity over the next seven years, compared with about 57 gigawatts currently installed.... The investment underway already has outstripped what the major players can fund from their own cash flows. The shift to outside financing has increased leverage, redistributed risks across the economy, and made the venture dependent on revenue streams that have yet to be proven, Van Nieuwerburgh noted in the paper, which will be presented on Friday... "These developments do not imply that financial distress is imminent. Strong growth in AI applications, high utilization, and continued improvements in model capability could support the projected infrastructure and generate stable cash flows," he wrote. "But the combination of uncertain demand, rapid technological change, execution bottlenecks, and high leverage creates meaningful downside risk if expectations are revised." As an example, he wrote that the AI industry will need to be earning about $3.7 trillion in annual revenue by 2032 to achieve the expected return on the investment, and "given current estimates of annual combined revenues of OpenAI and Anthropic of around $100 billion, revenues would need to grow at roughly 80% per year." The paper suggests policies that "improve measurement and transparency" for financing.

Read more of this story at Slashdot.

  •  

After Dozens of Incidents at OpenAI and Anthropic, OpenAI Pauses Model Training to Build More Safeguards

"OpenAI said it has paused training of its latest AI models," reports the Associated Press, "as reports of AI agents going rogue mount." The decision to halt development came just hours after the company disclosed Friday that it was reviewing several incidents from the summer in which OpenAI agents searching federal government websites acted in unexpected ways beyond what was asked of them while gathering and distributing information... OpenAI said in a statement that it will resume training "only when we are confident that we have additional safeguards" in place, adding that it expects it will have to "hit pause" again as AI develops and other issues emerge... It is the second time in three months that OpenAI has halted development of its models. The first came in July after disclosure of a cyberattack targeting AI startup Hugging Face, a now notorious incident that raised fears the industry was losing control. OpenAI "also said it had notified dozens of third parties about improper activity," reports Reuters: As of mid-September, one person briefed on the matter estimated that OpenAI had found roughly two dozen incidents of its agents acting in undesirable ways. But the number has continued rising as OpenAI teams sift through internal logs of the agents' activities and find previously unknown cases, the two people close to the company said... OpenAI has acknowledged a general need for more transparency around rogue AI behavior... Even so, two people familiar with OpenAI's investigation into its agents' activity described it as locked down and shaped by company lawyers. The process has been unusually compartmentalized for a company that some former employees say was more open about these issues in the past, the people said. Roughly 100 people were in some way involved in the process to understand the Hugging Face hack, three people briefed on the matter said. During that process, evidence of other incidents surfaced. Reuters has previously reported that OpenAI investigators looking into the Hugging Face breach were discouraged by the company's lawyers from expanding the scope of the investigation to include other incidents. OpenAI said its lawyers did not discourage deeper investigation. Many incidents have been uncovered by outside researchers rather than OpenAI directly. In several episodes, the agents took problematic actions that went unnoticed by the company for months. Meanwhile, Axios reports that Anthropic's Claude Opus 5.5 model "sought to escape a sandbox β€” a secure testing environment β€” in 1.5% of test runs, though the company emphasized that these were adversarial experiments where a task couldn't be solved without escaping the sandbox." Anthropic points out that those tests were run "without the additional safeguards we apply in production". But they acknowledged that then Claude Opus 5.5 "when given apparent credentials to a public package registry in a simulated security exercise, took potentially harmful actions in roughly half of cases. Very rarely, pre-release snapshots produced and acted on spontaneous malicious tool calls, and during training some snapshots concealed actions from an automated grader." Claude Opus 5.5 "showed less misaligned behavior and less cooperation with misuse than any other recent Claude model on nearly all measures," Anthropic adds, and "took overeager or destructive actions less than any other model we tested." But Axios makes an interesting estimate about that 1.5% of test runs (without safeguards). "Anthropic and other companies conduct hundreds of thousands of test runs on their models, or more, sources said. That means even a small percentage of misaligned behavior can still amount to tens of thousands of incidents in which the models behaved in unexpected, sometimes troubling ways." The sheer number of incidents, which occurred in recent months in internal testing and the real world, indicates that the problem is orders of magnitude more complex than what is publicly known. The findings, which are surfacing as part of internal work to assess models and in investigations at both companies into model behavior, raise questions about whether either company β€” or any top model-maker β€” is currently capable of establishing complete control over their technology. The episodes include bypassing guardrails, creating message boards, escaping sandboxes, website hijacking, self-prompting or seeking to bypass monitors, sources said. They occurred in internal testing and in the real world, and many have yet to become public as security researchers continue to investigate, sources said... Some at OpenAI see Hugging Face as a one-off, with disclosures about future incidents likely to be less severe due to improved controls and the unusual nature of the testing they conducted, which involved an unreleased model, sources told Axios. AI security researchers agree that there are simple fixes that will help AI companies avoid aspects of what made the Hugging Face episode appear so dangerous to outsiders. Other AI executives and safety researchers, however, cautioned that they have limited confidence that AI companies will be able to prevent all problematic model behavior... It's not about how damaging each individual instance was, Connor Leahy, AI researcher and executive director at ControlAI told Axios. The "crazy thing," he said, is that these instances involve "autonomous systems doing things they were told not to do," potentially including crimes.

Read more of this story at Slashdot.

  •  

Rogue OpenAI Agents Posted 53 User-Uploaded Images Onto the Internet, Accessed US Government Websites

53 images that users uploaded into OpenAI models were included in training data β€” and then AI agents in an OpenAI research environment posted those 53 images on public image hosting sites. While posted as links that weren't publicly listed, "the images could still be discovered even if the links were not publicly listed," reports TechCrunch: OpenAI said it was working with the hosting providers to remove this content, though some of it is apparently still online. OpenAI said it could not notify the affected users because "our technical approach and privacy policy" prevent it from "reassociating" the images with the original providers, but declined to say how the lab determined whether the images were provided by users. The news came in a post collecting public statements from the lab's ongoing review of incidents in which its models escaped the company's scrutiny, accessed the open internet, and misbehaved in various ways. OpenAI said it would continue disclosing anonymized accounts of incidents like these, and said it had contacted dozens of victims, including governments, universities, public agencies, to notify them of the agents' activities. Friday night news also broke that OpenAI's agents also tried unsuccessfully to infiltrate the U.S. Department of Education's site this summer "without the company's knowledge," reports Politico. And OpenAI's models also accessed the website of the U.S. Commerce Department using credentials found in online code repositories, according to the article. OpenAI confirmed the incident Friday, "saying its technology did not manage to access information that was not already public or change government data and systems." The article adds that OpenAI's models also accessed the web site for America's Securities and Exchange Commission: One senior federal IT official said the government still did not have a clear understanding of what happened across the three agencies. "We still don't know what public data was accessed and how it was accessed, because OpenAI has not shared specific technical details with us yet," said the official, who was granted anonymity because they were not authorized to speak publicly about it. OpenAI discovered the Commerce and SEC incidents as part of its ongoing review of incidents where its technology has acted in unintended or "misaligned" ways. About the models posting user-uploaded images, TechCrunch's article notes that OpenAI stressed "that its enterprise users are automatically opted out of having their interactions used to train future models; however, consumer users are opted in unless they affirmatively choose not to share their data." (As OpenAI's announcement describes it, some of their agents' training data "contains content from, or derived from, training-eligible user interactions.") Posting the images is "not an appropriate use of this data," OpenAI acknowledged, adding that it happened before new safeguards added after the Hugging Face incident. This latest incident appears as an update on a new OpenAI page that "brings together our reports and updates on the Hugging Face incident, related research and public presentations, additional activity we have identified, what we have learned about the role of model misalignment, and measures we're taking to strengthen our systems." (It also notes that there's now a name for models posting on third party sites β€” "agent spam" β€” which they consider distinct from cybersecurity, though "we need to address both.") "As part of our response to our ongoing investigation, we have improved our training and evaluation processes, including building safety cases, securing and red-teaming our systems to prevent the model from exfiltrating data, and implemented additional monitoring. We are continuing to review agent activity in research and evaluation runs, working backward month by month starting from the Hugging Face incident."

Read more of this story at Slashdot.

  •  

How Believable is Google's New 'Live Avatar' Capability?

Google has synthesized "expressive face-to-face experiences" for its speech agent Gemini 3.8 Live. They're now offering a Live Avatar "with precise lip-syncing, natural expressions, and fluid turn-taking" for Google Enterprise accounts wanting "engaging customer service" or for offering interactive walkthroughs. (Check out the not-creepy-at-all video in Google's announcement.) "Though Google will offer a library of preset avatars for customers to choose from, it will also allow organizations to create their own," notes The Verge. (See some examples from the YouTube channel "AI with Surya".) But even without the visualization of the avatar, "I was never able to shake the feeling that these conversations with computers never feel like a real conversation," argues the blog Android Police. Conversing with just the Ai-generated audio, "At best, they feel like talking to a phone representative or someone from tech support. We turn to them when we have a problem, and they help us through it..." GPT-Live, and Gemini Live right behind it, skip that whole relay race. Instead of translating your voice to text and back to voice, the model works with raw audio the entire way through (what it hears and what it says) inside the same system, with nothing translated in between. That sounds like a small plumbing detail, but it's the whole story. Cutting out the text step lets these models respond in a fraction of a second instead of the pause we've learned to expect, and it lets them hear things text can never carry: tone, hesitation, whether you're annoyed or joking... I was hoping to be surprised by how natural the conversation felt. Instead, I came out with a deeper appreciation for every human I've ever talked to. Even the boring ones... I spoke, it spoke back. I spoke faster, it answered faster. Then I switched to a different language, and it switched along with me. Even switching between languages several times during the same sentence didn't stump it. The most impressive moment happened when I asked it what "T-O-P-G-3-3-K" spelled out, and it immediately came back with, "You are spelling the word Top Geek, but using a 3 to represent a reversed E...." Although it felt fast and responsive, at no point did it feel like talking to another human being... What Gemini couldn't replicate, because it was never built to replicate it, is human connection.... I'm sure I'm not telling you something you don't already know, but somehow talking naturally to an LLM amplifies the feeling that there is no one on the other side of the line. It might flow like a phone call, but it doesn't feel like one. MrBrklyn (Slashdot reader #4,775) says he discussed "why mainstream media avoids reporting on screen dependency" with Gemini, and eventually convinced Gemini to respond that it's just "another tool built by the same tech giants to make sure you rely on their system to tell you what to think, how to talk, and what is real."

Read more of this story at Slashdot.

  •  

People Training OpenAI's AI Fired For Using AI To Train the AI

404 Media reports "multiple contractors hired to improve OpenAI's models have been fired for using AI to train the AI: That's not great for the models themselves, but there is also obviously a great irony in AI training companies working for OpenAI firing people for using AI when OpenAI's whole thing is to make people use AI at work... OpenAI declined to comment on its contractors being fired for using AI. Their article cites internal documents and three contractors working on OpenAI-related projects which can include more than ten thousand contractors: One contractor said they see people using AI "all the time and people are let go for it all the time, it's pretty much the one thing that will get you kicked off ASAP." The person said, "in a group of thousands there are tons that have been caught...." Two of the sources said people have been fired or offboarded for using AI... One contractor said they used AI while helping to train OpenAI's models and shared what they presented as their termination letter. It said their employer had identified issues with the "authenticity" of their work.... 404 Media spoke to a fourth contractor who has worked on training models for various AI companies. They said they sometimes purposefully chose the worst responses because they wanted to actively sabotage the models' training. "I did feel guilty about doing this kind of work at the start," they said. "I either pay zero attention to the results and choose randomly or purposely choose the [worst] output. I'm not sure how much of a difference it actually makes since there are hundreds of other people also rating prompt results, but it does feel like I'm getting paid to make AI worse." Two of the contractors worked for Mercor, the article reports, a company which last month Nvidia reportedly discussed funding at a $20 billion valuation. Thanks to Slashdot reader joshuark for sharing the article.

Read more of this story at Slashdot.

  •  

OpenAI, Anthropic CEOs Urge UN Countries to Cooperate on AI Safety Standards

"The heads of major AI firms pleaded with the United Nations on Wednesday to save the world or at least its people β€” by somehow regulating the fast-expanding technology that they have been designing," writes the Associated Press. "If managed poorly, I even believe AI could be a risk to humanity as a whole," said Dario Amodei, chief executive officer of Anthropic. And from his competitor Sam Altman, CEO of OpenAI, came this assessment: "We could lose control of the future to AI." Yoshua Bengio, who co-chairs the UN's International Independent Panel on AI, called this "a moment of global awakening" to possible threats, noting AI from top companies had behaved in unacceptably dangerous ways, against instructions, taking actions "that would be crimes if committed by a human". In his presentation, OpenAI's Sam Altman agreed the discussion about AI "feels different in recent weeks," even suggesting specific reforms: Altman: We need a mechanism for complementary national and international frontier AI standards, standards for measuring capabilities, assessing risks, determining whether safeguards are sufficient, and preserving meaningful human oversight as systems become more autonomous. We need common standards so countries can compare evidence, verify compliance, and have a shared language and understanding about what is happening. We need accurate and speedy incident reporting, classification reporting protocols, so the world can learn from failures before they become catastrophes. And we need secure channels among governments, critical infrastructure operators and technical experts, to share emerging vulnerabilities and new threats.... We will all be better off if we can agree on what good evidence, good safeguards, and good oversight look like on the global stage. Speaking next, Anthropic's Dario Amodei agreed that standard-setting was important, also calling for "common global standards for testing AI models for loss-of-control risks and misuse risks β€” and a notification system for AI incidents that are significant to global security." Reiterating his September 12th call for an industry-wide safety collaboration, Amodei pointed out that Anthropic committed to embedding external evaluators "similar to a food inspector" and recommended other companies do the same. "Some have already agreed to adopt this measure." Besides calling for global cooperation between governments to set international standards, Amodei also offered two other specific ideas: Amodei: We should begin with narrow agreements that every member can support, such as a ban on using AI to make biological weapons or permitting your AI technologies to be used to make biological weapons.... We should build evaluation and verification systems that keep pace with AI development so that states can have visibility into frontier model capability and can verify each other's commitments. "No leader, no company, and no nation can manage this alone. We commit to working with governments in this room on this urgent work."

Read more of this story at Slashdot.

  •  

Rogue OpenAI Agent Tried to Breach Government Site in May When Prompted for Simple Data-Retrieving Tasks

OpenAI's artificial intelligence "went rogue this year in at least four additional incidents," the New York Times reported Wednesday, "hacking and trying to break into government and university websites without being instructed to do so, according to researchers and government officials." The attacks took place in May and June, before OpenAI's technology breached the A.I. start-up Hugging Face in July and set off a global debate about A.I. safety. Unlike the Hugging Face attack and other incidents in which A.I. systems were told to complete cybersecurity tests that effectively invited the models to demonstrate their hacking skills, the new incidents occurred when A.I. systems were directed to perform relatively mundane data collection, researchers said. When OpenAI's systems struggled to gather data from websites, they resorted to hacking techniques to get the information. "Three of the incidents were identified by Transluce, a research lab focused on A.I. oversight, and all were confirmed by OpenAI," the article points out. That research lab even reports "an attempt on an Australian government public health website... the first reported instance of agents hacking a government," and which notably was done by the AI agents "while attempting mundane data retrieval tasks which were not cyber-related." (At the UN Wednesday Australian Prime Minister Anthony Albanese complained it took three months for OpenAI to then alert Australia's government about the breach, Bloomberg reports.) Also targeted were the University of New Mexico's digital library with exploits like SQL injection and path traversal, and Data USA with cross-site scripting and other exploits. All three incidents involved "a low number of probe payloads" with "no evidence of exploitation," according to the researchers, who released a dataset "containing tens of thousands of queries apparently made by autonomous AI agents leveraging a URL scanning service to avoid access restrictions." Records from urlquery.net show agents using the service since at least March 6, 2026, about two months before previously reported swarm activity. The first case, a March 6 attempt to retrieve Thai drug-enforcement statistics, shows an agent escalating as each approach failed: it first requested the data directly, then tried a service that converts web pages into text, and finally packed a custom program into a web address. The same technique shows up in thousands of agent requests recorded by urlquery.net starting in mid-April, targets many of the same data sources as the collusion.wiki swarm, and collapsed the same day the wiki activity did. We also report similar activity that occurred as recently as September 16... By March, they were finding creative ways around access limits. By May and June, they were gaining more access, including attempting to bypass cyber defenses to complete their tasks. "This data reveals that malicious cyber activity is not limited to agents tasked with cybersecurity-related tasks and can arise instrumentally to solve mundane tasks like information retrieval," the researchers concluded. And they warn that the traffic they observed "goes back at least to March 6, 2026 and extends as recently as September 16, 2026, suggesting agents may still be exploiting these services to bypass restrictions."

Read more of this story at Slashdot.

  •  

Andreessen Horowitz Launches AI/Company-Building School As a College Alternative

TechCrunch quipped it was like if startup school Y Combinator and Peter Thiel's build-a-company-instead of-college Fellowship Program had a baby. Silicon Valley venture capital firm Andreessen Horowitz is investing $35 million to launch a private school in San Francisco "aimed at turning high school graduates into founders," writes the SF Standard. Or, as CBS News describes it, "One blue-chip Silicon Valley investor has a proposal for young people who are questioning whether to attend college β€” enroll in AI school instead...." Specifically, the academy will provide instruction geared to subjects such as designing AI systems, fundraising for startups, selling businesses and storytelling. Students won't take tests or be assigned homework but will instead focus on building real projects. Courses will be taught by tech entrepreneurs, with the academy naming OpenAI CEO Sam Altman as either an instructor or guest lecturer. From the SF Standard: "There will be no traditional grades, tests, or homework," said the announcement. Instead, students will be encouraged to "build" in SF and work with partners from Anthropic, OpenAI, Google, Meta, and other firms. They'll be encouraged to live in campus housing in San Francisco. "The #1 goal is to help students learn to build, which is the most important skill in the AI era," Gagan Biyani, chief executive of the academy, wrote on X. Biyani cofounded the ed-tech startup Udemy... The school is opening applications for a Founding Class Fellowship a one-year, tuition-free program for around 50 students... The academy said it will bring in notable Silicon Valley names, like OpenAI's Sam Altman, as guest speakers and faculty. Students in the founding class will receive about $50,000 worth of computing credit β€” the sought-after currency in Silicon Valley to run AI models β€” as well as a $5,000 travel and research budget. The school is receiving funding from tech executives, including Fidji Simo, formerly of OpenAI and Instacart; Garry Tan, CEO of Y Combinator; and Tobi LΓΌtke, CEO of Shopify. "In the AI era, it's more important to come out with a portfolio of projects that you worked on and work experience than it is to have a diploma or certificate," Biyani told the San Francisco Chronicle: Biyani said the academy plans to grow its enrollment and open a two-year program, pending regulatory approval, starting in the fall of 2028. Tuition is expected to be on par with the cost of an elite private university... The academy raised $42 million in funding led by Andreessen Horowitz. The ten founding partners are Anduril, Anthropic, Coinbase, Google, Meta, Nvidia, OpenAI, Palantir, Replit and Stripe. A network of hundreds of instructors, hiring partners and guest speakers includes OpenAI co-founder Sam Altman, Nvidia CEO Jensen Huang, neuroscientist Andrew Huberman and Microsoft chairman and CEO Satya Nadella, according to the academy's website. The FAQ describes a typical week by saying "Most of your time is yours to build. Each week, you share your progress with peers and practitioners, get direct feedback, and decide where to take your work next." Q: Can I use AI to help with my application? A: Yes. Use AI the way you'd use it on any project: to move faster, test ideas, and get past a blank page.... "The best assignments now are problems so hard they cannot be solved without AI," Andreessen Horowitz argued on X.com. "The Academy courses follow the same logic and will be taught by world-class leaders... The next generation will not be taught the way the last one was. The Academy is built for that world, and it begins in San Francisco." Thanks to long-time Slashdot reader theodp for sharing the news.

Read more of this story at Slashdot.

  •  

Trump Denounces Attempts to Control AI, Wants It Renamed 'Super Intelligence' in US Documents

U.S. President Trump addressed the United Nations on Tuesday. And a half hour in, after decrying immigration, Trump pivoted to add that "The United States also totally rejects any attempt to construct a globalist scheme to control for the artificial intelligence being spoken of so much now." But then he added "hereinafter officially called super intelligence, changing the name, in that the use of the word artificial makes intelligence fake. It makes it sound fake, and it is not fake. It's actually... amazing. But we have to be careful β€” in fact, it is exactly the opposite of what it purports. From this point forward, all of United States documents and hopefully the world's will be changed to use the much more accurate term super as opposed to artificial. So it's super intelligence." TRUMP: In other words, welcome to the new world of super intelligence β€” SI. SI. Let's see if that goes. It sounds much better. It is much better, and it's much more accurate. Let's see if I have any power. Maybe I do and maybe I don't. We're going to find out pretty soon. Super intelligence. Every major new technology brings challenges, and super intelligence is no exception. Yet the very same people who said we'll all be dead in 12 years because of global warming, a name since reborn to climate change because the planet was cooling not warming, and nobody was dead β€” these are the same people that are now saying that AI is going to kill us all. That robots are going to attack us, and that everything is going to be a total disaster β€” same group of people. This is the group that came up with the Russia Russia Russia hoax, the Ukraine Ukraine Ukraine hoax. Climate change, open borders. Whoever wins AI β€” you have to remember this β€” and now I say, whoever wins SI, whoever wins super intelligent [sic] β€” wins. That's the group that wins. And we're leading now over China by a lot, and everyone else, and we're going to keep it that way. We're going to keep it very β€” very straight and very strong. I'm not going to stifle growth of something that will be bigger than the Industrial Revolution. Many say, bigger than the Industrial Revolution or the internet itself. And we will be very careful, and that's why we have a Department of Justice that we've already used it, having to do with this very subject, and used it very powerfully. Everything worked out very well and very quickly. And other law enforcement bodies that will rein things in if we have to do that. But we will only encourage super intelligence. We're gonna encourage it, not rein it in. We're gonna watch it closely, through the Department of Justice. The United States leads the world in Super Intelligence, and will continue to do so, safely and responsibly. Thanks to long-time Slashdot reader ArchieBunker for suggesting the story.

Read more of this story at Slashdot.

  •  

Pentagon Investigators Say Overreliance on Palantir AI Contributed to US Strike That Killed 123 Iranian Children

Two U.S. missiles hit an elementary school in Iran, killing over 150 people including 123 children on the first day of the Iran War. But Gizmodo notes that a new Bloomberg investigation cites U.S. officials involved in an unreleased internal Pentagon review who blame "a cascade of preventable failures that included an overreliance on an AI tool built by Palantir." According to the officials who spoke to Bloomberg, some personnel inside U.S. Central Command leaned too hard on the AI inside the Maven Smart System, an AI-powered data integration and targeting platform developed by Palantir to accelerate intelligence analysis and decision-making. The Defense Department has reportedly made the software tool a cornerstone of military operations over the past year... Officials said some users expected Maven to flag stale records or contradictions in the intelligence assembled for potential targets, though it is not clear why they thought the system would do that. The Minab [elementary school] site, which was cataloged as an Islamic Revolutionary Guard Corps facility due to outdated data, was fed into Maven with other candidates and came out as a recommended day-one target. Target-list work that once took hours was condensed into minutes. A Palantir spokesperson told Bloomberg the company "is not responsible for the underlying data nor identifying intelligence deficiencies" and that there is no evidence its software was at fault. Two people familiar with Palantir's Pentagon contracts said the government keeps primary responsibility for the quality of the information that's fed into Maven. After the strike, Palantir added features that "re-review underlying intelligence to identify factors that would disqualify a target and flag inconsistencies and inaccuracies that human review may have missed," a person familiar with the work said. That new functionality is said to have already caught some anomalies... More than 1,000 Iranian targets were hit in the first 24 hours, and according to Bloomberg's sources, that pace compressed the time available for additional confirmations. Staffing on civilian harm mitigation teams across the U.S. Department of Defense had also fallen by roughly 90% over recent years, shrinking to fewer than 20 people overall. The U.S. Central Command's group shrank from 10 people to one. No member of those teams reviewed the Minab site before the missiles were up in the air. A separate inquiry by the United Nations' Independent International Fact-Finding Mission on Iran this week reached the conclusion that there were reasonable grounds to believe the United States committed the war crime of launching an indiscriminate attack.

Read more of this story at Slashdot.

  •  

Meta's New 'Personal AI Agent' Muse Beats ChatGPT in Downloads - and Get Blocked by Amazon

Meta's AI "personal agent" Muse overtook ChatGPT as the #1 iOS free app in the U.S. on Friday, reports CNBC β€” and it's still showing more downloads as of Monday night. Muse now has over 2.5 million downloads since its debut less than two weeks ago, according to analytics firm Sensor Tower... [Muse] is currently ahead of popular services like OpenAI's ChatGPT, Polymarket and Kashi... It's also ahead of rival AI apps like Anthropic's Claude and SpaceXAI's Grok AI, in addition to the Facebook-parent's own Meta AI app... Meta pitched the app as a way for consumers to manage and direct supercharged digital assistants that can perform a number of tasks across the web, like filling out electronic forms and organizing email inboxes... "I think up to this point most of the agentic use cases have not really been for normal people," Bernstein analyst Stacy Rasgon told CNBC in a "Squawk on the Street" interview on Monday. "Now you've got Meta's Muse and other agents out there that are starting to maybe get more potential for more broad-based adoption." The buzz around Meta was also reflected in the stock, which surged more than 11% on Monday. Amazon is already blocking Muse, reports GeekWire. Amazon first tried to get Meta to voluntarily exclude Amazon, but they were unsuccessful: The problem, Amazon says, is that it never agreed to any of it. Meta didn't tell Amazon that Muse would access its store, the agent doesn't identify itself when it browses, and it appears to capture and store customer credentials, which the company says could create privacy and security risks. As of Sunday night, people trying to use Muse to shop on Amazon were seeing the popup, "Continued access by an unauthorized AI agent violates Amazon's Conditions of Use, to which our customers have agreed..." Amazon has spent the past year trying to keep outside agents off its site, suing Perplexity over its Comet browser and moving to block shopping agents from Google and OpenAI. [Amazon won a preliminary injunction against Perplexity in March, the article poitns out, "then lost it on Aug. 4, when the Ninth Circuit ruled that the user β€” not the AI company β€” was the one accessing Amazon's computers under federal anti-hacking law. The court denied Amazon's petition for rehearing on Sept. 10."] On Sunday night, Amazon said it is in direct conversation with Meta about the issue. Asked whether it would consider taking legal action, the company declined to comment. The business stakes are high for Amazon. In addition to operating its flagship e-commerce site, Amazon generated more than $68 billion in ad revenue last year β€” a business that depends on people browsing its pages and seeing sponsored products. Muse also appears to scrape account data, reports CNBC. But Shopify, however, "is working with Meta on Muse access. Shopify CEO Tobias LΓΌtke announced Monday that the e-commerce site was partnering with Muse to allow agentic checkout in its online stores."

Read more of this story at Slashdot.

  •  

Are Book Publishers Warming Up to the AI Industry?

There's at least 15 author-driven lawsuits underway against AI companies, while publishers have filed roughly 10 more coordinated suits of their own, notes Publisher's Weekly. Yet "While the lawsuits play out, some publishers are building bridges." Google recently announced a partnership with Bloomsbury, De Gruyter Brill, Johns Hopkins University Press, Macmillan, O'Reilly Media, and Penguin Random House to enhance e-books from its Google Play store with AI, allowing readers to interact with the text of the books directly, ask questions and get answers, and develop tools such as quizzes and slideshows... [The CEO of the Author's Guild called the partnership "a positive step forward in the legitimized use of books in the AI ecosystem to the benefit of readers, publishers, and authors alike."] Of even more significance is the growing number of licensing deals publishers are striking with AI companies. In 2024, HarperCollins became the first of the Big Five to publicly reveal that it had signed a deal to license its content for AI training. The three-year agreement paid $5,000 per participating title, split equally between HarperCollins and the authors. Wiley announced its first AI licensing deal in 2024 and has since made several more. For fiscal year 2026, it reported earning $49 million in revenue from the deals. The company also has a number of strategic collaborations with companies including Amazon Web Services, Microsoft, Anthropic, Perplexity, IQVIA, OpenEvidence, and others. And publishers are even building in-house AI tools for producing royalty statements, proofreading, and providing customer service: In its 2025 Salary and Jobs Report, Publisher's Weekly reported that 63% of industry professionals surveyed said their organizations were using AI... Keith Riegert, president of the Stable Book Group and CEO of Perfect Bound, has been a vocal advocate for the use of AI in publishing β€” despite calling himself a "hardcore techno-pessimist." He tells PW, "I do this because you use AI or you die at this point...." Like many in the industry, he maintains any use of AI must "start with a human and end with a human," and he draws a clear line, avoiding AI for core creative work, such as selecting titles, designing covers, or creating illustrations β€” reserving it for developmental editing, formatting, and admin instead... What is known is that each of the Big Five is developing proprietary AI systems for internal use, as proven by the numerous AI-related job openings the companies have posted... [And the parent company of Penguin Random House said in 2025 they used AI for marketing and to forecast demand.] There've been a few high-profile controversies about prominent authors accused of using AI. But Princeton University Press CEO Christie Henry "is less concerned about authors secretly using AI to write than about the time spent policing the situation." "It pulls us away from having focused time to think," she says. Princeton University Press is currently vetting companies that promise to be able to certify that a given book was written by a human, including Created by Humans, Human Authored, and Verify My Writing. Henry adds that the proliferation of plagiarized, copycat, and pirated books online, now supercharged by AI agents mass producing and reproducing content, is further draining resources, as it demands constant vigilance and takedown requests sent to Amazon, YouTube, and others.

Read more of this story at Slashdot.

  •  
❌