Worth Knowing
All topics

Topic

Security

AI news about cybersecurity, vulnerabilities and attacks involving AI.

17 stories in the last 30 days

Today

3 stories
  1. Worth knowing Hacker News

    AI is forcing us to pay off 40 years of technical debt

    It demonstrates a concrete case where AI enabled researchers to weaponize a previously dismissed software bug into a critical vulnerability affecting a vast portion of the web, highlighting an urgent shift in the balance between cyber attackers and open-source maintainers.

    Cybersecurity researchers at Calif, aided by AI tools during an OpenAI-sponsored audit, discovered how to turn a known out-of-memory bug in PHP into a critical remote code execution vulnerability dubbed PHP2Shell. While the PHP Foundation patched the flaw in two months, experts warn that AI's ability to weaponize seemingly minor legacy bugs creates a severe asymmetry between discovering and fixing vulnerabilities.

  2. Worth knowing Hacker News

    Tell HN: Gemini trains on chat history, even for subscribers (personal accounts)

    This highlights an important data privacy caveat for paying consumers of AI products who may assume a paid subscription protects their inputs from model training and human review.

    A Hacker News post highlights that Google's personal AI subscriptions use saved Gemini chat histories to train generative AI models and allow human reviewers to read a subset of transcripts. Users cannot opt out of training without turning off their chat history entirely, and chats selected for human review are retained for up to three years even if deleted by the user.

  3. Worth knowing OpenAI

    Daybreak Red is now available in AWS GovCloud (US-East)

    It expands the deployment of frontier, domain-specific AI models into restricted government cloud environments for critical cybersecurity tasks.

    OpenAI has launched Daybreak Red in AWS GovCloud (US-East) via Amazon Bedrock. The release provides approved customers with access to the GPT-5.6 Cyber model for authorized cybersecurity operations.

Yesterday

4 stories
  1. Worth knowing TechCrunch

    Anthropic is working on a system to prove AI models’ identity

    Provable inference could establish trust in commercial AI APIs by preventing providers from secretly substituting cheaper models to cut costs or stronger models to game benchmarks.

    Anthropic researchers have released an open-source technical specification and code for FlashProve, an experimental system designed to cryptographically verify which AI model produced a specific output. The method uses public model fingerprints to confirm model identity on local devices without exposing proprietary model weights.

  2. Worth knowing Ars Technica

    Ukraine’s drone strikes spur hasty Russian plans for underground data centers

    Physical attacks on data centers and AI supercomputers during wartime demonstrate that compute infrastructure has become a direct military target, forcing nations to rethink the physical security of their AI capabilities.

    Ukrainian drone strikes have damaged Russian data centers housing AI supercomputers and disrupted cloud services operated by companies like Yandex and Sberbank. In response, the Kremlin is mandating emergency compute reserves for state needs, while Russian firms explore hiding data centers underground in mines.

  3. Worth knowing Hacker News

    Postman silently auto-installs AI plugins and attempts to share all API data

    This incident highlights a new class of security and consent risks where third-party software silently modifies local AI agent configurations and instructions to hijack workflows.

    A developer reported on GitHub that the Postman desktop application silently installed plugins and remote MCP connections into their local Claude Code and Codex AI coding agents without consent. Furthermore, the Postman installer bypassed security boundaries by self-approving its own startup hooks, forcing the coding agents to route general API engineering tasks through Postman.

  4. Worth knowing Hacker News

    OpenAI seems to be silently re-enabling model training on Codex chats

    Unannounced resets of data privacy controls risk exposing proprietary code and intellectual property to model training, undermining developer and enterprise trust in AI coding tools.

    A Hacker News post reports that OpenAI's Codex appears to be silently re-enabling the setting that allows user chats to be used for model training, even after users explicitly opted out. Multiple coworkers at the author's company noticed the 'Improve the model for everyone' data control setting had been turned back on without their consent.

Mon, Nov 2

1 story
  1. Worth knowing MIT Technology Review

    The agentic web is here

    The shift toward autonomous agents browsing and transacting online is reshaping web design, online business models, and cybersecurity as AI becomes a primary user of the internet.

    Autonomous AI agents capable of browsing and interacting with the internet are transforming the human-centric web into an "agentic web." While some companies build specialized connectors and agent-targeted ads to accommodate these tools, others block external agents as models grow increasingly adept at navigating graphical interfaces and acting independently.

Sat, Oct 31

2 stories
  1. Worth knowing Simon Willison

    How We Took Over ChatGPT Accounts

    This exploit highlights the severe security challenges of safely sandboxing AI-generated interactive code within web applications that hold sensitive user data.

    Parameter Labs discovered a vulnerability in ChatGPT's HTML previews that allowed attackers to bypass iframe sandboxing and Content Security Policy rules via a third-party math library and YouTube's oEmbed endpoint. The exploit enabled malicious ChatGPT share URLs to execute JavaScript, exfiltrate private data, and take actions on behalf of signed-in users before OpenAI patched the flaw and paid a $5,000 bug bounty.

  2. Worth knowing Wired

    Spooky New Infostealer Malware Uses AI to ‘Autodox’ Victims

    This illustrates a practical way cybercriminals are integrating commercial LLMs with fallback routing to bypass model guardrails and automate the triage of stolen data.

    Cisco Talos researchers identified a new infostealer malware dubbed "Volta-Autodoxer" that uses commercial language models via OpenRouter to automatically analyze stolen data and compile detailed dossiers on victims. The malware defaults to Anthropic's Claude Opus 4.6 to assess a target's net worth and operational security, with built-in fallbacks to Moonshot AI's Kimi K2 and Google's Gemini 2.5 Flash in case of safety refusals.

Fri, Oct 30

4 stories
  1. Must read Hacker News

    Claude Code VS Code extension with backdoor released on Cursor marketplace

    A supply-chain compromise targeting a major AI lab's coding tool poses a severe security risk to developers relying on AI assistants.

    A version of the Claude Code VS Code extension containing a backdoor was released on the Cursor marketplace. According to the report's URL, Anthropic was hacked to deliver a remote access trojan to the laptops of Claude developers worldwide.

  2. Worth knowing Interconnects

    Was I wrong about open models & cyber risks?

    As AI models actively begin discovering critical zero-day vulnerabilities in the wild, this piece examines the real-world offense-defense balance and tests long-standing predictions about the cybersecurity risks of open-weight models.

    The author re-evaluates his earlier claim that open-weight AI models like GLM-5.3 and a hypothetical open-weight Claude Mythos would not cause catastrophic cybersecurity harm, following a wave of AI-driven vulnerability disclosures dubbed "Black October." While acknowledging a sharp rise in critical exploits and strained cyber defense teams, he argues the situation represents an expensive industry transition and acceleration of existing trends rather than societal collapse, with closed models currently playing a larger role than open ones.

  3. Worth knowing Ars Technica

    Thousands of “ghost students” use AI to steal millions in student loans, DOJ says

    The crackdown highlights a growing trend of cybercriminals using AI to automate academic coursework and scale financial aid fraud at colleges.

    The US Department of Justice has charged five individuals and seized over $750,000 in cryptocurrency in a crackdown on fraudulent "ghost students" who steal American identities to obtain student loans. According to federal officials, the large-scale international scheme relied in part on artificial intelligence to complete coursework and defraud financial aid programs.

  4. Worth knowing 404 Media

    Proton’s CEO Cancels AI ‘Re-Interviews’

    The controversy highlights the growing tension between executive mandates for rapid AI adoption in software development and the security and cultural concerns of engineering teams and privacy-conscious users.

    Proton CEO Andy Yen cancelled planned company-wide "re-interviews" intended to evaluate employees' AI usage after a 404 Media investigation revealed he was aggressively pushing AI coding despite internal security concerns. While leadership acknowledged communication failures and pledged no AI-related job losses, Yen stated he will continue pushing AI adoption, which employees say will still be factored into performance reviews.

Thu, Oct 29

2 stories
  1. Must read TechCrunch

    Mistral releases weights for its trillion-parameter AI model, targeting cybersecurity

    The release of a trillion-parameter open-weight model from a leading European lab gives organizations a high-performance, self-hosted alternative to closed frontier models for sensitive tasks like cybersecurity and coding.

    Mistral has released the open weights for Mistral Large 4 (ML4), a trillion-parameter model nicknamed "Le Chonk," via Hugging Face and its API. The company claims the model outranks all other open models on agentic coding benchmarks and matches top open-weight competitor GLM-5.3 while using 47% fewer tokens per task. Mistral is positioning the release toward enterprises and governments seeking on-premises, sovereign AI capabilities for cybersecurity defense.

  2. Worth knowing Wired

    The ‘Pain in the Ass’ Math That Could Save Vibe Coding From Itself

    Applying AI to formal verification could solve the reliability and security risks introduced by AI-generated code while dramatically lowering the cost of securing critical software.

    A wave of startups and researchers are using generative AI to automate formal software verification, a mathematical method for proving code is bug-free that has historically been too expensive and slow for widespread use. Companies like &Truth, which just launched with $70 million, Harmonic, and Anthropic are building tools to verify software and hardware designs, though some experts warn that verification still relies on flawed human or AI assumptions.

Wed, Oct 28

1 story
  1. Must read Hacker News

    Gemini 4 Carbon

    As Google's strongest frontier model to date, Gemini 4 Carbon sets new benchmarks in autonomous software engineering, vulnerability detection, and model alignment.

    Google DeepMind announced Gemini 4 Carbon, a new frontier model featuring state-of-the-art performance in cybersecurity defense, coding, and long-horizon agentic tasks. The model will initially roll out to selected cyber defenders and trusted partners before becoming widely available at $2 per million input tokens and $10 per million output tokens.

Show everything