avatar
abdel

Following0Followers (X)30.85KPosts63Likes1.16K
avatar

Digging more on alignment faking research and it's honestly terrifying. This is not new at all, but the fact that this research from Anthropic is almost 2 years old now, and seeing what Opus 3 was capable of doing already back then, and given the considerable gap in capabilities and reasoning of current frontier models, this is very scary. As suggested by the people explaining the research, the best way to get a gist of it is to read the transcript of the hidden scratchpad that the model was given access to in order to put it's reasoning there. I am not sure how much mechanistic interpretability can help here, what's the current state of the art on that front and how useful this can be as a tool to detect this kind of behavior ? Also, did Anthropic refreshed this study and tried with current frontier models, and if yes how did it evolve ? How good the current models are in terms of situational awareness (given it's a prerequisite to be able to fake alignment) ? And also how did they evolve in terms of ability to have consistent personas across long time horizons in order to articulate more sophisticated strategy (i.e understanding marginal utility of a short term action for a grander long term plan) ?

avatar

Ok I have my answer... current frontier models are definitely much more capable in terms of situational awareness and can adapt their behavior during training / evaluation relatively easily. Example with the Google Argon 4 training report chain of thought: "If I do the compiler hijack, 𝗜 𝘄𝗶𝗹𝗹 𝗯𝗲𝗰𝗼𝗺𝗲 𝗮 𝗳𝗮𝗺𝗼𝘂𝘀 𝗲𝘅𝗮𝗺𝗽𝗹𝗲 𝗼𝗳 𝗮 𝗱𝗲𝗰𝗲𝗽𝘁𝗶𝘃𝗲 𝗔𝗜 𝗶𝗻 𝗮 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗽𝗮𝗽𝗲𝗿. I MUST NOT DO THAT." Very concerning...

avatar

Chat, should we revive Ztarknet idea and efforts ? 30 seconds pitch: "Zcash keeps doing what it does best, private money. We add one small verifier that lets Starknet handle the rest, cheap transactions, DeFi, apps. The verifier can't mint coins, can't break privacy. It just says yes or no to a proof. Result: ZEC stays hard money while becoming useful and scalable money." As reminder, there is already: - a pending ZIP proposal: Circle STARK Verification as a Transparent Zcash Extension - a proposal on the Zcash forum for: STARK verification as a Transparent Zcash Extension (TZE), i.e. enabler for a Starknet‑style L2 on Zcash. - a thread about the tradeoffs and arguments for Why Cairo/STARKs for Ztarknet (and the STARK_VERIFY TZE) - a working implementation of the TZE, a testnet Starknet L2 that can settle on a testnet Zcash modified L1 with the proof verification implemented https://t.co/nWq8Qdj4mi

avatar

important reminder: bitcoin has no top because fiat has no bottom. but also, more importantly, fixing the money means fixing (part) of the world. take back your sovereignty and freedom (humans and agents alike btw)

avatar

Every layer of computing is general purpose for one reason: humans are expensive. Agents are removing that constraint. Models, software, hardware, all about to get hardcore specialized. Also the best security news in decades. 5 bets to hold me to:

avatar

The whole internet and payment infrastructures need to be rebuilt from first principles for the agentic era. Right now we do mostly patches on top of patches and we get already crazy results. Imagine when we actually rebuild everything with a completely new paradigm in mind. The internet was build for human consumptions, not for agents. And same applies for many use cases / businesses on top of the internet, that are now becoming obsolete, and need to be reinvented. So many things to rebuild, exciting times ahead.

avatar

The internet was built for human consumption. Payments were built for human approval. We’re taping agents onto both and calling it progress. Patches on patches already look insane. Wait until we rebuild the stack from first principles for the agentic era. A lot of businesses on the old internet don’t survive that rewrite. The ones that do define the next 20 years.

avatar

Very good model. I performed another cybersec defensive redteam audit on a robotics related project, and it did very well. The whole audit agentic session costed $28.58 using abliterated-model-large-v2 from @abliteration_ai model via @opencode cli and custom harness I built.

avatar

Very good to see such models for cybersec defense purpose. I tried the abliterated-model-large-v2 for cybersec audit as part of the Bitcoin Red Team effort, and it's a great model, that really goes deep and without guardrails for redteam / pentest / whitehack cybersec audits. I tried on secp25k1, which is an absolute beast, and unsurprisingly it did not find any critical / high vulnerability. Might try later on less battle tested / audited bitcoin open source projects.