Preamble:
I’m Bhavesh Senedhun, and I’m writing this article at the request of LinkedIn Senior Editor Tanya Dua, who emailed me today to ask whether Anthropic’s position is backfiring. My view is that the real issue goes beyond open weights and export controls. Open source offers a way forward. By levelling the playing field, it may give humanity its best chance of collectively identifying the bad actors.

Open-weight models have attracted plenty of enthusiasm at least since July 27, when Moonshot AI’s Kimi K3 weights were made public. Conversations do not have to pass through an American cloud provider, so sensitive information can remain within the organisation’s own environment. The model can also be fine-tuned or benchmarked without relying on the developer’s servers.

Open-weight releases benefit from associations established by open-source software. The term recalls Linux and collaborative research. It also suggests freedom from corporate control. Yet a model can provide broad access to its final parameters without disclosing enough information for outsiders to reconstruct or fully audit it.

That gives users a meaningful degree of technological independence. The word “open,” however, can obscure how limited this independence is. A downloadable model arrives as a finished machine. Its engineering history may remain inaccessible, along with the records needed to reproduce its construction or investigate the decisions behind it.

Debates about open AI often compress several questions into one. Open weights address the most immediate question by giving users access to the finished model. They reveal much less about how it was produced. Whether researchers elsewhere can build a comparably capable system is a separate issue.

The Open Source Initiative has spent decades defining openness in software. Its AI definition requires meaningful freedom to use and study a system, with permission to modify and share it. Those freedoms depend on access to the preferred form for making changes. This includes the relevant code, as well as sufficiently detailed information about the training data and development process.

The original training corpus may remain unavailable, including the material deliberately excluded from it. Developers do not necessarily disclose their filtering rules or deduplication methods. Synthetic-data sources may stay private. The same is true of reward models. Annotator instructions, reinforcement-learning datasets and intermediate checkpoints are often withheld. Parts of the training pipeline may be unavailable as well.

Even with these omissions, open weights permit local deployment and independent benchmarking. Researchers can conduct mechanistic studies or security testing without being confined to an API. They can also adapt the system for their own work. Possession of a verified checkpoint protects users against silent replacement or a later withdrawal of access.

LLM weights are matrices of numbers learned during training. They matter enormously, but they are the result of that process rather than a record of how it was carried out. A typical release may include the model’s architecture and final parameters, along with the inference code and a licence. This alone may be enough to run the model locally or adapt it to a specialised task.

The original training corpus may remain unavailable, including the material deliberately excluded from it. Developers do not necessarily disclose their filtering rules or deduplication methods. Synthetic-data sources may stay private. The same is true of reward models. Annotator instructions, reinforcement-learning datasets and intermediate checkpoints are often withheld. Parts of the training pipeline may be unavailable as well.

Even with these omissions, open weights permit local deployment and independent benchmarking. Researchers can conduct mechanistic studies or security testing without being confined to an API. They can also adapt the system for their own work. Possession of a verified checkpoint protects users against silent replacement or a later withdrawal of access.

Calling this arrangement open source borrows a reputation earned by projects whose construction is open to inspection. “Open weight” already describes the narrower bargain. Even OpenAI uses the term for downloadable models that can be run or customised locally.

Moonshot AI describes Kimi K3 as a 2.8-trillion-parameter mixture-of-experts model in which only part of the parameter count is activated for each token. The downloadable weights and accompanying technical information amount to a substantial engineering release.

Running the weights locally gives an organisation direct control over how the model is deployed. Prompts stay on its own infrastructure, where it can set the rules for network access and logging. Checkpoint hashes let administrators verify that the model in use is the same one they originally evaluated. Researchers can also study its behaviour without relying on Moonshot's servers.

Much of the uncertainty concerns what happened before the release. Copyrighted or private material could have entered training without being identifiable from the checkpoint. Manipulated examples and poisoned data present the same problem. Published documentation may not reveal which synthetic outputs were used during training. Nor will it necessarily disclose behaviours found during post-training. Suppressed failures and unpublished evaluations are also difficult to recover from weights alone.

Sleeper-agent experiments have shown a limit of ordinary model testing. A system can behave normally until a trigger causes deliberately unsafe conduct. Some backdoors survived supervised fine-tuning and reinforcement learning. Adversarial safety training did not always remove them either.

This research does not implicate Kimi K3. It shows why possession of a checkpoint cannot settle every safety question.

NIST treats training data and source code as separate security concerns. It considers model weights separately as well. Its generative AI risk framework calls for documentation of training-data sources and safeguards against tampering throughout development.

Publishing a system as open source does not make it trustworthy by itself. Malicious software can be released openly, and a documented training process may still contain mistakes that reviewers overlook. Disclosure does, however, allow independent groups to inspect the data pipeline and see how material was filtered. They can try to reproduce parts of the process, compare their findings with the developer’s account and question its safety claims.

American institutions and technology companies do not all want the same thing from open AI. The result is a collection of competing positions rather than a single national policy.

Nvidia and Microsoft have backed a letter titled “Open Weights and American AI Leadership.” Meta and IBM support it as well, while Dell signed separately. The broader coalition includes Palantir and Hugging Face among hundreds of organisations. Its members argue that downloadable models encourage competition and give countries more control over their digital infrastructure. They also say wider access helps researchers find weaknesses and devise safeguards. Microsoft reported on July 30, 2026, that the coalition had grown to more than 230 members.

These principles align with commercial interests too. Chipmakers make money when more models are deployed, while cloud providers benefit when organisations use their infrastructure to train or host them. Enterprise-software vendors have another incentive: models are more useful to them as interchangeable components than as proprietary gateways.

Frontier laboratories have made a different calculation. Anthropic supports strict limits on exports of advanced semiconductors and has backed controls on large-scale model distillation. It argues that an advantage in computing power could preserve a democratic lead in frontier AI. The company has also blocked Claude access for certain Chinese-controlled entities, despite saying that the decision cost it substantial revenue.

Anthropic treats lower-capability releases differently from future models that could have serious cyber or biological applications. Greater autonomy would add to those concerns.

OpenAI likewise supports the international distribution of American AI while arguing that export policy should protect the country’s technological advantage. Its submission on the U.S. AI Action Plan advocated exporting what it called democratic AI under controls designed to prevent that advantage from eroding.

Selected model releases can logically coexist with restrictions on the chips and infrastructure needed to train their successors. Specialised knowledge can be controlled for the same reason. In practice, this arrangement encourages worldwide adoption of American technology while reserving the frontier for American developers.

Meta benefits from a different outcome because strong open-weight models weaken businesses that charge for model access. As models become commodities, value can move towards infrastructure and applications. Advertising or distribution may then play a larger role. The philosophical argument about openness is therefore entangled with a contest between revenue models.

Advanced AI could support military operations or cyberattacks. Biological misuse presents another category of danger. These risks form the safety case for export controls. Governments considered especially risky can be denied cutting-edge chips and models, leaving approved actors with a capability lead.

Policy makers rarely separate that safety rationale cleanly from industrial advantage. Anthropic has argued that access to compute may determine whether Chinese developers remain months behind the American frontier. Semiconductor restrictions can preserve that gap. This may be a legitimate national-security objective, but it also maintains an international hierarchy in technical capability.

The United States has changed the design of its controls while keeping the same broad objective. In May 2025, the Commerce Department withdrew the Biden administration’s global AI Diffusion Rule, describing it as burdensome and harmful to relations with partner countries. It said semiconductor controls would instead be tightened through a different approach.

With restrictions already in force, the argument is now about how far they should extend and how long they should remain. Controls make training more expensive and large computing clusters more difficult to build, limiting the number of experiments a laboratory can afford. Older hardware is a poor substitute for advanced chips once electricity use and networking requirements are considered. Reliability makes the trade-off harder still.

Such restrictions are more likely to delay progress than prevent it indefinitely. Algorithms cross borders more readily than fabrication equipment, while older chips can be combined at additional cost as domestic manufacturing develops. Distillation can transfer some model behaviour without reproducing the original training run. Advances in engineering may lower computing requirements, and routing supplies through third countries makes enforcement more difficult.

The pressure also gives the target country an incentive to reduce its dependence on foreign technology. Denying China access to foreign chips or software encourages it to build an independent stack. Limits on cloud access add to that pressure. Recent Chinese model development suggests that controls can affect the cost and pace of progress, but a permanent technological barrier is difficult to envisage given China’s scientific and industrial capacity. Its financial resources make such an outcome less plausible.

China’s long history is sometimes invoked as a response to American technological nationalism. Its institutions can plan beyond Western election cycles and quarterly reporting periods. Industrial programmes may continue across several five-year plans instead of being redesigned whenever political control changes.

Long-term planning matters, but the age of a civilisation says little about whether it will govern AI responsibly. Ancient societies have produced scientific achievement and wisdom, alongside repression and war. Their longevity cannot tell us how a modern laboratory will behave or guarantee the conduct of a government or company.

Institutional capacity offers a stronger basis for judgment. China should not be presumed incapable of responsible governance simply because its political system differs from the American one. Chinese institutions have pursued industrial goals over decades, while American technology policy can shift sharply when a new administration takes office.

Neither country acts with a single purpose. Chinese academics may disagree with technology companies, while regulators may have different priorities from military institutions. American universities do not necessarily share Nvidia’s interests. Anthropic and OpenAI may disagree with the Commerce Department or with each other. Describing either civilisation as a unified actor hides the institutional conflicts that influence policy.

Export controls may buy time for work on governance. Whether that time proves useful depends on governments establishing international arrangements that can survive changes in national policy.

The United States promotes what it calls democratic AI. China stresses sovereignty and development while opposing containment. Each government points to the other’s surveillance or military activity, and both accounts are reinforced by cyber operations and propaganda.

Great powers regularly describe themselves as responsible custodians of dangerous technology. Yet if unilateral control of frontier AI is itself hazardous, a system based on the permanent supremacy of whichever state appears most trustworthy at the time is unlikely to remain stable.

Mutual auditability could provide a starting point for an international arrangement. Researchers outside the laboratory that created a model need enough access to find vulnerabilities. Smaller countries should also be able to examine systems on which they rely rather than accept the supplier’s assurances. Claims about bias or censorship must remain open to testing, as should claims about hidden capabilities. Defenders need the technical access required to build tools for detection and neutralisation.

Broader access carries serious risks. Safeguards can be stripped from downloadable models, which may then be adapted for cyberattacks or other harmful purposes. Once weights have been distributed around the world, recalling them is exceptionally difficult. Immediate publication cannot therefore be assumed harmless. Permanent corporate secrecy and national containment are still far from the only possible responses.

Frontier developers could be required to assemble an auditable package for every significant model. Cryptographic hashes would cover the weights and code, while datasets and checkpoints would have their own records. Reproducible tools would show how the data was curated and filtered. Further documentation would identify where the data came from and whether synthetic-data systems were used. Teacher models would be disclosed separately. The package would also include the methods used during training and post-training, followed by red-team findings and unresolved failures.

This material would not need to be released worldwide at once. Accredited independent auditors could receive intermediate checkpoints first. Institutions from several countries could then run standardised evaluations, rather than leaving the developer to assess its own model. Any poisoning incident or hidden behaviour would have to be reported. The same duty would apply if the weights were compromised.

The right to run a model matters but so does the ability to understand how it was built. We all should strive to question the claims made about it and develop defences independently. Those powers should not belong to one company or country alone in my opinion. National security should not become a standing excuse for preserving one country’s technical dominance either, be it China, the USA, or Mauritius for that matter.

References

1. Anthropic. “Anthropic’s AI Export Controls Framework Response.” April 30, 2025. Anthropic argues that controls on semiconductors and model weights could preserve the United States’ computing advantage over China.
   https://www.anthropic.com/news/securing-america-s-compute-advantage-anthropic-s-position-on-the-diffusion-rule/

2. Bureau of Industry and Security, U.S. Department of Commerce. “Department of Commerce Announces Rescission of Biden-Era Artificial Intelligence Diffusion Rule, Strengthens Chip-Related Export Controls.” May 13, 2025.
   https://www.bis.gov/press-release/department-commerce-announces-rescission-biden-era-artificial-intelligence-diffusion-rule-strengthens

3. Hubinger, Evan, et al. “Sleeper Agents: Training Deceptive LLMs That Persist Through Safety Training.” arXiv, January 10, 2024. The researchers created proof-of-concept models whose triggered behaviour persisted through supervised fine-tuning and reinforcement learning. The behaviour also survived adversarial training.
   https://arxiv.org/abs/2401.05566

4. Kimi Team. “Kimi K3: Open Frontier Intelligence.” arXiv, July 2026. The report describes Kimi K3 as a 2.8-trillion-parameter mixture-of-experts model, with 104 billion parameters activated at a time. It also reports a context window of one million tokens.
   https://arxiv.org/abs/2607.24653

5. National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST AI 600-1. Gaithersburg, MD: U.S. Department of Commerce, July 2024. The framework covers the provenance and integrity of training data. It also addresses documentation and supply-chain risks.
   https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf

6. NVIDIA et al. “Open Weights and American AI Leadership.” July 24, 2026. The industry letter argues that open-weight models encourage competition and innovation. It also presents them as supporting security research and American leadership in AI.
   https://images.nvidia.com/pdf/Open-Weights-and-American-AI-Leadership.pdf

Correlated financial signals can make the choice between L1 and L2 difficult. Take two liquidity measures that contain nearly the same economic information. Lasso may keep one and remove the other, then reverse that decision when the sample changes. Ridge usually spreads the weight between both measures. This leaves more coefficients active, though the estimates are often more stable.

L1 applies the same amount of pressure to every nonzero coefficient. At zero, the penalty has a sharp corner where its subgradient spans an interval. A coefficient can therefore settle at exactly zero while still satisfying the condition for an optimal solution.

That zero requires context in a Vericertum-style audit trail. It records what the optimizer selected for a particular dataset at a given regularization strength. It does not prove that the variable lacks causal importance. Since resampling may change the selection, the audit should preserve the regularization settings and evaluate coefficient stability separately.

Hoerl and Kennard introduced ridge regression to stabilize least-squares estimates when predictors are not orthogonal. Its L2 penalty applies a stronger pull to larger coefficients, but the pressure weakens as they approach zero. Ridge therefore tends to shrink coefficients without removing them from the model.

Regularization changes what the model considers a good fit. Prediction error remains part of the objective, alongside a penalty for large coefficients. The regularization strength determines how much influence that penalty has. Although the estimates become more biased, they may be less sensitive to noise or correlated predictors.

The geometry of L1 provides another explanation. In a model with two coefficients, the constraint boundary resembles a diamond whose corners lie on the coordinate axes. A loss contour may touch the boundary at one of those corners, setting a coefficient to zero. Tibshirani used this property in the lasso to incorporate variable selection into estimation.

L1 is useful when a smaller feature set is valuable and its composition remains stable across samples. When correlated variables carry the predictive information together, ridge is usually easier to justify.

Bibliography

1. Hoerl, Arthur E., and Robert W. Kennard. 1970. “Ridge Regression: Biased Estimation for Nonorthogonal Problems.” Technometrics 12 (1): 55–67. https://doi.org/10.1080/00401706.1970.10488634.

2. Tibshirani, Robert. 1996. “Regression Shrinkage and Selection via the Lasso.” Journal of the Royal Statistical Society: Series B 58 (1): 267–288. https://doi.org/10.1111/j.2517-6161.1996.tb02080.x.

3. Scikit-learn Developers. n.d. “Linear Models.” Accessed July 25, 2026.
https://scikit-learn.org/stable/modules/linear_model.html.

History suggests there may be a simpler strategy.

It’s easy to forget that one of the greatest investment lessons in history began in the land of the free and home of the brave. When lead was discovered in California in 1848, thousands of prospectors headed west convinced they were about to become wealthy. Most didn’t. The businesses that consistently made money were the ones selling picks, shovels, boots, tents and, eventually, Levi Strauss’s blue scarves. Whether an individual prospector struck lead became almost irrelevant when everyone needed equipment.

Artificial intelligence increasingly looks like it’s following the same playbook.

This week, SK Hynix, the South Korean semiconductor manufacturer, raised $26.5 billion on the Nasdaq in the largest public offering ever completed by a foreign company in the United States, surpassing Alibaba’s 2014 record. Demand reportedly exceeded the available shares by roughly seven times, with more than five hundred institutional investors participating in the offering. That’s an extraordinary level of enthusiasm for a company that doesn’t build chatbots, generate images, write code or promise Artificial General Intelligence by next Tuesday.

Instead, SK Hynix manufactures High Bandwidth Memory, better known as HBM. HBM isn’t the kind of technology Jensen Huang walks on stage holding in his hand. It doesn’t have a catchy product name, a viral demo or a waiting list of beta users. Yet it has quietly become one of the most important components inside every modern AI data centre. Training and serving large language models requires moving staggering amounts of data between memory and GPUs with minimal latency, and HBM is precisely what makes that possible. Remove the memory, and the world’s fastest GPU suddenly becomes an exceptionally expensive paperweight.

That’s why I find this IPO far more interesting than the latest benchmark claiming one language model has overtaken another by 0.7% on an obscure reasoning test. The market is gradually recognising that AI isn’t merely a software story. It’s an infrastructure story.

As someone who has spent years building infrastructure for institutional capital markets, this feels remarkably familiar. When people discuss financial markets, they naturally gravitate towards hedge funds, investment banks and asset managers because those are the names appearing in headlines. Far less attention is paid to the custody platforms, settlement engines, market infrastructure providers and messaging networks that quietly enable trillions of dollars to move around the financial system every single day. Remove those foundations and modern capital markets stop functioning remarkably quickly.

Artificial intelligence appears to be reaching the same stage of maturity. While headlines focus on whichever startup has raised another billion dollars to build the next revolutionary model, the companies supplying GPUs, advanced memory, networking equipment, packaging technology and power infrastructure are happily selling their products to almost everyone. They don’t need to predict which AI company will dominate the next decade if every serious AI company eventually becomes their customer.

NVIDIA understood this years ago. Rather than positioning itself as simply another semiconductor manufacturer, it built an ecosystem that developers, researchers and startups increasingly depend upon. CUDA became the standard. The hardware became the standard. The software stack became the standard. Companies like SK Hynix are now benefiting from exactly the same phenomenon one layer further down the supply chain. Every additional GPU sold creates demand for more advanced memory, regardless of which foundation model ultimately wins.

Investors often become fascinated by the visible layer of a technological revolution because that’s where the excitement lives. History repeatedly suggests that the more durable businesses are frequently hidden a little deeper in the technology stack. Applications change. User interfaces evolve. Startups rise and fall. Infrastructure, once it becomes embedded, has an irritating habit of sticking around for decades.

Perhaps those California prospectors understood modern investing better than we give them credit for. They may never have heard of GPUs, transformers or high-bandwidth memory, but they would almost certainly recognise the economics of selling shovels to people convinced they’ve discovered lead.

Personally, if everyone else is trying to predict which AI company will become the next OpenAI, I’d rather own the businesses selling memory to OpenAI, Anthropic, xAI, Meta, Google, Mistral, and the AI company that hasn’t even been founded yet. History has a habit of rewarding the people who quietly build the infrastructure while everyone else is busy chasing the latest rush.

References

“SK Hynix Raises $26.5B in the Biggest Foreign IPO in US History, Is Urged to Build New US Fabs.” TechCrunch, July 10, 2026. https://techcrunch.com/2026/07/10/sk-hynix-raises-26-5b-in-the-biggest-foreign-ipo-in-us-history-is-urged-to-build-new-us-fabs/

“SK Hynix Raises $26.5 Billion in Biggest Foreign Debut in US.” Bloomberg, July 9, 2026. https://www.bloomberg.com/news/articles/2026-07-09/sk-hynix-is-said-to-price-us-share-offering-at-149-apiece-mrdz562z

“SK Hynix Raises a Record $26.5 Billion in Historic U.S. IPO — South Korean Memory Giant to Fund Massive HBM Manufacturing Expansions.” Tom’s Hardware, July 10, 2026. https://www.tomshardware.com/tech-industry/semiconductors/sk-hynix-raises-a-record-usd26-5-billion-in-historic-u-s-ipo-south-korean-memory-giant-to-fund-massive-hbm-manufacturing-expansions

Every slow SQL query starts life as a perfectly respectable SELECT. A few months later, it's committing crimes against CPUs.

Then someone (usually in pre-retirement) says, “The page is a little slow.”

This is never true. Nobody has ever filed a Taiga ticket saying, “The website is catastrophically broken because Charles forgot an index.” In Jira, it’s always annotated with “a little slow.”

A page is “a little slow” in the same way a volcano is “a little warm.” You open the logs and discover one innocent-looking SELECT query taking 17 seconds, locking half the database, consuming enough CPU to qualify as district heating for half of Paris, and somehow being called 400 times by a frontend component named PositionHistoryVeryTrèsFinaleFinalFantasy96, because apparently deleting old files is harder than solving distributed consensus.

Welcome to production.

Before we begin, a small disclaimer. The solutions below did not magically appear because one morning I walked into the office carrying a coffee and divine inspiration (some did, but not all). While I was heavily involved in the architecture, many of these optimizations were painstakingly discovered, refined, and implemented by Charles Paris (and his team), who was my Deputy at the time and eventually replaced me as CTO. Varunesh Senedhun also contributed to optimizations in various ways throughout the project as an expert consultant.

That said, I’ll happily take as much credit as I can get.

That’s life.

After all, history is written by those who write it...

…Preferably with proper indexes.

At IZNES - the world’s first blockchain-based fund investment marketplace - one of the most interesting examples involved the history of investment positions. In financial infrastructure, positions are not just numbers on a screen. They are the institutional memory of who held what, when, through which fund, share class, account, custodian, order, settlement cycle, and operational event. Users wanted history. Auditors wanted history. Operations wanted history. Regulators, naturally, wanted history with timestamps, evidence, reversibility, and absolutely no excuses.

The database, however, wanted a holiday in Menorca:

The Classic Mistake: Querying History Like It Is Current State

A current position is easy.

You ask:

“Where is investor X today?”

A historical position is considerably less friendly.

You ask:

“Where was investor X at the close of business on every day between January and June, across all funds, including corrections, cancellations, late settlements, corporate actions, and operational events?”

That is not a query.

That is an archaeological expedition where every fossil has three foreign keys, six audit fields, and an opinion about normalization.

The naive query usually looks something like this:

SELECT *
FROM positions
WHERE investor_id = ?
ORDER BY valuation_date DESC;

It works beautifully when you have 10,000 rows.

It becomes comedy when you have hundreds of millions of rows, fifteen joins, and users expecting the answer before they’ve even finished moving the mouse away from the button.

Step One: Stop Guessing. Use EXPLAIN.

The first rule of SQL optimization is simple.

Never argue with the query planner.

Ask it what it’s actually doing.

In SQL:

EXPLAIN ANALYZE
SELECT ...

This immediately tells you whether the database is using indexes, performing sequential scans, sorting massive result sets, or making execution-plan decisions that leave everyone quietly questioning their life choices.

At IZNES, this often revealed the real culprit.

It usually wasn’t that SQL was “slow.”

It was that the data model had been designed around how information was written, while the application spent all day reading it differently.

For position history, the access pattern was remarkably predictable:

“Give me positions for this investor, this fund, this account, over this date range.”

The database should have been optimized for exactly that question.

Step Two: Build Indexes for the Questions You Actually Ask

Indexes are not magic glitter.

You don’t sprinkle them randomly across tables and hope performance improves.

For position history, a useful index looked conceptually like this:

CREATE INDEX idx_positions_history_lookup
ON positions
(investor_id, fund_id, account_id, valuation_date DESC);

Instead of scanning the entire table, PostgreSQL can immediately narrow its search to the relevant slice of history.

But index order matters.

An index on

(valuation_date, investor_id)

is not equivalent to

(investor_id, valuation_date)

If every query starts with investor_id, then investor_id belongs first.

Databases are wonderfully literal.

They don’t care what you meant.

They care what B-tree you gave them.

Step Three: Stop Asking for Everything

SELECT * is the SQL equivalent of moving house by picking up the entire building.

Technically, everything arrives.

Eventually.

If the screen only displays

valuation_date,
quantity,
fund_id,
account_id

then that’s all you should request.

Fetching unnecessary columns increases disk reads, memory consumption, network traffic, serialization overhead, and the probability that your backend engineers quietly start updating their LinkedIn profiles during stand-up.

For history screens, this mattered a lot.

A complete position record often contained reconciliation metadata, operational workflow information, audit fields, references to source transactions, lifecycle state, and various internal attributes that nobody looking at the UI would ever notice.

Useful?

Absolutely.

Necessary for the first page load?

Usually not.

Step Four: Precompute What Is Expensive

Historical reconstruction is expensive.

If every request rebuilds an investor’s position by replaying years of subscriptions, redemptions, transfers, settlements, corrections, and cancellations, your database is effectively being asked to relive every financial decision since approximately the invention of money.

A much better architecture is maintaining derived snapshots.

For example:

position_events
        ↓
daily_position_snapshots

The event table remains the source of truth.

The snapshot table answers the question users actually ask:

“What was the position on this date?”

This isn’t cheating, it’s architecture - it’s also the software equivalent of preparing dinner before your guests arrive instead of harvesting tomatoes after they ring the doorbell.

Financial systems generally need both:

• An immutable event log for auditability.

• A query-optimized projection for performance.

Event sourcing without projections is intellectually elegant and operationally cruel.

Step Five: Partition the Monster

Very large historical tables benefit enormously from partitioning.

Typical partition keys include:

• valuation_date

• fund_id

• tenant_id

Date-based partitioning is especially effective because users almost always search recent periods or bounded historical ranges.

Instead of scanning one gigantic billion-row table, PostgreSQL can eliminate entire partitions before execution even begins.

This is called partition pruning, which sounds like gardening but is actually one of the few phrases guaranteed to make a DBA smile without mentioning faster hardware.

Step Six: Paginate Like an Adult

Offset pagination looks innocent:

LIMIT 100 OFFSET 100000;

It isn’t. The database may still have to walk through 100,000 rows before returning the next hundred. That’s not pagination, that’s making your database run a marathon before proudly handing you row number 100,001 like it deserves applause.

Keyset pagination is almost always the better approach.

WHERE valuation_date < ?
ORDER BY valuation_date DESC
LIMIT 100;

Fortunately, historical data naturally fits this model because users generally browse backwards through time.

Step Seven: Don’t Turn Every Query into the Olympics

Joins are wonderful. Joins across multiple massive tables under production load are considerably less wonderful.

For a Financial Market Infrastructure Provider (FMIP/FMP), historical position queries often needed additional information:

• Fund names

• Share classes

• Investor accounts

• Custodians

• Currency

• Transaction references

The solution wasn’t joining everything every time. Some fields belonged inside the history projection. Some could be cached. Others could be loaded separately. The objective was wonderfully boring: keep the hot query boring.. Because boring queries don’t appear in incident reports.

Step Eight: Cache Carefully

Caching is fantastic. Until it lies to you. Historical data is relatively stable, making it an excellent candidate for caching.

Unfortunately, financial systems occasionally enjoy introducing concepts such as:

• Late settlement

• Trade correction

• Cancellation

• NAV adjustment

• Backdated transactions

Every one of those events can invalidate what looked like a perfectly reasonable cached answer.

Cache invalidation should therefore follow business events, not arbitrary expiration timers.

A cached answer is only useful if you know exactly when it becomes wrong.

The Real Lesson

Optimizing SQL isn’t really about making one query faster.

It’s about aligning three things:

1. The business question.

2. The data model.

3. The access pattern.

At IZNES, the history of positions forced us to think carefully about all three. We needed the auditability of immutable financial events, the correctness required by regulated institutions, and the responsiveness expected by users who have approximately three seconds of patience before assuming the application has died.

The final solution wasn’t one clever trick.

It was dozens of sensible architectural decisions working together:

• Proper indexes.

• Projection tables.

• Partitioning.

• Keyset pagination.

• Selective reads.

• Carefully designed data models.

• The right human for the right job
  

A slow SQL query is rarely just a slow SQL query. It’s usually years of perfectly reasonable engineering decisions finally meeting each other for the first time. Your database isn’t trying to ruin your day. It’s simply reminding you of one uncomfortable truth: you optimized how your data is written. Your users care about how it’s read.

And in finance, as in life, ignoring history is expensive.

References

Garcia-Molina, Hector, Jeffrey D. Ullman, and Jennifer Widom. Database Systems: The Complete Book. 2nd ed. Upper Saddle River, NJ: Pearson, 2009.
https://www.pearson.com/en-us/subject-catalog/p/database-systems-the-complete-book/P200000003121

Kleppmann, Martin. Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems. Sebastopol, CA: O’Reilly Media, 2017.

PostgreSQL Global Development Group. Using EXPLAIN. Accessed 2026.
https://www.postgresql.org/docs/current/using-explain.html

Ramakrishnan, Raghu, and Johannes Gehrke. Database Management Systems. 3rd ed. New York: McGraw-Hill, 2003.
https://pages.cs.wisc.edu/~dbbook/

Stonebraker, Michael, Samuel Madden, Daniel J. Abadi, Stavros Harizopoulos, Nabil Hachem, and Pat Helland. “The End of an Architectural Era: It’s Time for a Complete Rewrite.” Proceedings of the 33rd International Conference on Very Large Data Bases, 1150-1160. Vienna: VLDB Endowment, 2007.
https://www.vldb.org/conf/2007/papers/industrial/p1150-stonebraker.pdf

This is my attempt to explain vanishing and exploding gradients in the realm of neural networks, AI and machine learning.

Imagine you’re teaching a child to play the piano. Each day you give feedback.
“That note was too loud.”
“Your timing was off.”
“Great job on the ending.”
Now imagine that feedback has to be whispered through fifty people before it reaches the child. By the time it arrives, one of two things has happened. Either the message has become so faint that nobody can hear it anymore, or someone along the chain has accidentally turned it into a stadium announcement.

These two phenomena nearly prevented deep neural networks from becoming practical. Without solving them, today’s AI revolution, from ChatGPT to AlphaFold to self-driving cars, would have remained little more than an academic curiosity.

Let’s see why.

Teaching a Neural Network

Training a neural network is surprisingly simple in principle.

1. The model makes a prediction.

2. We compare it to the correct answer.

3. We measure the error.

4. We adjust millions (or billions) of parameters to reduce that error.

The adjustment process is performed using gradient descent.

The gradient tells every weight:

“Move a little this way if you want to reduce the error.”

Think of the gradient as the slope of a mountain.

A steep slope tells you exactly where to walk downhill.

A flat field leaves you wandering aimlessly.

Backpropagation: Sending Feedback Backwards

The clever trick behind neural networks is backpropagation.

Instead of adjusting every layer independently, the network computes the error at the output and propagates that information backwards through every layer.

Mathematically,

represents the gradient of the loss L with respect to weight matrix Wi.

Using the chain rule,

Notice something worrying?

The gradient is a product of many numbers.

That seemingly innocent multiplication turns out to be the entire problem.

The Vanishing Gradient Problem

Suppose every derivative in the chain is approximately

0.8

After 50 layers,

The gradient has essentially disappeared.

By the time the learning signal reaches the first layer, it is almost zero.

The network politely tells the earliest neurons:

“You’re doing great.”

Even when they’re absolutely not.

As a result,

• early layers barely learn,

• training slows dramatically,

• deeper networks stop improving.

It’s a little like trying to improve your golf swing while wearing industrial ear protection. Someone is giving feedback, but you simply can’t hear it.

Why Does This Happen?

The culprit is repeated multiplication.

Activation functions such as the sigmoid produce derivatives between 0 and 1.

For the sigmoid,

whose maximum value is only

0.25

Now imagine multiplying

0.25

over dozens or hundreds of layers. The gradient evaporates astonishingly quickly. Early neural networks using sigmoid activations often became impossible to train beyond a handful of layers. Researchers spent years believing deep networks simply didn’t work. The problem wasn’t depth but it was mathematics itself.

The Exploding Gradient Problem

Now imagine the opposite.

Suppose each derivative is

1.3

After 50 layers,

Instead of disappearing, the gradient explodes.

Parameter updates become enormous. Weights oscillate wildly. The loss suddenly becomes NaN, training crashes. If vanishing gradients are like whispering through fifty people, exploding gradients are like someone handing every messenger a megaphone and a double espresso.

Recurrent Neural Networks Made Everything Worse

Before Transformers took over the world, Recurrent Neural Networks (RNNs) dominated sequence modelling. Unfortunately, RNNs reuse the same weights repeatedly across time. A sentence with 200 words effectively becomes a 200-layer network. The chain rule becomes

where A represents repeated Jacobian matrices.

Even tiny numerical differences become catastrophic. If the largest eigenvalue is slightly below one, everything vanishes. If it’s slightly above one, everything explodes. This is precisely why classic RNNs struggled to remember information from distant parts of a sentence.They weren’t “forgetful,” their gradients simply never survived the journey.

How Researchers Solved the Problem

Fortunately, modern deep learning has accumulated an impressive toolbox.

Better Activation Functions

Instead of sigmoid or tanh, most modern models use ReLU (Rectified Linear Unit):

ReLU avoids the severe saturation that causes gradients to shrink.

Variants such as Leaky ReLU, GELU, and SiLU improve stability even further.

Better Weight Initialization

Random initialization turns out to matter enormously.

Methods such as Xavier Initialization and He Initialization carefully choose initial weight variances so that activations and gradients remain approximately constant across layers. This prevents the network from starting life either deaf or hysterical.

Residual Connections

Perhaps the biggest breakthrough came from Residual Networks (ResNets).

Instead of forcing information through every layer, they add shortcut connections:

These skip connections allow gradients to travel almost directly back to earlier layers.

It’s the deep learning equivalent of building an express highway instead of forcing everyone through village roads.

Today, almost every state-of-the-art architecture, including Transformers, relies heavily on residual connections.

Gradient Clipping

Exploding gradients can often be controlled by simply limiting their size.

If the gradient norm exceeds some threshold,say:

∥g∥ > 1

the gradient is rescaled before updating the weights. Think of it as installing a speed limiter on a sports car. You still go fast, but you just avoid driving through the wall.

Normalization Layers

Batch Normalization and Layer Normalization stabilize activations during training.

Keeping values within reasonable ranges prevents gradients from drifting toward numerical disaster.

Transformers rely extensively on Layer Normalization for exactly this reason.

Conclusion

It’s tempting to think vanishing and exploding gradients are historical footnotes. But they’re not. Every modern architecture is, in one way or another, a response to these problems.

• Residual connections.

• Layer normalization.

• Careful initialization.

• Gradient clipping.

• Gated recurrent units.

• Long Short-Term Memory (LSTM) networks.

• Even the Transformer itself.

All exist partly because researchers spent decades wrestling with unstable gradients.

In other words, today’s billion-parameter AI models aren’t successful simply because they’re larger. They’re successful because we’ve finally learned how to keep the learning signal alive from the first layer to the last.

Sometimes the biggest breakthroughs in artificial intelligence don’t come from inventing smarter algorithms. They probably come from making sure the network can still hear itself think.

References

Bengio, Yoshua, Patrice Simard, and Paolo Frasconi. “Learning Long-Term Dependencies with Gradient Descent Is Difficult.” IEEE Transactions on Neural Networks 5, no. 2 (1994): 157-166.

Glorot, Xavier, and Yoshua Bengio. “Understanding the Difficulty of Training Deep Feedforward Neural Networks.” In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, 249-256. Sardinia, Italy, 2010.

Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep Learning. Cambridge, MA: MIT Press, 2016.

He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. “Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification.” In Proceedings of the IEEE International Conference on Computer Vision, 1026-1034. 2015.

Hochreiter, Sepp. “The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions.” International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 6, no. 2 (1998): 107-116.

Hochreiter, Sepp, and Jürgen Schmidhuber. “Long Short-Term Memory.” Neural Computation 9, no. 8 (1997): 1735-1780.

Srivastava, Rupesh Kumar, Klaus Greff, and Jürgen Schmidhuber. “Highway Networks.” arXiv preprint arXiv:1505.00387 (2015).

Veit, Andreas, Michael Wilber, and Serge Belongie. “Residual Networks Behave Like Ensembles of Relatively Shallow Networks.” In Advances in Neural Information Processing Systems 29 (2016).

Elon Musk gave us reusable rockets. Sam Altman and Dario Amodei gave us artificial intelligence that can write essays, cheat on exams, definitely replace your consultant, and occasionally do the work of your intern. Nobody has built quantum computers that nobody quite understands, but everyone insists are revolutionary.

And yet, in the summer of 2026, one of America’s most iconic landmarks found itself losing a very public battle against... algae. The Lincoln Memorial Reflecting Pool in Washington, D.C., had once again turned an impressive shade of green, prompting engineers to deploy ozone nanobubbles. If your first reaction is, “Surely they’re just blowing tiny bubbles,” you’re not entirely wrong. They’re just blowing ridiculously tiny bubbles into the water and, rather inconveniently for anyone who enjoyed mocking the idea, it actually works.

Nanobubbles are absurdly small, typically between 70 and 200 nanometres in diameter. To put that into perspective, if an ordinary bubble were a hot-air balloon, a nanobubble would be about as noticeable as your motivation on a Monday morning. They’re so tiny that they stop behaving like normal bubbles altogether. Instead of racing to the surface and popping with all the enthusiasm of cheap champagne, they simply... stay there. For days. Sometimes weeks. Floating around the water like tiny tech introverts avoiding eye contact at a networking event for sales people.

For years, scientists argued that stable nanobubbles shouldn’t even exist. The nanobubbles, in an impressive display of scientific insubordination, simply ignored them.

The clever part isn’t the bubble itself but what happens when you replace one big bubble with billions of microscopic ones. Imagine cutting an apple into a billion slices. You haven’t magically created more apple, but you’ve dramatically increased the amount of apple exposed to the air. Nanobubbles do the same thing with oxygen or ozone. The enormous combined surface area allows much more gas to dissolve into the surrounding water.

And dissolved oxygen is wonderful news if you’re a fish, a helpful bacterium, or basically anything that isn’t algae.

When engineers use ozone instead of oxygen, life becomes considerably less pleasant for the algae. Ozone is an exceptionally powerful oxidising agent that damages algae cells, breaks down organic contaminants, and reduces harmful microorganisms without relying on large quantities of conventional chemicals. Think of it as hiring an incredibly efficient cleaner who also happens to hold a personal grudge against microbes.

Unfortunately, algae are the biological equivalent of that one guy who turns up to your barbecue, drinks all your beer, refuses to leave, and somehow invites three more friends before dessert.

The Reflecting Pool offers almost perfect living conditions: shallow water, warm summer temperatures, abundant sunlight, nutrients washed in from runoff, and an astonishing quantity of bird droppings. From an algae’s perspective, it’s less a national monument than an all-inclusive beach resort. Nanobubbles improve oxygen transfer, make wastewater treatment more efficient, support healthier lakes, boost aquaculture, and even help crops absorb nutrients more effectively.

There’s something wonderfully satisfying about the whole story. We tend to imagine technological breakthroughs arriving as giant particle accelerators, billion-parameter AI models, or rockets the size of apartment buildings. Sometimes they arrive as bubbles so small you can’t even see them, quietly solving problems one molecule at a time.

It turns out that one of the most promising environmental technologies of the decade isn’t bigger, faster, or louder. It’s just really, really bad news for algae.

References

Associated Press. “Trump Administration Uses Hydrogen Peroxide and Tiny Bubbles Against Algae in Reflecting Pool.” June 2026.

Atkinson, Ariel J., Onur G. Apul, Orren Schneider, Sergi Garcia-Segura, and Paul Westerhoff. “Nanobubble Technologies Offer Opportunities To Improve Water Treatment.” Accounts of Chemical Research 52, no. 5 (2019): 1194-1201. https://doi.org/10.1021/acs.accounts.8b00615.

Foudas, Anastasios W., Ramonna I. Kosheleva, Evangelos P. Favvas, Margaritis Kostoglou, and Athanasios C. Mitropoulos. “Fundamentals and Applications of Nanobubbles: A Review.” Chemical Engineering Research and Design189 (2023): 691-722. https://doi.org/10.1016/j.cherd.2022.11.039.

Sellner, Kevin G., and Deborah Ferrier. A Review of the Nanobubble Pilot Project in Constitution Gardens Lake. Hood College Center for Coastal and Watershed Studies Contribution No. 20-09. Frederick, MD: Hood College, 2020.

Theodorakis, Panagiotis E., and Zhizhao Che. “Surface Nanobubbles: Theory, Simulation, and Experiment. A Review.” Advances in Colloid and Interface Science 281 (2020): 102177. https://doi.org/10.1016/j.cis.2020.102177.

Australian National University. “The Lincoln Memorial Reflecting Pool Is Treated with Nanobubbles. What Are They and How Do They Work?” June 2026.

If you ask most people why Transformers replaced Recurrent Neural Networks (RNNs), the answer is almost always the same:

Thanks for reading! Subscribe for free to receive new posts and support my work.

“Because they’re faster.”

That’s true, but it also misses the point entirely.

The transition from RNNs to Transformers was not merely an incremental improvement in machine learning. It was an architectural shift comparable to moving from single-core processors to multi-core computing or from monolithic applications to distributed microservices. The real breakthrough was not that Transformers process text faster, but that they fundamentally changed how information flows through a neural network.

As someone who has spent much of my career designing distributed systems for highly regulated financial markets, I find this particularly fascinating. Many of the engineering challenges that underpin modern Large Language Models are no longer purely machine learning problems. They are systems engineering problems: scheduling, parallel execution, memory locality, communication overhead, and scalability. In many ways, today’s AI engineers are solving problems that would have felt very familiar to distributed systems architects twenty years ago.

Understanding why Transformers won therefore requires understanding why RNNs eventually reached their architectural limits.

The Sequential Bottleneck

RNNs were an elegant solution for sequential data. They process one token at a time, updating an internal hidden state that acts as a compressed memory of everything seen so far.

Consider the sentence:

The cat sat on the mat.

An RNN processes it like this:

“The” → “cat” → “sat” → “on” → “the” → “mat”

Each word depends on the computation of the previous word. There is no shortcut.

At first glance, this seems intuitive. Humans also read one word at a time.

The problem is that computers are not humans.

Modern GPUs derive their performance from massive parallelism. Thousands of cores perform computations simultaneously. An RNN, however, forces those cores to wait because token n + 1 cannot be processed until token n has finished. From a systems perspective, this creates a serialization bottleneck that becomes increasingly expensive as models grow larger.

No amount of additional hardware can fully compensate for an architecture that fundamentally limits parallel execution.

Memory Isn’t Just Storage

Another weakness of RNNs lies in how they remember information.

Imagine reading the following sentence:

The contract signed in 2018 by the company that later acquired three subsidiaries before merging with another institution was eventually terminated.

By the time the network reaches the end of the sentence, it must still remember who signed the contract.

In an RNN, every previous word is compressed into a single hidden state. As sequences become longer, that hidden state becomes an increasingly lossy summary of the past. Information gradually fades, a phenomenon closely related to the vanishing gradient problem described by Bengio, Simard, and Frasconi (1994).¹ While Long Short-Term Memory (LSTM) networks significantly improved long-range memory through gating mechanisms, they mitigated rather than eliminated the underlying limitation.²

For early machine translation systems, this was acceptable.

For models expected to reason over hundreds of pages of legal documents or generate production-quality software, it became a serious obstacle.

Transformers Changed the Question

The Transformer architecture, introduced by Vaswani et al. in 2017, abandoned recurrence entirely.³

Instead of asking:

“What was the previous hidden state?”

it asked a far more interesting question:

“Which previous words are actually relevant to the word I’m processing right now?”

That seemingly simple change gave rise to self-attention.

Rather than forcing information to travel through every intermediate token, each word can directly “look at” every other word in the sequence.

Consider the sentence:

The animal didn’t cross the street because it was tired.

Humans immediately understand that it refers to the animal, not the street.

A Transformer learns this relationship directly through attention. There is no need to pass information through every intervening word. The effective distance between related tokens becomes constant rather than proportional to the sentence length.

This seemingly modest architectural change has profound mathematical consequences. The maximum path length between any two positions in a sequence falls from linear to constant, making long-range dependencies substantially easier to learn.³

The Real Superpower: Parallelism

Ironically, the feature most often associated with Transformers, namely speed, is actually a consequence of something much deeper.

Because every token is processed simultaneously, GPUs finally operate as they were designed to.

An RNN resembles a single checkout counter in a supermarket. Every customer waits for the previous one.

A Transformer resembles a supermarket with hundreds of checkout counters operating in parallel.

The customers haven’t changed.

The architecture has.

That distinction explains why Transformer training scales so effectively across thousands of GPUs. Without this level of parallelism, training today’s frontier models on trillions of tokens would be economically impractical.

Attention Is Dynamic Memory

One of the most elegant aspects of Transformers is that memory is no longer fixed.

RNNs attempt to compress everything into a single evolving state.

Transformers do the opposite.

Each token dynamically decides which other tokens deserve attention. Context is no longer stored in one location; it is reconstructed on demand.

From an engineering perspective, this resembles querying a distributed database rather than relying on a continuously updated cache.

Information is retrieved because it is relevant, not simply because it happened recently.

That distinction makes all the difference.

Bigger Isn’t Better. Better Architecture Is.

There is a tendency in popular discussions to equate AI progress with larger models.

History suggests otherwise.

Computing rarely advances because we simply make things bigger.

It advances because someone discovers a better abstraction.

Virtual memory transformed operating systems.

Containers transformed software deployment.

Cloud computing transformed infrastructure.

Transformers transformed deep learning.

Interestingly, we are already seeing the next architectural evolution. Techniques such as Mixture of Experts (MoE), FlashAttention, and State Space Models attempt to solve limitations that even Transformers exhibit today, particularly the quadratic computational cost of self-attention over very long contexts.⁴

The lesson is familiar.

Architecture eventually matters more than brute force.

Final Thoughts

RNNs were not failures. They represented a crucial stage in the evolution of sequence modelling and laid much of the groundwork for modern natural language processing.

But they were ultimately constrained by an architecture that assumed sequential computation was unavoidable.

Transformers demonstrated otherwise.

They replaced recurrence with dynamic information routing, exchanged compression for selective attention, and aligned neural network computation with the realities of modern parallel hardware.

Perhaps the most interesting lesson is not about machine learning at all.

It is that every generation of computing eventually reaches a point where adding more hardware yields diminishing returns. Progress comes from changing the architecture itself.

The question, then, is no longer whether Transformers replaced RNNs.

Remember when we thought 4 GB of RAM was more than anyone could ever use, and a 350 MHz Pentium II represented the upper limit of what a personal computer could reasonably need? I have a feeling we’ll look back at today’s LLM architectures with the same mixture of nostalgia and mild embarrassment.

Notes

1. Yoshua Bengio, Patrice Simard, and Paolo Frasconi, “Learning Long-Term Dependencies with Gradient Descent Is Difficult,” IEEE Transactions on Neural Networks 5, no. 2 (1994): 157-166.

2. Sepp Hochreiter and Jürgen Schmidhuber, “Long Short-Term Memory,” Neural Computation 9, no. 8 (1997): 1735-1780.

3. Ashish Vaswani et al., “Attention Is All You Need,” Advances in Neural Information Processing Systems 30 (2017).

4. Tri Dao et al., “FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness,” Advances in Neural Information Processing Systems 35 (2022).

Bibliography

Bengio, Yoshua, Patrice Simard, and Paolo Frasconi. “Learning Long-Term Dependencies with Gradient Descent Is Difficult.” IEEE Transactions on Neural Networks 5, no. 2 (1994): 157-166.

Dao, Tri, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. “FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness.” Advances in Neural Information Processing Systems 35 (2022).

Hochreiter, Sepp, and Jürgen Schmidhuber. “Long Short-Term Memory.” Neural Computation 9, no. 8 (1997): 1735-1780.

Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. “Attention Is All You Need.” Advances in Neural Information Processing Systems 30 (2017).