An empirical exploration of why arithmetic models sometimes neither memorize nor generalize (Part 2 of an ongoing exploration into how small language models learn arithmetic) In the previous post, I described an experiment that started with a simple goal:to observe memorization and eventual grokking in a small language model trained on arithmetic operations. The setupContinueContinue reading “Between Memorization and Meaning: When Neural Networks Learn, But Not the Way We Expect”
Category Archives: technology
Why arithmetic models look dumb long after they’ve learned the rule
An experiment in memorization, grokking, and misleading loss curves This post documents an experiment that didn’t go the way I expected.What started as a simple attempt to observe memorization and grokking in arithmetic models turned into a deeper lesson about how misleading loss curves can be — especially for algorithmic tasks. What I expected toContinueContinue reading “Why arithmetic models look dumb long after they’ve learned the rule”
Trust – a social fabric
Preface – Smart contracts and decentralization One of the fundamental ideologies of a blockchain ecosystem is “trust”, rather “trustless”(ness). With smart contracts, a block of code would have the ability to run automatically setting the terms of a “contract” as per protocol, and would immutably persisted in the blockchain. In this post, I am goingContinueContinue reading “Trust – a social fabric”
Get paid for paying attention
How many times have you been irritated by spam emails and random ad campaign in your inbox when you open your email? Almost often, right? Or think of a time when you are consumed in a piece of content on a website and you get interrupted by an ad banner selling you things that youContinueContinue reading “Get paid for paying attention”
Barter to Bitcoin – an evolution story
What is money? At hind sight, it’s a piece of paper that can buy you anything from a cup of coffee to a house. But why is money “money”? In other words, why money has to be a unit of transaction? Why does it have a value? Seriously, what’s stopping us from printing shit tonContinueContinue reading “Barter to Bitcoin – an evolution story”
The Reactive Future
Human nature has been reactive since the beginning of time. If there is there is a need to travel farther and without necessarily indulging into too much inconvenience, we invent machines (read steam engine, cars, trains and the likes). When there isn’t enough land space for new construction, we cut forests. When we needed certainContinueContinue reading “The Reactive Future”
Extract-Transform-Load fast with Golang and Postgres
The PitchSo, I have joined a travel startup recently, and since my first day here, I have been focusing on organizing the humongous data from different sources that fuel the business, using data pipelines and sharding strategies. The ProblemA part of the problem was to solve this huge data import problem. All we needed wasContinueContinue reading “Extract-Transform-Load fast with Golang and Postgres”
Simple URL Shortener
We all have come across social and professional networking sites, and have encountered posts containing minified (shortened) links that look something like bit.ly/bla or goo.gl/blah or lnkd.in/blahhh. We have got so used to these that we almost never pay attention to logic or design behind this philosophy. Alteast, I didn’t, until recently when someone askedContinueContinue reading “Simple URL Shortener”
Ignite with Mongodb
Thanks to the previous posts, we now know how to query data in Ignite. So far, we were either persisting data in memory (in which case, it would go off as soon as the node is down) or on file-system. However, we may also use the database of our choice as the persistence layer. One suchContinueContinue reading “Ignite with Mongodb”
Querying with Apache Ignite
In my last blog, I covered the basics of starting an ignite node or a cluster. We also wrote a simple Hello-World Program in Java that starts an ignite node as jar, and how we can put data into cache(cache.put) and retrieve it based on a key(cache.get) i.e a basic lookup. However, ignite is not limitedContinueContinue reading “Querying with Apache Ignite”
Wisdombox 