An experiment in memorization, grokking, and misleading loss curves This post documents an experiment that didn’t go the way I expected.What started as a simple attempt to observe memorization and grokking in arithmetic models turned into a deeper lesson about how misleading loss curves can be — especially for algorithmic tasks. What I expected toContinueContinue reading “Why arithmetic models look dumb long after they’ve learned the rule”
Tag Archives: technology
Extract-Transform-Load fast with Golang and Postgres
The PitchSo, I have joined a travel startup recently, and since my first day here, I have been focusing on organizing the humongous data from different sources that fuel the business, using data pipelines and sharding strategies. The ProblemA part of the problem was to solve this huge data import problem. All we needed wasContinueContinue reading “Extract-Transform-Load fast with Golang and Postgres”
Simple URL Shortener
We all have come across social and professional networking sites, and have encountered posts containing minified (shortened) links that look something like bit.ly/bla or goo.gl/blah or lnkd.in/blahhh. We have got so used to these that we almost never pay attention to logic or design behind this philosophy. Alteast, I didn’t, until recently when someone askedContinueContinue reading “Simple URL Shortener”
Ignite with Mongodb
Thanks to the previous posts, we now know how to query data in Ignite. So far, we were either persisting data in memory (in which case, it would go off as soon as the node is down) or on file-system. However, we may also use the database of our choice as the persistence layer. One suchContinueContinue reading “Ignite with Mongodb”
Querying with Apache Ignite
In my last blog, I covered the basics of starting an ignite node or a cluster. We also wrote a simple Hello-World Program in Java that starts an ignite node as jar, and how we can put data into cache(cache.put) and retrieve it based on a key(cache.get) i.e a basic lookup. However, ignite is not limitedContinueContinue reading “Querying with Apache Ignite”
Apache Ignite: An Introduction
Apache Ignite was born in a company named GridGain, and was later contributed to Apache as community project. The official documentation reads: Ignite is an in-memory computing platform that is durable, strongly consistent, and highly available with powerful SQL, key-value and processing APIs We can think of ignite as an in-memory data grid, though theyContinueContinue reading “Apache Ignite: An Introduction”
Monitoring a Filesystem – Part 3
Monitoring a filesystem, as it turns out, is not that simple afterall. Given the plethora of considerations, like infinite data volume, less performance overhead, fast, light-weight etc. that we have already seen in the preceding posts of this series. By this time it was very clear that scanning through directories at regular intervals would not scale whenContinueContinue reading “Monitoring a Filesystem – Part 3”
Monitoring a Filesystem – Part 2
So, in my previous post of this series, I shared my experience where we were continuously polling the filesystem for any updates. We have also seen what were the disadvantages of such systems. We started exploring other tools, that were similar in nature and were thus, discarded. One such option that we tried was the ApacheContinueContinue reading “Monitoring a Filesystem – Part 2”
Monitoring a Filesystem – Part 1
So, one fine day at work, I got bumped into this simple task of detecting any changes on a specific folder on the filesystem, and sync any new/modified files to an S3 bucket in AWS. In common terms, a file watcher. Not a big deal at all… Here was my approach.I wrote a simple shellContinueContinue reading “Monitoring a Filesystem – Part 1”
3D image spin in browser
Preface In my earlier blog, I have explained how to create an interactive image (turn-by-turn) as you drag L-R or R-L. In case, you have not gone through it, I would highly recommend you start with it, as I have covered some of the basic principles there.Let’s now try to build a little more complex one.ContinueContinue reading “3D image spin in browser”
Wisdombox 