System Design Case Study #2: How GitHub Indexes Code For Blazing Fast Search & Retrieval
GitHub coded their search engine from scratch in Rust called Project Blackbird because existing off-the-shelf solutions didn’t fit their requirements at the scale they were operating. The engine supports features like searching with identifiers, punctuations, substrings, regular expressions, wildcards, etc., which are specific to code...
Database Architecture – Part 2 – NoSQL DB Architecture with ScyllaDB (Shard Per Core Design)
In the first post of database architecture, I discussed the internal architecture of databases/database management systems, taking a peek into the architectures of MySQL and CockroachDB databases. In another article, a continuation of that article, I discussed how modern cloud servers run parallel compute to...
Parallel Processing: How Modern Cloud Servers Leverage Different System Architectures to Optimize Parallel Compute
Modern cloud servers leverage several system architectures to process data parallelly, which increases throughput, minimizes latency and optimizes resource consumption. In this article, I’ll discuss those system architectures and we’ll get an insight into how distributed services leverage these architectures to scale. With that being said....
Database Architecture – A Deep Dive – Part 1
This article is a deep dive into the internal architecture of databases/DBMS (Database Management Systems). I’ll begin with a standard architecture relational databases have; will then take a peek into the architectures of a couple of real-world SQL databases and then, in the subsequent articles,...
Wide-column Database, Column Databases – A Deep Dive
Wide-column, column-oriented and column-family databases belong to the NoSQL family of databases built to store and query massive amounts of data, aka BigData. They are highly available and scalable, built to work in a distributed environment. Most of the wide-column databases do not support joins...
YouTube database – How does it store so many videos without running out of storage space?
YouTube is one of the most popular websites on the planet. As of May 2019, more than 500 hours of video content is uploaded to the platform every single minute. With over 2 billion users, the video-sharing platform is generating billions of views with over...
Distributed Data Processing 101 – A Deep Dive
This write-up is an insight into distributed data processing. It addresses the frequently asked questions on it such as: What is distributed data processing? How different is it in comparison to centralized data processing? What are the pros and cons? What are the various approaches...
What Database Does Twitter Use? – A Deep Dive
This write-up is an insight into the persistence layer of Twitter. It covers topics such as what database does it use? Is it SQL, NoSQL, or a polyglot persistent system? How does it store hundreds of millions of tweets sent every single day? and more....
Instagram architecture & database – How does it store & search billions of images
Instagram is the most popular photo-oriented social network on the planet today. With over a billion users, it has become the first choice for businesses to run their marketing campaigns on. This write-up is a deep dive into its platform architecture and addresses questions like...
Facebook database [Updated] – A thorough insight into the databases used @Facebook
This write-up is a comprehensive insight into the persistence layer of Facebook and the first of the distributed systems write-ups on this blog. It takes a deep dive into questions like what database does Facebook use? Is it a NoSQL or a SQL DB? Is it...
Follow Me On Social Media