I kept hearing about the term wire protocol especially Postgres wire protocol in the recent days (Looking at you cockroachdb, yugabytedb - in a good way) but never really quite understood it. Decided to implement something simple in Go to understand it better. As always, if you find anything wrong or I misunderstood please correct me.

In simple terms,

“wire” - something over network generally (but PG also supports over domain sockets)

Citus: Distributed PostgreSQL for Data-Intensive Applications paper can be downloaded here.

Recently, our team got a request to provide a solution to shard Postgres. One of the solutions that we discussed was Citus. I have heard about the product and seen their blogs related to Postgres in the past but never used it. I thought it would be fun to read about its internal workings.

If you find something wrong on the notes, please send a pull request. Before digging their white paper, let’s take a step back and ask what is sharding and why do we need sharding?

TL;DR:

Connect with people whom you admire. Things can take longer so stay in touch. At the right moment, don’t hesitate to ask.

I recently became an Engineering Manager after being an Individual Contributor for almost 8 years. If you are interested to know more about the reasoning behind the change. As like many other jobs in Tech, there is no formal training for an Engineering Manager. Reminds me of my dad’s saying “How do you teach one to swim? Just throw on the water !”. I felt it would be great to have a mentor with whom I can chat/discuss regularly around management, leadership, politics, team building etc etc…

Capturing thoughts from https://twitter.com/viggy28/status/1530800893842444289

1. When you are doing major DML changes, other than locks one more thing to keep in mind is replication lag. Especially if you use your replicas in hot standby mode.

2. When you need to delete most of the records in a massive table, its better to create a new table and just copy the records that you need to preserve. When you need to delete all the records in a massive table, just truncate it instead of deleting them.

Before I forget let me write this down here. $$ log_stament - none - all

log_min_duration_statment - millisecond value

When you set log_statement=none and log_min_duration_statement=1 then any statement which takes longer than 1 millisecond will be logged.

When you set log_statement=all and log_min_duration_statement=1 then all statements are logged; however it only shows duration on statements longer than 1 millisecond.

Last week, I got promoted to Engineering Manager (EM) of the Database team at Cloudflare. I joined the team almost 3 years ago. A senior team member and myself were hired to form a then dedicated database team. I have been enjoying it and the team is amazing to work with. Great people. Opportunities came to either advance on the Individual Contributor (IC) or the leadership side. I picked the latter. My thoughts on why did I transition to leadership.