Pensieve

February 17, 2022

1-2 Oblivious Transfer

I learnt about oblivious transfer when reading up about garbled circuits. As an engineer this feels like a fascinating, almost magical protocol. And it’s not just me. Everyone I talked to about this protocol had the same reaction – disbelief. Here’s an engineer’s perspective. Objective Let’s create two fictional characters – Alice and Bob. Alice […]
November 23, 2021

Bowen Lookout and Yew Lake Loop

We had a fun Saturday, exploring live music scenes in Vancouver. The top recommendation in the group was Guilt & Co, which turned out to be an excellent pick! We stayed for three sets, and the second artist was really good. Okay, the lyrics didn’t matter since I was lost in the music they were […]
July 11, 2020

Making Sense | Is Life Actually Worth Living?

This was an interesting discussion. It was a learning opportunity for me, as I have no prior opinion on anti-natalism. David has a unique perspective where he assigns different values to creation of life and continuation of life. He believes, on average, life is overall painful and it’s cruel to bring a new life into existence. […]
December 24, 2019

Rise of Skywalker | Balance Restored?

I watched the final part of the Skywalker saga, and overall liked it. It had plenty of elements to invoke nostalgia and make me feel emotionally comfortable. However, I think the ending could have been better, but don’t we all feel so about most of the fantasy series out there? Without caring about the wider […]
July 11, 2018

Writing Into Dynamic Partitions Using Spark

Hive has this wonderful feature of partitioning — a way of dividing a table into related parts based on the values of certain columns. Using partitions, it’s easy to query a portion of data. Hive optimizes the data load operations based on the partitions. Writing data into partitions is very easy. You have two options: […]
May 24, 2017

Parse Json in Hive Using Hive JSON Serde

In an earlier post I wrote a custom UDF to read JSON into my table. Since then, I have also learnt about and used the Hive-JSON-Serde. I will use the same example as before. Now, using the Hive-JSON-Serde you can parse the above JSON record as: This is really great! I can now parse more […]
April 26, 2017

SPOJ | NICEDAY — The Day of the Competitors

Problem Contestants are evaluated in 3 competitions. We say that: A contestant A is better than B if A is ranked above B in all of the three competitions, they were evaluated in. A is an excellent contestant if no other contestant is better than A. Given the ranks of all the contestants that participated […]
April 4, 2017

Writing UDF To Parse JSON In Hive

Sometimes we need to perform data transformation in ways too complicated for SQL (even with the Custom UDF’s provided by hive). Let’s take JSON manipulation as an example. JSON is widely used to store and transfer data. Hive comes with a built-in json_tuple() function that can extract values for multiple keys at once. But if […]
April 4, 2017

Project Euler | The Millionth Lexicographic Permutation Of The Digits

The 24th problem of Project Euler wanted the one-millionth lexicographic permutation of the digits 0, 1, 2, 3, 4, 5, 6, 7, 8 and 9. If all of the permutations are listed numerically or alphabetically, we call it lexicographic order. The lexicographic permutations of 0, 1 and 2 are: This was yet another problem which I solved […]
April 4, 2017

Project Euler | Maximum Sum Traversing Top To Bottom In A Triangle

The 18th and the 67th problems in Project Euler are one and the same. The only difference is in the input test case. The problem 18 has a smaller input and 67 has a large input. For the explanation purpose, I’ll consider problem 18. The code without any modification can be extended to 67th problem. Given a triangle of […]