Executive Summary

• Engineering quality is paramount. Focus on clean and clear code. Code is the primary medium of communication for any engineer. Write beautiful code and hold people to high standards.
• Adding head count has vast hidden costs and often brings down output. The reason for this is the exponential increase in communication required to align everyone to common goals.
• Similarly, pods cannot scale unless they can own small, independent components.
• Monoliths are normal and good when the company is small. Refactoring the monolithic model into different components allows us to scale pods and org.
• Carve monoliths only when it hits critical mass. Then, identify the pain and pull it out into it's own pod.
• "Relationship as a service" or "Please get this done for me over and above other stuff on your plate"
  • "Traffic congestion" (A team which is too busy) and structural failure can lead to the impression that the system is bureaucratic and political.
• Criteria when hiring Engineers
  • Computer science (Can computers understand your code?)
    • Ability to grasp large complex systems
    • Understanding of implications of design choices on that system.
  • Software Engineering (Can humans understand your code?)
    • How well do you communicate through your code? This is the metric that enables building good systems.
    • The hygiene you show in code is the hygiene you will enforce on others.
  • Good behavioral traits (Can you grow?)
    • Curious
    • Wants to learn
    • Can accept feedback
    • "Strong opinions, Weakly held."
    • "Pride without attachment."
• Find the full video here!
• The rest of this post is detailed notes on the video

The video

Introductions

• CEO: Nadiem Makarim
• CTO: Niranjan Paranjape
• Tech Recruitment Head: Sidu Ponnappa
• C42 Engineering: Sequoia SWAT Team for solving tech problems
  • They were the rare "other Clojure shop" in India in 2010.
• Common consulting milestone for them was "the Client just tried to poach me". When Gojek tried this with Niranjan, they sold the entire company to Gojek.
• Today, Gojek does $9 BILLION in transactions with 300 engineers.

How do you get so much done with so few engineers

• Adding headcount does not increase output.
• With every engineer you add, you actually slow down work (exponentially)
• A good finance person will challenge you on every item of spend because they understand that that's how they lose money.
• Similarly, a good engineering leader will challenge you on every addition of headcount. The hidden cost is enormous and impacts ability to deliver quickly and at quality.

Collaboration, Communication and Team Size

• Every person you add exponentially increases the combinatorics of communication you need in order to agree on what exactly it is that you are building.
• Code is communication between humans. Keep it clean and simple.
• Upto 4 people, productivity grows linearly with addition of people. Beyond that it starts plateauing and dipping.
  • The person who observed this started slicing and dicing teams so that at any point in time a subsystem was being worked on by no more than 4 people.
  • By Conway's Law, he was forcing system architecture to be set up in a certain way.
• We chose to map out our systems based on what a 5 person team could do.

What about "number of pods"? Will infinite teams of 5 scale infinitely?

• As an organization, we find similar diminishing returns with growing the number of pods.
• The company's ability to add pods is also limited by ability of the pods to understand the underlying system.
• If your pods are independent (have to collaborate only once in 2 to 4 weeks), then you are in a manageable situation.
• Multiple pods working on the same problem leads to pushing for local goals (evolving the system to maximize your objectives based on limited understanding)

Complexity of organisations

• "Don't slow me down" from CEO to engineering
  • Philosopher poet engineers
  • Engineering had to educate the CEO on the bad effects of not on-boarding new comers fast enough
• Adding Kafka to our stack allowed us to scale
  • Helped us decouple communication
  • Tap into our data, you don't need to ask me anymore.
• Refactoring the monolithic model into different components allowed us to put more people on the independent teams.

Monolith to SOA

• We first pulled out the allocation system
  • matching driver to customer
• Next we pulled out Go Pay
  • This allowed us to staff the team of Go Pay without them having to understand the entire monolith.
• Identify a problem, pull it out, establish a pod around it.
• CTO was pushing CEO:
  • Before we move into this new thing, we need to get rid of the monolith first.
  • The choice they made was never stall growth. This meant hanging on to the monolith for longer than they would have liked, and refactoring it in bits and pieces rather than in one single effort.
• When starting out, it makes complete sense to have a monolith.
• When you carve a system out of a monolith, eventually that will become a monolith.
  • Now you need to split it again.
  • Identify when the monolith reaches critical mass, and then refactor your team and code-base in lockstep.

Relationship as a service AKA How politics enters your system

• I need something from your system, so I'm going to lean on our relationship to unblock me and ignore your own priorities.
• "Hey I really need this done fast, because X".
• This will grow until you are completely busy trying to unblock other people and are unable to focus on any of your own priorities.
• Now you have a bureaucracy. The person with the back-channel to the blocked system gets to jump the queue. This inherently does not scale because people without the relationship back-channel get more and more frustrated.
• "Our organization is political." I don't have privileged access, therefore that other guy must be master politician.
• The actual underlying problem is traffic congestion.
  • If you are trying to do your best, you are going to use every resource at your disposal, including your relationships, to get work done
• Even in the most well-intentioned and non-egotistical organizations, structural failure leads to the perception of favouritism

Is there such a thing as a 10x engineer?

• As system complexity grows, for you to meaningfully contribute, you need to know the system!
• Criteria to select engineers
  • Computer science (can computers understand your code?)
    • Ability to grasp large complex systems
    • Understanding of implications of design choices on that system.
  • Software Engineering (can humans understand your code?)
    • How well do you communicate through your code?
    • Communication problems are the hard blockers on building good systems.
    • Well written code provides contextual logic and clarity.
  • Good behavioral traits:
    • curious
    • wants to learn
    • reads books
    • can accept feedback
    • ^ This is the kind of person who can grow.

Good practices in coding

• Every code-base comes with a glossary
• Terms have specific meaning, and naming is critical in the dividends it pays off in the future.
• Every engineer at every level has to take a written test
• We are not looking at whether you can solve the complex problem. We are judging how you express yourself. You are a programmer, and your code should speak well for you.
• Hygiene becomes more and more important as a programmer becomes more and more senior.
  • The hygiene you show in code is the hygiene you will enforce on others.
• High standards are important.

Differences between good engineers and 10x engineers

• 10x outcome (not output)
• As you grow your organization, individual outcome is less important. Team outcome is what you need to measure.
• Engineering is completely a team sport.
• You can have deeply capable and extremely competent individual contributors who like to work alone. As long as their code fits beautifully with other systems, they are multiplying everyone's capacity.

How do you spot red flags in a new hire?

• Every good engineer is highly opinionated. A good engineer has strong opinions weakly held.
  • You should be able to change your opinion on being presented facts.
• If you are unable to deal with criticism of your code, that's a smell.
  • We ask junior people to interview senior engineers. This shows the grace they have when dealing with others.
• We routinely wind up in situations where you have young people owning complex systems. Senior engineers need to be hired to now deal with these systems, and they will be on-boarded by juniors. They should be able to deal with this fact.
• CTO: "When I lay out a design, I expect people to give me rational criticism so that I can learn from them."
• If you had to hire someone with deep experience vs great behavioral foundations, what would you choose?
  • It Depends!
  • Sometimes, you need to bring deep experience to the table to build something out and learn from the guru.
  • Behavioral traits means that someone is teachable. Outside the Valley, this is extremely important because of the lack of access to experienced engineering.

Onboarding / Engineering bootcamp

• Structured decision making
• Basic coding hygiene
• Collaboration
  • How do you engage with someone else to meaningfully decide reasonably quickly what something in code should be.
• Heavy focus on coding during the bootcamp.
• We bring in coaches from every team and teach. (regardless of hire). This helps us build empathy, which is critical for future collaboration.
• Pride without attachment.
  • Pride allows you to focus on creating beautiful code.
  • Attachment means you will stand in the way of the evolution of your beautiful baby. You need to let it go and let others change it.
  • If you look back at code you wrote 6 months back and don't notice how bad you were, you aren't growing.
• Repeat the same problem every Monday throughout the whole bootcamp. Ask them to review every week and see if they find that feeling of progress.

Gojek culture

• Even the most high-performing team has to deal with intense time pressure.
• What makes or breaks teams under pressure?
  • Show them the impact that they are making.
  • You don't need to push engineers, you need to give them the rush! External force will not drive the maker as much as the impact of work will.
  • The value you create is not proportional to the time you put in, it's proportional to the state of your mind in that time you put in.
    • If you are burned out, or your team mate is burned out then there is going to be a huge negative spiral.
  • To harness creativity, you need balance and psychological safety. The hours that you put in, you need to be in a state of flow. This requires you to take short breaks after days of intense activity.
• You can have a 10x engineer amidst you, but you may never know it because you did not give them sufficient leeway to unleash their art.

Cross-posted from my interview published on Helpshift's Medium blog

• Name: Vedang Manerikar
• Current Role: Principal Engineering Manager, Backend
• How long you have been with Helpshift?: 7 years

What does your day-to-day at Helpshift look like?

My day-to-day is unblocking people and getting things done!

I review architecture, read code, enable customer service, help product guys understand the engineering pros and cons of their product decisions and prioritize work for maximum productivity. I work with an amazing team. They make all of this possible and achievable.

That means I also get to hang out with my best friends, catch up on their day-to-day, laugh at inside jokes that have been running for years and overall have a blast. Seven years at a place lets you build deep relationships and puts a smile on your face when you think about work!

What has been your favorite off-site team oriented event, and why?

When we were cash-strapped and raising funds, a team off-site was the running joke of the company. "After Series A!", we would raise our glasses and say. As time went on without funding in sight, every request turned into "After Series A!".

So when we did raise our first round, we treated ourselves to a long weekend at a spectacular villa in Goa. It was a heady celebration and the world was our oyster. It remains my favorite off-site because the funding felt like validation.

What role(s) did you start in at Helpshift and how have you graduated/migrated into different roles?

I came to Helpshift with a strong systems background: writing kernel modules and working on storage replication. I started writing Python at Helpshift and then graduated to Clojure (in 2011, when almost no one else was writing Clojure). I've since written and helped design large chunks of the Helpshift architecture, and have learnt interesting things about distributed systems through trial-by-fire. Today, I make an impact by making sure my team has clear, unambiguous goals and all the resources to achieve these goals.

Favourite thing about working for an SF tech startup:

I have a lot of "favorite things" about working at Helpshift (We keep our workplace fun!). A small example of this is that on any given day someone or the other will bring home-made snacks and goodies for the others to enjoy.

What advice would you have for prospective Helpshift candidates?

We're still a small enough company that everyone knows everyone else by name. If you're the kind of person who wants to get in on the ground floor and grow with a company, we're perfect for you. Come with a no-nonsense, no-bullshit attitude. Be a kind person.

What is your proudest moment at Helpshift?

My colleagues and I have taken this company from serving 0 requests per second to serving 50,000 requests per second. We're installed in over 2 Billion devices in the world. That means that there's a 50% chance that you (reader) have us installed in one of the apps on your phone. That sentence is a source of great pride for me.

How would you describe the company culture and what do you like most about it?

As a company, we've always looked for attitude over smarts. We want people who are good listeners and good communicators. We want people who are passionate and hold strong opinions. We want people who care about quality. This is the culture that we've always striven to maintain at Helpshift.

Superpower (real or fake): Obscure Emacs shortcuts.

Give a man Emacs and he'll get some programming done. Teach a man how to tweak it and play with it and he'll happily shave yaks for the rest of his life.

Slightly addicted to: Tea.

Walking out of the office to have road-side tea in the evening is an old Helpshift tradition. I'm always on the lookout for great Masala Chai, and if you come to my home I'll whip up a fantastic cup of tea for us to chat over!

Always on the search for the perfect: new book to read.

Reading is one of my favorite ways to unwind. I read anything that I can get my hands on: fantasies, travelogues, thrillers, pop psychology, autobiographies, technical books, comics, you name it!

If you could interview one person, dead or alive, who would it be and why?

Isaac Newton or Albert Einstein or Alan Turing! (Sorry I cheated.) These were phenomenal people with world-changing ideas. How did they get them? How did they remain productive all their life? What did "work" mean for them? What was their work day like? What did they do to unwind? What did they do for leisure? Who did they look up to? I'd love to know these answers!