Edge Web Server Testing at Swiftype
28 Apr2018

This article has been originally posted on Swiftype Engineering blog.


For any modern technology company, a comprehensive application test suite is an absolute necessity. Automated testing suites allow developers to move faster while avoiding any loss of code quality or system stability. Software development has seen great benefit come from the adoption of automated testing frameworks and methodologies, however, the culture of automated testing has neglected one key area of modern web application serving stack: web application edge routing and multiplexing rulesets.

From modern load balancer appliances that allow for TCL based rule sets; local or remotely hosted varnish VCL rules; or in the power and flexibility that Nginx and OpenResty make available through LUA, edge routing rulesets have become a vital part of application serving controls.

Over the past decade or so, it has become possible to incorporate more and more logic into edge web server infrastructures. Almost every modern web server has support for scripting, enabling developers to make their edge servers smarter than ever before. Unfortunately, the application logic configured within web servers is often much harder to test than that hosted directly in application code, and thus too often software teams resort to manual testing, or worse, customers as testers, by shipping their changes to production without edge routing testing having been performed.

In this post, I would like to explain the approach Swiftype has taken to ensure that our test suites account for our use of complex edge web server logic
to manage our production traffic flow, and thus that we can confidently deploy changes to our application infrastructure with little or no risk.

Read the rest of this entry


Compliance-Driven Development or the Story Behind Swiftype’s SOC2 Certification
18 Jan2018

This article has been originally posted on Swiftype Engineering blog.


Based on my experience, just a decade ago not many people within the Silicon Valley startup community considered compliance an important stepping stone in a company’s development roadmap. And when it came to compliance for startups, it was nearly synonymous with PCI/DSS — mandatory certification used by the credit card industry. Over the last few years though, the rise in the number of startups working with large amounts of private and confidential data (fintech, healthcare, etc) and subsequently the rise in the magnitude of data breaches, led our industry to accept the idea that compliance and certifications are not just for the “big guys”. Nowadays, even very small companies are pressed to go through formal certifications if they want people to trust them with private or confidential data.

That is exactly what happened to Swiftype at the beginning of 2017. While preparing for a public release of our latest product (Swiftype Enterprise Search), we understood that it was going to involve a lot of confidential information and we would need to be able to assure our customers of our capabilities to protect their data. In addition to the marketing aspect, there was a security angle to the problem as well: we were looking for a standard framework that could be used by our small team to ensure the safety of customer data, guiding us through the process. Based on those considerations, we decided to go through a formal SOC 2 certification. In this article, I will describe our journey towards the certification and our findings along the way.

 

Read the rest of this entry


My Favourite Books in 2017
2 Jan2018

Following the very ambitious and successful 2016 challenge, I have decided to keep the goal at the same level of 36 books for 2017 to prove to myself that it is sustainable and wasn’t a one-off success. Surprising myself, I have crushed the goal and finished 39 books this year. Below is summary of the best of those books.

Business, Management and Leadership

After changing my job at the beginning of 2017 and returning to Swiftype to focus on Technical Operations team leadership, I continued working on improving my skills in this area and read a number of truly awesome books:

  • The Effective Executive: The Definitive Guide to Getting the Right Things Done” by Peter F. Drucker — this classic has immediately become one of my favourite leadership books of all time. There are many useful lessons I learned from it (like the notion that all knowledge workers should consider themselves executives in some sense), but the most powerful was the part on executive time management.
  • Hatching Twitter: A True Story of Money, Power, Friendship, and Betrayal” by Nick Bilton — A truly horrifying “Game of Thrones”-like story behind the early years of Twitter. I didn’t think shit like that actually happened in real life… I guess the book made me grow up a little and realize, that simply doing your best to push your company forward is not always enough. I’d highly recommend this book to anybody working in a fast growing company or thinking about starting a VC-backed business.
  • Shoe Dog: A Memoir by the Creator of NIKE” by Phil Knight — a great story of a great company built by regular people striving for quality results. Heavily reinforces the notion that to be an entrepreneur you need to be a bit crazy and slightly masochistic. Overall, a very fascinating tale of a multi-decade development of a company — a strong contrast with all the modern stories about internet businesses. A must read for people thinking about starting a business.

Health, Medicine and Mortality

I have always been fascinated by the history of medicine, medical stories and the inner workings of the modern medical system. Unfortunately, this year I’ve had to interact with it a lot and that made me seriously consider the fact of our mortality. This has led me upon a quest to learn more about the topics of medicine, mortality and philosophy.

  • When Breath Becomes Air” by Paul Kalanithi — Fantastic memoir! Terrifying, depressing, beautifully described story of a young neurosurgeon, his cancer diagnosis, his battle with the horrible disease and up to the very end of his life. I found the story of Paul very relatable and just like with Atul Gawande’s book I’ve read last year, it brought forth very important questions on how should we deal with our own mortality. Paul gave us a great example of one of the options for how we may choose to spend our last days — the same way we may want to spend our lives: “You can’t reach perfection, but you can believe in an asymptote toward which you are ceaselessly striving”.
  • The Emperor of All Maladies” by Siddhartha Mukherjee — probably the best book on cancer out there (based on my limited research). The author takes us on a long, very interesting and terrifying trip through the dark ages of human war against cancer and explains why after so much time we are still only starting to understand how to deal with it and there is still a long road ahead. Highly recommended to anybody interested in the history of medicine or wants to understand more about the reason behind a malady that kills more than 8 million people each year.
  • Complications: A Surgeon’s Notes on an Imperfect Science” by Atul Gawande — once again, one of my favourite authors manages to explain a hard problem of complications in healthcare and give us a sobering look at the limits and fallibilities of modern medicine.
  • Bonus: “On The Shortness Of Life” by Seneca — It is amazing how something written 2000 years ago can have such profound relevance today. I found this short book really inspiring and it has led me to start my road to adapting some of Stoic techniques including mindfulness and meditation.

Miscellaneous

Few more books I found very interesting:

  • Born a Crime: Stories From a South African Childhood” by Trevor Noah — Listened to this book on Audible and absolutely loved it! Hearing Noah’s voice describing his crazy childhood in South Africa mixing fun and absolutely horrifying details of his life there and the struggles he had to endure being a coloured kid under and right after Apartheid.
    Even though it was never as scary as what Noah is describing in his book, I have found in his stories a lot of things I could relate to based on my childhood in late USSR and then in 1990s Ukraine which was going through an economic meltdown with all of the usual attributes like crime and crazy unemployment.
  • I Can’t Make This Up: Life Lessons” by Kevin Hart — I have never been a particular fan of Kevin Hart. Not that I disliked him, just didn’t really follow his career. This book (I absolutely recommend the audiobook version!) ended up being one of the biggest literary surprises ever for me: it is the funniest inspirational read and the most inspiring comic memoir I’ve ever read (or, in this case, listened to). Kevin’s dedication to his craft, his work ethic and perseverance are truly inspiring and his success is absolutely well-earned.
  • Kingpin: How One Hacker Took Over the Billion-Dollar Cybercrime Underground” by Kevin Poulsen — Terrifying read… I’ve never realized how close the early years of my career as a systems administrator and developer took me to the crazy world of underground computer crime that was unfolding around us.
    I’ve spent a few weeks week wondering if doing what Max and other people in this story did is the result of an innate personality trait or just a set of coincidences, a bad hand the life deals a computer specialist, turning them into a criminal. For many people working in this industry, it is always about the craft, the challenge of building systems (just like the bind hack was for Max) and I am not sure there is a point in one’s career when you make a conscious decision to become a criminal. Unfortunately, even after finishing the book I don’t have an answer to this question.
    The book is a fascinating primer on the effects of bad and the need for good security in today’s computerized society and I’d highly recommend it to everybody working with computers on a daily basis.
  • Modern Romance” by Aziz Ansari — very interesting insight into the crazy modern world of dating and romance. Made me really appreciate the fact that I have already found the love of my life and hope will never need to participate in the technology-driven culture today’s singles have to deal with. Really recommend listening to the audiobook, Aziz is very funny even when he’s talking about a serious topic like this.
  • The Year of Living Danishly: My Twelve Months Unearthing the Secrets of the World’s Happiest Country” by Helen Russell — Really liked this book. It offers a glimpse into a society surprisingly different from what many modern North Americans would consider normal. Reading about all kinds of Danish customs, I would think back to the times I grew up in USSR and realize, that modern Danish life is very close to what was promised by the party back then. The only difference — they’ve managed to make it work long term.
    Even though not many of us could or want to relocate to Denmark or to affect our government policies, there is a lot in this book that many of us could apply in our lives: trusting people more, striving for a better work-life balance, exercising more, surrounding ourselves with beautiful things, etc.

I hope you enjoyed this overview of the best books I’ve read in 2017. Let me know you liked it!


My Favourite Books in 2016
6 Jan2017

I’ve planned to read 36 books in 2016 and managed to hit that number a few hours before the NY! The best of those 36 books are listed below.

Business, Management and Leadership

Considering the new role I’ve started in January 2016 (first-time CTO of a growing startup company), my reading last year was heavily geared towards business, management and leadership topics. Here are my favourite books in this category:

  • The Hard Thing About Hard Things: Building a Business When There Are No Easy Answers” — in my opinion, a must-read book for anybody interested in starting a company or already building one. A treasure trove of great advice for startup founders on building and managing their teams.
  • Crossing the Chasm: Marketing and Selling High-Tech Products to Mainstream Customers — the author explains why so many companies, that find an initial product-market fit, subsequently fail to grow into leaders of their respective markets and often die a slow and painful death. The concept of a chasm and, especially, the idea of the whole product were very powerful for my understanding of what I felt in many companies I worked for — mainstream customers could not use your product unless they are provided with a minimum set of components and services to solve their problem. Very important read for leaders of modern SaaS companies, especially for API/platform enterprises.
  • Turn the Ship Around!: A True Story of Turning Followers into Leaders” — an inspiring story of a navy captain transforming one of the worst-performing crews in the fleet into a perfectly functioning team by pushing control down to individual team members.
  • The Score Takes Care of Itself” — inspiring story of one of the best sport team transformations and the man behind it, legendary coach Bill Walsh.

Miscellaneous

Few more books I found very interesting:

  • The Collapse of Parenting: How We Hurt Our Kids When We Treat Them Like Grown-Ups” — maybe it is just a confirmation bias, but I absolutely loved this book. The author focuses on a few serious problems in today’s parenting and the resulting decline in the achievement and psychological health of American children. He finally managed put into words something that was bothering me for 10 years since moving to Canada. Now that I became a parent and would have to raise a child in this environment, I was glad to hear that I wasn’t crazy not to agree with the approach that is being pushed on modern parents by American society.
  • Being Mortal: Medicine and What Matters in the End” — one of my favourite authors, Atul Gawande, explores the current state of end of life care in the USA, Canada, and Western Europe. Terrifying at first, the book makes you consider your own mortality and think about the choices you are bound to make eventually for yourself and, potentially, for your close family members.
  • Sapiens: A Brief History of Humankind” — a captivating overview of our history as human species: from 70,000 years ago until the 20th century: how we evolved, how we affected other species on the planet and how did we end up where we are today. A long, but very interesting read!
  • The Road To Sparta: Retracing the Ancient Battle and Epic Run that Inspired the World’s Greatest Foot Race” — fascinating story of Dean Karnazes (one of the most famous ultra-distance runners in the world) and his exploration of the legend of the Marathon. Highly recommended to anybody interested in running.
  • Catastrophic Care: How American Health Care Killed My Father — and How We Can Fix It” — very detailed overview of what is broken in US healthcare today. Even if you don’t have anything to do with US healthcare market, the book is a great collection of stories about side-effects of what initially looked like great ideas, but ended up screwing the system even further.

Fiction

I was always a huge fan of sci-fi fiction and this past year I have discovered a few real gems that ended up on my all time favourite list:

  • Remembrance of Earth’s Past (aka The Three Body Problem)” series by a Chinese author Liu Cixin — huge universe, highly-detailed and powerful characters, timeline spanning centuries — you can find all of it here. But on top of the standard components of a good space opera, there is this previously unknown to me layer of Chinese culture, language, philosophy.
    This trilogy has become an instant classic for me and is in the top-10 of my all time favourites next to Asimov’s “Foundation” and Peter F. Hamilton’s “Void”.
  • Everything from Niel Gainman! Up until this year when I got exposed to his writing, I’ve never realized how much pleasure one could get from reading prose. I’m not sure how he does it, but if he were to publish a book of obituaries or classifieds, I’d be willing to read that too — I enjoyed his English so much! Favourite books so far: “The Graveyard Book” and “The Ocean at the End of the Lane”.

I hope you enjoyed this overview of the best books I’ve read in 2016. Let me know you liked it!


DbCharmer Development: I Give Up
14 Nov2014

About 6 years ago (feels like an eternity in Rails world) working at Scribd I’ve started working on porting our codebase from some old version or Rails to a slightly newer one. That’s when I realized, that there wasn’t a ruby gem to help us manage MySQL connections for our vertically sharded databases (different models on different servers). I’ve started hacking on some code to replace whatever we were using back then, finished the first version of the migration branch and then decided to open the code for other people to use. That’s how the DbCharmer ruby gem was born.

For the next few years a lot of new functionality we needed has been added to the gem, making it more complex and immensely more powerful. I’ve enjoyed working on it, developing those features, contributing to the community. But then I left Scribd, stopped being a user of DbCharmer and the situation drastically changed. For quite some time (years) I would keep fighting to make the code work with newer and newer versions of Rails, struggling to wrap my head around more and more (sometimes useless) abstractions Rails Core team decided to throw into ActiveRecord.

Finally, in the last 2 years (while trying to make DbCharmer compatible with Rails 4.0) it has become more and more apparent, that I simply do not want to do this anymore. I do not need DbCharmer to support Rails 4.0+, while it is very clear that many users need it and constant nagging in the issues and the mailing list, asking for updates generated a lot of anxiety for me, anxiety I couldn’t do much about (the worst kind). As the result, since I simply do not see any good reasons to keep fighting this uphill battle (and developing stuff like this for ActiveRecord IS a constant battle!) I officially give up.

Read the rest of this entry