top of page

The Zen Interview for Data Scientists

Using Eastern philosophies (and Python Programming) to optimize your talent selection

“If the mind is empty, it is always ready for anything. In the beginner’s mind there are many possibilities, but in the expert’s mind there are few.” – Suzuki Roshi


In today’s competitive marketplace, companies are struggling to optimize their process for identifying candidates, completing competency-based interviews, and making effective decisions. Though technology is useful, the key factor in hiring talent is the interview panel. Ultimately, the ability of SMEs to develop a success profile, identify quality CVs, ask probing questions, and incorporate a scoring rubric is critical.


As data-scientists, it’s our responsibility to provide People Partners with the insights needed to make effective decisions. In addition to utilizing statistical methods, it’s important to structure the process in a differential manner. This means the panel should be diverse, the scoring rubric objective, and the questions…well, highly unique. This last part runs contrary to conventional theory. Namely, the whole purpose of the competency-based interview is to standardize the candidate questions, allowing for comparability and fairness. However, the downside of this process is that it’s easy to prepare for. As such, deciding between finalists, each of whom have polished answers and the requisite experience can be difficult. To get around this, a useful approach is to incorporate Zen teachings.


Zen and the Interview Process


Before I explain further, the purpose of this article is not to discuss Zen’s spiritual philosophy. Instead, the goal is to highlight the practical aspects of the teachings, especially how they relate to interviewing. In Zen, the discussion between the Abbot and the disciple is a core process. As part of this, it’s common for teachers to use (Koans) or parables to test the student’s understanding. The interesting part of a Koan is that it can’t be solved through logical reasoning. Instead, it requires an emotional breakthrough. Something that can’t be taught, nor accelerated. A Koan is only solved through years of practice, where the answer is discovered through skill, experience, and effort.


The greatest strength of a Koan is that it differentiates student progress. Even if two disciples meditate the same amount of time, only one may progress. If standard questions are asked, the Abbot would have little understanding how either student is learning. However, the Koan shatters the traditional approach. Through the question, the truth about the student shines through (good or bad).


Sample Koan:


“When both hands are clapped, a sound is produced; listen to the sound of one hand clapping.”


Now, let’s see how to apply this approach to a professional setting. Don’t worry, the application is straightforward. Regardless of the domain, these principles help your team drive candidate differential.


Zen Principles for Interviewers


The Buddha had disciples. Real talent always has a group of followers.

A great question to ask, “In twenty seconds, list five names we can call for references.” A great follow-up question, “In twenty seconds, list five more names.” Keep iterating this process until the candidate runs out of names. As your final question, ask if the candidate listed their direct and skip-level boss. What is the point of this exercise? Well, exceptional leaders and SMEs have fans. If a prospective candidate struggles to provide names, that says something. Likewise, if they choose not to list their boss or a skip-level leader, that says something. However, if they smile and provide between 6-10 names within the allotted time, you can be assured they are worth evaluating in more detail.


** For this question, speed is the key. Only the names and keep iterating.


Good posture requires decades of practice, but anyone can buy a meditation rug.

During your competency-based interviews, make sure the questions cannot be answered by purchasing a mediation rug. Too often, panels ask the standard question, “Describe a time when you led a project with diverse stakeholders and tight deadlines…” Ask yourself, does this question challenge the candidate? Remember, a goal of interviewing is to establish candidate differentials. In other words, how does X candidate compare to Y. To achieve this, you must craft your questions in a way that a meditation rug won’t suffice. In Zen, developing the right back posture takes years of practice. Applying this to Human Resources, your questions should challenge even highly qualified candidates. In this way, consider incorporating scenarios and trade-offs into your dialogue. Ask questions that relate to the candidate’s background, but apply follow-up that challenges their business acumen, CSR, and diversity skills.


The wisdom of knowing, the wisdom of not-knowing, and the willingness to admit it.

Great candidates almost always have exceptional self-awareness. Knowing your strengths and weaknesses, alongside those of your team is a mark of leadership. Unfortunately, most interview panels do not properly challenge a candidate’s self-awareness. To fix this, try probing job transitions and listed achievements a bit more, using a contrarian approach.


In other words, design your question using, “why this, instead of that…”

The best candidates will briefly pause, then provide an answer that shows profound self-awareness. This should include why he/she made the decision, the downside of doing so, and whether the benefit of retrospect would have caused changes. Remember, a candidate that practiced their answers will struggle in this section. They will answer in the prepared manner, while failing to handle the follow-up questions. This is a great approach to utilize during the final round.


Mindfulness emphasizes the moment. Be cautious of the post-interview calibration.

Be cautious of post-interview calibration sessions. Ultimately, the feeling “in-the-moment” is important. Though debrief sessions are useful, they should not be used to generate candidate scores. In the end, each stakeholder should score participants based on how he/she felt during the interview. This approach ensures that panelists engage in a mindful process, instead of relying on group think.


Compassion inspires trust. An authentic conversation requires significant trust.

In Zen, compassion is a core principle. Engaging in mindful practice, finding ways to offer kindness and support, that is a sign of a great employee. In the interview process, it’s critical for panelists to establish trust prior to engaging in challenging dialogue. To this point, the process for establishing trust requires deliberate actions. This means offering reassurances before and after challenging questions. Remember the reference test, letting a candidate know that 7 is a great number will generate some comfort. Overall, the key is to use visual and verbal ques to let the candidate know they are safe and respected.


Incorporating Data Science


Avoid excel reports; automate worksheet functions using Pandas

Often, teams utilize a scoring rubric to assess competency-based interviews. Though some organizations are transitioning to online platforms, a majority still rely on Excel to manually score and consolidate their interview results. In many respects, Excel is a powerful tool because most HR Professionals maintain a competency in that platform. As such, data scientists should consider creating a back-end solution to automate the scoring process, while allowing excel to remain as the front-end interface. This is where Python’s Pandas becomes useful. Specifically, the process for linking workbooks is relatively simple. You create data-frames for each workbook, use the (.loc and .isin) to filter columns, and consolidate the combined result into a new workbook.


Web-scraping identifies candidate sentiment, alongside potential interview questions

The internet is the world’s largest database. As such, smart organizations look for ways to incorporate external data-points into their interview process. To this point, Python can be used to conduct candidate sentiment analysis scraping sites such as LinkedIn, google scholar, professional societies, and YouTube. For example, the Beautiful Soup and Selenium packages allow programmers to scrap HTML texts from websites. Depending on the domain, data scientists can extract a candidate’s public comments and compare those made by executives and industry influencers. Think of it this way, executive speeches have an underlying sentiment that reflects the company’s culture. The goal is to determine if a candidate’s public comments positively align with that sentiment.


** Check out this PIP install package

For Python, consider the SpaCy package to parse the word and sentence structure. Though the package has structural limitations, it allows organizations to build scalable models for candidate sentiment.


Utilize Principal Component Analysis (PCA) to understand the interview scoring:

Once you’ve automated the scoring rubric, the next step is to distill interview results into key themes. Though most organizations complete this step subjectively, a data scientist can add value by incorporating Principal Component Analysis (PCA). Using Pandas and the scikit-learn tools in Python, organizations can identify the principal components associated with scoring rubrics. There are many articles that highlight the process for completing this analysis. However, the key is to understand the underlying strategy. Overall, scoring rubrics should help identify themes across a broad range of candidates. This requires correlating the candidate’s skills, work experience and academic background with their answers, while comparing the results against a normative set of High Performers.


Sample questions that PCA helps answer: do engineering or finance backgrounds answer risk management questions better; what career experiences correlate with a successful competency-based interview; and what interview themes can be incorporated into the CV screening process? As data-points are collected, PCA can help interview committees understand which backgrounds produce the most favorable answers. From this, you can establish a normative scoring range to better understand how individual performance compares relative to peers and/or industry domain.


If you found this topic interesting, please subscribe to my YouTube Channel (David Swanagon) and reach out to me if you’d like more information on people analytics.


23 views0 comments
bottom of page