This article is sponsored by Pieces and was written, edited, and published in alignment with our Emerj sponsored content guidelines. Learn more about our thought leadership and content creation services on our Emerj Media Services page.
The synergy between software development and data science teams enhances decision-making, improves efficiency, fosters innovation, and aligns efforts toward shared business objectives.
Recent research from institutions like the University of Rio de Janeiro highlights the collaboration dynamics between software engineers and analytics leaders in machine learning projects, revealing challenges such as differing technical expertise and unclear role definitions. The study suggests cultivating a collaborative culture and enhancing communication as potential solutions to improve teamwork.
Another paper published by research professionals from Microsoft and a professor from the University of California indicates that integrating data science into software development can enhance productivity and quality by leveraging user behavior data for better strategic planning. Data scientists increasingly use diverse tools like SQL, R, and Python but face challenges related to compatibility and data quality.
Emerj Senior Editor Matthew DeMello recently sat down with Yigal Edery, SVP of Product and Strategy at Sisense, and Tsavo Knott, Technical Co-founder & CEO of Pieces, to talk about the integration of AI tools in development workflows to enhance coding efficiency, manage errors effectively, and foster collaboration among data science and software teams.
Pieces is an AI-driven software company that builds a platform to enable software developers to work more efficiently. Sisense is an API-first analytics platform that modernizes businesses by seamlessly integrating in-context analytics within applications, leading to action-oriented insights that enhance decision intelligence. Together, they shed light on both the limitations and possibilities for collaboration between enterprise professionals from both disciplines.
The following analysis examines two critical insights from their conversation:
- Fostering a collaborative data culture: Encouraging data walk-through sessions among product managers, data engineers, and data scientists to analyze datasets and generate new insights for continuous improvement collectively.
- Implementing AI-driven error management systems: Collecting and analyzing error data through AI-driven systems to streamline resolution processes and enable developers to tackle similar challenges efficiently.
Guest: Yigal Edery, SVP of Product and Strategy, Sisense
Expertise: Analytics, Cloud Infrastructure, Networking, Security
Brief Recognition: Yigal Edery is the SVP of Product and Strategy at Sisense. Previously, he worked with NVIDIA, Kameleon, and Microsoft as senior director of products, VP of products, and group program manager. He earned his Bachelor’s in computer science and economics from Bar-Ilan University.
Guest: Tsavo Knott, Technical Co-founder & CEO of Pieces
Expertise: Coding, Software Development, Entrepreneurship, Interactive Media, Computer Science
Brief Recognition: Tsavo graduated from Miami University in 2018 with a Bachelor’s Degree in Game and Interactive Media Design as well as Computer Science. Before co-founding Pieces in 2020, he served as vice president and co-founder of Accent.ai, a language learning platform.
Fostering a Collaborative Data Culture
Yigal opens the conversation by explaining that his organization has roughly 100 engineers and a team of data scientists with PhDs in data science and data analysis who are focused on tasks requiring their expertise.
He differentiates between software engineers, who build infrastructure for data science, and data scientists. He does so noting that, while both fields overlap (software engineers may learn some data science and data scientists know programming), they are distinct disciplines.
In the organization’s process, he mentions that product managers developing a new feature consider the data signals they need, then work with the data engineers to make the data available in the data lake. The data science team and the product managers use this data to derive insights, returning feedback into the product plans.
Yigal highlights the importance of product managers being able to conduct their own data analyses to reduce reliance on others while ensuring solid collaboration and harmony between their teams:
“The process is really a three-way dialogue. To make everything work effectively, it’s essential to define and capture the right signals from the start. Data engineers know how to gather and model these signals, while data scientists and product managers know how to analyze and interpret them. It’s a collaborative effort, not a linear process – everyone needs to be aligned and actively involved for success.”
– Yigal Edery, SVP of Product and Strategy at Sisense
He further emphasizes the need for a clear goal when working with data to avoid getting derailed and collecting the wrong information. He highlights the importance of aligning goals with the organization’s broader strategy to ensure impactful results. Without alignment, team members may end up completing random tasks that, while generally productive, don’t make an impact.
Tsavo builds on Yigal’s point about the importance of alignment by emphasizing the complexity of managing custom-built data pipelines. He highlights the challenge of consolidating scattered data across various platforms like Crash Analytics, Google Analytics, and Mixpanel.
Yigal agrees with Tsavo’s point about managing different data sources and adds that having multiple sources is inevitable. The key challenge, he says, is figuring out how to correlate these sources effectively. He explains that you need a clear linkage, whether it’s the user, account, or another relevant factor, to connect the different data sets. Proving these correlations allows you to make sense of the data by linking, for example, product data with CRM and sales data. Reinforcing these even more significant correlations ensures that the various data sources can be used together in a meaningful way.
Yigal suggests making data analysis a part of the company culture by holding “data walk sessions,” where teams come together to review data. He emphasizes that these sessions often lead to new questions and insights that drive improvements in how data is used.
Yigal is often surprised by the different insights his team members derive from the same data, reinforcing the value of collective analysis and feeding a solid feedback loop for continuous improvement.
Implementing AI-Driven Error Management Systems
Tsavo highlights how tools like GitHub Copilot and Codium are changing the game for developers, accelerating code writing and increasing overall output. But with faster coding comes a shift in what developers need to focus on. It’s no longer just about writing code; the speed-up creates pressure in other areas like code review, collaboration, and architectural planning.
This shift means developers are moving beyond the code itself, engaging more in cross-functional teamwork and future-focused architecture discussions. Tsavo notes the need for developers to have broader knowledge and understanding, even when using tools like GitHub Copilot. Yigal agrees and stresses that, while these tools can help speed up coding, developers must critically evaluate AI-generated code since they are ultimately responsible for its application.
Yigal shares that Sisense leverages AI throughout its products to upskill all the different personas using the product. He also noted that the Sisense platform uses AI to generate code for embedding its analytics into applications, through the use of the platform’s Compose SDK.
Tsavo discusses how the flow of productivity for developers gets interrupted when encountering errors in code and the data necessary to drive the kind of AI systems that can help pick up the pieces:
“So, for example, you run into an error, and basically, work stops. You’re in the browser; you’re trying to Stack Overflow it. You ask ChatGPT, whatever else; maybe you send something to your team. You ask, ‘Has anyone seen this error?’
So, the sites you visit, the people you talk to, and the project it was associated with: ‘How do we capture that?’ So then, in the future, you’re asking, “When was the last time I ran into this error, and how did I solve it?” And then, in a team environment, you end up asking: ‘When was the last time anyone ran into this error, and how did they solve it?'”
–Tsavo Knott, Technical Co-founder and CEO of Pieces
In terms of the kinds of regular workflows best suited for running machine learning models to notice anomalies in behavioral history, Tsavo notes that as developers increasingly work across various programming languages and new environments, they encounter more errors. The compounding scenarios amplify the need for a system that helps retain context about past mistakes and their solutions, making it easier to navigate similar challenges in the future.
Yigal responds by highlighting that this specific situation at Sisense differs from what Tsavo described. While Tsavo focuses on individual developer workflows and error resolution, Yigal emphasizes how Sisense leverages AI in broader contexts, particularly in customer support.
He explains that when customers encounter errors, Sisense collects data on various errors and use cases. They then train a language model on the dataset to help navigate similar issues more quickly in the future.
Yigal suggests that this approach of using AI to streamline error handling could be beneficial in the context of individual developer workflows as well, implying that a similar system could help developers manage and resolve their coding errors more efficiently.
