A few months ago, we spoke about three rules of thumb for assessing companies and trying to figure out if they are really doing AI or just using it for marketing purposes in our How to Cut Through the Artificial Intelligence Hype piece. Since it was so popular, I was inspired to write this follow-up.
We look at thousands of companies and interview many of them one on one. These companies are a mix of those buying AI solutions, selling AI solutions, and developing new ways to use AI. This puts us in a unique position to make assessments about an AI vendor’s legitimacy.
Undeniably these rules will upset some people. That isn’t our intention. The intention is to give our readers a simple and short process to filter for bogus “AI” vendors.
We found that some companies are truly finding more ways to use artificial intelligence to drive business goals. However, it is also true that other companies are developing ingenious tactics to use artificial intelligence purely as a marketing strategy to make them look more interesting or exciting, either for potential customers or investors.
In this article, we will lay out the seven trust signals with which to assess a company purporting to have AI in its DNA. They are as follows:
- Investors
- Academic Background
- Work Experience
- Talent Density
- Hiring Practices
- Company History
- Full-time Leadership
In most cases, one will have to look from the bottom up–that is, assess the people. These trust signals will help business leaders determine if a company is genuinely innovating with artificial intelligence or if it is really only using it as a marketing strategy.
While we can understand why companies would want to show themselves in an exciting light, we do not believe they should do so fraudulently. We are here to keep executive readers that might not have an understanding of how to analyze these companies from falling prey to what ultimately comes down to deceitful branding and marketing.
Our job is not only to stay on top of the new applications, but to stay on the cutting edge of how companies are misleading potential clients. We aim to educate our audience so they can make better decisions for themselves.
(Note: The “signals” listed below are just signals. Over the past 2 years we’ve developed this set of simple “filters” for examining companies, and we suspect that 90% of the time, they can spot a deceptive firm properly. Companies who don’t have these signals aren’t necessarily lying about using AI, but on the aggregate, we believe you’ll find these signals to be remarkably useful in quickly filtering vendors on their legitimacy.)
Trust Signal: Smart Investors
A good way to determine the legitimacy of an AI company is to check the type of investors it has. We want to see top venture capitalist companies onboard, preferably those mentioned in cutting-edge technology sites such as TechCrunch.
These will most probably include VC companies operating out of Silicon Valley or other tech hubs such as Boston, San Francisco, or New York. While the US is further along than most other countries when it comes to AI, we may also consider companies with VC backing from major tech cities globally, such as Tokyo, Berlin, Shanghai, or Tel Aviv as a good sign.
This is primarily because successful VC companies focusing on technology will go deep in their research before investing in any company, including (or perhaps most particularly) AI companies. It is very unlikely a company only using AI as a marketing ploy will be able to get any kind of commitment from these types of investors.
In other words, if an AI company lists Menlo Park-based Sequoia Capital as a major investor, then there is a good chance the company has some legitimacy in the AI solution market.
The short and uncouth version of this trust signal could be summed up thus:
So-called AI companies who raise millions from non-idiots are likely to be non-idiots themselves.
Trust Signal: Academic Background of Technical Leadership
We want to see key people in an AI company to have at least a master’s degree in computer science, data science, applied math, or hard sciences like physics or chemistry. These fields tend to breed the kind of math skills required for a technical leader in an AI company.
Bonus points for degrees from a hyper-reputable university such as Cornell, Stanford, or MIT, or from an institution of equivalent notoriety outside the US.
This isn’t to say that “non-prestigious” schools aren’t training their students well, but rather, that – from a simple secondary research perspective – these lesser-known schools don’t carry the same degree of trust (i.e. We can safely presume that to get a PhD in computer science from MIT implies that you are not a moron. We cannot as safely make this assumption for a school without a reputation for rigor or innovation in the fields mentioned above).
It is easy enough to find out. We can check the company’s profile on LinkedIn and look for employees with “data,” “AI,” “machine,” or “intelligence” in their job title. We can then check their academic backgrounds to see if they have a master’s or doctorate degree in the fields mentioned above from reputable institutions. If we find them, that is one more indication that the company is truly leveraging AI.
Trust Signal: Examine Past Work Experience
If a person has the requisite knowledge for AI work, they can basically go out and be a data scientist wherever they are. However, the legitimacy of a data scientist role has to connect with past work experience.
Again, it is easy enough to do a search using LinkedIn. When we have a list of data scientists and data analysts, we want to check their past work experience. We want to see people that have previously had a similar role in a marquee company.
This is not some type of business snobbery. Many people want to work for marquee companies because they have better chances of advancement and it looks good on their resumes. As a result, the hiring process in these companies tends to be rather stringent.
Again, this is using the efforts of a larger company with more resources than we are likely to have. It is highly unlikely a company like Google will hire someone for a position for which he or she is not qualified.
Therefore, if we see that a data analyst in the company we are looking into was a data analyst for Google Cloud Services or for some other marquee company, we have some degree of confidence that person has the qualifications to do the job. It is a good signal that the company has the right kind of people for AI work. Ergo, it is a good sign the company is actually leveraging AI.
On the other hand, if the previous work of a data analyst is as the SEO analyst for some unknown company, or as a salesperson for a marquee company, the dots do not connect. To use work experience as a trust signal, it needs to be proof in the pudding. A data scientist or data analyst with relevant work experience from a marquee company is solid proof of that AI pudding.
In other words:
If all your “data scientists” are straight out of getting English or Art History degrees from nondescript arbitrary universities, you have no damned data scientists.
Trust Signal: AI Talent Density
We have so far looked at individual people in terms of their academic background and past work experience to determine the company’s legitimacy. At this point, we should start looking at the big picture.
An AI-based company should have the right people doing the key roles, but finding one or two people that fit the bill is not quite enough to establish legitimacy. The next step in this assessment is to discover the proportion of the AI talent to the general list of employees.
If we are looking for a company that makes T-shirts, for example, we will have a contingent of people with no experience whatsoever in making a T-shirt. However, we want everyone on the production floor to have the skill and experience in making T-shirt. The same applies when assessing the legitimacy of an AI company. When roughly half of the staff has these credentials, it is likely that a company has AI in its DNA.
LinkedIn again provides a good measure of this. All we have to do is go to the company profile, filter for all employees with “data,” “machine,” “artificial,” “AI,” or “ML” in their job title, and compute the percentage of all these people against the total number of employees. If we have an AI company where 18 to 20 out of the 40 employees has the academic background and work experience in AI, there is a high likelihood we are dealing with a company with AI at its core.
On the other hand, if we find only two or three people have the requisite AI credentials in a 100-person company, it is very unlikely that AI is a part of their core product. It is more likely that the company is using AI for marketing purposes and not actually leveraging artificial intelligence as a real part of their offering.
At this point, it would also pay to be circumspect. If seven people in a 30-person company have AI qualifications, one may think this is not a bad ratio. However, we still need to check if these seven people actually pass the litmus test of both academic background and work experience as detailed above.
A fresh graduate of computer science from MIT will still have a long way to go to contribute significantly to AI solution development, so if most or all of these data scientists or analysts are fresh graduates, it is still not a good trust signal.
Trust Signal: Hiring Data Scientists
The next two trust signals have to do with the company as a whole. The first one is in their hiring practices. To investigate the legitimacy of an AI-based company, it is often a good idea to check the career pages or job boards of the company. We want to see if they are looking for people with the type of machine learning credentials we want them to have. If they are, it is a good sign the company is truly leveraging AI.
On the other hand, if the company seems to be hiring fresh graduates with no academic credentials or work experience in AI, it is highly probable that they are simply giving random people the job title of data scientists or data analysts just to give the right impression to potential clients or investors.
We might think that an AI company should already have the talent they need. However, it is a fact that qualified data scientists and data analysts are much in demand, so it is not at all easy to have a full team of talent at once or even all the time. We can expect an AI-based company to deal with attrition, just like any industry with high demand for specialist talent. This should be apparent in their hiring practices.
Trust Signal: Company History
Assuming that talent turnover is a fact in AI, we might expect that a particular data scientist may have only been with a company for a couple of months. However, we should find out how long certain job positions have been around relative to the age of the company. If data scientist and analyst jobs are a relatively recent addition to the company, then chances are the products are not predicated on AI.
For example, if a company that has been around for six years creates a department of machine learning or data science in the last three to six months, it is almost certain that machine learning and AI are not at the core of the products. This is something one should consider when assessing an AI company.
It is perfectly possible that the company has diversified lately into AI. It may actually be doing some serious innovating in AI despite being a late entrant into the game; however, there is a better than likely chance that the company had simply tacked on an AI department as a marketing strategy and is not doing AI in any significant way.
Trust Signal: Full-time Technical Leadership
One of the main things we want to see in an AI company is AI competence in the leadership team. Generally, if a company has a CTO or Head of Machine Learning who has excellent academic credentials and work experience, then that is a good trust signal.
However, we need to qualify if this is true leadership. Just because the CTO or Head of Machine Learning has the requisite credentials for AI doesn’t always mean that person is actually contributing to the development of the AI product.
One of the sneaky things some companies do is to get an eminently qualified AI person on its leadership team somehow as the CTO or head of ML, but purely as a nominal position. It may be as a favor to a good friend, or for monetary compensation, or both. That AI person does not actually do anything to contribute to the development and design of an AI solution.
We can make a search online for the names of those CTOs or heads of ML to see if they are associated with any other companies. If that is the case, there is a good chance they are just lending their names to some of these companies to give them credibility.
As CTO or head of ML, it would take a lot of their time to make any quality contributions to anyone AI company. It is unlikely that they have the time to do the same for other companies, especially if they also work for marquee companies or their own startups.
Of course, there is a chance that the company we are assessing is the one to which a specific AI professional devotes a majority of their time. However, this can be hard to determine, unless perhaps that person is also a founder of the company.
Straight up:
If your “CTO” has a full time job elsewhere, you have no CTO.
You’re in the game or you’re not.
Caveat for Considering AI Vendor Solutions
While we hope that some of these tools and tactics are useful for combing through the sea of artificial intelligence companies out there in the market now, it is not necessarily a top priority for a majority of businesses. The vendor companies business leaders should spend any time on are those most likely to deliver results now and in the future for their business.
We want to emphasize that artificial intelligence-oriented vendor companies are not the only ones that can drive value in a business. When looking for a solution when assessing different vendors, it should not be on the idea that artificial intelligence is necessary for solving a specific business problem. Making a buying decision based purely on who is using AI is misguided.
Our purpose is not to steer people towards only doing business with companies predicated on artificial intelligence. We simply want to make sure that if an AI solution is necessary for a business, its leaders have the tools to help them assess if a company is truly AI capable, or simply using the term as a marketing ploy. Hopefully, you, as a business leader, found some of these additional tools useful as well
Header Image Credit: Strange Notions
