Target was in the litigation e-discovery space, parsing SEC and other rigid-form documents. We were asked for an initial “is it real?” machine learning / artificial intelligence (ML/AI) assessment. We found two serious problems: a) some of what they we’re calling “machine learning” was actually heuristic models and sophisticated rules engines; these can sometimes be applied to similar problem sets as machine learning, but are not machine learning as claimed; and b) the ML/AI they did have was typical for, say, an undergraduate student and riddled with bias and overfitting errors the team didn’t realize were there. Client pulled the plug and walked away.

Target was in robotic process automation (RPA) space and was using ML/AI to watch employee tasks and then intelligently mimic. We found the ML/AI to be real but average for, say, someone with 2-3 professional years’ experience. Target used the right tools, understood which data features mattered most, and built privacy into training sets from the start. But target was using simplistic models and lacked accuracy feedback mechanisms. Client proceeded to next stage of due diligence (and ultimately concluded the deal after further due diligence).

Target was in end-point security space using artificial intelligence to block intrusions by finding anomalous device behavior. Target had recently signed long-term services contract with an “ML company” with a dozen PhD’s who claimed sophisticated ML expertise. But the claims were untrue and despite the high number of PhD’s in math and statistics, not one had more than a few month’s of self-taught experience in ML/AI. Client proceed with the deal with target terminating the “ML company” contract and agreeing to build new and real ML/AI capabilities in-house within 12 months.