The World’s First Real-time Novelty Detector For Categorical Data

An Industry First: Novelty Detection for Categorical Data in Real-Time

thatDot is excited to share the general availability of the world’s first system for real-time categorical anomaly detection. Data Engineers, Developers, and Data Scientists can now generate a real-time score showing the novelty of any categorical data—greatly extending the science of anomaly detection beyond the mainstay of time-series numerical analysis, and opening up zettabytes of data for new insights.

Most of the big data collected globally is not numerical data; so traditional tools don’t apply. File names, email addresses, postal addresses, demographic groupings, IP addresses, given names, and other identifiers are all examples of categorical data that cannot be natively processed by existing anomaly detection technology. thatDot’s Novelty Detector is expanding the frontier of what data can be analyzed in real-time for anomalous signals.

Turn High-Volume Data into High-Value Data

“Big data comes with a curse: most of it is useless, but you can’t tell which data is valuable and which is mundane. We’re changing that. Early users of thatDot’s Novelty Detector have quickly made critical discoveries and found new value in both existing datasets and incoming streams of new data. Unlocking the insight in real-time data streams is helping our customers accelerate product development and operational issue analysis, benefiting both the revenue and costs sides of their business” said Ryan Wright, Founder and CEO of thatDot, Inc.

Built upon Quine’s stateful streaming data engine, thatDot’s Novelty Detector easily scales the dynamic graphical models used to find true anomalies and explains why they stand out. The broad contextual information available to our graph processing engine dramatically reduces false positives so users don’t waste of time and resources with unneeded verification and issue resolution.

Use Case: Real-time Access Log Fingerprinting

Cloud services are powerful and ubiquitous, but each service is used differently, by different users in different places, and for different reasons. How can you tell if one of those user’s credentials are compromised? You’d need a system to “learn” what is normal for each service, for each user, in each location. Creating that training data would be nearly impossible. thatDot Novelty Detector trains itself (no training data required!) and immediately flags the compromised usage in real-time. By using the values from log data: [Service name, REST API endpoint, User ID, Country, City] and any other relevant information like time-of-day or specific service information, anomalous access patterns become immediately apparent. And with the context-aware explanations, thatDot Novelty detector will tell you what was so unusual about that anomaly.

thatDot Categorical Anomaly Detection

Categorical data provides insight into user behavior, system flows, and config changes all in real time.

Simple to Use

thatDot Novelty Detector is available as a container for rapid deployment in common container management platforms such as Kubernetes or AWS Elastic Container Service (ECS). Turn it on and thatDot Novelty Detector offers a simple REST API to ingest data—as a continual stream, or as a batch—and return a novelty score for each data observation. Together with the summary score, a valuable set of additional information explaining why that observation is anomalous is delivered with every data observation.

No tuning, training, or setting hyper-parameters is needed with thatDot Novelty Detector. It is ready to run immediately, allowing rapid use all the way from research projects to large scale industrial applications.

In support of data exploration and presentation, a graphical visualization of the observed data is provided via the included user interface.

Free to Try

thatDot Novelty Detector is available now on the AWS Marketplace or you can contact thatDot.com for information about custom deployments. All users receive a free tier of usage on either platform and discounted annual commitment pricing is available for high volume use.

Media interest, please contact

Robert Malnati

thatDot, Inc.

info@thatdot.com