Show HN: Memory-Based Anomaly Detection in Multi-Aspect Streams
MemStream detects anomalies from a multi-aspect data stream. MemStream is a memory augmented feature extractor, allows for quick retraining, gives a theoretical bound on the memory size for effective drift handling, is robust to memory poisoning, and outperforms 11 state-of-the-art streaming anomaly detection baselines. After an initial training of the feature extractor on a small subset of normal data, MemStream processes records in two steps: It outputs anomaly scores for each record by querying the memory for K-nearest neighbours to the record encoding and calculating a discounted distance and It updates the memory, in a FIFO manner, if the anomaly score is within an update threshold . Please unzip and place the files in the data folder of the repository. Command line options -dataset: The dataset to be used for training. Dev: Pytorch device to be used for training like "Cpu", "Cuda:0" etc. MemStream expects the input multi-aspect record stream to be stored in a contains , separated file.