Optimization Approach to Selecting Methods of Detecting Anomalies in Homogeneous Text Collections
Keywords:
anomaly detection, novelty detection, outlier detection, homogeneous text collections, sparse space dimension reduction, topic modelingAbstract
The problem of detecting anomalous documents in text collections is considered. The existing methods for detecting anomalies are not universal and do not show a stable result on different data sets. The accuracy of the results depends on the choice of parameters at each step of the problem solving algorithm process, and for different collections different sets of parameters are optimal. Not all of the existing algorithms for detecting anomalies work effectively with text data, which vector representation is characterized by high dimensionality with strong sparsity. The problem of finding anomalies is considered in the following statement: it is necessary to checking a new document uploaded to an applied intelligent information system for congruence with a homogeneous collection of documents stored in it. In such systems that process legal documents the following limitations are imposed on the anomaly detection methods: high accuracy, computational efficiency, reproducibility of results and explicability of the solution. Methods satisfying these conditions are investigated. The paper examines the possibility of evaluating text documents on the scale of anomaly by deliberately introducing a foreign document into the collection. A strategy for detecting novelty of the document in relation to the collection is proposed, which assumes a reasonable selection of methods and parameters. It is shown how the accuracy of the solution is affected by the choice of vectorization options, tokenization principles, dimensionality reduction methods and parameters of novelty detection algorithms. The experiment was conducted on two homogeneous collections of documents containing technical norms: standards in the field of information technology and railways. The following approaches were used: calculation of the anomaly index as the Hellinger distance between the distributions of the remoteness of documents to the center of the collection and to the foreign document; optimization of the novelty detection algorithms depending on the methods of vectorization and dimensionality reduction. The vector space was constructed using the TF-IDF transformation and ARTM topic modeling. The following algorithms have been tested: Isolation Forest, Local Outlier Factor and One-Class SVM (based on Support Vector Machine). The experiment confirmed the effectiveness of the proposed optimization strategy for determining the appropriate method for detecting anomalies for a given text collection. When searching for an anomaly in the context of topic clustering of legal documents, the Isolating Forest method is proved to be effective. When vectorizing documents using TF-IDF, it is advisable to choose the optimal dictionary parameters and use the One-Class SVM method with the corresponding feature space transformation function.
References
2. Ghosal T. et al. Novelty goes deep. A deep neural solution to document level novelty detection. In Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, August 20–26, 2018, pp. 2802–2813.
3. Zhao L., Zhang M., Ma S. The nature of novelty detection. Information Retrieval. 2006. vol. 9. no. 5. С. 521–541.
4. Guzman J., Poblete B. Online relevant anomaly detection in the Twitter stream: an efficient bursty keyword detection model. Proceedings of the ACM SIGKDD workshop on outlier detection and description. 2013. pp. 31-39.
5. Lau J. H. et al. Word sense induction for novel sense detection. Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics. 2012. pp. 591-601.
6. Gurina A.O., Guzev O.Ju., Eliseev V.L. [Detection of anomalous events on the host using an autoencoder] Obnaruzhenie anomal'nyh sobytij na hoste s ispol'zovaniem avtokodirovshhika. International Journal of Open Information Technologies. 2020. vol. 8. no. 8.
7. Goldstein M., Dengel A. Histogram-based outlier score (HBOS): A fast unsupervised anomaly detection algorithm. KI-2012: Poster and Demo Track. 2012. pp. 59-63.
8. Zhao Y., Nasrullah Z., Li Z. Pyod: A python toolbox for scalable outlier detection. arXiv preprint. arXiv:1901.01588. 2019.
9. Denning D.E. An intrusion-detection model. IEEE Transactions on software engineering. 1987. no. 2. pp. 222-232.
10. Markou M., Singh S. Novelty detection: a review—part 1: statistical approaches. Signal processing. 2003. vol. 83. no. 12. pp. 2481-2497.
11. Chandola V., Banerjee A., Kumar V. Anomaly detection: A survey. ACM computing surveys (CSUR). 2009. vol. 41. no. 3. pp. 1-58.
12. Pimentel M.A.F. et al. A review of novelty detection. Signal Processing. 2014. vol. 99. pp. 215-249.
13. Faria E.R. et al. Novelty detection in data streams. Artificial Intelligence Review. 2016. vol. 45. no. 2. pp. 235-269.
14. Ruff L. et al. A unifying review of deep and shallow anomaly detection. Proceedings of the IEEE. 2021.
15. Hendrycks D., Mazeika M., Dietterich T. Deep anomaly detection with outlier exposure. arXiv preprint. arXiv:1812.04606. 2018.
16. Gorokhov O., Petrovskiy M., Mashechkin I. Convolutional neural networks for unsupervised anomaly detection in text data. International Conference on Intelligent Data Engineering and Automated Learning. Springer, Cham, 2017. pp. 500-507.
17. Yang Y. et al. Topic-conditioned novelty detection. Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining. 2002. pp. 688-693.
18. Ng K.W. et al. Novelty detection for text documents using named entity recognition. 2007 6th international conference on information, communications and signal processing. IEEE, 2007. pp. 1-5.
19. Amplayo R.K., Hong S.L., Song M. Network-based approach to detect novelty of scholarly literature. Information Sciences. 2018. vol. 422. pp. 542-557.
20. Li Z. et al. COPOD: copula-based outlier detection. arXiv preprint. arXiv:2009.09463. 2020.
21. Mikolov T., Yih W., Zweig G. Linguistic regularities in continuous space word representations. Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies. 2013. pp. 746-751.
22. Krasnov F.V., Smaznevich I.S. [The explicability factor of the algorithm in the problems of searching for the similarity of text documents]. Vychislitel'nye tehnologii. [Computational technologies]. 2020. vol. 25. no. 5. pp. 107-123.
23. Schubert E., Gertz M. Intrinsic t-stochastic neighbor embedding for visualization and outlier detection. International Conference on Similarity Search and Applications. Springer, Cham, 2017. pp. 188-203.
24. McInnes L., Healy J., Melville J. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint. arXiv:1802.03426. 2018
25. Narayan A., Berger B., Cho H. Density-preserving data visualization unveils dynamic patterns of single-cell transcriptomic variability. bioRxiv. 2020.
26. Campos G.O. et al. On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study. Data mining and knowledge discovery. 2016. vol. 30. №. 4. pp. 891-927.
27. Amarbayasgalan T., Jargalsaikhan B., Ryu K.H. Unsupervised novelty detection using deep autoencoders with density-based clustering. Applied Sciences. 2018. vol. 8. no. 9. P. 1468.
28. Campello R.J.G.B. et al. Hierarchical density estimates for data clustering, visualization, and outlier detection. ACM Transactions on Knowledge Discovery from Data (TKDD). 2015. vol. 10. no. 1. pp. 1-51.
29. Ankerst M. et al. OPTICS: Ordering points to identify the clustering structure. ACM Sigmod record. 1999. vol. 28. no. 2. pp. 49-60.
30. Karypis G., Han E. H., Kumar V. Chameleon: Hierarchical clustering using dynamic modeling. Computer. 1999. vol. 32. no. 8. pp. 68-75.
31. Karypis G., Kumar V. A software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices. University of Minnesota, Department of Computer Science and Engineering, Army HPC Research Center, Minneapolis, MN. 1998. vol. 38.
32. Kannan R. et al. Outlier detection for text data. Proceedings of the 2017 siam international conference on data mining. Society for Industrial and Applied Mathematics, 2017. pp. 489-497.
33. Zhang J., Ghahramani Z., Yang Y. A probabilistic model for online document clustering with application to novelty detection. Advances in neural information processing systems. 2004. vol. 17. pp. 1617-1624.
34. Manevitz L. M., Yousef M. One-class SVMs for document classification. Journal of machine Learning research. 2001. vol. 2. no. Dec. pp. 139-154.
35. Zimek A., Campello R.J.G.B., Sander J. Ensembles for unsupervised outlier detection: challenges and research questions a position paper. ACM SIGKDD Explorations Newsletter. 2014. vol. 15. no. 1. pp. 11-22.
36. Marques H. O. et al. Internal evaluation of unsupervised outlier detection. ACM Transactions on Knowledge Discovery from Data (TKDD). 2020. vol. 14. no. 4. pp. 1-42.
37. Liu F.T., Ting K.M., Zhou Z.H., Isolation Forest. 2008 Eighth IEEE international conference on data mining. IEEE, 2008. pp. 413-422.
38. Krasnov F.V. [Comparative Analysis of the Accuracy of Methods for Visualizing the Structure of a Text Collection]. International Journal of Open Information Technologies. 2021. vol. 9. no. 4. pp. 79-84. (In Russ.)
39. Pimenov V.I., Voronov M.V. [Formalization of regulatory texts]. Informatika i avtomatizacija. [Computer Science and Automation]. 2021. no. 3(20). pp. 562–590. (In Russ.)
Published
How to Cite
Section
Copyright (c) Ирина Сергеевна Смазневич
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms: Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).