ReducIng time of assessment of text documents similarity in the heterogenious multiprocessor computer system

Abstract

ReducIng time of assessment of text documents similarity in the heterogenious multiprocessor computer system

Serov S.S., Andreev A.E., Kravchenya P.D., Gushchin R. I., Chebotarev P.P.

Incoming article date: 15.06.2015

We consider the parallel implementation of a simplified shingles algorithm for the speed-up of text documents comparison on heterogeneous computing system based on multi-core processors and the Many Integrated Core (MIC) accelerators. A step-by-step gradual modification of the original single-threaded program is presented as well as the architecture of distributed software web-service for document comparison. The results of testing of the developed prototype, showing the possibility of reducing the time of comparison of documents up to 12 times using multiple accelerators in the system are presented.

Keywords: shingles algoritm, CRC32, Intel Xeon Phi ™, MIC, OpenMP, Mono, ASP.NET