Convolutional neural network with semantically meaningful activations for speech analysis

Abstract

Semantic analysis of speech is more prospective compared to analysis of text since speech contains more in- formation that is important for understanding. The most im- portant distinguishing feature of speech is intonation, which is inaccessible in the text analysis. For successful semantic analysis of speech it is necessary to transform the speech signal into features with semantic interpretation. The mathematical apparatus of convolutional neural networks (CNN) seems suitable to implement this kind of transformation. However there is a scalability problem that makes it hard to combine many CNN’s in a single solution. To overcome this we propose to develop a CNN model with semantically meaningful activations i.e. the model that is capable of semantic interpretation of its internal states. The ultimate goal of the transform is to extract all semantically meaningful information, however the present work is confined to voice activity detection (VAD) and intonation extraction. Unlike other VADs based on artificial neural networks, the proposed model does not require a lot of computing resources and has a comparable or even better performance.