Isodata-Based Method for Clustering Surveys Responses with Mixed Data: The 2021 StackOverflow Developer Survey

Abstract

Massive amounts of data are generated every day from all kinds of sources, from numerical data generated by sensors to veiled messages on social networks. Transforming these data into properly organized pieces of information and transforming it into resources for decision-making is complicated, not only because of the speed and volume at which it is produced, but due to the fact the high complexity of the context in which it is generated. Often, the first step in analyzing the data is to separate it into categories that correspond to segments of interest in that context. However, in many real cases, the limits of these segments and even the number of existing segments is unknown. Clustering techniques allow defining the classes of entities in a data set with sufficient relevance. However, those techniques usually work only with numerical data. Surveys are a very useful tool for collecting data in ill-defined contexts, but these data usually contain values that are not only numerical but of a very diverse nature. This paper presents a modification to the Isodata method to process data with mixed numerical and categorical values. The resulting algorithm is tested by analyzing the results of the 2021 Stack Overflow developer survey. The results obtained in the clustering of such data are sound and show that the Isodata method, with the proposed adaptations, can be successfully employed to discover patterns in complex mixed data.