Datasets and tools

Datasets and tools

Find out more about the datasets and the tech stack at the core of this project.

Datasets

Datasets generated in the course of this project are available for download in the dedicated section of this website. After more comprehensive quality checks, they are formally released.

Here are the textual datasets that have been released on Discuss Data:

Giorgio Comai (2021). kremlin_en - A textual dataset based on the contents published on the English-language version of the Kremlin’s website, v. 1.0, Discuss Data, <doi:10.48320/5EB1481E-AE89-45BF-9C88-03574910730A>.

Giorgio Comai (2024). Russian state institutions full-text datasets – A collection of corpora based on contents extracted from the websites of Russian state institutions, v. 1.0, Discuss Data, https://doi.org/10.48320/0578D7FE-35F7-4E9E-A29D-926618A5C6BD

Giorgio Comai (2025). zavtra.ru_ru - Full text corpus based on the website of Russian weekly newspaper ‘Zavtra’ (in Russian, 1996-2024) – Full text of all articles published on zavtra.ru with metadata, in tabular format, v. 1.1, Discuss Data, https://doi.org/10.48320/EF3EBDDA-53CC-4078-88D9-BC9F3E7CF970

Tools and software

There are many great solutions, both proprietary and open source, to process and analyse data. Most of them either assume that the user already has textual data in a structured format, or that they know how to get them. Others allow for text mining on a large scale, but make it somewhat difficult to customise or refine the output.

Textual datasets collected during the course of this project are created with an updated iteration of castarter - Content analysis starter toolkit for R - a package for the R programming language aimed at facilitating the process for users who may have only limited familiarity with R.

The package also allows for sharing datasets through interactive web interfaces that allow for basic analyses without requiring any technical knowledge.

Documentation, tutorial and publicly available interfaces will published in the course of this project.

The R package castarter is available on GitHub. While still under active development, it is mostly functional. Both documentation and functionalities of the package are in active development. Some tutorials are already available in the dedicated section of this website.

More about this project