Data Dojo Würzburg 27
DataDojo@Lunch - hybrid
April 2024
- When: Wednesday, April 17nd, 2024 at 11:00am until 12:30pm (90 minutes)
- Where: CCTB or online (CCTB Seminar Zoom Link)
- Info: DataDojo Website, Repo
Dataset
CCTB Cluster Usage
Minutely CPU and memory usage of several CCTB servers monitored via sysstats over a period of a month (March 2024).
We will use pre-converted tsv files for memory and CPU.
Potential questions
- What was the avarage CPU load of each CCTB server?
- What was the avarage memory usage of each CCTB server?
- Where there any events/periods of particularly high usage?
Further ideas
- Extract other pieces of information from the sar files (e.g. disk or network usage)
- Gather and visualize the data in a timeseries database (TSDB, e.g. InfluxDB)
Collaborative Tools and Workflow
For Notebooks (R, python, julia, js, …) with real time collaboration CoCalc seems to be the best option right now. It worked great the last couple of times so we’ll stick to it for now. You need to register an account there (it is free).
Future Suggestions
Add your suggestions to the list and :+1: to the end of a line you are interested in
Concrete suggestions
- Build a citation and/or collaboration network for the CCTB using the OpenAlex API
Data Sets
- DFG Chancengleichheits-Monitoring
- Würzburg Baumkataster, Würzburger Klimabäume
- National Registry of Exonerations
- Bee Varroa Image Classification :bee:
- Mattermost Chat History - e.g. analyze the messages and reactions from the lunch channel
- Wordbank - data of children learning to talk
- All Birds :bird:
- Results of the Bundestagswahl 2021
- Weather data throughout Germany over time (incl. temperature, precipitation, …): https://www.dwd.de/DE/leistungen/cdc_portal/cdc_portal.html
- German Mikrozensus
- Kaggle Titanic or Tabular Playground or Meta Kaggle
- World Trade Data (Open Trade Statistics)
- Open Citation Data
- Top 100 charts + Audio Features
- Emoji Usage :hugging_face::heart::laughing:
- Observable Curated Datasets
- Abgeordnetenwatch - Data on German elected officials in EU-Parlament, Bundes- and Landtag (Election History, Ausschusszugehörigkeit, Side jobs, etc)
- Button Men Game Results
- Extract structured data from pdfs (Example: population statistics in Leinach from Gemeindeblatt)
- DSA Transparency Database - daily archives of statements of reasons for take downs from social media
- Dürremonitor Deutschland
Data Sources
all data types are welcome, including tables, images, videos, sounds, DNA, …
- Frag den Staat API
- Polizeiliche Kriminalstatistik (PKS)
- OpenData Bayern
- TidyTuesday
- Our World in Data (R package: owidR), Sustainable Development Goals
- Open Data Initiatives (Würzburg, Germany, Statistisches Bundesamt, Europe, APIs)
- Data is plural
- Awesome Public Datasets
- Kaggle Datasets or Competitions, e.g. SLICED
- tsibbledata: Time Series Datasets
- R-text-data: Text Datasets, ready to use in R
- data.world
- Statista - the University of Würzburg has a campus license
- Open Legal Data
- Bundestag Data (e.g. poll results, deputies, wahl-o-mat, inspirational blog post)
- Deutsche Digitale Bibliothek (API, old newspapers from Germany)
- Earth Observation: Satellite Image Time Series
- Machine Learning Datasets
- Internation (Student) Assessment Data (TIMSS, PIRLS, PISA, …)
- (Medical) Imaging Datasets, MedMNIST
- Inspirational Notebooks on Observable
- Ski resort statistics :skier: