SIPA Data Carpentries workshop

Columbia University Libraries

Online

Jan 31-Feb 1, 2025

9AM-5PM (Friday), 10AM-6PM (Saturday)

Instructors: Ann James, Irene Morse, Paul Selmants, Marion Walton

Helpers: Ben Chiewphasa, Nkemjika Emuchay, Esther Jackson, Elizabeth Kwon, Omer Imran, Dan Woulfin

General Information

The Carpentries project comprises the Software Carpentry, Data Carpentry, and Library Carpentry communities of Instructors, Trainers, Maintainers, helpers, and supporters who share a mission to teach foundational computational and data science skills to researchers.

Want to learn more and stay engaged with The Carpentries? Carpentries Clippings is The Carpentries' biweekly newsletter, where we share community news, community job postings, and more. Sign up to receive future editions and read our full archive: https://carpentries.org/newsletter/

Data Carpentry develops and teaches workshops on the fundamental data skills needed to conduct research. Its target audience is researchers who have little to no prior computational experience, and its lessons are domain specific, building on learners' existing knowledge to enable them to quickly apply skills learned to their own research. Participants will be encouraged to help one another and to apply what they have learned to their own research problems.

For more information on what we teach and why, please see our paper "Good Enough Practices for Scientific Computing".

Who: The course is aimed at graduate students and other researchers. You don't need to have any previous knowledge of the tools that will be presented at the workshop.

Where: This training will take place online. The instructors will provide you with the information you will need to connect to this meeting.

When: Jan 31-Feb 1, 2025; 9AM-5PM (Friday), 10AM-6PM (Saturday) Add to your Google Calendar.

Requirements: Participants must have access to a computer with a Mac, Linux, or Windows operating system (not a tablet, Chromebook, etc.) that they have administrative privileges on. They should have a few specific software packages installed (listed below).

Accessibility: We are committed to making this workshop accessible to everybody.

We are dedicated to providing a positive and accessible learning environment for all. We do not require participants to provide documentation of disabilities or disclose any unnecessary personal information. However, we do want to help create an inclusive, accessible experience for all participants. We encourage you to share any information that would be helpful to make your Carpentries experience accessible. To request an accommodation for this workshop, please fill out the accommodation request form. If you have questions or need assistance with the accommodation form please email us.

Glosario is a multilingual glossary for computing and data science terms. The glossary helps learners attend workshops and use our lessons to make sense of computational and programming jargon written in English by offering it in their native language. Translating data science terms also provides a teaching tool for Carpentries Instructors to reduce barriers for their learners.

Contact: Please email dw3105@columbia.edu for more information.

Roles: To learn more about the roles at the workshop (who will be doing what), refer to our Workshop FAQ.


Code of Conduct

Everyone who participates in Carpentries activities is required to conform to the Code of Conduct. This document also outlines how to report an incident if needed.


Surveys

Please be sure to complete these surveys before and after the workshop.

Pre-workshop Survey

Post-workshop Survey


Schedule

Day 1

Before starting Pre-workshop survey
9:00AM EST Data Organization in Spreadsheets
10:15AM ESTMorning break
10:30AM ESTData Organization in Spreadsheets (continued)
12:00PM ESTLunch break
1:00PM ESTIntroduction to R
2:30PM ESTAfternoon break
2:45PM ESTIntroduction to R (cont)
4:30PM ESTEnd of day 1

Day 2

10:00AM EST OpenRefine for Data Cleaning
11:15AM ESTMorning break
11:30AM ESTOpenRefine for Data Cleaning (continued)
1:00PM ESTLunch break
2:00PM ESTIntroduction to R (cont)
3:30PM ESTAfternoon break
3:45PM ESTIntroduction to R (cont)
6:00PM EST Post-workshop survey
END

Setup

To participate in a Data Carpentry workshop, you will need access to software as described below. In addition, you will need an up-to-date web browser.

We maintain a list of common issues that occur during installation as a reference for instructors that may be useful on the Configuration Problems and Solutions wiki page.

Install the videoconferencing client

If you haven't used Zoom before, go to the official website to download and install the Zoom client for your computer.

Set up your workspace

Like other Carpentries workshops, you will be learning by "coding along" with the Instructors. To do this, you will need to have both the window for the tool you will be learning about (a terminal, RStudio, your web browser, etc..) and the window for the Zoom video conference client open. In order to see both at once, we recommend using one of the following set up options:

This blog post includes detailed information on how to set up your screen to follow along during the workshop.


Data

You can download all of the data used in this workshop as a single zip file. The file is 206 KB.

Clicking the download link will automatically download all of the files to your default download directory as a single compressed (.zip) file. To expand this file, double click the folder icon in your file navigator application (for Macs, this is the Finder application).

For a full description of the data used in this workshop see the data page.


Spreadsheets

For this lesson you will need either Microsoft Excel or LibreOffice Calc. Libreoffice is a free open source office suite.

  1. Download software from this website. You will go to a page that asks about a donation, but you don’t need to make one.
  2. Once the installer is downloaded, double click on the .exe file to start the installation(you may need to open your Downloads folder).
  3. The Windows installer has a setup wizard with options like "Typical" or "Custom" installation.
  4. You may need to accept terms and conditions and might be prompted to choose a default language or features.
  1. Download software from this website. You will go to a page that asks about a donation, but you don’t need to make one.
  2. Once the installer is downloaded, double click on it (you may need to open your Downloads folder).
  3. Move the LibreOffice icon to the Applications folder. You may need to restart your computer for it to take effect.

MacOS often blocks software from unverified developers. If this happens to you:

  • Go to System Preferences → Security & Privacy → General, and click Open Anyway for LibreOffice. After this, the app will open normally on subsequent uses.

Linux often has LibreOffice available in the default repositories, so the website download step may not be necessary.

  • Website download
    1. Download software from this website. You will go to a page that asks about a donation, but you don’t need to make one.
    2. Once the installer is downloaded, double click on it (you may need to open your Downloads folder) and LibreOffice should install.
  • Terminal installation
    • Debian-based (e.g., Ubuntu): Users can often install LibreOffice via the terminal using the command: sudo apt update && sudo apt install libreoffice
    • RPM-based (e.g., Fedora): Users might use: sudo dnf install libreoffice

OpenRefine

For this lesson you will need OpenRefine and a web browser. Note: this is a Java program that runs on your machine (not in the cloud). It runs inside a web browser, but no web connection is needed.

  1. Check that you have either the Firefox or the Chrome browser installed and set as your default browser. OpenRefine runs in your default browser. It will not run correctly in Internet Explorer.
  2. Download software from http://openrefine.org/
  3. Create a new directory called OpenRefine.
  4. Unzip the downloaded file into the OpenRefine directory by right-clicking and selecting "Extract ...".
  5. Go to your newly created OpenRefine directory.
  6. Launch OpenRefine by clicking openrefine.exe (this will launch a command prompt window, but you can ignore that - just wait for OpenRefine to open in the browser).
  7. If you are using a different browser, or if OpenRefine does not automatically open for you, point your browser at http://127.0.0.1:3333/ or http://localhost:3333 to use the program.
  1. Check that you have either the Firefox or the Chrome browser installed and set as your default browser. OpenRefine runs in your default browser. It may not run correctly in Safari.
  2. Download software from http://openrefine.org/.
  3. Create a new directory called OpenRefine.
  4. Unzip the downloaded file into the OpenRefine directory by double-clicking it.
  5. Go to your newly created OpenRefine directory.
  6. Launch OpenRefine by dragging the icon into the Applications folder.
  7. Use Ctrl-click/Open ... to launch it.
  8. If you are using a different browser, or if OpenRefine does not automatically open for you, point your browser at http://127.0.0.1:3333/ or http://localhost:3333 to use the program.
  1. Check that you have either the Firefox or the Chrome browser installed and set as your default browser. OpenRefine runs in your default browser.
  2. Download software from http://openrefine.org/.
  3. Make a directory called OpenRefine.
  4. Unzip the downloaded file into the OpenRefine directory.
  5. Go to your newly created OpenRefine directory.
  6. Launch OpenRefine by entering ./refine into the terminal within the OpenRefine directory.
  7. If you are using a different browser, or if OpenRefine does not automatically open for you, point your browser at http://127.0.0.1:3333/ or http://localhost:3333 to use the program.

R

R is a programming language that is especially powerful for data exploration, visualization, and statistical analysis. To interact with R, we use RStudio.

Install R by downloading and running this .exe file from CRAN. Also, please install the RStudio IDE. Note that if you have separate user and admin accounts, you should run the installers as administrator (right-click on .exe file and select "Run as administrator" instead of double-clicking). Otherwise problems may occur later, for example when installing R packages.

Video Tutorial

Instructions for R installation on various Linux platforms (debian, fedora, redhat, and ubuntu) can be found at <https://cran.r-project.org/bin/linux/>. These will instruct you to use your package manager (e.g. for Fedora run sudo dnf install R and for Debian/Ubuntu, add a ppa repository and then run sudo apt-get install r-base). Also, please install the RStudio IDE.


More complete setup instructions for the Data Carpentry Social Sciences workshops (with R) can be found at the workshop overview site.