autoharp

# Autoharp Instructor Grading App

A Shiny application designed to help instructors grade student R/Rmd scripts
more easily when using the autoharp package.

## Features

### Tab 0: Session Management

This tab allows you to save and resume grading sessions:

1. **Load Previous Session**: Upload a previously saved session file (.rds) to resume grading
2. **Current Session Info**: View information about your active grading session
3. **Manual Save**: Save your session manually at any time
4. **Auto-Save**: Sessions are automatically saved after:
   - Completing automated grading (Tab 1)
   - Saving plot grades (Tab 2)
   - Completing autoharp grading (Tab 4)

**Session Data Includes**:
- Solution template and objects
- Student files list
- All grading results
- Plot grades and comments
- Current grading position
- Output directory configuration

### Tab 1: Automated Object Testing

This tab allows instructors to:

1. **Load a Solution Template**: Upload a plain RMD file containing the correct solutions (no autoharp annotations needed)
2. **Specify Student Scripts Folder**: Point to a folder containing student R/Rmd submissions (use the "Browse..." button for a graphical folder selector, or type the path manually)
3. **Specify Output Directory**: Configure where all outputs (plots, results, session files) are saved (default: "grading_output")
4. **Specify Working Directory**: **NEW!** Set the working directory from which student files are executed (default: student file location)
5. **Select Objects to Test**: Choose which objects from the solution should be tested against student submissions
6. **Run Automated Grading**: Execute comprehensive tests on all student submissions

**Note**: When you select a folder in Tab 1, the same folder path is automatically populated in Tab 4 (Autoharp Grading) for convenience. You can change it in Tab 4 if needed.

### Tab 2: Plot Grading

This tab provides an interface for manually grading student-generated plots:

1. **Navigate Through Students**: Use Previous/Next buttons or dropdown to select students
2. **View Student Plots**: See all plots generated by each student (plots are saved to a `student_plots` folder during grading)
3. **Assign Grades**: Enter numeric grades (0-100) for each student
4. **Add Comments**: Provide optional feedback comments
5. **Track Progress**: Visual progress bar shows grading completion status

**Note**: Plots are automatically rendered and saved to a `student_plots` subfolder within the configured output directory during the automated grading process. Sessions are auto-saved after each grade entry.

### Tab 3: Template Download

This tab allows you to generate an autoharp-compatible solution template with test chunks:

1. **Upload Solution**: Upload a solution RMD file in the 'Object Testing' tab
2. **Select Objects**: Choose which objects to include in tests
3. **Generate Template**: Create the autoharp version with test chunks
4. **Download Template**: Download the generated template for use with `render_one()`

The generated template includes:
- Proper R Markdown chunks with `test` prefix names
- `autoharp.objs` option containing solution objects for comparison
- `autoharp.scalars` option containing test result scalars
- Comprehensive type-based tests for all selected objects

### Tab 4: Autoharp Grading

This tab allows you to use the autoharp `render_one` function for comprehensive grading:

1. **Upload Template File**: Upload an autoharp-compatible template file with test chunks (generated from Tab 3)
2. **Specify Student Scripts Folder**: Point to a folder containing student scripts (use the "Browse..." button or type the path). This field is automatically populated with the folder selected in Tab 1, but you can change it if needed.
3. **Specify Output Directory**: Configure where render_one outputs are saved (default: "student_out")
4. **Specify Working Directory**: **NEW!** Set the working directory from which student files are executed (default: student file location)
5. **Specify Test Pattern**: Enter the pattern to identify test chunks (default: "test")
6. **Allow Package Installation**: Choose whether to allow automatic package installation during grading
7. **Run Autoharp Grading**: Execute `render_one` on all student scripts using `lapply`
8. **View Results**: See results in a comprehensive table including:
   - Run status (SUCCESS/FAIL)
   - Run time and memory usage
   - All correctness check results
   - Plot grades from Tab 2 (if available)
9. **Download Excel**: Export all results as an Excel file with two sheets:
   - Sheet 1: Grading Results (autoharp correctness checks)
   - Sheet 2: Plot Grades (manual plot grades and comments from Tab 2)

**Configuration**:
- Template file: Upload via file input (supports .Rmd and .qmd files)
- Student scripts: Specified via folder path (automatically synced from Tab 1)
- Output directory: Configurable (default: "student_out")
- Working directory: Configurable (default: student file location)

**Workflow**:
1. First, use Tab 1 to upload a solution and select the student scripts folder
2. Configure output directory in Tab 1 or Tab 4 (where all outputs will be saved)
3. Use Tab 3 to generate/download an autoharp template
4. In Tab 4, upload the generated autoharp template
5. The student scripts folder is automatically populated from Tab 1, but can be changed if needed
6. Optionally, use Tab 2 to manually grade plots
7. Use Tab 4 to run autoharp grading with the specified test pattern
8. Download comprehensive results as an Excel file with two sheets
9. Your session is automatically saved throughout - return anytime via Tab 0!

---

## Output Directory Management

You can configure where all grading outputs are stored:

- **Default Locations**: 
  - Tab 1: `grading_output/` (for object testing results and plots)
  - Tab 4: `student_out/` (for autoharp render_one outputs)
- **Custom Directories**: Specify any path for organized storage
- **Structure**:
  ```
  <output_directory>/
  ├── student_plots/        # Student-generated plots
  │   ├── student_01/
  │   └── student_02/
  ├── sessions/            # Auto-saved session files
  │   ├── session_20260123_120000.rds
  │   └── session_20260123_130000.rds
  └── [render_one outputs]  # HTML, logs, etc.
  ```

---

## Session Management

 Sessions are automatically saved and can be resumed:

### Auto-Save Behavior
- Sessions are saved automatically after:
  - Running automated grading (Tab 1)
  - Saving each plot grade (Tab 2)
  - Completing autoharp grading (Tab 4)
- Session files are stored in `<output_directory>/sessions/`
- Each session has a unique ID with timestamp

### Resuming a Session
1. Go to the **Session** tab
2. Click "Upload Session File" and select a `.rds` file
3. Click "Load Session" to restore:
   - All grading results
   - Plot grades and comments
   - Student files
   - Configuration settings
   - Your position in the grading workflow

### Manual Save
- Use the "Save Session Now" button in the Session tab anytime
- Useful before closing the app or switching tasks

---

## Comprehensive Default Tests

The app includes comprehensive built-in tests based on object types:

### Numeric Vectors

| Test | Description | Details |
|------|-------------|---------|
| `length` | Compares length of vectors | `length(obj) == length(.obj)` |
| `mean` | Compares mean values | Tolerance: 1e-6, handles NA |
| `sd` | Compares standard deviation | Tolerance: 1e-6, handles NA |
| `min` | Compares minimum values | Tolerance: 1e-6, handles NA |
| `max` | Compares maximum values | Tolerance: 1e-6, handles NA |
| `sum` | Compares sum of values | Tolerance: 1e-6, handles NA |
| `na_count` | Compares count of NA values | `sum(is.na(obj)) == sum(is.na(.obj))` |
| `identical` | Checks if all values match | Uses `all.equal()` with tolerance |

### Character Vectors

| Test | Description | Details |
|------|-------------|---------|
| `length` | Compares length of vectors | `length(obj) == length(.obj)` |
| `identical` | Checks exact match | Uses `identical()` |
| `unique_count` | Compares unique value count | `length(unique(obj)) == length(unique(.obj))` |
| `nchar_total` | Compares total character count | `sum(nchar(obj)) == sum(nchar(.obj))` |

### Logical Vectors

| Test | Description | Details |
|------|-------------|---------|
| `length` | Compares length of vectors | `length(obj) == length(.obj)` |
| `identical` | Checks exact match | Uses `identical()` |
| `sum_true` | Compares sum of TRUE values | `sum(obj, na.rm = TRUE) == sum(.obj, na.rm = TRUE)` |
| `na_count` | Compares count of NA values | `sum(is.na(obj)) == sum(is.na(.obj))` |

### Data Frames

| Test | Description | Details |
|------|-------------|---------|
| `nrow` | Compares number of rows | `nrow(obj) == nrow(.obj)` |
| `ncol` | Compares number of columns | `ncol(obj) == ncol(.obj)` |
| `colnames` | Checks if column names match | Uses `setequal()` |
| `coltypes` | Checks if column types match | Compares `sapply(obj, class)` |
| `column_mean_*` | Compares means of numeric columns | For each numeric column (up to 3) |
| `column_sd_*` | Compares SDs of numeric columns | For each numeric column |
| `factor_levels_*` | Compares factor levels | For each factor column |
| `correlation_matrix` | Compares correlation structure | Max correlation difference < 0.01 |

### Matrices

| Test | Description | Details |
|------|-------------|---------|
| `nrow` | Compares number of rows | `nrow(obj) == nrow(.obj)` |
| `ncol` | Compares number of columns | `ncol(obj) == ncol(.obj)` |
| `dim` | Compares dimensions | `identical(dim(obj), dim(.obj))` |
| `rownames` | Checks if row names match | Uses `identical()` |
| `colnames` | Checks if column names match | Uses `identical()` |
| `mean` | Compares overall mean | Tolerance: 1e-6 |
| `max_diff` | Maximum element difference | `max(abs(obj - .obj)) < 1e-6` |

### Lists

| Test | Description | Details |
|------|-------------|---------|
| `length` | Compares number of elements | `length(obj) == length(.obj)` |
| `names` | Checks if element names match | Uses `setequal()` |
| `element_*` | Tests each named element | Uses `all.equal()` for each (up to 3) |
| `element_class_*` | Checks element classes | For each named element |
| `element_length_*` | Checks element lengths | For atomic and list elements |
| `element_mean_*` | Checks element means | For numeric elements |

### Functions

| Test | Description | Details |
|------|-------------|---------|
| `num_args` | Compares number of arguments | `length(formals(obj)) == length(formals(.obj))` |
| `arg_names` | Checks if argument names match | Uses `setequal()` |
| `is_function` | Verifies it's a function | `is.function(obj)` |
| `body_length` | Compares body complexity | Info only - line count comparison |

### Factors

| Test | Description | Details |
|------|-------------|---------|
| `length` | Compares length | `length(obj) == length(.obj)` |
| `nlevels` | Compares number of levels | `nlevels(obj) == nlevels(.obj)` |
| `levels` | Checks if levels match | Uses `setequal()` |
| `level_order` | Checks if level order matches | Uses `identical()` |
| `frequencies` | Compares level frequencies | Compares `table()` output |

---

## How to Run

### From the app directory:

```r
library(shiny)
runApp("app")
```

### Or from any R session:

```r
library(shiny)
runApp("/path/to/autoharp/app")
```

## Usage Instructions

1. **Start the App**: Run the app using one of the methods above

2. **Resume or Start New Session** (Optional):
   - Go to the **Session** tab to load a previous session
   - Or start fresh by going directly to Tab 1

3. **Configure Output Directory**:
   - In Tab 1 or Tab 4, specify your desired output directory
   - All grading outputs, plots, and session files will be saved here
   - Defaults: `grading_output` (Tab 1) or `student_out` (Tab 4)

4. **Load Solution Template**: 
   - Upload an RMD file containing the correct solutions
   - This can be a plain RMD file without any autoharp-specific annotations
   - The app will execute the RMD and extract all objects created

5. **Specify Student Folder**:
   - Enter the full path to the folder containing student R/Rmd files
   - The app will find all `.R`, `.r`, `.Rmd`, and `.rmd` files

6. **Select Objects**:
   - After loading the solution, checkboxes will appear for each object
   - Select which objects should be tested against student submissions

7. **Run Grading**:
   - Click "Run Automated Grading" to process all student files
   - View results in the summary table and detailed test output
   - Session is automatically saved after completion

8. **Grade Plots (Tab 2)**:
   - Navigate through students using the selector or buttons
   - View each student's plots
   - Assign grades and comments
   - Click "Save Grade" to record the grade (auto-saves session)

9. **Generate Template (Tab 3)**:
   - Click "Generate Template" to create an autoharp-compatible solution
   - Preview the generated template with test chunks
   - Download the template for use with `render_one()`

10. **Download Results**:
    - Click "Download Results as CSV" to export all data
    - The CSV includes both automated test scores and manual plot grades

11. **Pause and Resume**:
    - Your session is automatically saved throughout the grading process
    - Close the app at any time
    - Return later and load your session from the Session tab to continue

## Generated Template Format

The generated template adds test chunks in the following format:


```{r test_01_X, autoharp.objs=c("X"), autoharp.scalars=c("X_length_check", "X_mean_check", "X_identical_check")}
# Tests for X (numeric)
# Length check
X_length_check <- length(X) == length(.X)

# Mean check (with tolerance)
X_mean_check <- isTRUE(all.equal(mean(X, na.rm = TRUE), mean(.X, na.rm = TRUE), tolerance = 1e-6))

# Identical values check
X_identical_check <- isTRUE(all.equal(X, .X))
```

This format is compatible with autoharp's `render_one()` function for automated grading.

## Technical Notes

- Student scripts are executed in isolated environments
- Errors during execution are caught and recorded
- Tests use tolerance of 1e-6 for numeric comparisons
- Plot collection works by saving figures to a dedicated `student_plots` directory during knitting
- Tab 4 uses the autoharp `populate_soln_env` and `render_one` functions for comprehensive grading
- **Sessions are saved as RDS files** containing all grading state, allowing you to pause and resume work
- **Output directories are configurable** for flexible organization of grading materials

## File Structure

```
app/
├── app.R              # Main Shiny application
├── www/helpers.R      # Test functions and utilities
└── README.md          # This file
```

**Generated directories during grading (configurable):**
```
<output_directory>/          # User-specified output directory
├── student_plots/           # Contains rendered plots from each student
│   ├── student_01/
│   │   ├── plot1.png
│   │   └── plot2.png
│   └── student_02/
│       └── plot1.png
├── sessions/                # Auto-saved session files
│   ├── session_20260123_120000.rds
│   └── session_20260123_130000.rds
├── [student_name].html      # Rendered output from render_one
├── [student_name].knit.md   # Knitted markdown files
└── render_one.log           # Log file from render_one
```

## Dependencies

The app uses base R functions and requires:
- shiny (for the UI)
- DT (for data tables)
- knitr (for knitting RMD files)
- rmarkdown (for rendering Rmd files)
- base64enc (for plot encoding)
- autoharp (for render_one and populate_soln_env functions)
- shinyFiles (optional but recommended, for enhanced folder selection UI - will gracefully degrade if not available)
- openxlsx (optional, for Excel export in Tab 4 - will prompt to install if not available)

These are already dependencies of the autoharp package, except for `shinyFiles` and `openxlsx` which are optional.

**Note:** This version no longer requires `shinydashboard` package. If you're upgrading from a previous version, you can remove the `shinydashboard` dependency from your environment.