Skip to content

Commit

Permalink
Merge pull request #52 from Imageomics/update-readme
Browse files Browse the repository at this point in the history
updated readme descriptions and links
  • Loading branch information
egrace479 authored Dec 20, 2024
2 parents c4545eb + aa93378 commit 5f4740e
Showing 1 changed file with 7 additions and 7 deletions.
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,15 +59,16 @@ frame-by-frame behavior annotation.

#### To create mini-scenes, we first must perform the detection step, by drawing bounding boxes around each animal in frame.

See [data/mini_scenes](https://huggingface.co/imageomics/x3d-kabr-kinetics/tree/main/data/mini_scenes) in HuggingFace for example mini-scenes.
See [data/mini_scenes](https://huggingface.co/imageomics/x3d-kabr-kinetics/tree/main/data/mini_scenes) on Hugging Face for example mini-scenes.

### Step 2A: Perform detections to create tracks
#### Option 1: Manual detections in CVAT
![](images/cvat_annotation_tool.png)
**Figure 4:** Simplified CVAT annotation tool interface
<br>

Upload your raw videos to [CVAT](https://www.cvat.ai/) and perform the detections by drawing bounding boxes manually. This can be quite consuming, but has the advantage of generating highly accurate tracks.
Depending on the resolution of your raw video, you may encounter out of space issues with CVAT. You can use [helper_scripts/downgrade.sh](helper_scripts/downgrade.sh) to reduce the size of your videos.
Depending on the resolution of your raw video, you may encounter out of space issues with CVAT. You can use [downgrade.sh](helper_scripts/downgrade.sh) to reduce the size of your videos.


#### Option 2: Automatic detections with YOLO
Expand All @@ -93,10 +94,10 @@ tracks_extractor --video path_to_videos --annotation path_to_annotations [--trac
```

## Step 3: Label mini-scenes with behavior
You can use the [KABR model](https://huggingface.co/imageomics/x3d-kabr-kinetics) to label the mini-scenes with behavior. See the [ethogram](ethogram) folder for the list of behaviors used to label the zebra videos.
You can use the [KABR model](https://huggingface.co/imageomics/x3d-kabr-kinetics) on Hugging Face to label the mini-scenes with behavior. See the [ethogram](ethogram) folder for the list of behaviors used to label the zebra videos.


To use the [KABR model](https://huggingface.co/imageomics/x3d-kabr-kinetics), download `checkpoint_epoch_00075.pyth.zip`, unzip `checkpoint_epoch_00075.pyth`, and install [SlowFast](https://github.com/facebookresearch/SlowFast). Then run [miniscene2behavior.py](miniscene2behavior.py).
To use the [KABR model](https://huggingface.co/imageomics/x3d-kabr-kinetics), download `checkpoint_epoch_00075.pyth.zip` from Hugging Face, unzip `checkpoint_epoch_00075.pyth`, and install [SlowFast](https://github.com/facebookresearch/SlowFast). Then run [miniscene2behavior.py](miniscene2behavior.py).

Label the mini-scenes:
```
Expand All @@ -109,12 +110,12 @@ miniscene2behavior [--config path_to_config] --checkpoint path_to_checkpoint [--
- If `gpu_num` is 0, the model will use CPU. Using at least 1 GPU greatly increases inference speed. If you're using OSC, you can request a node with one GPU by running `sbatch -N 1 --gpus-per-node 1 -A [account] --time=[minutes] [bash script]`.
- mini-scenes are clipped videos focused on individual animals and video is the raw video file from which mini-scenes have been extracted.

See [these csv files](https://huggingface.co/imageomics/x3d-kabr-kinetics/tree/main/data/mini_scene_behavior_annotations) in HuggingFace for examples of annotated mini-scene outputs.
See [these csv files](https://huggingface.co/imageomics/x3d-kabr-kinetics/tree/main/data/mini_scene_behavior_annotations) in Hugging Face for examples of annotated mini-scene outputs.


## Step 4: Calculate time budgets

See [time budgets example](/examples/time_budget.ipynb) to code to create these visualizations.
See the [time budgets notebook](/notebooks/time_budget.ipynb) for the code to create these visualizations.


<!-- Add two photos side by side -->
Expand All @@ -128,7 +129,6 @@ See [time budgets example](/examples/time_budget.ipynb) to code to create these

![](images/timebudget.png)
<br>

**Figure 6:** Overall time budget for duration of 10 minute observation


Expand Down

0 comments on commit 5f4740e

Please sign in to comment.