Add ndjson logging for training #10970
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
👋 Hello @akx, thank you for submitting a YOLOv5 🚀 PR! To allow your work to be integrated as seamlessly as possible, we advise you to:
- ✅ Verify your PR is up-to-date with
ultralytics/yolov5masterbranch. If your PR is behind you can update your code by clicking the 'Update branch' button or by runninggit pullandgit merge masterlocally. - ✅ Verify all YOLOv5 Continuous Integration (CI) checks are passing.
- ✅ Reduce changes to the absolute minimum required for your bug fix or feature addition. "It is not daily increase but daily decrease, hack away the unessential. The closer to the source, the less wastage there is." — Bruce Lee
|
@glenn-jocher Mind taking a look at this too? 😄 |
akx
force-pushed
the
ndjson-logging
branch
2 times, most recently
from
48332f0 to
fc08e35
Compare
|
👋 Hello there! We wanted to let you know that we've decided to close this pull request due to inactivity. We appreciate the effort you put into contributing to our project, but unfortunately, not all contributions are suitable or aligned with our product roadmap. We hope you understand our decision, and please don't let it discourage you from contributing to open source projects in the future. We value all of our community members and their contributions, and we encourage you to keep exploring new projects and ways to get involved. For additional resources and information, please see the links below:
Thank you for your contributions to YOLO 🚀 and Vision AI ⭐ |
|
@glenn-jocher Still hoping this could get reviewed... |
|
@akx hi there, Thank you for your patience and for bringing this to our attention. We apologize for the delay in reviewing your contribution. Our team has been working diligently to address all the open pull requests and we appreciate your understanding. While we try our best to review all contributions, we receive a high volume of pull requests and sometimes it takes longer than expected to review each one. We truly value your contribution and will do our best to review it as soon as possible. Thank you for your understanding and your continued support of YOLOv5. |
|
@glenn-jocher Thank you for your comment! If everyone else on the team is currently busy, could you perhaps review this? It's not a very long PR, and it's pretty straightforward in what it does. 😃 |
|
@akx absolutely, I'd be happy to help! I'll take a look and get back to you as soon as possible. Thank you for your understanding and for your contribution! |
|
@glenn-jocher I'm looking forward to that! After all, this pull request has been open for over 9 months now! 🙂 For your convenience, I also rebased this just now! |
|
Thank you for the heads up, @akx. I'll prioritize reviewing this rebased pull request to ensure it's addressed promptly. We appreciate your patience and understanding as we work through the backlog of contributions. Thank you for your continued support! |
|
Hello, @glenn-jocher! I understand you're probably very busy with your review queue and all, but it's been nearly a fortnight since I've heard from you. I rebased this once more for your convenience. Looking forward to the review! |
|
Thank you for your understanding, @akx! I'll make sure to review the latest rebase as soon as possible. I appreciate your patience and your dedication to this pull request. Thank you for your continued support! |
|
@glenn-jocher Hello! It's been a couple of weeks since your last comment – I understand you're probably very busy, but could you please either review this PR, or close it if you deem it is unwelcome here? 😃 |
|
Absolutely, @akx! Thanks for your continued patience. I will prioritize reviewing this PR to provide you with feedback as soon as possible. Your understanding and dedication are truly appreciated! |
This adds support for NDJSON (newline-delimited JSON) metrics logging, for both console (stdout) output and a file (like the current CSV file). NDJSON can be easily grepped from the output and/or parsed with e.g. `jq`. The feature is enabled with the `--ndjson-console` and `--ndjson-file` switches to `train.py`.
for more information, see https://pre-commit.ci
|
Hello, @glenn-jocher! It's been another three weeks with no sign of a review, and this PR is closing in on its first birthday! What do you think, should we throw it a party? A review party?! 😁 Happy new year! |
|
@akx PR merged! Thank you for your contributions. |
* Add ndjson logging for training This adds support for NDJSON (newline-delimited JSON) metrics logging, for both console (stdout) output and a file (like the current CSV file). NDJSON can be easily grepped from the output and/or parsed with e.g. `jq`. The feature is enabled with the `--ndjson-console` and `--ndjson-file` switches to `train.py`. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This adds support for NDJSON (newline-delimited JSON) metrics logging, for both console (stdout) output and a file (like the current CSV file).
NDJSON can be easily grepped from the output and/or parsed with e.g.
jq.An example JSON line is e.g.
{"epoch": 0, "train/box_loss": 0.047209735, "train/obj_loss": 0.07622885704040527, "train/cls_loss": 0.03480850160121918, "metrics/precision": 0.7328301877186243, "metrics/recall": 0.6168343308792175, "metrics/mAP_0.5": 0.7002523536111919, "metrics/mAP_0.5:0.95": 0.45269375994535394, "val/box_loss": 0.03693515807390213, "val/obj_loss": 0.04277978464961052, "val/cls_loss": 0.009943380020558834, "x/lr0": 0.07046875, "x/lr1": 0.00328125 "x/lr2": 0.00328125}The feature is enabled with the
--ndjson-consoleand--ndjson-fileswitches totrain.py.🛠️ PR Summary
Made with ❤️ by Ultralytics Actions
🌟 Summary
Enhanced logging capabilities in YOLOv5 with support for NDJSON (Newline Delimited JSON).
📊 Key Changes
--ndjson-consoleand--ndjson-filefor enabling NDJSON logging.Loggersclass to support NDJSON logging, with helper functions for JSON formatting.train.py) to initialize logging based on user preferences.🎯 Purpose & Impact