Introduce PTS for video

[sscobici]

There is a need to introduce PTS to the packet structure for proper video handling. Currently, the packet includes TS, which corresponds to DTS.

Options:

1. Add PTS and populate it only for video packets.
2. Add PTS and populate it for both audio and video packets, with PTS equal to TS for audio.
3. Add PTS and rename TS to DTS for clarity.

I’d like to work on a PR and would appreciate guidance on the desired target state. It’s possible to progress incrementally from option 1 to 2, and then to 3.

Option 1: Requires a small PR and can include separate commits for MKV and MP4.

Option 2: Provides no benefit for audio processing.

Option 3: Clarifies that TS is actually DTS for video packets, but requires changes in many existing applications. If we want this clarity, it’s better to implement it in version 0.6, which is already incompatible with 0.5.

@pdeljanov Please provide your opinion.

[pdeljanov]

Updating Packet is one of the last major breaking items I had on the to-do list for 0.6, but haven't gotten to it yet as I'm still working through metadata changes. So thanks for being willing to take this on.

Historically, the timestamp in Packet was named pts. I believe I changed it to ts in version 0.5, years ago now. I suppose it could've been called dts since for audio PTS and DTS are equal, however, PTS makes more sense to me since that's the presentation time for the audio.

Seems like we'll need to have both PTS and DTS fields, and ditch the single TS field. This'll be a breaking change, but 0.6 already breaks a lot, and at least no one will accidentally use the wrong timestamp because we repurposed the old field. For audio packets, I think both the PTS and DTS fields should be populated with the same value. Selectively populating PTS or DTS means making one an Option, which only wastes space (an extra 8 bytes).

This is, of course, if we want to have a Packet that is fully generic for audio, video, and subtitle packets. Another option would be for Packet to carry non codec-type specific information: track ID, PTS, duration, start/end trim, and data buffer. Then have a codec-type specific enum that carries things like the DTS for video, etc etc. I'm not sure if this is worthwhile, though. If it's just DTS, then then former solution is likely enough.

---

On the topic of updates to Packet. I have the two other open questions for it:

1. Does the packet need to carry multiple data planes instead of a single buffer?
2. Does the packet need the capability to carry side data?

---

One last thing to mention, I will be landing a major change to the MKV reader fairly soon, so it may be worthwhile to prototype this only on MP4 for now.