Discussion re OnlineCLScenario

[HamedHemati]

Hi everyone!

I've been recently working with OnlineCLScenario, which was recently added to Avalanche, to implement an online strategy. I have some comments/suggestions/questions on that.

Lazy generator: In general, the lazy stream generator is pretty fast and I think we can say the performance issue that we previously had is solved.

Bugs: A minor bug would return the main experience instead of sub-experiences when creating a lazy stream. I've fixed it (+some other changes) in a local branch which I will push.

Attributes: I've also made the attributes accessible in train mode. This was needed for detecting task switches (for example is_first_subexp in model adaptation) for some online supervised classification tasks.

Limitations:
Here are some limitations that I found with the current version:

1. To create an online stream, you need to provide OnlineCLScenario with an existing stream, which means it takes all experiences in that stream and creates a lazy stream that cannot be sliced. The problem here is that we won't able to call the train function separately for each sub-stream (belonging to each original experience). This can make eval tricky as we might want to evaluate after each "original" experience.

Therefore to have something like:

for exp in original_train_stream.experience:
     online_sub_stream = OnlineCLStream([exp])
     strategy.train(online_sub_stream)
     strategy.eval(test_stream)

We need to change OnlineCLStream such that it will (also) accept a list of experiences instead of a stream.

2. The second thing is the stream names after creating an online stream. As discussed previously, OnlineCLScenario creates a dictionary of streams by adding the prefix original_ to the main train and test streams and calling the online stream train_stream. This causes errors when trying to iterate through the original streams via the online scenario due to name discrepancy. We can instead call the online stream online_train_stream and keep the original streams with the same names.

I would appreciate your comments on the current limitations (if I'm correct and not missing something).
Thanks!

[AntonioCarta]

Thanks for the work @HamedHemati. You caught a lot of bugs. If you can, try to add a small test for them.

Attributes: I've also made the attributes accessible in train mode. This was needed for detecting task switches (for example is_first_subexp in model adaptation) for some online supervised classification tasks.

This should be a parameter, with False as the default value. And in particular, remember that this:

for example is_first_subexp in model adaptation

is not possible in many OCL scenarios.

The problem here is that we won't able to call the train function separately for each sub-stream (belonging to each original experience). This can make eval tricky as we might want to evaluate after each "original" experience.

Is this a real problem? most of the times you want to eval on the entire stream. I guess we can add it, but it's low priority. Task switches can still be detected from the full stream.

The second thing is the stream names after creating an online stream. As discussed previously, OnlineCLScenario creates a dictionary of streams by adding the prefix original_ to the main train and test streams and calling the online stream train_stream. This causes errors when trying to iterate through the original streams via the online scenario due to name discrepancy. We can instead call the online stream online_train_stream and keep the original streams with the same names.

This looks like a bug. The original streams should be left untouched. OCL should make a copy and change the name of the copy.

[HamedHemati]

Hey @AntonioCarta and others!

I’m implementing the OnlineSupervisedTemplate with the recent changes and decided to go with the option that you would sub-class OnlineSupervisedTemplate from BaseOnlineSGDTemplate instead of BaseSGD to keep things cleaner. The online BaseOnlineSGD is similar to OnlineSGD with the difference that it now has (after_/before_)training_pass instead of “epochs” and the PeriodicEval also needs to change correspondingly. I also noticed that the Clock needs an additional counter for the number of passes I’ve added. However, for the Plugins, I would like to know your opinion on whether it would be better to keep the core plugins and only add the (after/before) training passes to the SGDPlugin or to create a separate plugin for OnlineSGD and OnlineSupervised. The second option would make user implementations more structured in terms of strategy type, but there will be many callback replicates. What do you think?

[AntonioCarta]

This is the problem with renaming stuff. I see two options:

• (option 1) I see why epochs is not the correct term, but can we use iterations instead? Basically, for each experience you can do one or more iterations (passes). As long as the doc is clear it should be ok.

• (option 2) If you really want to rename things, you could also add an alias, something like:

def num_passes(self):
    return self.num_num_epochs

and old stuff will still work.

Personally, I think option 1 is still easy to understand while being easier to implement and maintain. Option 2 is ok, but it adds some callbacks and removes some other, and this in general makes things harder to reason about.

With option 1 we just need to add to the PeriodicEval the option to count iterations instead of epochs.

[HamedHemati]

@AntonioCarta Agreed. We can wrap the code inside train_pass (originally train_epoch) with a for loop like this:

for self.pass in range(self.n_passes):
     ...
     

This way, the before_iteration and after_iteration callbacks do the work for before_pass and after_pass so no additional callbacks would be needed, and the user can access the current pass using self.pass.

I've also changed the names of the streams for OnlineCLScenario as below:

• online_train_stream
• train_stream
• test_stream

Now there is no naming discrepancy, and I think having the prefix online_ makes it easier to follow.

[AntonioCarta]

I've also changed the names of the streams for OnlineCLScenario as below:

* online_train_stream

* train_stream

* test_stream

Now there is no naming discrepancy, and I think having the prefix online_ makes it easier to follow.

Isn't this a bit confusing? If I use ocl_benchmark.train_stream I expect an online stream instead of the original one. I fear this may result in bugs because users end up training on the wrong stream.

[HamedHemati]

Isn't this a bit confusing? If I use ocl_benchmark.train_stream I expect an online stream instead of the original one. I fear this may result in bugs because users end up training on the wrong stream.

I thought that keeping train_stream and test_stream as they are, and naming the online version online_train_stream, is more like : this is an online version of the train_stream.

I assume that everyone using OnlineCLStream would read the documentation and see the difference between the train_tream and online_train_stream when calling strategy.train() in an example.

[AntonioCarta]

I assume that everyone using OnlineCLStream would read the documentation and see the difference between the train_tream and online_train_stream when calling strategy.train() in an example.

It makes sense but I find it confusing. Names are part of the documentation so they should be descriptive. Personally, I would never check the documentation to confirm that train_stream is really the stream I should use for training.