Dev meeting August 2023

[SimonHeybrock]

Agenda

• Retrospective
• Backlog grooming
• New developer starting -> plan
• Problems with gathering detailed ESS requirements

Retrospective

What went well

• Europython was great, let's go again next year
• Whiteboard discussions about software design are good
• We finally have a framework for building workflows
• Common effort to automate things in CI and across repos (copier)
• tof was well received
• CI more stable than it has ever been
• Enjoying creating material for the summer school
• Successful start for Sunyoung

What could be improved

• Too much time spent on interviews: can we do things any better?

  • Not everyone goes to all interviews? -> but it is important that everyone sees the technical interview
  • Do screening differently? Can we just have a small code review exercise?

• Messy PRs in beamlime when project was young. Why was that?

  • Could we do a prototype where the dev is free to do what they want, no reviews, and once design is approved by the team we start again in a new project an implement cleanly with reviews?
  • Did we try to add too many features to start with for hypothetical problems?
  • Moving target PRs were difficult to review: maybe open draft PRs and wait a day or so before marking them as ready?

• Confliuence did not work for gathering requirements and information

  • use something else? JIRA? Gitlab?
  • not sure the technology is the thing getting in the way
  • information on Kafka was scattered all over confluence

• Taking breaks together:

  • say "coffee breaks in 15 min"

• Use Kanban board more consistently:

  • Always put what we are working on, even if it is just a note

• We don't have a good way of setting tasks that represent a sequence of tasks, e.g. 1. release scipp, 2. update/release scippneutron, 3. update/release ess

  • maybe blocked tasks on the board?

• Motivation has gone down in doing proper code reviews, and as a result reviews have become quite superficial, especially when reviewing a PR in a repo that we are not so familiar with

  • if you are not familiar, you have to really dive into the project to understand it, and that takes a lot of time
  • we find ourselves skimming over changes, they look ok, but we don't pay too much intention
  • would doing more walkthroughs help?
    • when sitting in a walkthrough, I don't have time to think about the fine details, and still have to dive back into it
  • have small sprints days where we all work on Scitacean or beamlime?
  • or pair programming where the package author is watching and the unfamiliar person is typing?
  • once per month: one day sprint on one project

• Feel that sometimes we are falling behind on software architecture, that we just wrote a giant class with everything in it, or very long functions

  • how to we keep up to date with design patterns? (video semainars?)
  • sometimes it's just good to go back to basics, make things modular, better structured/smaller classes
    • is there anything concrete we can do? Youtube channel Arjan (?) are excellent refreshers

• Debugging the Scipp C++ is not fun (cannot display variable values)

  • Is there a way to move almost everything into python without losing performance?
  • write a rust version of scipp?

• Feel isolated with Scitacean:

  • not much reviewing from the swap side
  • not many contributions from other scipp members as well
  • suggested monthly sprints would help

• Deployment and especially docs builds not working super well

  • many problems are probably to do with keeping the different versions on the docs. suggest removing that are keeping archives of old docs. Do gh release assets count towards repo size?
  • py versions in CI: only running on lowest supported, should we also run on latest python? Have a nightly build

• Getting ECDC to do things differently is difficult:

  • changes to info in NeXus are being turned down (logical layout of detectors)
  • nexus geometries are inefficiently stored and not version controlled
  • no cooperation/collaboration on nightly testing
  • what can be done?

• Moving to DTU

  • Should we do some sort of risk analysis and what mitigations we can come up with if a team member leaves?

• We still don't have digital twins, at least for the first instruments

  • there is something for dream, but not easily/publicly available
  • there are detector tests for loki
  • odin?
  • how do we push for this?

All hope is lost

• Too much effort migrating Scitacean to new Scicat backend

  • However, now that this is all done, we are in a better place than before, with better tests, better generator, and should be easier if there ever is another change of backend
  • It would probably have been easier if starting with a small prototype, instead of doing too much in one go?
  • Lower level tests would also have been good because we only had high level tests which needed the whole thing to work before we could run the tests?
  • Some similar issues in Plopp tests, where many tests depend on some internal classes. Could mocking help here?

• Nightly tests are dead

To Do

• WFM work made no progress in months