|-handling

[marcelklehr]

No description provided.

[marcelklehr]

@michael-franke I believe neither magpie-base nor magpie-backend currently have code that handles submissions containing |-chars in items of |-delimited lists.

[michael-franke]

I have a vague memory that there might have been functionality to recognize and process (somehow) submissions with bars in magpie V2, maybe as part of the frontend wrangling the data before submission? I remember discussing some rule like this: if N is the maximum number of “|” occurring in direct sequence (e.g., “||||” -> N=4) in the input to be demarcated with “|”, then use N+1 bars (“|||||”) as demarcator. That’s hideous in worst-case scenarios, but should work alright in practice. But, of course, better solutions exist.

…

On 19 Apr 2022, at 17:41, Marcel Klehr ***@***.***> wrote: @michael-franke I believe neither magpie-base nor magpie-backend currently have code that handles submissions containing |-chars in items of |-delimited lists. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[x-ji]

FYI, on the backend what currently happens is that a JSON array will be printed in the output CSV with "|" as the separator. I guess it will indeed be some processing on the frontend at submission time that figures out if the response is supposed to be submitted as an array.

[marcelklehr]

I remember discussing some rule like this: if N is the maximum number of “|” occurring in direct sequence (e.g., “||||” -> N=4) in the input to be demarcated with “|”, then use N+1 bars (“|||||”) as demarcator. That’s hideous in worst-case scenarios, but should work alright in practice.

should this then be used on the whole dataset or only in the specific rows where the | occurs?

[michael-franke]

Only where we insert | due to array conversion.

…

On Apr 20, 2022, at 19:25, Marcel Klehr ***@***.***> wrote:  I remember discussing some rule like this: if N is the maximum number of “|” occurring in direct sequence (e.g., “||||” -> N=4) in the input to be demarcated with “|”, then use N+1 bars (“|||||”) as demarcator. That’s hideous in worst-case scenarios, but should work alright in practice. should this then be used on the whole dataset? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[marcelklehr]

Only where we insert | due to array conversion.

Usually array conversion will have to happen in every row though, right?

e.g. for this dataset...

a,b
['a text', 'another text'], 3
['a text with |-chars', 'another text'], 4

do we output...

a,b
a text|another text, 3
a text with |-chars||another text, 4

...or...

a,b
a text||another text, 3
a text with |-chars||another text, 4

[michael-franke]

Hm, yes, and when we insert multiple |s in this way for one participant, all previous submissions will still have a single |, making this really painful in later data analysis. It would be better to replace all occurrences of “|” in array inputs with some dummy like “RPLCD-VRTCL-BAR” or some such (unlikely monster).

…

On 20 Apr 2022, at 19:42, Marcel Klehr ***@***.***> wrote: Only where we insert | due to array conversion. Usually array conversion will have to happen in every row though, right? e.g. for this dataset... a,b ['a text', 'another text'], 3 ['a text with |-chars', 'another text'], 4 do we output... a,b a text|another text, 3 a text with |-chars||another text, 4 ...or... a,b a text||another text, 3 a text with |-chars||another text, 4 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.