Skip to content

3.1.0-preview2

Pre-release
Pre-release
Compare
Choose a tag to compare
@znorman-harris znorman-harris released this 18 Jul 22:09

Further improve non-full-parse parsing to reduce time and memory usage. Here is before/after performance information:

const before = {
  lines: 2252881,
  app_platform: 'win32',
  app_arch: 'x64',
  app_cpus: 20,
  app_ram: 32,
  app_ram_used: 2.5,
  num_workers: 6,
  parse_time: 13.49,
  parse_rate: 167015,
  num_pro: 7816,
  num_save: 830,
  num_idl_task: 15,
  num_envi_task: 0,
  num_idl_json: 0,
};

const after = {
  lines: 2252881,
  app_platform: 'win32',
  app_arch: 'x64',
  app_cpus: 20,
  app_ram: 32,
  app_ram_used: 1.75,
  num_workers: 6,
  parse_time: 7.86,
  parse_rate: 286567,
  num_pro: 7816,
  num_save: 830,
  num_idl_task: 15,
  num_envi_task: 0,
  num_idl_json: 0,
};