Skip to content

Commit

Permalink
3.3.1
Browse files Browse the repository at this point in the history
  • Loading branch information
hyunwoongko committed Oct 29, 2021
1 parent 55174cd commit e4aa1b7
Show file tree
Hide file tree
Showing 4 changed files with 13 additions and 9 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Kss: A Toolkit for Korean sentence segmentation
<img alt="latest version" src="https://img.shields.io/badge/latest%20version-3.3.0-blue.svg"/> <a href="https://github.com/hyunwoongko/kss/blob/master/LICENSE"><img alt="BSD 3-Clause" src="https://img.shields.io/badge/license-BSD%203%20Clause-blue.svg"/></a>
<img alt="latest version" src="https://img.shields.io/badge/latest%20version-3.3.1-blue.svg"/> <a href="https://github.com/hyunwoongko/kss/blob/master/LICENSE"><img alt="BSD 3-Clause" src="https://img.shields.io/badge/license-BSD%203%20Clause-blue.svg"/></a>
<a href="https://github.com/hyunwoongko/kss/issues"><img alt="Issues" src="https://img.shields.io/github/issues/hyunwoongko/kss"/></a>

This repository contains the source code of Kss, a representative Korean sentence segmentation toolkit. I also conduct ongoing research about Korean sentence segmentation algorithms and report the results to this repository.
Expand Down
5 changes: 4 additions & 1 deletion docs/UPDATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,4 +179,7 @@

>>> split_sentences(text, backend="none")
['부디 만수무강 하옵소서 천천히 가세용~', '너 밥을 먹는구나 응 맞아 난 근데 어제 이사했음 그랬구나 이제 마지막임 응응'
```
```

#### Kss 3.3.1
- Disable multiprocessing if length of input list is 1.
13 changes: 7 additions & 6 deletions kss/kss.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,11 +98,6 @@ def split_sentences(
num_workers = get_num_workers(num_workers)
results = []

if num_workers in [0, 1]:
pool = None
else:
pool = Pool(max_workers=num_workers)

max_recover_step = length_constraints(
text,
max_recover_length,
Expand All @@ -118,6 +113,11 @@ def split_sentences(
else:
_text = text

if num_workers in [0, 1]:
pool = None
else:
pool = Pool(max_workers=num_workers)

if pool:
preprocessed_list = pool.map(build_preprocessed_list, _text)
else:
Expand All @@ -137,7 +137,8 @@ def split_sentences(

mp_postprocessing.append(out)

if pool:
print(mp_input_texts)
if pool and len(mp_input_texts) >= 2:
results += pool.map(
partial(
_split_sentences,
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ def read_file(filename, cb):

setup(
name="kss",
version="3.3.0",
version="3.3.1",
author="Hyunwoong Ko",
author_email="[email protected]",
url="https://github.com/hyunwoongko/kss",
Expand Down

0 comments on commit e4aa1b7

Please sign in to comment.