Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor import #212

Draft
wants to merge 36 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
7b9e609
Enable test execution by calling "composer test"
abrain Nov 5, 2020
4e8708e
Switch to a more general approach to pages in the admin area
abrain Nov 5, 2020
ad7d4ab
Use the predefined mocks for translation functions
abrain Nov 6, 2020
755df8e
Define the menu slug on the class itself
abrain Nov 6, 2020
2e986a8
Test for wp-einsatz import source
abrain Nov 6, 2020
dc5bef2
Implement basic page loading
abrain Nov 7, 2020
b4cf944
Add a few simple tests for the CSV source
abrain Nov 7, 2020
5931551
Simplify how the menu slug for a page is set / retrieved
abrain Nov 7, 2020
dce8970
Use proper escaping
abrain Nov 7, 2020
eee34e7
Add new class for reading CSV files
abrain Nov 9, 2020
32055c1
Use proper objects for representing the import steps
abrain Nov 11, 2020
5b33e42
Carry over settings from step to step
abrain Nov 25, 2020
536c44e
Added switch for the actions and adjusted the file chooser
abrain Nov 30, 2020
e310906
Add the analysis step again
abrain Nov 30, 2020
0a36deb
Clean up a bit
abrain Dec 1, 2020
66e39f0
Make the CSV source use the new CsvReader class
abrain Dec 2, 2020
ba889ed
Fix the tests
abrain Dec 2, 2020
f31f51e
Generate and check the mapping
abrain Dec 2, 2020
650e8e2
Merge branch 'develop' into feature/refactor_import
abrain Jan 3, 2021
28a6127
Merge branch 'develop' into feature/refactor_import
abrain Jan 29, 2021
04ed3f3
Fix deprecated use of the implode function
abrain Jan 29, 2021
672c42a
Merge branch 'develop' into feature/refactor_import
abrain Feb 5, 2021
876482d
Merge branch 'develop' into feature/refactor_import
abrain Feb 6, 2021
12287fb
Make a tiny change, to trigger a new build of the PR and check the re…
abrain Feb 6, 2021
2a896a5
Merge branch 'develop' into feature/refactor_import
abrain Feb 6, 2021
fd38449
No need for bash
abrain Feb 6, 2021
255c8e0
Send the test report in the same script, or the env variable is no lo…
abrain Feb 6, 2021
b8f0f28
Merge branch 'develop' into feature/refactor_import
abrain Feb 11, 2021
4470e91
Merge branch 'develop' into feature/refactor_import
abrain Mar 21, 2021
db058dd
Merge branch 'develop' into feature/refactor_import
abrain Sep 25, 2021
2a34a6e
Merge branch 'develop' into feature/refactor_import
abrain Oct 14, 2021
141b648
Merge branch 'develop' into feature/refactor_import
abrain Aug 24, 2024
0f0fc2a
Fix composer.json
abrain Aug 24, 2024
4a3a736
Add missing translators comments
abrain Aug 24, 2024
2395f3e
Merge remote-tracking branch 'origin/develop' into feature/refactor_i…
abrain Aug 24, 2024
e61630e
Improve CSV reader error handling and tests
abrain Aug 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions patchwork.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"redefinable-internals": [
"feof",
"fgetcsv"
]
}
6 changes: 3 additions & 3 deletions src/Admin/Initializer.php
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
use abrain\Einsatzverwaltung\Core;
use abrain\Einsatzverwaltung\Data;
use abrain\Einsatzverwaltung\Export\Tool as ExportTool;
use abrain\Einsatzverwaltung\Import\Tool as ImportTool;
use abrain\Einsatzverwaltung\Import\Page as ImportPage;
use abrain\Einsatzverwaltung\Options;
use abrain\Einsatzverwaltung\PermalinkController;
use abrain\Einsatzverwaltung\Settings\MainPage;
Expand Down Expand Up @@ -57,8 +57,8 @@
add_action('admin_menu', array($mainPage, 'addToSettingsMenu'));
add_action('admin_init', array($mainPage, 'registerSettings'));

$importTool = new ImportTool($utilities, $data);
add_action('admin_menu', array($importTool, 'addToolToMenu'));
$importPage = new ImportPage();
add_action('admin_menu', array($importPage, 'registerAsToolPage'));

Check warning on line 61 in src/Admin/Initializer.php

View check run for this annotation

Codecov / codecov/patch

src/Admin/Initializer.php#L60-L61

Added lines #L60 - L61 were not covered by tests

$exportTool = new ExportTool();
add_action('admin_menu', array($exportTool, 'addToolToMenu'));
Expand Down
101 changes: 101 additions & 0 deletions src/AdminPage.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
<?php
namespace abrain\Einsatzverwaltung;

use function add_management_page;
use function esc_html;
use function printf;

/**
* Base class for pages in the admin area.
*
* @package abrain\Einsatzverwaltung
*/
abstract class AdminPage
{
/**
* @var string
*/
private $menuSlug;

/**
* @var string
*/
private $pageTitle;

/**
* @param string $pageTitle
* @param string $menuSlug
*/
public function __construct(string $pageTitle, string $menuSlug)

Check warning on line 29 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L29

Added line #L29 was not covered by tests
{
$this->pageTitle = $pageTitle;
$this->menuSlug = $menuSlug;

Check warning on line 32 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L31-L32

Added lines #L31 - L32 were not covered by tests
}

abstract protected function echoPageContent();

/**
* Prints an error message.
*
* @param string $message The message text.
*/
public function printError(string $message)

Check warning on line 42 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L42

Added line #L42 was not covered by tests
{
printf('<p class="notice notice-error">%s</p>', esc_html($message));

Check warning on line 44 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L44

Added line #L44 was not covered by tests
}

/**
* Prints an informational message.
*
* @param string $message The message text.
*/
public function printInfo(string $message)

Check warning on line 52 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L52

Added line #L52 was not covered by tests
{
printf('<p class="notice notice-info">%s</p>', esc_html($message));

Check warning on line 54 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L54

Added line #L54 was not covered by tests
}

/**
* Prints a success message.
*
* @param string $message The message text.
*/
public function printSuccess(string $message)

Check warning on line 62 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L62

Added line #L62 was not covered by tests
{
printf('<p class="notice notice-success">%s</p>', esc_html($message));

Check warning on line 64 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L64

Added line #L64 was not covered by tests
}

/**
* Prints a warning message.
*
* @param string $message The message text.
*/
public function printWarning(string $message)

Check warning on line 72 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L72

Added line #L72 was not covered by tests
{
printf('<p class="notice notice-warning">%s</p>', esc_html($message));

Check warning on line 74 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L74

Added line #L74 was not covered by tests
}

/**
* Registers this page under the Tools menu.
*/
public function registerAsToolPage()

Check warning on line 80 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L80

Added line #L80 was not covered by tests
{
add_management_page(
$this->pageTitle,
esc_html($this->pageTitle),
'manage_options',
$this->menuSlug,
array($this, 'render')

Check warning on line 87 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L82-L87

Added lines #L82 - L87 were not covered by tests
);
}

/**
* Generates the output of the page.
*/
public function render()

Check warning on line 94 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L94

Added line #L94 was not covered by tests
{
echo '<div class="wrap">';
printf("<h1>%s</h1>", esc_html($this->pageTitle));
$this->echoPageContent();
echo '</div>';

Check warning on line 99 in src/AdminPage.php

View check run for this annotation

Codecov / codecov/patch

src/AdminPage.php#L96-L99

Added lines #L96 - L99 were not covered by tests
}
}
11 changes: 11 additions & 0 deletions src/Exceptions/FileReadException.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
<?php
namespace abrain\Einsatzverwaltung\Exceptions;

/**
* Class FileReadException
* @package abrain\Einsatzverwaltung\Exceptions
*/
class FileReadException extends \Exception
{

}
13 changes: 13 additions & 0 deletions src/Exceptions/ImportCheckException.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<?php
namespace abrain\Einsatzverwaltung\Exceptions;

use Exception;

/**
* Class ImportCheckException
* @package abrain\Einsatzverwaltung\Exceptions
*/
class ImportCheckException extends Exception
{

}
129 changes: 129 additions & 0 deletions src/Import/CsvReader.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
<?php
namespace abrain\Einsatzverwaltung\Import;

use abrain\Einsatzverwaltung\Exceptions\FileReadException;
use function array_key_exists;
use function fclose;
use function feof;
use function fgetcsv;
use function fopen;
use function is_array;
use function sprintf;

/**
* Makes CSV files accessible line by line
* @package abrain\Einsatzverwaltung\Import
*/
class CsvReader
{
/**
* @var string
*/
private $delimiter;

/**
* @var string
*/
private $enclosure;

/**
* @var string
*/
private $filePath;

/**
* CsvReader constructor.
*
* @param string $filePath
* @param string $delimiter
* @param string $enclosure
*/
public function __construct(string $filePath, string $delimiter, string $enclosure)
{
$this->delimiter = $delimiter;
$this->enclosure = $enclosure;
$this->filePath = $filePath;
}

/**
* @param int $numLines How many lines to read.
* @param int[] $columns Array of 0-based column indices which should be returned. Empty array gets all columns.
* @param int $offset How many lines to skip before reading, default 0.
*
* @return string[][]
* @throws FileReadException
*/
public function getLines(int $numLines, array $columns = [], int $offset = 0): array
{
$handle = fopen($this->filePath, 'r');
if ($handle === false) {
// translators: 1: file path
$message = sprintf(__('Could not open file %s', 'einsatzverwaltung'), $this->filePath);
throw new FileReadException($message);
}

// If an offset is defines, some lines should be skipped
if ($offset > 0) {
// The CSV parsing has to be used here as well, as line breaks could appear inside field delimiters
$this->readLines($handle, $offset, []);
}

$lines = $this->readLines($handle, $numLines, $columns);

fclose($handle);
return $lines;
}

/**
* @param resource $handle
* @param int $numLines
* @param array $columns
*
* @return array
* @throws FileReadException
*/
private function readLines($handle, int $numLines, array $columns): array
{
$linesRead = 0;
$lines = array();
while ($numLines === 0 || $linesRead < $numLines) {
$line = fgetcsv($handle, 0, $this->delimiter, $this->enclosure);

// Error while reading, most likely EOF
if ($line === false) {
break;
}

$linesRead++;

// Empty line in the file, skip this
if ($line == [null]) {
continue;
}

// Return entire line when all columns have been requested
if (empty($columns)) {
$lines[] = $line;
continue;
}

// Return only the requested columns
$filteredLine = array();
foreach ($columns as $columnIndex) {
// If the line has less columns than expected, the value will be an empty string
$filteredLine[] = array_key_exists($columnIndex, $line) ? $line[$columnIndex] : '';
}
$lines[] = $filteredLine;
}

if (($numLines === 0 || $linesRead < $numLines) && feof($handle) === false) {
throw new FileReadException(sprintf(
// translators: 1: number of lines
_n('Reading was aborted after %d line', 'Reading was aborted after %d lines', $linesRead, 'einsatzverwaltung'),
$linesRead
));
}

return $lines;
}
}
Loading