Refactor internal schema function api

Can enable running individual tests or supllying any other arguments.
Will remove the default argument `--cov`.

.woodpecker.yml

@@ -1,86 +0,0 @@
-branches: main
-
-pipeline:
-  code_lint:
-    image: python
-    commands:
-      - pip install poetry
-      - poetry install
-      - pip install black
-      - echo "----------------- running lint ------------------"
-      - python --version && poetry --version && black --version
-      - poetry run black .
-
-  unit_tests:
-    image: thekevjames/nox
-    commands:
-      - pip install poetry
-      - poetry install
-      - echo "----------------- running tests ------------------"
-      - python --version && poetry --version && nox --version
-      - poetry run nox
-
-  static_analysis:
-    image: python
-    commands:
-      - pip install poetry
-      - poetry install
-      - pip install mypy
-      - echo "----------------- running analysis ------------------"
-      - python --version && poetry --version && mypy --version
-      - poetry run mypy .
-
-  build_dist:
-    image: python
-    commands:
-      - pip install poetry
-      - poetry install
-      - echo "----------------- running analysis ------------------"
-      - python --version && poetry --version
-      - poetry build
-    when:
-      branch: main
-
-  release_prep:
-    image: python
-    commands:
-      - echo "----------------- preparing release ------------------"
-      - python tools/extract-changelog.py
-
-  gitea_release:
-    image: plugins/gitea-release
-    settings:
-      api_key: 
-        from_secret: gitea_release_token
-      base_url: https://git.martyoeh.me
-      files: dist/*
-      title: NEWEST_VERSION.md
-      note: NEWEST_CHANGES.md
-    when:
-      event: tag
-      tag: v*
-
-  pypi_release:
-    image: python
-    commands:
-      - pip install poetry
-      - poetry install
-      - echo "----------------- publishing to pypi ------------------"
-      - poetry publish --username "$PYPI_USERNAME" --password "$PYPI_PASSWORD"
-    when:
-      event: tag
-      tag: v*
-
-  notify_matrix:
-    image: plugins/matrix
-    settings:
-      homeserver: https://matrix.org
-      roomid:
-        from_secret: matrix_roomid
-      userid:
-        from_secret: matrix_userid
-      accesstoken:
-        from_secret: matrix_token
-    when:
-      status: [ success, failure ]
-

CHANGELOG.md

@@ -5,19 +5,13 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project tries to adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-<!-- ## [Unreleased] -->
-
-## [0.4.1] - 2022-01-05
-
-### Added
-
-* Added pypi release publication
+## [Unreleased]
 
 ### Changed
 
 * Compatible with Python stretching back to version 3.7
 
-## [0.4.0] - 2021-12-06
+## [0.4] - 2021-12-06
 
 ### Added 
 

README.md

@@ -1,34 +1,15 @@
-# habitmove
+# habit-migrate
 
-Takes habit in one habit-tracking application and transforms them ready to use for another.
+Can take an export of nomie habits in json format and convert it to be importable in Loop Habit Tracker.
 
-Currently can take an export of nomie habits in json format and convert it to be importable in Loop Habit Tracker.
-Plans for reverse migration are on the roadmap, and ultimately this tool ideally understands more and more habit formats to prevent app lock-in.
-
-Confirmed working for nomie version 5.6.4 and Loop Habit Tracker version 2.0.2 and 2.0.3. 
+Confirmed working for nomie version 5.6.4 and Loop Habit Tracker version 2.0.2. 
 Presumably works for other nomie 5.x versions and other Loop 2.x versions as well, 
 but that is untested.
 
-## Installation
-
-Installation can be accomplished through *pip*:
-
-```bash
-pip install habitmove
-```
-
-Requirements:
-
-`habitmove` requires at least Python 3.7. 
-It has only been tested on GNU/Linux (amd64) though it should work on other platforms.
-
 ## Usage 
 
-Run as a cli utility `habitmove` currently takes a single argument: the nomie database `.json` file to import habits from.
-
-Invoked like: `habitmove nomie-export.json`.
-
-The output as a Loop Habit Tracker database will be written to `output.db` in the present working directory.
+Run as a commandline utility habit migrate currently takes a single argument, the nomie database `.json` file.
+The output as importable Loop Habit Tracker database will be written to `output.db` in present working directory.
 
 Can also take an existing Loop Habit database (exported from the application), 
 and add the nomie exported habits and checkmarks to it.
@@ -37,28 +18,34 @@ it will not (should not™️) overwrite anything.
 If there are any duplicated habits however, 
 it will add duplications of the existing repetitions into the database.
 
-## Development
+Invoked like: `python run.py nomie-export.json`.
+Note, however, that -- until a packaged version is released -- you will need to have some packages in your environment.
+If you wish to run it un-packaged, install [poetry](https://python-poetry.org/) and let it do all dependency management by doing:
 
-To enable easy development on the app, 
-install [poetry](https://python-poetry.org/) and let it do all dependency management for you by doing:
-
-```bash
+```
 poetry install
 poetry run habitmove <nomie-json>
 ```
 
-To see a set up more closely resembling the final cli environment, 
-with its libraries loaded as environmental dependencies enter the poetry shell:
+In the future there might be an easier road to using this package but that's the way it is for now.
 
-```bash
-poetry shell 
-```
+The package can also be used as a library to load nomie data
+or move data into Loop Habit Tracker.
 
-The package can eventually also be used as a library to load nomie data to work with in Python,
-or to move data into Loop Habit Tracker.
-Take a look at the `Parser` and `Transformer` interfaces respectively.
 
-To run tests for the app, simply invoke `pytest` through `poetry run pytest` or from within the `poetry shell`.
-To run larger scale test automation, make sure you habe nox installed and run `poetry run ` or again through the shell.
+## Roadmap
+
+* [ ] clean up README
+* [x] begin adding tests
+    * [x] add some unit tests for various functions
+    * [ ] and at least an integration test for the stable database (loop-2021-12-02.db or equivalent)
+    * [ ] test most components
+* [ ] abstract migration target away from loop
+* [ ] abstract import away from nomie
+* [ ] abstract importer/migrator themselves to work with other targets
+* [ ] allow migration to/from nomie/loop
+* [ ] cmdline options for
+    * [ ] ignoring/adding/overwriting duplicated timestamps
+    * [ ] output file
+    * [ ] from/to
 
-You can exclude integration tests that take longer and inspect the complete database output of the program through the parameters `-m "not e2e"` for both `pytest` and `nox` (which also does it automatically).

pyproject.toml

@@ -1,10 +1,7 @@
 [tool.poetry]
 name = "habitmove"
-version = "0.4.1"
+version = "0.4.0"
 description = "migrate nomie data to loop habits tracker"
-license="GPL-3.0-only"
-readme="README.md"
-repository="https://git.martyoeh.me/Marty/habit-migrate"
 authors = ["Marty Oehme <marty.oehme@gmail.com>"]
 packages = [
     { include = "habitmove", from = "src"},
@@ -38,13 +35,3 @@ show_missing = true
 [build-system]
 requires = ["poetry-core>=1.0.0"]
 build-backend = "poetry.core.masonry.api"
-
-[[tool.mypy.overrides]]
-module = [
-    "click",
-    "click.testing",
-    "pytest",
-    "nox",
-    "importlib-metadata"
-]
-ignore_missing_imports = true

src/habitmove/__init__.py

@@ -4,6 +4,6 @@ import sys
 try:
     from importlib.metadata import version as metadata_version
 except ImportError:
-    from importlib_metadata import version as metadata_version  # type: ignore
+    from importlib_metadata import version as metadata_version
 
 __version__ = str(metadata_version(__name__))

src/habitmove/repetitions.py

@@ -55,10 +55,9 @@ def habit_list_add_ids(c: sqlite3.Cursor, habitlist: list[Habit]) -> dict[int, H
     :return habit_id_dict: The habit collection as a dict with the keys
                                  consisting of the habit's sqlite database ID.
     """
-    with_id: dict[int, Habit] = {}
+    with_id = {}
     for h in habitlist:
         sql_id = fetch_habit_id(c, h.uuid or "")
-        if sql_id is not None:
         with_id[sql_id] = h
 
     return with_id
@@ -75,8 +74,6 @@ def fetch_habit_id(cursor: sqlite3.Cursor, uuid: str) -> Optional[int]:
     if id is not None:
         return id[0]
 
-    return None
-
 
 def add_to_database(
     cursor: sqlite3.Cursor, habits: dict[int, Habit], repetition: Repetition

tools/extract-changelog.py

@@ -1,56 +0,0 @@
-import re
-
-## Extracts the version and newest changes from a semantic changelog.
-#
-# Important, it only works with three-parted version numbers
-# a-la 1.2.3 or 313.01.1888 -- needs \d.\d.\d to work.
-#
-# The version number and changeset will be put in `NEWEST_VERSION.md`
-# and `NEWEST_CHANGES.md` respectively, for further use in releases.
-OUTPUT_FILE_VERSION = "NEWEST_VERSION.md"
-OUTPUT_FILE_CHANGES = "NEWEST_CHANGES.md"
-
-
-def getVersion(file):
-    for line in file:
-        m = re.match(r"^## \[(\d+\.\d+\.\d+)\]", line)
-        if m and m.group(1):
-            return m.group(1)
-
-
-def getSection(file):
-    inRecordingMode = False
-    for line in file:
-        if not inRecordingMode:
-            if re.match(r"^## \[\d+\.\d+\.\d+\]", line):
-                inRecordingMode = True
-        elif re.match(r"^## \[\d+\.\d+\.\d+\]", line):
-            inRecordingMode = False
-            break
-        elif re.match(r"^$", line):
-            pass
-        else:
-            yield line
-
-
-def toFile(fname, content):
-    file = open(fname, "w")
-    file.write(content)
-    file.close()
-
-
-with open("CHANGELOG.md") as file:
-    title = getVersion(file)
-    print(title)
-    toFile(OUTPUT_FILE_VERSION, title)
-
-with open("CHANGELOG.md") as file:
-    newest_changes_gen = getSection(file)
-    newest_changes = ""
-    for line in newest_changes_gen:
-        newest_changes += line
-    print("[Extracted Changelog]")
-    print(newest_changes)
-    toFile(OUTPUT_FILE_CHANGES, newest_changes)
-
-file.close()