Add pypi release automation

Added basic continuous integration tests to run on any push:
On main branch, the python program is built.
On tagged commit, a gitea release is created.

Fixed first detected build pipeline issues:
Fixing mypy library stubs missing for some imported libraries.
Fixed two small typing errors for Repetitions.

The steps run on basic python containers, onto which the ci steps simply
install poetry.
This takes a little more processing time during pipeline running (~16s
per step),
but also gives a lot of flexibility in container usage.

Added script which assists in creating an automatic release by
extracting the current version and newest changes from the semantic
changelog.
This is then used in the gitea release preparation as title and
content of the release message.
The files built in the dist directory by poetry will be attached.

.woodpecker.yml

@@ -0,0 +1,86 @@
+branches: main
+
+pipeline:
+  code_lint:
+    image: python
+    commands:
+      - pip install poetry
+      - poetry install
+      - pip install black
+      - echo "----------------- running lint ------------------"
+      - python --version && poetry --version && black --version
+      - poetry run black .
+
+  unit_tests:
+    image: thekevjames/nox
+    commands:
+      - pip install poetry
+      - poetry install
+      - echo "----------------- running tests ------------------"
+      - python --version && poetry --version && nox --version
+      - poetry run nox
+
+  static_analysis:
+    image: python
+    commands:
+      - pip install poetry
+      - poetry install
+      - pip install mypy
+      - echo "----------------- running analysis ------------------"
+      - python --version && poetry --version && mypy --version
+      - poetry run mypy .
+
+  build_dist:
+    image: python
+    commands:
+      - pip install poetry
+      - poetry install
+      - echo "----------------- running analysis ------------------"
+      - python --version && poetry --version
+      - poetry build
+    when:
+      branch: main
+
+  release_prep:
+    image: python
+    commands:
+      - echo "----------------- preparing release ------------------"
+      - python tools/extract-changelog.py
+
+  gitea_release:
+    image: plugins/gitea-release
+    settings:
+      api_key: 
+        from_secret: gitea_release_token
+      base_url: https://git.martyoeh.me
+      files: dist/*
+      title: NEWEST_VERSION.md
+      note: NEWEST_CHANGES.md
+    when:
+      event: tag
+      tag: v*
+
+  pypi_release:
+    image: python
+    commands:
+      - pip install poetry
+      - poetry install
+      - echo "----------------- publishing to pypi ------------------"
+      - poetry publish --username "$PYPI_USERNAME" --password "$PYPI_PASSWORD"
+    when:
+      event: tag
+      tag: v*
+
+  notify_matrix:
+    image: plugins/matrix
+    settings:
+      homeserver: https://matrix.org
+      roomid:
+        from_secret: matrix_roomid
+      userid:
+        from_secret: matrix_userid
+      accesstoken:
+        from_secret: matrix_token
+    when:
+      status: [ success, failure ]
+

CHANGELOG.md

@@ -5,7 +5,19 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project tries to adhere to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+<!-- ## [Unreleased] -->
+
+## [0.4.1] - 2022-01-05
+
+### Added
+
+* Added pypi release publication
+
+### Changed
+
+* Compatible with Python stretching back to version 3.7
+
+## [0.4.0] - 2021-12-06
 
 ### Added 
 
@@ -14,6 +26,11 @@ and this project tries to adhere to [Semantic Versioning](https://semver.org/spe
   e.g. #smoking #smoking. This does not work in Loop, but we import
   them as separate counter instances instead.
 
+### Changed
+
+* Begin rewrite of much of the internals to be more class-based and have more solid type checking
+* If used as library, most of internal functionality has changed with more changes upcoming
+
 ### Fixed
 
 * Create missing PRAGMA values for Loop SQLite database, fixing failing import

README.md

@@ -1,44 +1,64 @@
-# habit-migrate
+# habitmove
 
-Can take an export of nomie habits in json format and convert it to be importable in Loop Habit Tracker.
+Takes habit in one habit-tracking application and transforms them ready to use for another.
 
-Confirmed working for nomie version 5.6.4 and Loop Habit Tracker version 2.0.2. 
+Currently can take an export of nomie habits in json format and convert it to be importable in Loop Habit Tracker.
+Plans for reverse migration are on the roadmap, and ultimately this tool ideally understands more and more habit formats to prevent app lock-in.
+
+Confirmed working for nomie version 5.6.4 and Loop Habit Tracker version 2.0.2 and 2.0.3. 
 Presumably works for other nomie 5.x versions and other Loop 2.x versions as well, 
 but that is untested.
 
+## Installation
+
+Installation can be accomplished through *pip*:
+
+```bash
+pip install habitmove
+```
+
+Requirements:
+
+`habitmove` requires at least Python 3.7. 
+It has only been tested on GNU/Linux (amd64) though it should work on other platforms.
+
 ## Usage 
 
-Run as a commandline utility habit migrate currently takes a single argument, the nomie database `.json` file.
+Run as a cli utility `habitmove` currently takes a single argument: the nomie database `.json` file to import habits from.
-The output as importable Loop Habit Tracker database will be written to `output.db` in present working directory.
+
+Invoked like: `habitmove nomie-export.json`.
+
+The output as a Loop Habit Tracker database will be written to `output.db` in the present working directory.
 
 Can also take an existing Loop Habit database (exported from the application), 
 and add the nomie exported habits and checkmarks to it.
 Simply put the exported Loop database in the same directory and call it `output.db`, 
 it will not (should not™️) overwrite anything.
+If there are any duplicated habits however, 
+it will add duplications of the existing repetitions into the database.
 
-Invoked like: `python run.py nomie-export.json`.
+## Development
-Note, however, that -- until a packaged version is released -- you will need to have some packages in your environment.
-If you wish to run it un-packaged, install [poetry](https://python-poetry.org/) and let it do all dependency management by doing:
 
-```
+To enable easy development on the app, 
+install [poetry](https://python-poetry.org/) and let it do all dependency management for you by doing:
+
+```bash
 poetry install
 poetry run habitmove <nomie-json>
 ```
 
-In the future there might be an easier road to using this package but that's the way it is for now.
+To see a set up more closely resembling the final cli environment, 
+with its libraries loaded as environmental dependencies enter the poetry shell:
 
-The package can also be used as a library to load nomie data
+```bash
-or move data into Loop Habit Tracker.
+poetry shell 
+```
 
+The package can eventually also be used as a library to load nomie data to work with in Python,
+or to move data into Loop Habit Tracker.
+Take a look at the `Parser` and `Transformer` interfaces respectively.
 
-## Roadmap
+To run tests for the app, simply invoke `pytest` through `poetry run pytest` or from within the `poetry shell`.
-
+To run larger scale test automation, make sure you habe nox installed and run `poetry run ` or again through the shell.
-* [ ] clean up README
-* [ ] begin adding tests
-    * [ ] add some unit tests for various functions
-    * [ ] and at least an integration test for the stable database (loop-2021-12-02.db or equivalent)
-* [ ] abstract migration target away from loop
-* [ ] abstract import away from nomie
-* [ ] abstract importer/migrator themselves to work with other targets
-* [ ] allow migration to/from nomie/loop
 
+You can exclude integration tests that take longer and inspect the complete database output of the program through the parameters `-m "not e2e"` for both `pytest` and `nox` (which also does it automatically).

habitmove/__init__.py

@@ -1,4 +0,0 @@
-import habitmove.schema as schema
-import habitmove.habits as habits
-import habitmove.repetitions as rep
-import habitmove.nomie as nomie

noxfile.py

@@ -0,0 +1,9 @@
+import nox
+
+
+@nox.session(python=["3.7", "3.8", "3.9"])
+def tests(session):
+    args = session.posargs or ["--cov"]
+    session.run("poetry", "install", external=True)
+    session.run("pytest", *args)
+    pass

poetry.lock

@@ -1,8 +1,322 @@
-package = []
+[[package]]
+name = "atomicwrites"
+version = "1.4.0"
+description = "Atomic file writes."
+category = "dev"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*"
+
+[[package]]
+name = "attrs"
+version = "21.2.0"
+description = "Classes Without Boilerplate"
+category = "dev"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[package.extras]
+dev = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "zope.interface", "furo", "sphinx", "sphinx-notfound-page", "pre-commit"]
+docs = ["furo", "sphinx", "zope.interface", "sphinx-notfound-page"]
+tests = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins", "zope.interface"]
+tests_no_zope = ["coverage[toml] (>=5.0.2)", "hypothesis", "pympler", "pytest (>=4.3.0)", "six", "mypy", "pytest-mypy-plugins"]
+
+[[package]]
+name = "click"
+version = "8.0.3"
+description = "Composable command line interface toolkit"
+category = "main"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+colorama = {version = "*", markers = "platform_system == \"Windows\""}
+importlib-metadata = {version = "*", markers = "python_version < \"3.8\""}
+
+[[package]]
+name = "colorama"
+version = "0.4.4"
+description = "Cross-platform colored terminal text."
+category = "main"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[[package]]
+name = "coverage"
+version = "6.2"
+description = "Code coverage measurement for Python"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+tomli = {version = "*", optional = true, markers = "extra == \"toml\""}
+
+[package.extras]
+toml = ["tomli"]
+
+[[package]]
+name = "importlib-metadata"
+version = "0.23"
+description = "Read metadata from Python packages"
+category = "main"
+optional = false
+python-versions = ">=2.7,!=3.0,!=3.1,!=3.2,!=3.3"
+
+[package.dependencies]
+zipp = ">=0.5"
+
+[package.extras]
+docs = ["sphinx", "rst.linker"]
+testing = ["packaging", "importlib-resources"]
+
+[[package]]
+name = "iniconfig"
+version = "1.1.1"
+description = "iniconfig: brain-dead simple config-ini parsing"
+category = "dev"
+optional = false
+python-versions = "*"
+
+[[package]]
+name = "packaging"
+version = "21.3"
+description = "Core utilities for Python packages"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+pyparsing = ">=2.0.2,<3.0.5 || >3.0.5"
+
+[[package]]
+name = "pluggy"
+version = "1.0.0"
+description = "plugin and hook calling mechanisms for python"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+importlib-metadata = {version = ">=0.12", markers = "python_version < \"3.8\""}
+
+[package.extras]
+dev = ["pre-commit", "tox"]
+testing = ["pytest", "pytest-benchmark"]
+
+[[package]]
+name = "py"
+version = "1.11.0"
+description = "library with cross-python path, ini-parsing, io, code, log facilities"
+category = "dev"
+optional = false
+python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
+
+[[package]]
+name = "pyparsing"
+version = "3.0.6"
+description = "Python parsing module"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.extras]
+diagrams = ["jinja2", "railroad-diagrams"]
+
+[[package]]
+name = "pytest"
+version = "6.2.5"
+description = "pytest: simple powerful testing with Python"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+atomicwrites = {version = ">=1.0", markers = "sys_platform == \"win32\""}
+attrs = ">=19.2.0"
+colorama = {version = "*", markers = "sys_platform == \"win32\""}
+importlib-metadata = {version = ">=0.12", markers = "python_version < \"3.8\""}
+iniconfig = "*"
+packaging = "*"
+pluggy = ">=0.12,<2.0"
+py = ">=1.8.2"
+toml = "*"
+
+[package.extras]
+testing = ["argcomplete", "hypothesis (>=3.56)", "mock", "nose", "requests", "xmlschema"]
+
+[[package]]
+name = "pytest-cov"
+version = "3.0.0"
+description = "Pytest plugin for measuring coverage."
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+coverage = {version = ">=5.2.1", extras = ["toml"]}
+pytest = ">=4.6"
+
+[package.extras]
+testing = ["fields", "hunter", "process-tests", "six", "pytest-xdist", "virtualenv"]
+
+[[package]]
+name = "pytest-mock"
+version = "3.6.1"
+description = "Thin-wrapper around the mock package for easier use with pytest"
+category = "dev"
+optional = false
+python-versions = ">=3.6"
+
+[package.dependencies]
+pytest = ">=5.0"
+
+[package.extras]
+dev = ["pre-commit", "tox", "pytest-asyncio"]
+
+[[package]]
+name = "toml"
+version = "0.10.2"
+description = "Python Library for Tom's Obvious, Minimal Language"
+category = "dev"
+optional = false
+python-versions = ">=2.6, !=3.0.*, !=3.1.*, !=3.2.*"
+
+[[package]]
+name = "tomli"
+version = "2.0.0"
+description = "A lil' TOML parser"
+category = "dev"
+optional = false
+python-versions = ">=3.7"
+
+[[package]]
+name = "zipp"
+version = "3.6.0"
+description = "Backport of pathlib-compatible object wrapper for zip files"
+category = "main"
+optional = false
+python-versions = ">=3.6"
+
+[package.extras]
+docs = ["sphinx", "jaraco.packaging (>=8.2)", "rst.linker (>=1.9)"]
+testing = ["pytest (>=4.6)", "pytest-checkdocs (>=2.4)", "pytest-flake8", "pytest-cov", "pytest-enabler (>=1.0.1)", "jaraco.itertools", "func-timeout", "pytest-black (>=0.3.7)", "pytest-mypy"]
 
 [metadata]
 lock-version = "1.1"
-python-versions = "^3.9"
+python-versions = "^3.7"
-content-hash = "ce2aa767160f871dd3652615ba0a0dceb7733d62eb8cb4665b87f30a562e3adf"
+content-hash = "fec236ab912efe582781c62e4cd2c4cd7e609557e284e067fa875f30ab806d93"
 
 [metadata.files]
+atomicwrites = [
+    {file = "atomicwrites-1.4.0-py2.py3-none-any.whl", hash = "sha256:6d1784dea7c0c8d4a5172b6c620f40b6e4cbfdf96d783691f2e1302a7b88e197"},
+    {file = "atomicwrites-1.4.0.tar.gz", hash = "sha256:ae70396ad1a434f9c7046fd2dd196fc04b12f9e91ffb859164193be8b6168a7a"},
+]
+attrs = [
+    {file = "attrs-21.2.0-py2.py3-none-any.whl", hash = "sha256:149e90d6d8ac20db7a955ad60cf0e6881a3f20d37096140088356da6c716b0b1"},
+    {file = "attrs-21.2.0.tar.gz", hash = "sha256:ef6aaac3ca6cd92904cdd0d83f629a15f18053ec84e6432106f7a4d04ae4f5fb"},
+]
+click = [
+    {file = "click-8.0.3-py3-none-any.whl", hash = "sha256:353f466495adaeb40b6b5f592f9f91cb22372351c84caeb068132442a4518ef3"},
+    {file = "click-8.0.3.tar.gz", hash = "sha256:410e932b050f5eed773c4cda94de75971c89cdb3155a72a0831139a79e5ecb5b"},
+]
+colorama = [
+    {file = "colorama-0.4.4-py2.py3-none-any.whl", hash = "sha256:9f47eda37229f68eee03b24b9748937c7dc3868f906e8ba69fbcbdd3bc5dc3e2"},
+    {file = "colorama-0.4.4.tar.gz", hash = "sha256:5941b2b48a20143d2267e95b1c2a7603ce057ee39fd88e7329b0c292aa16869b"},
+]
+coverage = [
+    {file = "coverage-6.2-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:6dbc1536e105adda7a6312c778f15aaabe583b0e9a0b0a324990334fd458c94b"},
+    {file = "coverage-6.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:174cf9b4bef0db2e8244f82059a5a72bd47e1d40e71c68ab055425172b16b7d0"},
+    {file = "coverage-6.2-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:92b8c845527eae547a2a6617d336adc56394050c3ed8a6918683646328fbb6da"},
+    {file = "coverage-6.2-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:c7912d1526299cb04c88288e148c6c87c0df600eca76efd99d84396cfe00ef1d"},
+    {file = "coverage-6.2-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:d5d2033d5db1d58ae2d62f095e1aefb6988af65b4b12cb8987af409587cc0739"},
+    {file = "coverage-6.2-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:3feac4084291642165c3a0d9eaebedf19ffa505016c4d3db15bfe235718d4971"},
+    {file = "coverage-6.2-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:276651978c94a8c5672ea60a2656e95a3cce2a3f31e9fb2d5ebd4c215d095840"},
+    {file = "coverage-6.2-cp310-cp310-win32.whl", hash = "sha256:f506af4f27def639ba45789fa6fde45f9a217da0be05f8910458e4557eed020c"},
+    {file = "coverage-6.2-cp310-cp310-win_amd64.whl", hash = "sha256:3f7c17209eef285c86f819ff04a6d4cbee9b33ef05cbcaae4c0b4e8e06b3ec8f"},
+    {file = "coverage-6.2-cp311-cp311-macosx_10_14_x86_64.whl", hash = "sha256:13362889b2d46e8d9f97c421539c97c963e34031ab0cb89e8ca83a10cc71ac76"},
+    {file = "coverage-6.2-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.whl", hash = "sha256:22e60a3ca5acba37d1d4a2ee66e051f5b0e1b9ac950b5b0cf4aa5366eda41d47"},
+    {file = "coverage-6.2-cp311-cp311-win_amd64.whl", hash = "sha256:b637c57fdb8be84e91fac60d9325a66a5981f8086c954ea2772efe28425eaf64"},
+    {file = "coverage-6.2-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:f467bbb837691ab5a8ca359199d3429a11a01e6dfb3d9dcc676dc035ca93c0a9"},
+    {file = "coverage-6.2-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2641f803ee9f95b1f387f3e8f3bf28d83d9b69a39e9911e5bfee832bea75240d"},
+    {file = "coverage-6.2-cp36-cp36m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:1219d760ccfafc03c0822ae2e06e3b1248a8e6d1a70928966bafc6838d3c9e48"},
+    {file = "coverage-6.2-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:9a2b5b52be0a8626fcbffd7e689781bf8c2ac01613e77feda93d96184949a98e"},
+    {file = "coverage-6.2-cp36-cp36m-musllinux_1_1_aarch64.whl", hash = "sha256:8e2c35a4c1f269704e90888e56f794e2d9c0262fb0c1b1c8c4ee44d9b9e77b5d"},
+    {file = "coverage-6.2-cp36-cp36m-musllinux_1_1_i686.whl", hash = "sha256:5d6b09c972ce9200264c35a1d53d43ca55ef61836d9ec60f0d44273a31aa9f17"},
+    {file = "coverage-6.2-cp36-cp36m-musllinux_1_1_x86_64.whl", hash = "sha256:e3db840a4dee542e37e09f30859f1612da90e1c5239a6a2498c473183a50e781"},
+    {file = "coverage-6.2-cp36-cp36m-win32.whl", hash = "sha256:4e547122ca2d244f7c090fe3f4b5a5861255ff66b7ab6d98f44a0222aaf8671a"},
+    {file = "coverage-6.2-cp36-cp36m-win_amd64.whl", hash = "sha256:01774a2c2c729619760320270e42cd9e797427ecfddd32c2a7b639cdc481f3c0"},
+    {file = "coverage-6.2-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:fb8b8ee99b3fffe4fd86f4c81b35a6bf7e4462cba019997af2fe679365db0c49"},
+    {file = "coverage-6.2-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:619346d57c7126ae49ac95b11b0dc8e36c1dd49d148477461bb66c8cf13bb521"},
+    {file = "coverage-6.2-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:0a7726f74ff63f41e95ed3a89fef002916c828bb5fcae83b505b49d81a066884"},
+    {file = "coverage-6.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:cfd9386c1d6f13b37e05a91a8583e802f8059bebfccde61a418c5808dea6bbfa"},
+    {file = "coverage-6.2-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:17e6c11038d4ed6e8af1407d9e89a2904d573be29d51515f14262d7f10ef0a64"},
+    {file = "coverage-6.2-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:c254b03032d5a06de049ce8bca8338a5185f07fb76600afff3c161e053d88617"},
+    {file = "coverage-6.2-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:dca38a21e4423f3edb821292e97cec7ad38086f84313462098568baedf4331f8"},
+    {file = "coverage-6.2-cp37-cp37m-win32.whl", hash = "sha256:600617008aa82032ddeace2535626d1bc212dfff32b43989539deda63b3f36e4"},
+    {file = "coverage-6.2-cp37-cp37m-win_amd64.whl", hash = "sha256:bf154ba7ee2fd613eb541c2bc03d3d9ac667080a737449d1a3fb342740eb1a74"},
+    {file = "coverage-6.2-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:f9afb5b746781fc2abce26193d1c817b7eb0e11459510fba65d2bd77fe161d9e"},
+    {file = "coverage-6.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:edcada2e24ed68f019175c2b2af2a8b481d3d084798b8c20d15d34f5c733fa58"},
+    {file = "coverage-6.2-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:a9c8c4283e17690ff1a7427123ffb428ad6a52ed720d550e299e8291e33184dc"},
+    {file = "coverage-6.2-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:f614fc9956d76d8a88a88bb41ddc12709caa755666f580af3a688899721efecd"},
+    {file = "coverage-6.2-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:9365ed5cce5d0cf2c10afc6add145c5037d3148585b8ae0e77cc1efdd6aa2953"},
+    {file = "coverage-6.2-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:8bdfe9ff3a4ea37d17f172ac0dff1e1c383aec17a636b9b35906babc9f0f5475"},
+    {file = "coverage-6.2-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:63c424e6f5b4ab1cf1e23a43b12f542b0ec2e54f99ec9f11b75382152981df57"},
+    {file = "coverage-6.2-cp38-cp38-win32.whl", hash = "sha256:49dbff64961bc9bdd2289a2bda6a3a5a331964ba5497f694e2cbd540d656dc1c"},
+    {file = "coverage-6.2-cp38-cp38-win_amd64.whl", hash = "sha256:9a29311bd6429be317c1f3fe4bc06c4c5ee45e2fa61b2a19d4d1d6111cb94af2"},
+    {file = "coverage-6.2-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:03b20e52b7d31be571c9c06b74746746d4eb82fc260e594dc662ed48145e9efd"},
+    {file = "coverage-6.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:215f8afcc02a24c2d9a10d3790b21054b58d71f4b3c6f055d4bb1b15cecce685"},
+    {file = "coverage-6.2-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl", hash = "sha256:a4bdeb0a52d1d04123b41d90a4390b096f3ef38eee35e11f0b22c2d031222c6c"},
+    {file = "coverage-6.2-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl", hash = "sha256:c332d8f8d448ded473b97fefe4a0983265af21917d8b0cdcb8bb06b2afe632c3"},
+    {file = "coverage-6.2-cp39-cp39-win32.whl", hash = "sha256:6e1394d24d5938e561fbeaa0cd3d356207579c28bd1792f25a068743f2d5b282"},
+    {file = "coverage-6.2-cp39-cp39-win_amd64.whl", hash = "sha256:86f2e78b1eff847609b1ca8050c9e1fa3bd44ce755b2ec30e70f2d3ba3844644"},
+    {file = "coverage-6.2-pp36.pp37.pp38-none-any.whl", hash = "sha256:5829192582c0ec8ca4a2532407bc14c2f338d9878a10442f5d03804a95fac9de"},
+    {file = "coverage-6.2.tar.gz", hash = "sha256:e2cad8093172b7d1595b4ad66f24270808658e11acf43a8f95b41276162eb5b8"},
+]
+importlib-metadata = [
+    {file = "importlib_metadata-0.23-py2.py3-none-any.whl", hash = "sha256:d5f18a79777f3aa179c145737780282e27b508fc8fd688cb17c7a813e8bd39af"},
+    {file = "importlib_metadata-0.23.tar.gz", hash = "sha256:aa18d7378b00b40847790e7c27e11673d7fed219354109d0e7b9e5b25dc3ad26"},
+]
+iniconfig = [
+    {file = "iniconfig-1.1.1-py2.py3-none-any.whl", hash = "sha256:011e24c64b7f47f6ebd835bb12a743f2fbe9a26d4cecaa7f53bc4f35ee9da8b3"},
+    {file = "iniconfig-1.1.1.tar.gz", hash = "sha256:bc3af051d7d14b2ee5ef9969666def0cd1a000e121eaea580d4a313df4b37f32"},
+]
+packaging = [
+    {file = "packaging-21.3-py3-none-any.whl", hash = "sha256:ef103e05f519cdc783ae24ea4e2e0f508a9c99b2d4969652eed6a2e1ea5bd522"},
+    {file = "packaging-21.3.tar.gz", hash = "sha256:dd47c42927d89ab911e606518907cc2d3a1f38bbd026385970643f9c5b8ecfeb"},
+]
+pluggy = [
+    {file = "pluggy-1.0.0-py2.py3-none-any.whl", hash = "sha256:74134bbf457f031a36d68416e1509f34bd5ccc019f0bcc952c7b909d06b37bd3"},
+    {file = "pluggy-1.0.0.tar.gz", hash = "sha256:4224373bacce55f955a878bf9cfa763c1e360858e330072059e10bad68531159"},
+]
+py = [
+    {file = "py-1.11.0-py2.py3-none-any.whl", hash = "sha256:607c53218732647dff4acdfcd50cb62615cedf612e72d1724fb1a0cc6405b378"},
+    {file = "py-1.11.0.tar.gz", hash = "sha256:51c75c4126074b472f746a24399ad32f6053d1b34b68d2fa41e558e6f4a98719"},
+]
+pyparsing = [
+    {file = "pyparsing-3.0.6-py3-none-any.whl", hash = "sha256:04ff808a5b90911829c55c4e26f75fa5ca8a2f5f36aa3a51f68e27033341d3e4"},
+    {file = "pyparsing-3.0.6.tar.gz", hash = "sha256:d9bdec0013ef1eb5a84ab39a3b3868911598afa494f5faa038647101504e2b81"},
+]
+pytest = [
+    {file = "pytest-6.2.5-py3-none-any.whl", hash = "sha256:7310f8d27bc79ced999e760ca304d69f6ba6c6649c0b60fb0e04a4a77cacc134"},
+    {file = "pytest-6.2.5.tar.gz", hash = "sha256:131b36680866a76e6781d13f101efb86cf674ebb9762eb70d3082b6f29889e89"},
+]
+pytest-cov = [
+    {file = "pytest-cov-3.0.0.tar.gz", hash = "sha256:e7f0f5b1617d2210a2cabc266dfe2f4c75a8d32fb89eafb7ad9d06f6d076d470"},
+    {file = "pytest_cov-3.0.0-py3-none-any.whl", hash = "sha256:578d5d15ac4a25e5f961c938b85a05b09fdaae9deef3bb6de9a6e766622ca7a6"},
+]
+pytest-mock = [
+    {file = "pytest-mock-3.6.1.tar.gz", hash = "sha256:40217a058c52a63f1042f0784f62009e976ba824c418cced42e88d5f40ab0e62"},
+    {file = "pytest_mock-3.6.1-py3-none-any.whl", hash = "sha256:30c2f2cc9759e76eee674b81ea28c9f0b94f8f0445a1b87762cadf774f0df7e3"},
+]
+toml = [
+    {file = "toml-0.10.2-py2.py3-none-any.whl", hash = "sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b"},
+    {file = "toml-0.10.2.tar.gz", hash = "sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f"},
+]
+tomli = [
+    {file = "tomli-2.0.0-py3-none-any.whl", hash = "sha256:b5bde28da1fed24b9bd1d4d2b8cba62300bfb4ec9a6187a957e8ddb9434c5224"},
+    {file = "tomli-2.0.0.tar.gz", hash = "sha256:c292c34f58502a1eb2bbb9f5bbc9a5ebc37bee10ffb8c2d6bbdfa8eb13cc14e1"},
+]
+zipp = [
+    {file = "zipp-3.6.0-py3-none-any.whl", hash = "sha256:9fe5ea21568a0a70e50f273397638d39b03353731e6cbbb3fd8502a33fec40bc"},
+    {file = "zipp-3.6.0.tar.gz", hash = "sha256:71c644c5369f4a6e07636f0aa966270449561fcea2e3d6747b8d23efaa9d7832"},
+]

pyproject.toml

@@ -1,18 +1,50 @@
 [tool.poetry]
 name = "habitmove"
-version = "0.3.1"
+version = "0.4.1"
 description = "migrate nomie data to loop habits tracker"
+license="GPL-3.0-only"
+readme="README.md"
+repository="https://git.martyoeh.me/Marty/habit-migrate"
 authors = ["Marty Oehme <marty.oehme@gmail.com>"]
+packages = [
+    { include = "habitmove", from = "src"},
+]
 
 [tool.poetry.dependencies]
-python = "^3.9"
+importlib-metadata = {version = "^0.23", python = "<3.8"}
+python = "^3.7"
+click = "^8.0"
 
 [tool.poetry.dev-dependencies]
+pytest = "^6.2"
+coverage = {extras = ["toml"], version = "^6.2"}
+pytest-cov = "^3.0.0"
+pytest-mock = "^3.6.1"
 
 [tool.poetry.scripts]
-habitmove = "run:main"
+habitmove = "habitmove.cli:main"
+
+[tool.coverage.paths]
+source = ["src", "*/site-packages"]
+
+[tool.coverage.run]
+branch = true
+source = ["habitmove"]
+
+[tool.coverage.report]
+show_missing = true
+# fail_under = 80  # if we want pytest to automatically fail if not enough tests supplied
 
 [build-system]
 requires = ["poetry-core>=1.0.0"]
 build-backend = "poetry.core.masonry.api"
 
+[[tool.mypy.overrides]]
+module = [
+    "click",
+    "click.testing",
+    "pytest",
+    "nox",
+    "importlib-metadata"
+]
+ignore_missing_imports = true

src/habitmove/__init__.py

@@ -0,0 +1,9 @@
+# init.py
+import sys
+
+try:
+    from importlib.metadata import version as metadata_version
+except ImportError:
+    from importlib_metadata import version as metadata_version  # type: ignore
+
+__version__ = str(metadata_version(__name__))

src/habitmove/cli.py

@@ -5,8 +5,8 @@ import habitmove.repetitions as rep
 import habitmove.nomie as nomie
 from habitmove.nomiedata import NomieImport
 
-
+import click
-import sys
+from . import __version__
 
 
 def migrate(data: NomieImport):
@@ -25,9 +25,11 @@ def migrate(data: NomieImport):
     db.close()
 
 
-def main():
+@click.command()
-    file = sys.argv[1]
+@click.version_option(version=__version__)
-    data = nomie.get_data(file)
+@click.argument("inputfile")
+def main(inputfile):
+    data = nomie.get_data(inputfile)
     migrate(data)
 
 

src/habitmove/habits.py

@@ -1,7 +1,12 @@
+from __future__ import annotations
+
+import sqlite3
+
+from habitmove.nomiedata import Tracker
 from habitmove.loopdata import Habit
 
 
-def migrate(db, trackers):
+def migrate(db: sqlite3.Connection, trackers: list[Tracker]):
     c = db.cursor()
     habits = trackers_to_habits(trackers)
     for habit in habits:
@@ -23,7 +28,7 @@ def trackers_to_habits(trackers):
         habits.append(
             Habit(
                 archived=t.hidden,
-                color=11 if t.score != "-1" else 0,
+                color=0 if t.score == -1 else 11,
                 description=t.tag,
                 name=f"{t.emoji} {t.label}",
                 unit="" if t.uom == "num" else t.uom,
@@ -34,7 +39,9 @@ def trackers_to_habits(trackers):
             habits[-1].type = 1
             # nomie only has concept of max value,
             # use a percentage of it for Loop range target
-            habits[-1].target_value = int(t.max) // NOMIE_MAX_TO_TARGET_VALUE_RATIO
+            habits[-1].target_value = (
+                t.goal or int(t.max) // NOMIE_MAX_TO_TARGET_VALUE_RATIO
+            )
     return habits
 
 

src/habitmove/loopdata.py

@@ -20,7 +20,6 @@ class Habit:
     position: int = 0
     uuid: str = ""
 
-    # TODO test post init uuid setting
     def __post_init__(self):
         if not self.uuid or self.uuid == "":
             self.uuid = uuid4().hex

src/habitmove/nomie.py

@@ -2,6 +2,7 @@
 
 import json
 import re
+from click import secho, echo
 from habitmove.nomiedata import Tracker, Event, Activity, NomieImport
 
 
@@ -35,13 +36,13 @@ def verify_continue(data: NomieImport):
     for e in data.events:
         activity_count += len(e.activities) if e.activities else 0
 
-    print(f"Exporting from nomie {data.version}:")
+    secho(f"Exporting from nomie {data.version}:", fg="green")
-    print(f"Found trackers: {trackers}")
+    echo(f"Found trackers: {trackers}")
-    print(
+    echo(
         f"Found events: {len(data.events)} entries, containing {activity_count} individual activities."
     )
     if not confirmation_question("Do you want to continue?", default_no=False):
-        print("Aborted.")
+        echo("Aborted.")
         exit(0)
 
 

src/habitmove/nomiedata.py

@@ -1,30 +1,37 @@
-from typing import Optional, Any
+from __future__ import annotations
+
+from typing import Any, Union
 from dataclasses import dataclass, field
+import re
 
 # A nomie habit tracker. Tracks anything whose value can be encapsulated in a numerical value.
 @dataclass(frozen=True)
 class Tracker:
-    color: str
-    emoji: str
-    hidden: bool
-    id: str
-    ignore_zeros: bool
-    label: str
-    math: str
-    one_tap: bool
     tag: str
-    type: str
+    label: str
-    uom: str
+    id: str
+    one_tap: bool = True
+    color: str = "#000080"
+    emoji: str = ""
+    hidden: bool = False
+    ignore_zeros: bool = False
+    math: str = "mean"
+    type: str = "tick"  # tick or range mostly
+    uom: str = ""
     # TODO no idea what include does
-    include: str
+    include: str = ""
     min: int = 0
     max: int = 0
     goal: int = 0
     default: int = 0
-    # TODO score can be string (if custom) or int (if simple good/bad)
+    score: Union[int, str] = 1  # score can be string ('custom') or int
-    score: str = ""
     score_calc: list[dict[str, Any]] = field(default_factory=lambda: [])
 
+    def __post_init__(self):
+        # ensure save as int if not 'custom' scoring
+        if re.match(r"^-?[0-9]+$", str(self.score)):
+            object.__setattr__(self, "score", int(self.score))
+
 
 @dataclass(frozen=True)
 class Activity:

src/habitmove/repetitions.py

@@ -1,3 +1,5 @@
+from __future__ import annotations
+
 import sqlite3
 from typing import Optional
 from datetime import datetime
@@ -53,10 +55,11 @@ def habit_list_add_ids(c: sqlite3.Cursor, habitlist: list[Habit]) -> dict[int, H
     :return habit_id_dict: The habit collection as a dict with the keys
                                  consisting of the habit's sqlite database ID.
     """
-    with_id = {}
+    with_id: dict[int, Habit] = {}
     for h in habitlist:
         sql_id = fetch_habit_id(c, h.uuid or "")
-        with_id[sql_id] = h
+        if sql_id is not None:
+            with_id[sql_id] = h
 
     return with_id
 
@@ -72,6 +75,8 @@ def fetch_habit_id(cursor: sqlite3.Cursor, uuid: str) -> Optional[int]:
     if id is not None:
         return id[0]
 
+    return None
+
 
 def add_to_database(
     cursor: sqlite3.Cursor, habits: dict[int, Habit], repetition: Repetition
@@ -92,7 +97,8 @@ def add_to_database(
                     (sql_id, repetition.timestamp, repetition.value),
                 )
             except sqlite3.IntegrityError:
-                # TODO better error handling
+                # FIXME better error handling
+                # TODO think about adapting this to allow importing into existing databases
                 print(
                     f"{sql_id}, {habit.name}: timestamp {datetime.fromtimestamp(repetition.timestamp/1000)} not unique, moving timestamp slightly."
                 )

src/habitmove/schema.py

@@ -1,7 +1,8 @@
 import sqlite3
+import sys
 
 
-def create_database(name):
+def create_database(db_file: str = ":memory:") -> sqlite3.Connection:
     """create a database connection to the SQLite database
         specified by db_file
     :param db_file: database file
@@ -9,16 +10,14 @@ def create_database(name):
     """
     conn = None
     try:
-        conn = sqlite3.connect(name)
+        conn = sqlite3.connect(db_file)
         return conn
     except sqlite3.Error as e:
         print(e)
-
+        sys.exit(1)
-    return conn
 
 
-def create_tables(db):
+def create_tables(c: sqlite3.Cursor):
-    c = db.cursor()
     c.execute(
         """ CREATE TABLE IF NOT EXISTS Habits (
                                         id integer PRIMARY KEY AUTOINCREMENT,
@@ -51,8 +50,7 @@ def create_tables(db):
     )
 
 
-def create_constraints(db):
+def create_constraints(c: sqlite3.Cursor):
-    c = db.cursor()
     c.execute(
         """ CREATE UNIQUE INDEX IF NOT EXISTS idx_repetitions_habit_timestamp 
             on Repetitions( habit, timestamp);
@@ -60,15 +58,15 @@ def create_constraints(db):
     )
 
 
-def create_pragma(db):
+def create_pragma(c: sqlite3.Cursor):
-    c = db.cursor()
     c.execute(""" PRAGMA user_version = 24; """)
     c.execute(""" PRAGMA schema_version = 30; """)
 
 
-def migrate(name):
+def migrate(fname):
-    db = create_database(name)
+    db = create_database(fname)
-    create_tables(db)
+    c = db.cursor()
-    create_constraints(db)
+    create_tables(c)
-    create_pragma(db)
+    create_constraints(c)
+    create_pragma(c)
     return db

tests/test_cli.py

@@ -0,0 +1,15 @@
+import click.testing
+import pytest
+
+from habitmove import cli
+
+
+@pytest.fixture
+def runner():
+    return click.testing.CliRunner()
+
+
+def test_cli_fails_without_file(runner):
+    result = runner.invoke(cli.main)
+    assert result.exit_code == 2
+    assert "Missing argument" in result.output

tests/test_habits.py

@@ -0,0 +1,66 @@
+import pytest
+from habitmove import loopdata, nomiedata
+from habitmove import habits
+
+
+@pytest.fixture
+def trackerlist():
+    return [
+        nomiedata.Tracker(
+            hidden=False,
+            score=-1,
+            tag="testtrack",
+            emoji="🧪",
+            label="Testing",
+            uom="kilotest",
+            id="12345",
+        ),
+        nomiedata.Tracker(
+            hidden=False,
+            tag="testtrack",
+            emoji="🧪",
+            label="Testing",
+            id="54321",
+            one_tap=False,
+            color="#FF0000",
+            ignore_zeros=False,
+            math="mean",
+            type="range",
+            uom="megatest",
+            min=0,
+            max=10,
+            goal=6,
+            default=2,
+            score=1,
+        ),
+    ]
+
+
+def test_simple_habit_transform_from_tracker(trackerlist):
+    result = habits.trackers_to_habits([trackerlist[0]])
+    assert result == [
+        loopdata.Habit(
+            name="🧪 Testing",
+            description="testtrack",
+            unit="kilotest",
+            uuid="12345",
+            archived=False,
+            color=0,
+        )
+    ]
+
+
+def test_range_habit_transform_from_tracker(trackerlist):
+    result = habits.trackers_to_habits([trackerlist[1]])
+    assert result == [
+        loopdata.Habit(
+            name="🧪 Testing",
+            description="testtrack",
+            unit="megatest",
+            uuid="54321",
+            archived=False,
+            color=11,
+            type=1,
+            target_value=6,
+        )
+    ]

tests/test_loopdata.py

@@ -0,0 +1,6 @@
+from habitmove.loopdata import Habit
+
+
+def test_uuid_sets_automatically():
+    sut = Habit(name="testhabit")
+    assert sut.uuid

tests/test_nomiedata.py

@@ -0,0 +1,18 @@
+from habitmove import nomiedata
+
+
+def test_score_numerical_becomes_int():
+    sut = nomiedata.Tracker(label="Int checking", tag="isint", id="1337", score="-1")
+    assert type(sut.score) == int
+
+
+def test_score_invalid_int_stays_string():
+    sut = nomiedata.Tracker(label="Int checking", tag="isint", id="1337", score="-1.3")
+    assert type(sut.score) == str
+
+
+def test_score_string_stays_string():
+    sut = nomiedata.Tracker(
+        label="Int checking", tag="isint", id="1337", score="custom"
+    )
+    assert type(sut.score) == str

tools/extract-changelog.py

@@ -0,0 +1,56 @@
+import re
+
+## Extracts the version and newest changes from a semantic changelog.
+#
+# Important, it only works with three-parted version numbers
+# a-la 1.2.3 or 313.01.1888 -- needs \d.\d.\d to work.
+#
+# The version number and changeset will be put in `NEWEST_VERSION.md`
+# and `NEWEST_CHANGES.md` respectively, for further use in releases.
+OUTPUT_FILE_VERSION = "NEWEST_VERSION.md"
+OUTPUT_FILE_CHANGES = "NEWEST_CHANGES.md"
+
+
+def getVersion(file):
+    for line in file:
+        m = re.match(r"^## \[(\d+\.\d+\.\d+)\]", line)
+        if m and m.group(1):
+            return m.group(1)
+
+
+def getSection(file):
+    inRecordingMode = False
+    for line in file:
+        if not inRecordingMode:
+            if re.match(r"^## \[\d+\.\d+\.\d+\]", line):
+                inRecordingMode = True
+        elif re.match(r"^## \[\d+\.\d+\.\d+\]", line):
+            inRecordingMode = False
+            break
+        elif re.match(r"^$", line):
+            pass
+        else:
+            yield line
+
+
+def toFile(fname, content):
+    file = open(fname, "w")
+    file.write(content)
+    file.close()
+
+
+with open("CHANGELOG.md") as file:
+    title = getVersion(file)
+    print(title)
+    toFile(OUTPUT_FILE_VERSION, title)
+
+with open("CHANGELOG.md") as file:
+    newest_changes_gen = getSection(file)
+    newest_changes = ""
+    for line in newest_changes_gen:
+        newest_changes += line
+    print("[Extracted Changelog]")
+    print(newest_changes)
+    toFile(OUTPUT_FILE_CHANGES, newest_changes)
+
+file.close()