Merge pull request #1 from breadlysm/v0.6.0-isp-&-location-Influxdb-tags

V0.6.0 isp & location influxdb tags

.env

@@ -0,0 +1,17 @@
+# Comment out any variables you don't need except those labeled under required. 
+# Required 
+INFLUX_DB_ADDRESS = 192.168.1.1
+# Required 
+INFLUX_DB_PORT = 8086
+
+#INFLUX_DB_USER = 
+#INFLUX_DB_PASSWORD = 
+INFLUX_DB_DATABASE = speedtest
+# tag names will be used to add tags to your speedtests. They can be used to compare data overtime. These should be comma seperated. 
+INFLUX_DB_TAGS = isp,vpn_enabled, external_ip, server_id, server_name, server_location, speedtest_url
+
+# Speedtest Settings
+# Required. Time between tests (in minutes, converts to seconds).
+SPEEDTEST_INTERVAL = 60
+# Required. Time before retrying a failed Speedtest (in minutes, converts to seconds).
+SPEEDTEST_FAIL_INTERVAL = 5

Dockerfile

@@ -2,7 +2,7 @@ FROM python:3.8-slim-buster
 
 LABEL maintainer="Josh Smith" \
     description="Original by Aiden Gilmartin. Speedtest to InfluxDB data bridge"
-#wget "https://ookla.bintray.com/download/$(wget https://ookla.bintray.com/download/ -q -O - | grep x86_64-linux.tgz\" | grep -Po "(?<=href=\")[^^\"]*" | cut -d ":" -f 2)"
+
 # Install dependencies
 ENV DEBIAN_FRONTEND=noninteractive
 RUN apt-get update

README.md

@@ -1,12 +1,14 @@
 # Speedtest to InfluxDB
 
-This is a small Python script that will continuously run the Speedtest CLI application by Ookla, reformat the data output and forward it on to an InfluxDB database.
+This is a Python script that will continuously run the official Speedtest CLI application by Ookla, takes input from environment variables, formats data and writes it to an InfluxDB database.
 
-You may want to do this so that you can track your internet connections consistency over time. Using Grafana you can view and explore this data easily.
+This script will allow you to measure your internet connections speed and consistency over time. It uses env variables as configuration. It's as easy to use as telling your Docker server a 1 line command and you'll be set. Using Grafana you can start exploring this data easily. 
 
 ![Grafana Dashboard](https://i.imgur.com/8cUdMy7.png)
 
-## Using the script
+There are some added features to allow some additional details that Ookla provides as tags on your data. Some examples are your current ISP, the interface being used, the server who hosted the test. Overtime, you could identify if some serers are performing better than others. 
+
+## Configuring the script
 
 The InfluxDB connection settings are controlled by environment variables.
 
@@ -16,32 +18,78 @@ The variables available are:
 - INFLUX_DB_USER = user
 - INFLUX_DB_PASSWORD = pass
 - INFLUX_DB_DATABASE = speedtest
-- SPEEDTEST_INTERVAL = 1800
+- INFLUX_DB_TAGS = *comma seperated list of tags. See below for options*
-- SPEEDTEST_FAIL_INTERVAL = 300
+- SPEEDTEST_INTERVAL = 60
+- SPEEDTEST_FAIL_INTERVAL = 5
 
+### Variable Notes
+- Intervals are in minutes. *Script will convert it to seconds.*
+- If any variables are not needed, don't declare them. Functions will operate with or without most variables. 
+- Tags should be input without quotes. *INFLUX_DB_TAGS = isp, interface, external_ip, server_name, speedtest_url*
+  
+### Tag Options
+The Ookla speedtest app provides a nice set of data beyond the upload and download speed. The list is below. 
+
+| Tag Name 	| Description 	|
+|-	|-	|
+| isp 	| Your connections ISP 	|
+| interface 	| Your devices connection interface 	|
+| internal_ip 	| Your container or devices IP address 	|
+| interface_mac 	| Mac address of your devices interface 	|
+| vpn_enabled 	| Determines if VPN is enabled or not? I wasn't sure what this represented 	|
+| external_ip 	| Your devices external IP address 	|
+| server_id 	| The Speedtest ID of the server that  was used for testing 	|
+| server_name 	| Name of the Speedtest server used  for testing 	|
+| server_country 	| Country where the Speedtest server  resides 	|
+| server_host 	| Hostname of the Speedtest server 	|
+| server_port 	| Port used by the Speedtest server 	|
+| server_ip 	| Speedtest server's IP address 	|
+| speedtest_id 	| ID of the speedtest results. Can be  used on their site to see results 	|
+| speedtest_url 	| Link to the testing results. It provides your results as it would if you tested on their site.  	|
+
+### Additional Notes
 Be aware that this script will automatically accept the license and GDPR statement so that it can run non-interactively. Make sure you agree with them before running.
 
-### 1. No Container
+## Running the Script
 
-1. [Install the Speedtest CLI application by Ookla.](https://www.speedtest.net/apps/cli)
+### Ideal option, run as a Docker container. 
-
-    NOTE: The `speedtest-cli` package in distro repositories is an unofficial client. It will need to be uninstalled before installing the Ookla Speedtest CLI application with the directions on their website.
-
-2. Install the InfluxDB client for library from Python.
-
-    `pip install influxdb`
-
-3. Run the script.
-
-    `python3 ./main.py`
-
-### 2. Run with Docker or Podman
 
 1. Build the container.
 
-    `docker build -t aidengilmartin/speedtest-influx ./`
+    `docker build -t breadlysm/speedtest-to-influxdb ./`
 
 2. Run the container.
+    ```
+    docker run -d --name speedtest-influx \
+    -e 'INFLUX_DB_ADDRESS'='_influxdb_host_' \
+    -e 'INFLUX_DB_PORT'='8086' \
+    -e 'INFLUX_DB_USER'='_influx_user_' \
+    -e 'INFLUX_DB_PASSWORD'='_influx_pass_' \
+    -e 'INFLUX_DB_DATABASE'='speedtest' \
+    -e 'SPEEDTEST_INTERVAL'='1800' \
+    -e 'SPEEDTEST_FAIL_INTERVAL'='60'  \
+    breadlysm/speedtest-to-influxdb
+    ```
+### No Container
 
-    `docker run -d --name speedtest-influx aidengilmartin/speedtest-influx`
+1. Clone the repo 
 
+    `git clone https://github.com/breadlysm/speedtest-to-influxdb.git`   
+
+2. Configure the .env file in the repo or set the environment variables on your device. 
+
+3. [Install the Speedtest CLI application by Ookla.](https://www.speedtest.net/apps/cli)
+
+    NOTE: The `speedtest-cli` package in distro repositories is an unofficial client. It will need to be uninstalled before installing the Ookla Speedtest CLI application with the directions on their website.
+
+4. Install the InfluxDB client for library from Python.
+
+    `pip install influxdb`
+
+5. Run the script.
+
+    `python3 ./main.py`
+
+
+
+This script looks to have been originally written by https://github.com/aidengilmartin/speedtest-to-influxdb/blob/master/main.py and I forked it from https://github.com/martinfrancois/speedtest-to-influxdb. They did the hard work, I've continued to modify it though to fit my needs.

main.py

@@ -10,10 +10,13 @@ DB_PORT = int(os.environ.get('INFLUX_DB_PORT'))
 DB_USER = os.environ.get('INFLUX_DB_USER')
 DB_PASSWORD = os.environ.get('INFLUX_DB_PASSWORD')
 DB_DATABASE = os.environ.get('INFLUX_DB_DATABASE')
+DB_TAGS = os.environ.get('INFLUX_DB_TAGS')
 
 # Speedtest Settings
-TEST_INTERVAL = int(os.environ.get('SPEEDTEST_INTERVAL'))  # Time between tests (in seconds).
+# Time between tests (in minutes, converts to seconds).
-TEST_FAIL_INTERVAL = int(os.environ.get('SPEEDTEST_FAIL_INTERVAL'))  # Time before retrying a failed Speedtest (in seconds).
+TEST_INTERVAL = int(os.environ.get('SPEEDTEST_INTERVAL')) * 60
+# Time before retrying a failed Speedtest (in minutes, converts to seconds).
+TEST_FAIL_INTERVAL = int(os.environ.get('SPEEDTEST_FAIL_INTERVAL')) * 60
 
 influxdb_client = InfluxDBClient(
     DB_ADDRESS, DB_PORT, DB_USER, DB_PASSWORD, None)
@@ -26,13 +29,49 @@ def init_db():
         influxdb_client.create_database(
             DB_DATABASE)  # Create if does not exist.
     else:
-        influxdb_client.switch_database(DB_DATABASE)  # Switch to if does exist.
+        # Switch to if does exist.
+        influxdb_client.switch_database(DB_DATABASE)
+
+
 def pkt_loss(data):
     if 'packetLoss' in data.keys():
         return data['packetLoss']
-    else: 
+    else:
         return 0
 
+
+def tag_selection(data):
+    tags = DB_TAGS
+    if tags is None:
+        return None
+    # tag_switch takes in _data and attaches CLIoutput to more readable ids
+    tag_switch = {
+        'isp': data['isp'],
+        'interface': data['interface']['name'],
+        'internal_ip': data['interface']['internalIp'],
+        'interface_mac': data['interface']['macAddr'],
+        'vpn_enabled': (False if data['interface']['isVpn'] == 'false' else True),
+        'external_ip': data['interface']['externalIp'],
+        'server_id': data['server']['id'],
+        'server_name': data['server']['name'],
+        'server_location': data['server']['location'],
+        'server_country': data['server']['country'],
+        'server_host': data['server']['host'],
+        'server_port': data['server']['port'],
+        'server_ip': data['server']['ip'],
+        'speedtest_id': data['result']['id'],
+        'speedtest_url': data['result']['url']
+    }
+    
+    options = {}
+    tags = tags.split(',')
+    for tag in tags:
+        # split the tag string, strip and add selected tags to {options} with corresponding tag_switch data
+        tag = tag.strip()
+        options[tag] = tag_switch[tag]
+    return options
+
+
 def format_for_influx(cliout):
     data = json.loads(cliout)
     # There is additional data in the speedtest-cli output but it is likely not necessary to store.
@@ -73,8 +112,13 @@ def format_for_influx(cliout):
             }
         }
     ]
-
+    tags = tag_selection(data)
-    return influx_data
+    if tags is None:
+        return influx_data
+    else:
+        for measurement in influx_data:
+            measurement['tags'] = tags
+        return influx_data
 
 
 def main():
@@ -99,4 +143,4 @@ def main():
 
 if __name__ == '__main__':
     print('Speedtest CLI Data Logger to InfluxDB')
-    main()
+    main()