Merge pull request #1 from breadlysm/v0.6.0-isp-&-location-Influxdb-tags
V0.6.0 isp & location influxdb tags
This commit is contained in:
breadlysm
GitHub
commit
7207aeaea9
4 changed files with 138 additions and 29 deletions
17
.env
Normal file
17
.env
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
# Comment out any variables you don't need except those labeled under required.
|
||||
# Required
|
||||
INFLUX_DB_ADDRESS = 192.168.1.1
|
||||
# Required
|
||||
INFLUX_DB_PORT = 8086
|
||||
|
||||
#INFLUX_DB_USER =
|
||||
#INFLUX_DB_PASSWORD =
|
||||
INFLUX_DB_DATABASE = speedtest
|
||||
# tag names will be used to add tags to your speedtests. They can be used to compare data overtime. These should be comma seperated.
|
||||
INFLUX_DB_TAGS = isp,vpn_enabled, external_ip, server_id, server_name, server_location, speedtest_url
|
||||
|
||||
# Speedtest Settings
|
||||
# Required. Time between tests (in minutes, converts to seconds).
|
||||
SPEEDTEST_INTERVAL = 60
|
||||
# Required. Time before retrying a failed Speedtest (in minutes, converts to seconds).
|
||||
SPEEDTEST_FAIL_INTERVAL = 5
|
||||
|
|
@ -2,7 +2,7 @@ FROM python:3.8-slim-buster
|
|||
|
||||
LABEL maintainer="Josh Smith" \
|
||||
description="Original by Aiden Gilmartin. Speedtest to InfluxDB data bridge"
|
||||
#wget "https://ookla.bintray.com/download/$(wget https://ookla.bintray.com/download/ -q -O - | grep x86_64-linux.tgz\" | grep -Po "(?<=href=\")[^^\"]*" | cut -d ":" -f 2)"
|
||||
|
||||
# Install dependencies
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
RUN apt-get update
|
||||
|
|
|
|||
90
README.md
90
README.md
|
|
@ -1,12 +1,14 @@
|
|||
# Speedtest to InfluxDB
|
||||
|
||||
This is a small Python script that will continuously run the Speedtest CLI application by Ookla, reformat the data output and forward it on to an InfluxDB database.
|
||||
This is a Python script that will continuously run the official Speedtest CLI application by Ookla, takes input from environment variables, formats data and writes it to an InfluxDB database.
|
||||
|
||||
You may want to do this so that you can track your internet connections consistency over time. Using Grafana you can view and explore this data easily.
|
||||
This script will allow you to measure your internet connections speed and consistency over time. It uses env variables as configuration. It's as easy to use as telling your Docker server a 1 line command and you'll be set. Using Grafana you can start exploring this data easily.
|
||||
|
||||
|
||||
|
||||
## Using the script
|
||||
There are some added features to allow some additional details that Ookla provides as tags on your data. Some examples are your current ISP, the interface being used, the server who hosted the test. Overtime, you could identify if some serers are performing better than others.
|
||||
|
||||
## Configuring the script
|
||||
|
||||
The InfluxDB connection settings are controlled by environment variables.
|
||||
|
||||
|
|
@ -16,32 +18,78 @@ The variables available are:
|
|||
- INFLUX_DB_USER = user
|
||||
- INFLUX_DB_PASSWORD = pass
|
||||
- INFLUX_DB_DATABASE = speedtest
|
||||
- SPEEDTEST_INTERVAL = 1800
|
||||
- SPEEDTEST_FAIL_INTERVAL = 300
|
||||
- INFLUX_DB_TAGS = *comma seperated list of tags. See below for options*
|
||||
- SPEEDTEST_INTERVAL = 60
|
||||
- SPEEDTEST_FAIL_INTERVAL = 5
|
||||
|
||||
### Variable Notes
|
||||
- Intervals are in minutes. *Script will convert it to seconds.*
|
||||
- If any variables are not needed, don't declare them. Functions will operate with or without most variables.
|
||||
- Tags should be input without quotes. *INFLUX_DB_TAGS = isp, interface, external_ip, server_name, speedtest_url*
|
||||
|
||||
### Tag Options
|
||||
The Ookla speedtest app provides a nice set of data beyond the upload and download speed. The list is below.
|
||||
|
||||
| Tag Name | Description |
|
||||
|- |- |
|
||||
| isp | Your connections ISP |
|
||||
| interface | Your devices connection interface |
|
||||
| internal_ip | Your container or devices IP address |
|
||||
| interface_mac | Mac address of your devices interface |
|
||||
| vpn_enabled | Determines if VPN is enabled or not? I wasn't sure what this represented |
|
||||
| external_ip | Your devices external IP address |
|
||||
| server_id | The Speedtest ID of the server that was used for testing |
|
||||
| server_name | Name of the Speedtest server used for testing |
|
||||
| server_country | Country where the Speedtest server resides |
|
||||
| server_host | Hostname of the Speedtest server |
|
||||
| server_port | Port used by the Speedtest server |
|
||||
| server_ip | Speedtest server's IP address |
|
||||
| speedtest_id | ID of the speedtest results. Can be used on their site to see results |
|
||||
| speedtest_url | Link to the testing results. It provides your results as it would if you tested on their site. |
|
||||
|
||||
### Additional Notes
|
||||
Be aware that this script will automatically accept the license and GDPR statement so that it can run non-interactively. Make sure you agree with them before running.
|
||||
|
||||
### 1. No Container
|
||||
## Running the Script
|
||||
|
||||
1. [Install the Speedtest CLI application by Ookla.](https://www.speedtest.net/apps/cli)
|
||||
|
||||
NOTE: The `speedtest-cli` package in distro repositories is an unofficial client. It will need to be uninstalled before installing the Ookla Speedtest CLI application with the directions on their website.
|
||||
|
||||
2. Install the InfluxDB client for library from Python.
|
||||
|
||||
`pip install influxdb`
|
||||
|
||||
3. Run the script.
|
||||
|
||||
`python3 ./main.py`
|
||||
|
||||
### 2. Run with Docker or Podman
|
||||
### Ideal option, run as a Docker container.
|
||||
|
||||
1. Build the container.
|
||||
|
||||
`docker build -t aidengilmartin/speedtest-influx ./`
|
||||
`docker build -t breadlysm/speedtest-to-influxdb ./`
|
||||
|
||||
2. Run the container.
|
||||
```
|
||||
docker run -d --name speedtest-influx \
|
||||
-e 'INFLUX_DB_ADDRESS'='_influxdb_host_' \
|
||||
-e 'INFLUX_DB_PORT'='8086' \
|
||||
-e 'INFLUX_DB_USER'='_influx_user_' \
|
||||
-e 'INFLUX_DB_PASSWORD'='_influx_pass_' \
|
||||
-e 'INFLUX_DB_DATABASE'='speedtest' \
|
||||
-e 'SPEEDTEST_INTERVAL'='1800' \
|
||||
-e 'SPEEDTEST_FAIL_INTERVAL'='60' \
|
||||
breadlysm/speedtest-to-influxdb
|
||||
```
|
||||
### No Container
|
||||
|
||||
`docker run -d --name speedtest-influx aidengilmartin/speedtest-influx`
|
||||
1. Clone the repo
|
||||
|
||||
`git clone https://github.com/breadlysm/speedtest-to-influxdb.git`
|
||||
|
||||
2. Configure the .env file in the repo or set the environment variables on your device.
|
||||
|
||||
3. [Install the Speedtest CLI application by Ookla.](https://www.speedtest.net/apps/cli)
|
||||
|
||||
NOTE: The `speedtest-cli` package in distro repositories is an unofficial client. It will need to be uninstalled before installing the Ookla Speedtest CLI application with the directions on their website.
|
||||
|
||||
4. Install the InfluxDB client for library from Python.
|
||||
|
||||
`pip install influxdb`
|
||||
|
||||
5. Run the script.
|
||||
|
||||
`python3 ./main.py`
|
||||
|
||||
|
||||
|
||||
This script looks to have been originally written by https://github.com/aidengilmartin/speedtest-to-influxdb/blob/master/main.py and I forked it from https://github.com/martinfrancois/speedtest-to-influxdb. They did the hard work, I've continued to modify it though to fit my needs.
|
||||
54
main.py
54
main.py
|
|
@ -10,10 +10,13 @@ DB_PORT = int(os.environ.get('INFLUX_DB_PORT'))
|
|||
DB_USER = os.environ.get('INFLUX_DB_USER')
|
||||
DB_PASSWORD = os.environ.get('INFLUX_DB_PASSWORD')
|
||||
DB_DATABASE = os.environ.get('INFLUX_DB_DATABASE')
|
||||
DB_TAGS = os.environ.get('INFLUX_DB_TAGS')
|
||||
|
||||
# Speedtest Settings
|
||||
TEST_INTERVAL = int(os.environ.get('SPEEDTEST_INTERVAL')) # Time between tests (in seconds).
|
||||
TEST_FAIL_INTERVAL = int(os.environ.get('SPEEDTEST_FAIL_INTERVAL')) # Time before retrying a failed Speedtest (in seconds).
|
||||
# Time between tests (in minutes, converts to seconds).
|
||||
TEST_INTERVAL = int(os.environ.get('SPEEDTEST_INTERVAL')) * 60
|
||||
# Time before retrying a failed Speedtest (in minutes, converts to seconds).
|
||||
TEST_FAIL_INTERVAL = int(os.environ.get('SPEEDTEST_FAIL_INTERVAL')) * 60
|
||||
|
||||
influxdb_client = InfluxDBClient(
|
||||
DB_ADDRESS, DB_PORT, DB_USER, DB_PASSWORD, None)
|
||||
|
|
@ -26,13 +29,49 @@ def init_db():
|
|||
influxdb_client.create_database(
|
||||
DB_DATABASE) # Create if does not exist.
|
||||
else:
|
||||
influxdb_client.switch_database(DB_DATABASE) # Switch to if does exist.
|
||||
# Switch to if does exist.
|
||||
influxdb_client.switch_database(DB_DATABASE)
|
||||
|
||||
|
||||
def pkt_loss(data):
|
||||
if 'packetLoss' in data.keys():
|
||||
return data['packetLoss']
|
||||
else:
|
||||
return 0
|
||||
|
||||
|
||||
def tag_selection(data):
|
||||
tags = DB_TAGS
|
||||
if tags is None:
|
||||
return None
|
||||
# tag_switch takes in _data and attaches CLIoutput to more readable ids
|
||||
tag_switch = {
|
||||
'isp': data['isp'],
|
||||
'interface': data['interface']['name'],
|
||||
'internal_ip': data['interface']['internalIp'],
|
||||
'interface_mac': data['interface']['macAddr'],
|
||||
'vpn_enabled': (False if data['interface']['isVpn'] == 'false' else True),
|
||||
'external_ip': data['interface']['externalIp'],
|
||||
'server_id': data['server']['id'],
|
||||
'server_name': data['server']['name'],
|
||||
'server_location': data['server']['location'],
|
||||
'server_country': data['server']['country'],
|
||||
'server_host': data['server']['host'],
|
||||
'server_port': data['server']['port'],
|
||||
'server_ip': data['server']['ip'],
|
||||
'speedtest_id': data['result']['id'],
|
||||
'speedtest_url': data['result']['url']
|
||||
}
|
||||
|
||||
options = {}
|
||||
tags = tags.split(',')
|
||||
for tag in tags:
|
||||
# split the tag string, strip and add selected tags to {options} with corresponding tag_switch data
|
||||
tag = tag.strip()
|
||||
options[tag] = tag_switch[tag]
|
||||
return options
|
||||
|
||||
|
||||
def format_for_influx(cliout):
|
||||
data = json.loads(cliout)
|
||||
# There is additional data in the speedtest-cli output but it is likely not necessary to store.
|
||||
|
|
@ -73,8 +112,13 @@ def format_for_influx(cliout):
|
|||
}
|
||||
}
|
||||
]
|
||||
|
||||
return influx_data
|
||||
tags = tag_selection(data)
|
||||
if tags is None:
|
||||
return influx_data
|
||||
else:
|
||||
for measurement in influx_data:
|
||||
measurement['tags'] = tags
|
||||
return influx_data
|
||||
|
||||
|
||||
def main():
|
||||
|
|
|
|||
Loading…
Reference in a new issue