Ticket #1079 (new)

Opened 6 years ago

Last modified 5 years ago

Send and export sensor data to other data platforms

Reported by: valentt Owned by: kostko
Priority: normal Milestone: Ideas for the future
Component: nodewatcher/extra Version:
Keywords: projectideas, biggerproject Cc:
Related nodes: Realization state:
Blocking: Effort: normal
Blocked by: Security sensitive: no

Description (last modified by valentt) (diff)

Currently sensor data (temperature and others) is collected centrally in nodewatcher, but it would be great to integrate data sensor inputs with other open database platforms like:

The best way would probably be that nodewatcher still collects data and then it pushes data to those services. Additionally, an OpenWrt package could be developed which would allow node itself to take the data collected through local nodewatcher scripts and push them to other data platforms.

Change History

comment:1 in reply to: ↑ description Changed 6 years ago by kostko

Replying to valentt:

Currently sensor data (temperature) isn't collected, only lossy rrd database is available which is not optimal solution for data analysis.

This is not correct. Sensor data in nodewatcher 3.0 is collected via datastream backends which can be anything (currently only a MongoDB backend is implemented). These services you are talking about should then probably be implemented as datastream backends.

As for the old nodewatcher (v2), the data is also collected in MongoDB (at least at wlan slovenia installation), so we have it all archived to be converted to new datastream format after the new nodewatcher is ready for deployment.

References:

comment:2 Changed 6 years ago by valentt

Thanks a lot for clarifying how Nodewatcher collects data. I can share how I push data via few bash scripts, but I can't code beside that :(

Users who wish these features they would still need to manually open web accounts on those services, I can also make step-by-step guide on how to open an account and register new sensors.

All data is sent via wget or curl http push commands that contain your API key, specific sensor key and data.

Here is my bash script that pushes data into Sen.se and Cosm data platforms:
https://github.com/valentt/Internet-of-Things/blob/master/sensors.sh

This script doesn't work on openwrt devices because wget and curl that are in openwrt don't support https :(

Last edited 6 years ago by valentt (previous) (diff)

comment:3 Changed 6 years ago by mitar

  • Description modified (diff)

comment:4 Changed 6 years ago by mitar

  • Owner set to kostko
  • Component changed from other to nodewatcher/extra
  • Milestone set to Ideas for the future

comment:5 Changed 6 years ago by valentt

This ticket seams like duplicate - #1045
Weatherunderground could be also another source for sending (verified) sensor data.

Last edited 6 years ago by mitar (previous) (diff)

comment:6 Changed 6 years ago by mitar

It is for other data repository, it is not a exact duplicate.

comment:7 Changed 6 years ago by valentt

I just tried and sending data to open.sen.se works! I had issues with previously that curl POST command didn't work on openwrt, but now it works great!

Here is the script I used to post directly to open.sen.se:

#!/bin/ash

# api key for open.sen.se
apikey=
roomfeed=

roomtemp=$(digitemp_DS9097 -a -q -o"%.2C")

# post to open.sen.se
curl 'http://api.sen.se/events/?sense_key='$apikey -X POST -H "Content-type: application/json" -d '[{"feed_id": '$roomfeed',"value": '$roomtemp'}]'

If you are curious take a peek what is the temperature in my flat ;)

comment:8 Changed 6 years ago by valentt

  • Description modified (diff)

comment:9 Changed 5 years ago by valentt

  • Description modified (diff)

comment:10 Changed 5 years ago by mitar

#1045 was marked as duplicate of this ticket.

comment:11 Changed 5 years ago by mitar

A very interesting project is also RIPE Atlas.

comment:12 Changed 5 years ago by mitar

  • Keywords gsoc removed

Removing for now from GSoC ideas list.

comment:13 Changed 5 years ago by mitar

  • Keywords projectideas, biggerproject added
Note: See TracTickets for help on using tickets.