Ticket #1145 (new)

Opened 6 years ago

Last modified 3 years ago

3.0b quick TODO summary

Reported by: kostko Owned by: mitar
Priority: normal Milestone: 3.0b
Component: nodewatcher/modules Version:
Keywords: Cc:
Related nodes: Realization state:
Blocking: Effort: high
Blocked by: Security sensitive: no

Description (last modified by kostko) (diff)

This is a quick summary of things that need to be done for 3.0b.

  • General
    • Some minor refactorings all around to improve code structure.
    • There is still some legacy code under legacy.nodes.templatetags.* that should be removed or moved to their respective modules.
    • Using keywords in APIs and display name through Django translation. #1344
  • Migrations
    • We already have migrations from the 2.0 database schema.
    • We need migrations that populate the datastream database? Could be a problem if there are performance issues (see below).
  • modules.monitor.datastream (and related datastream library)
    • Need a RateField implementation, based on two streams – one stores the counter's absolute value and the other holds the derivatives.
    • There were some performance issues mentioned, these need to be addressed.
    • Visualization:
      • Client-side graph rendering, #487.
      • Client-side topology rendering, #736.
    • Insertion of NULL values when there is no data for a stream for a specific node (even for nodes that are filtered out by network monitoring processors).
  • modules.devices
    • Currently, many router descriptors are untested and probably not completely right. They need to be checked against actual hardware (#1256).
    • We should add device descriptors for the x86 architecture (#949).
  • modules.monitor.*
    • Various modules that implement 2.0 functionality on the new nodewatcher framework (populating the monitoring schema and validating node configuration).
    • #1144
    • Currently a node's timezone settings are hardcoded in the CGM. Should it be added to the configuration schema or handled somewhat differently?
  • modules.frontend.*
    • Display of node status, graphs, events, generating images, etc. (also see #1028).
    • This thing still need to be defined, we should probably start with something similar to 2.0 and then change later on.
  • core.frontend.static
    • Client-side registry code (registry.js) currently doesn't have proper error handling and simply puts up an alert. This should be refactored when building the frontend modules.
    • #1121
  • core.models
    • A node's name in GeneralConfig should be validated by a specific regular expression so that we allow the same names as in 2.0.
  • Admin
    • Interface for editing IP allocation pools.
    • Interface for editing projects.
  • Support for configuration of password authentication (see #1241).
  • Tests of everything.
  • Extract some code into packages:
    • Django registry.
    • Our support for pools and registration pattern for various components.
    • Based on that also package for menus.
    • And another for frontend components.
    • Smaller things could go into django-missing.
  • Importing of data into datastream:
    • Old data.
    • All KORUZA data (also high resolution measurements).
    • New real-time data while the importing is in progress.
  • Importing of data:
    • Users.
    • Nodes.
    • Other stuff (projects, permissions).
  • wlan slovenija homepage needs:
    • New global stats widget (nodes, users).
    • Network map.
    • Integration of user system.
  • grow and dev Trac also have user system integration.
  • Easy registration of node nodes:
    • Default config for typical types of nodes.
    • Cloning of nodes.
    • Simple node registration.
  • Documentation of everything.
  • Signal-to-noise computation in datastream.
  • E-mail sending for all notifications.
    • When image has been generated. An e-mail with link to the image page.
    • When there are errors.
  • Google Maps satellite layer for node map.
  • Nice URLs for nodes (UUID + slug).
  • Network statistics.
  • Node list should have client count.
  • Global client list.
  • Events page for the node.
  • Peers table for the node.
  • Subnets table for the node.
  • User registration:
    • Default project should be Slovenija.
    • For now only English language should be available.
    • Email subject is at the moment not set (it is empty).
    • User should have permissions to add new nodes by default.
  • Support for assigning node permissions to others (at least to change who is a maintainer).
  • Use django channels to provide real-time feedback on image generation, so that one does not have to wait and reload.

If there is something missing, this list should be updated.

Change History

comment:1 Changed 6 years ago by kostko

  • Effort changed from normal to high
  • Milestone set to 3.0b

comment:2 Changed 6 years ago by kostko

  • Description modified (diff)

comment:3 Changed 6 years ago by kostko

  • Description modified (diff)

comment:4 Changed 6 years ago by kostko

  • Description modified (diff)

comment:5 Changed 6 years ago by kostko

  • Description modified (diff)

comment:6 Changed 6 years ago by kostko

  • Description modified (diff)

comment:7 Changed 6 years ago by kostko

Monitoring module for core.interfaces now done. The following monitoring modules for data collection are still needed (based on node monitoring core schema):

  • core.general – should be straightforward to implement.
  • core.interfaces.limits – first requires support for reporting per-interface qos in the firmware telemetry provider.
  • core.interfaces.network – first requires support for reporting per-interface IP addresses in the firmware telemetry provider.
  • system.resources.network – first requires support for reporting network resource status in the firmware telemetry provider.
  • system.checks – should be straightforward to implement.
  • network.measurement.rtt – see #1144. Also the schema assumes each measurement has a known source Node – this means that the server running nodewatcher must have a corresponding Node entry and it must somehow be aware of it.
  • network.clients – first requires support for better reporting of client leases in the firmware telemetry provider.
Last edited 5 years ago by kostko (previous) (diff)

comment:8 Changed 6 years ago by kostko

  • Description modified (diff)

Router device descriptors have now been moved from modules.platforms.openwrt to modules.devices (as they are actually platform-independent).

comment:9 Changed 6 years ago by kostko

  • Description modified (diff)

comment:10 Changed 6 years ago by kostko

  • Description modified (diff)

comment:11 Changed 6 years ago by kostko

  • Description modified (diff)

comment:12 Changed 6 years ago by kostko

  • Description modified (diff)

It seems that the type field is not missing, we have just moved it into TypeConfig and forgot to update the schema.

comment:13 Changed 6 years ago by kostko

  • Description modified (diff)

comment:14 Changed 6 years ago by kostko

  • Description modified (diff)

comment:15 Changed 5 years ago by kostko

  • Description modified (diff)

comment:16 Changed 5 years ago by kostko

  • Description modified (diff)

comment:17 Changed 5 years ago by mitar

About node names. We currently have the idea that name is almost automatically generated from the location. But there could be multiple locations in different cities or projects. So one possibility would be that we would have names unique per project and URL then nested under the project.

One other option would be that we would have a name of the node being arbitrary string and then we slugify it. And by default we simply name the node same as location.

comment:18 Changed 5 years ago by kostko

  • Description modified (diff)

comment:19 Changed 5 years ago by kostko

  • Description modified (diff)

comment:20 Changed 5 years ago by kostko

  • Description modified (diff)

comment:21 Changed 5 years ago by kostko

We have now switched to using nodewatcher-agent (see #1207) by default in nodewatcher v3. All new modules will be using the new format.

comment:22 Changed 5 years ago by kostko

  • Description modified (diff)

comment:23 Changed 5 years ago by kostko

  • Description modified (diff)

comment:24 Changed 5 years ago by kostko

  • Description modified (diff)

comment:25 Changed 5 years ago by kostko

  • Description modified (diff)

comment:26 Changed 5 years ago by kostko

  • Description modified (diff)

comment:27 Changed 5 years ago by kostko

comment:28 Changed 5 years ago by kostko

  • Description modified (diff)

comment:29 Changed 5 years ago by kostko

  • Description modified (diff)

comment:30 Changed 5 years ago by mitar

Do we require node names to be unique? Do we want to require that?

What about our current code which generates node names automatically from the location? How we will support that?

comment:31 Changed 5 years ago by kostko

Currently, in v3, we don't require node names to be unique. In #873 we discussed the idea that node names should be unique "per project". This can be achieved in the projects module by performing an additional validation step, ensuring name uniqueness only inside a project.

Node names are also used for nicer URLs (instead of UUIDs). If node names are not unique, then such resolution will not be possible for non-unique node names.

comment:32 Changed 5 years ago by mitar

OK, let's move discussion there.

comment:33 Changed 5 years ago by kostko

  • Description modified (diff)

comment:34 Changed 5 years ago by mitar

  • Description modified (diff)

comment:35 Changed 5 years ago by kostko

  • Description modified (diff)

comment:36 Changed 5 years ago by kostko

  • Description modified (diff)

comment:37 Changed 4 years ago by mitar

BTW, see goals for 3.0b. Are we still targeting all of them?

comment:38 Changed 4 years ago by mitar

For node names, we discussed that we should just generate slugs from them (using slugify2), allowing names to be anything. And then node URLs would be something like /node/<UUID>/<slug>/. And we would redirect always to canonical URL. So if node is renamed, old URLs would still work.

comment:39 Changed 4 years ago by mitar

  • Description modified (diff)

comment:40 Changed 4 years ago by mitar

  • Description modified (diff)

comment:41 Changed 4 years ago by mitar

  • Description modified (diff)

comment:42 Changed 4 years ago by kostko

  • Description modified (diff)

comment:43 Changed 4 years ago by mitar

  • Description modified (diff)

comment:44 Changed 4 years ago by mitar

You should run migrate and it will add node maintainers group to all users. It will also add it automatically from now on to all users.

comment:45 Changed 4 years ago by mitar

  • Description modified (diff)

comment:46 Changed 3 years ago by mitar

  • Description modified (diff)

comment:47 Changed 3 years ago by valentt

One small but important usability issue: https://dev.wlan-si.net/ticket/1399
IP address in peers table was really useful.

From pure users point of view biggest issues is UI/UX in current version:
https://dev.wlan-si.net/ticket/1367

Last edited 3 years ago by valentt (previous) (diff)

comment:48 Changed 3 years ago by valentt

Node List page is loading to slow. Can it be made to load faster?

Each time any client loads Node List page list of all nodes is send to clients browser. Is that really necessary?

comment:49 Changed 3 years ago by kostko

  • Description modified (diff)
Note: See TracTickets for help on using tickets.