Ticket #1088 (closed: fixed)

Opened 6 years ago

Last modified 4 years ago

Migrate from Skype to a more open communication platform

Reported by: valentt Owned by:
Priority: normal Milestone:
Component: other Version:
Keywords: Cc:
Related nodes: Realization state:
Blocking: Effort: normal
Blocked by: Security sensitive: no

Description (last modified by mitar) (diff)

Skype is great because it "just works"tm for people, but it is closed proprietary platform. Before Microsoft bought it it was peer2peer but now that it is completely centralised it is really easy for anybody to record and eavesdrop voice and video conversations...

wlan slovenija as an open project promotes open platforms so it is natural that internal communication should be also on open platform.

There are two solutions that most similar communities use: IRC and XMMP (Jabber).

Both solutions are very flexible and extensive.

Both offer multiple ways for communication, multiple clients for every platform, web clients, logging, administration of channel, archive...

So why not?

Attachments

wlansi-chat.py (2.2 KB) - added by mitar 4 years ago.

Change History

comment:1 Changed 6 years ago by Musti

The only argument against is, how easily can a common user join such a chat. For more technically involved individuals, this is not a problem anyhow.

Eavesdropping of at least our public chat is ok, i mean it is public anyhow :)

I am all for a free channel, however we need to find a simple and effective way of doing it.

comment:2 Changed 6 years ago by igcek

Agree with both of the opinions.

I'm also inclined to changing skype for something else. As I see it it has to be easy to use, support as much as possible ways of loging and using it, open, has to have more ways of communication not just text (video, voice...)

When you look in to this you have to think what do we expect from user to do. So if we use skype we are expecting from a user to have a skype account. And if they don't have it to go and make it... or stay out of the party. But making an account on skype just to talk with us may turn somebody down.

So my recomendation would be to use jabber.

  • it can use video, voice...
  • it is open
  • an most importantly it supports webchat (so our expectations from the clientside would only be a browser and thats it)

If it can be done so that you can join chat from browser as anonymous user... then i think jabber is a clear winner.

comment:3 Changed 6 years ago by kostko

One nice thing about Skype is that when you come online all the chats you missed while you were offline are synced and you can see them without any effort. This is a big plus as far as I am concerned. Is this supported by Jabber out of the box?

Because not having this feature greatly slows down discussions. Also a "logging bot" that you have to query is not a solution as this introduces additional effort if you want to see past messages.

I agree that switching to a more open protocol like XMPP would be best, but only if it provides all the required functionality.

comment:4 Changed 6 years ago by igcek

It seems that it support's chat history.

http://xmpp.org/extensions/xep-0045.html#enter-history

With this xmpp or jabber when it comes to functionality sky is the limit :)

comment:5 Changed 6 years ago by mitar

  • Description modified (diff)

comment:6 Changed 6 years ago by mitar

  • Description modified (diff)

comment:7 Changed 6 years ago by mitar

So the plan was in fact, that we develop PiplMesh and then move to that platform for chat as well. But frankly, I am not so sure when it would be ready.

I think we are open, but are also practical. We are also using proprietary WiFi drivers if this works the best for us. And same for chat. Skype simply works.

But I agree that webchat is an useful feature. There are some for Skype, BTW.

And backhistory is also important. The issue with XMPP above, as far as I can tell, is that it is not that you get all history from the point you were last time online, but just the amount server sends to you. Is there any way to configure XMPP server so that it remembers for each user when it was online and sends the rest?

One more feature I like on Skype: you can edit past messages.

comment:8 follow-up: ↓ 10 Changed 6 years ago by valentt

I wouldn't like to push you to do something you don't like, but it also depends what you plan for the long term with this project.

If you plan to develop it for next few years without expecting new people to join then it doesn't matter if you keep using skype or not.

If you plan to get new people engaged then it also matters which tools are used for developers and their communication.

comment:10 in reply to: ↑ 8 Changed 6 years ago by mitar

Replying to valentt:

I wouldn't like to push you to do something you don't like, but it also depends what you plan for the long term with this project.

No, don't get me wrong. I don't argue based on what I like or dislike. I argue based on features needed for good communication. For now, Skype provides the features I see useful:

  • easy for new people to join (Skype is easy to install)
  • works on multiple platforms (works on mobile and desktop)
  • easy for novice to hang around and read what has been said in the meantime
  • you can edit posts which in a collaborative environment has proven to me very useful, because people don't use that for trolling, but for making communication better
  • most people all already running Skype anyway

So maybe the difference is just who I see main users. I see novice users. Users who need some basic help and want to ask. I believe that developers are capable of using any software we decide to use. But end users might not be. So while XMPP would maybe be closer in ideology, it is further for end-users. As far as I know there is no out-of-the-box client which does all of the above? It is? If it is, please tell. I don't mind if we have to configure server so that it works like that, but I would like that it just works for end users. And for XMPP I would then like that we can have a web client so that they don't have to install anything new.

So one other solution I was thinking about is that we would have our own real-time social network installation. As far as I know, Diaspora is not real-time. Or it is? But pump.io seems to be. identi.ca is migrating to pump.io and we might go for that? And integrate some ideas from PiplMesh into it? Like allowing to post only to users of particular node. So we would have to try out pump.io and see if it allows real-time chat (like one posts a post and then we have real-time comments on it). That it allows editing of posts. And that users can see past communications.

comment:11 Changed 6 years ago by mitar

So yes, it might be that it would be better that we don't reinvent the wheel with PiplMesh, just that we integrate something with our wireless network nature. The other idea I got was also to implement it very fast with Meteor. It is really fast to implement such things with it.

comment:12 Changed 6 years ago by mitar

Interesting: PSYC and Secure Share.

I tested pump.io, it needs a bit more work. :-)

comment:13 Changed 6 years ago by mitar

Diaspora does not seem to have real-time features.

comment:14 Changed 6 years ago by mitar

I believe that one quick and fast solution could be really done in Meteor. Anyone want to learn it and try it? JavaScript is the only language required.

comment:15 follow-up: ↓ 18 Changed 6 years ago by valentt

I understand your reasons for using Skype, but we could then better switch to Facebook because most users have Facebook and no software in necessary for communication. I know that Facebook doesn't it doesn't have rooms, but if primary motivation is end users that is where they are most of the time anyway...

Regarding XMPP or any other service hosted by wlan-si, I would recommend we try this out as a beta, not to switch to it immediately, if it works great, if it doesn't then skype is still here...

Historically most wifi networks started with their own services, not-connected and not dependent on the internet. That is really great thing to have.

Imagine having wlan-si type project installed in any country that has internet censorship problems, and when anybody disconnects internet they can't communicate even if their mesh nodes are connected.

Now imagine same scenario but they have xmpp/irc/whatever in their own network, and they can communicate even when internet gets cut off...

Regarding XMPP, it has some really powerful features, and most of them are implemented in one way or another in most servers.

We can try few and see how they work and if they satisfy none/any/most/all things you need.

Here is a nice place to start:
http://en.wikipedia.org/wiki/Comparison_of_XMPP_server_software
http://arstechnica.com/information-technology/2011/01/deciphering-the-jibber-jabber-getting-started-with-your-own-self-hosted-xmpp-server/

and I researched few features you asked for:
http://xmpp.org/extensions/xep-0045.html#enter-history
http://xmpp.org/extensions/xep-0045.html#enter-managehistory

Most multi-protocol IM clients have support for XMPP, I prefer Pidgin. I belive we could create pre-configured pidgin that would work out-of-the-box for any platform.

There is also an option to have web client so nothing is needed for clients to install, for example:
http://prosody.im/chat/

Last edited 6 years ago by valentt (previous) (diff)

comment:16 follow-up: ↓ 20 Changed 6 years ago by valentt

I had a nice chat with Prosody developers and their community on their web chat, I must say that it was easier and nicer that any Skype room experience I have, but of course that is totally subjective ;)

Here is log from there:

Valent: Is there a feature that would show me all room history since last time I logged in to room?
Zash: Yes. Since forever.
Zash: Valent: If prosody is configured to keep more than the default 20 lines and the client asks nicely, it'll be given what it wants.
Valent: nice ;)
SkyBlue: Valent: http://prosody.im/doc/modules/mod_muc#configuration
SkyBlue: max_history_messages –> change this
Valent: nice
Valent: Nice. So prosody is no problem, but it depends on the client...
Zash: Yeah
Valent: Does anybody know of any client that has feature "show me all room history since last time I was logged in" ?
Zash: Gajim does
Valent: nice
HAL: SkyBlue: Zash is running Swift version 2.0 on an unknown platform
Zash: But you need to change some settings deep in the advanced section
SkyBlue: !version Valent
HAL: SkyBlue: Valent is running Firefox version 21 on Linux
Zash: Swift also does it nicely, but only if you never close it.
Zash: Swift 3 will probably do it across restarts, once local history supports is done.
Zash: Pidgin might, libpurple does support the relevant parameters
Valent: I see Gajim could work for me, but there are also Mac users, any client that would work for them?
Zash: In Gajim, in the Advanced Config Editor, muc_restore_lines and _timeout are the settings
Zash: Swift is cross platform
SkyBlue: For Mac users: http://adium.im/

comment:17 Changed 6 years ago by mitar

Just to make sure: I really really appreciate your persistence and arguments so please don't take in any negative way that I might not agree with you (or at least with priorities). Even more, me opposing something might not even matter: you might convince others. When I argue I often write things which I believe are a common view of the community. If I am proven wrong I tend to stop arguing. :-)

In any case: if we don't talk about such things we will never see that we might not agree on them. So I really like that.

comment:18 in reply to: ↑ 15 ; follow-up: ↓ 26 Changed 6 years ago by mitar

Replying to valentt:

I understand your reasons for using Skype, but we could then better switch to Facebook because most users have Facebook and no software in necessary for communication. I know that Facebook doesn't it doesn't have rooms, but if primary motivation is end users that is where they are most of the time anyway...

No, there are multiple motivations. Facebook solves the issue with no special application needed, but it still is centralized (which is the issue with Skype) and require account (which is again issue with Skype). So two main issues we have with Skype are not solved by it.

Regarding XMPP or any other service hosted by wlan-si, I would recommend we try this out as a beta, not to switch to it immediately, if it works great, if it doesn't then skype is still here...

Sure. We just have to find which one. :-) As I am saying, I have some features I would like to have. Personally I don't see migration from Skype as so pressing issue so I would be willing to stick with it for some time until we find such technology, or find technology we can adapt to our needs, or we implement our own approach.

Historically most wifi networks started with their own services, not-connected and not dependent on the internet. That is really great thing to have.

Historically in Croatia. :-) I was really amazed to learn about that. How you put such networks up to create ways for gaming. :-) And just later on connected to the Internet. :-)

In some other networks (like Germany) Internet connectivity was the primary goal.

And I would really like to have more services in the future in the network.

Imagine having wlan-si type project installed in any country that has internet censorship problems, and when anybody disconnects internet they can't communicate even if their mesh nodes are connected.

This is why the plan is to migrate to L2 mesh network. Then users will be able to use Bonjour/ZeroConf (which is also available in Skype clients) to find each other without Internet connectivity. Like it works in your home local area network.

Our solution would anyway not work without Internet connectivity, because it would be centralized on some server in the network which is probably the one with Internet uplink anyway. For such solution we would need something running on the nodes themselves. Or as I said, a L2 network and then you can use any existing tools for that.

So we should understand what is topic of this ticket: how to use a network-centralized service for communication instead of cloud-based service. But it is still not good enough in the case of censorship where they kill the main server.

For this second use case there is this ticket: #846. I think it is better to talk about emergency communication then censorship. Censorship is only one type of possible reasons why Internet would not be available. We need to solve the issue for any such situation.

Regarding XMPP, it has some really powerful features, and most of them are implemented in one way or another in most servers.

Does it have end-to-end encryption? This is a good analysis of existing technologies, for a long-term scalable solution. Probably in our case anything would work. But then the question is really what we are searching for? Just something to replace Skype because of the ideological reasons? And we don't care about privacy, security, potential scalability, anonymity, censorship-resistance?

If we don't care about all that, what is really wrong with Skype?

and I researched few features you asked for:
http://xmpp.org/extensions/xep-0045.html#enter-history
http://xmpp.org/extensions/xep-0045.html#enter-managehistory

About room members: a member joins the room every time user's client connects to the server, or does a member joins once, even if client disconnects?

There is also an option to have web client so nothing is needed for clients to install, for example:
http://prosody.im/chat/

I am maybe trying to force two incompatible things together, but I see that the network needs two things:

  • a communication channel for developers and others interested in the network to communicate
  • a info portal for users to see when they connect to the network and can interact between each other, see location-aware information, share information, etc. (what is PiplMesh idea about)

What I would like to see, if we are migrating from Skype, that we would combine this two things. The reason is that I see this as a good way to bootstrap the second thing as at least we would be regularly using it. And at the same time it allows easy way to use by anybody as it will be a website, and a website anyone using the network would already be connected to.

So I just see this as a way that if we are going to invest the time and energy to go from Skype, that we could do this together. This is why I am a bit reluctant about XMPP (it can still be the underlying protocol, but Activity Streams or oStatus might be more suitable.

comment:19 Changed 6 years ago by mitar

Kostko, you played with XMPP, no? What is your take on it?

comment:20 in reply to: ↑ 16 ; follow-up: ↓ 24 Changed 6 years ago by mitar

Replying to valentt:

I had a nice chat with Prosody developers and their community on their web chat, I must say that it was easier and nicer that any Skype room experience I have, but of course that is totally subjective ;)

Ha ha. I opened that web interface and I couldn't:

  • click on username of somebody to communicate directly with the person
  • how can I edit my past messages?
  • how can I send files to other users?
  • the web interface does not support showing all the history since you last logged in?

(Not that we could not add those features in. :-) I am just trying to evaluate the possible work.)

Last edited 6 years ago by mitar (previous) (diff)

comment:21 follow-up: ↓ 23 Changed 6 years ago by mitar

comment:22 follow-up: ↓ 27 Changed 6 years ago by valentt

Whoow, nice conversation we got going here ;)

Well guys, I wouldn't ask for this ticket if I didn't see that using skype is ideologically wrong for this project, but still if it works for you and it would break how you are used to do stuff and if if would make core team be less effective then the hell with ideology ;) And you can close this ticked right away as far as I'm concerned ;)

If not lets talk further.

Yes wifi network in Croatia started as there was no other way to be connected for longer times and not pay lots and lost of money each month for dialup. Gaming and P2P file transfers were biggest motivators, for sure, but at least it wasn not motivated purely by pr0n (we all know Internet is for pr0n). Just kidding. Internet was thought as "the devil" because it destroyed wifi communities. Why? People who wanted "internet" and not "wifi" were end users not interested in participating, and didn't care for the wifi community, they just jelled "where is my internet" when something didn't work. And that killed most wifi communities as soon as they introduced internet into their networks...

Not back at the point...

I actually installed prosody on wr741nd router and it worked ;) I could do that again and connect it via ethernet to one node so it would become visible to everybody in the network and we can test it how many clients could this kind of "mini" server handle ;)

Perfect solution would be that every node would also be a node in communication clusters (as some kind of communication mesh) so that there is no one point of failure for communication, but that is currently very hard, but still an idea...

Only jabber server that supports clusters is ejabberd2

Regarding your idea for connecting local people, that could work, I thought to experiment to have my node open webchat window (running on local jabber server) instead of first page people visit, just to see who connect to my node and to talk with them ;)

For developers this would same way as it currently Skype works, you would have common room, could talk to people privately or publicly... same principles apply.

Last edited 6 years ago by valentt (previous) (diff)

comment:23 in reply to: ↑ 21 Changed 6 years ago by valentt

Replying to mitar:

Some more info.

Oh, nice, I wasn't expecting this feature to work. Here is what prosody developers said:

Valent: has anybody tried these "skype like" features (editing messages) on prosody?
Valent: http://superuser.com/questions/463112/can-xmpp-jabber-support-editing-of-messages-and-offline-message-like-in-skype
mathieui: message editing is a client thing
mathieui: it works well, actually
mathieui: offline messages have worked for as far as I can remember
mathieui: Valent, and for message correction, you have to use either swift, gajim (trunk) or poezio
MattJ: I also keep meaning to implement them in the web interface you're using :)
mathieui: >whatever isn't supported by GTalk in GMail is virtually non-existent in the Jabber world, this includes message editing. looks like FUD to me

comment:24 in reply to: ↑ 20 ; follow-up: ↓ 28 Changed 6 years ago by valentt

Replying to mitar:

Ha ha. I opened that web interface and I couldn't:

  • click on username of somebody to communicate directly with the person
  • how can I edit my past messages?
  • how can I send files to other users?
  • the web interface does not support showing all the history since you last logged in?

(Not that we could not add those features in. :-) I am just trying to evaluate the possible work.)

This is very, very basic web interface, I could ask prosody developers if there are other more advanced ones. I got a reply that they are working on implementing message editing for this web interface, which is really nice news for you message editing junkies :)

I never edit my messages, just blast a new one ;)

And most geeks on irc use regex that shows how they would search&replace text in previous message, that is how real men edit messages :)

comment:25 Changed 6 years ago by valentt

DISCLAIMER:

I have no special preference towards any protocol or server, we are using jabber for almost 10 years and it works great, has lots and lots of plugins, is really extensible, can be encrypted, logged, could have different clients...

Prosody jabber server I mention because we are currently using Wildfire jabber server, but it is a java beast. It uses more and more memory each month, and has memory leaks allover the place, so when I got prosody working on 1% of resources that we currently use for our wifi community I was amazed.

Last edited 6 years ago by valentt (previous) (diff)

comment:26 in reply to: ↑ 18 Changed 6 years ago by valentt

Replying to mitar:

About room members: a member joins the room every time user's client connects to the server, or does a member joins once, even if client disconnects?

I think members join once, not like on irc where you see yourself connected three times if your connection broke for few seconds...

comment:27 in reply to: ↑ 22 Changed 6 years ago by mitar

Replying to valentt:

I actually installed prosody on wr741nd router and it worked ;) I could do that again and connect it via ethernet to one node so it would become visible to everybody in the network and we can test it how many clients could this kind of "mini" server handle ;)

You can do that of course. :-)

Perfect solution would be that every node would also be a node in communication clusters (as some kind of communication mesh) so that there is no one point of failure for communication, but that is currently very hard, but still an idea...

Yes. I agree. This would be a great thing to have. Something small running on all nodes. A small webclient.

Hm, in fact it could be a simple combination of running Avahi on the node and then having a simple webchat over that. So you could see all other users in the network who are using Zeroconf to chat and even use it without a special client, just from the browser.

Here is something more written about Zeroconf and chat. So that you can see what I have in mind.

How big is Avahi OpenWrt package? But all this what we are writing here would be more suitable for #846.

Regarding your idea for connecting local people, that could work, I thought to experiment to have my node open webchat window (running on local jabber server) instead of first page people visit, just to see who connect to my node and to talk with them ;)

Yes, and now imagine such thing on the network scale. That you can see who else is connected to the network, communicate with them. This makes easier to build a community. You can read more about my ideas here.

comment:28 in reply to: ↑ 24 Changed 6 years ago by mitar

Replying to valentt:

And most geeks on irc use regex that shows how they would search&replace text in previous message, that is how real men edit messages :)

But when you are helping a novice over the chat, it makes it much better when you can fix your own mistakes in commands you are writing to somebody.

Also, XMPP allows only the last message to be edited. And is there a concept of room admin? Which could edit messages of others as well?

comment:30 Changed 6 years ago by valentt

I found one awesome IRC setup that shows how all (or almost all) current Skype features can be replicated and you get even more freedom and choice because you own the whole stack and can experiment with new features...

comment:31 follow-up: ↓ 38 Changed 6 years ago by mitar

Yes. I am using ZNC for my IRC connections and we have it installed on our server. But I am not so sure about user friendliness of it. So you would suggest that we provide it for everybody in the network?

comment:32 Changed 6 years ago by mitar

An interesting alternative is to use a permanent Google Hangout.

Last edited 5 years ago by mitar (previous) (diff)

comment:33 Changed 5 years ago by igcek

This one is also interesting: http://retroshare.sourceforge.net/

comment:35 Changed 4 years ago by mitar

I really like phoenix. A distributed system.

comment:36 Changed 4 years ago by mitar

comment:37 Changed 4 years ago by valentt

I have started testing znc service (sort of irq message queue proxy) for irc as skype replacement, and I'm really impressed so far

znc also has push plugin so if enabled (per user choice) you get push notifications to your android or ios device when somebody sends private message or mentions you

tested it with colloquy client on my iPhone and it works great

also nice is that users who don't want to install anything can use web client - http://webchat.freenode.net?channels=%23otvorenamreza.org

if you want to test znc connect to http://95.85.26.162:49161 with username:test password:test123 - this is an empty account so you need to add irc network and rooms which you want to connect to.

comment:38 in reply to: ↑ 31 Changed 4 years ago by valentt

Replying to mitar:

Yes. I am using ZNC for my IRC connections and we have it installed on our server. But I am not so sure about user friendliness of it. So you would suggest that we provide it for everybody in the network?

Probably only really active members will use znc (I see around 10 max active and regular people in current skype chat, others just come once and never return), others can use just regular irc clients or even web clients so they don't need to install anything.

comment:39 Changed 4 years ago by kostko

I have just set up Rocket Chat on our own server, check it out:

https://chat.wlan-si.net

comment:40 Changed 4 years ago by valentt

I'm really likeing Rocket.Chat, but as it is so new there is no mention of encryption and privacy, I hope they add end to end encryption to Rocket.Chat.

Also EFF has a nice secure messaging scorecard - https://www.eff.org/secure-messaging-scorecard

And just stumbled upon Tox.im p2p secure messenger, it looks really, really interesting -

Last edited 4 years ago by valentt (previous) (diff)

comment:41 Changed 4 years ago by valentt

Just talked with a coleague who says there are quite a fet Web frontends for Jabber/XMMP that look like Slack, so it is also interesting thing to know.

comment:42 Changed 4 years ago by mitar

but as it is so new there is no mention of encryption and privacy, I hope they add end to end encryption to Rocket.Chat.

What do you mean? It goes over SSL over our servers. What more would you want? Which type of privacy for a public chat?

Changed 4 years ago by mitar

comment:43 Changed 4 years ago by mitar

I attached our Linux Skype connecting script to this ticket to archive it.

comment:44 Changed 4 years ago by mitar

  • Status changed from new to closed
  • Resolution set to fixed

We migrated to open source Rocket.Chat for which we host the server. We are deprecating our Skype chat.

comment:45 Changed 4 years ago by mitar

Also interesting open source group chat from Dropbox: https://zulip.org/

Note: See TracTickets for help on using tickets.