PubSubQfrom SopherApps

Details

  • Name PubSubQ
  • Summary A simple Pub/Sub that uses websockets with opt-in persistence
  • Price (Monthly) 2,500,000 UGX
  • Free Trial 7 days
  • Category PubSub
  • Tags websockets pubsub
  • Download

Description

A simple Pub/Sub that uses websockets with opt-in persistence

Usage

Here is a sample of potential use cases

  • When there are multiple data pipelines running in separate apps, or even on separate server yet they are serving the same front end client
  • When there are multiple data pipelines where some of them depend on the output of other pipelines
  • When there are multiple data sources, each with their own unique business logic
  • When there are multiple data pipelines being developed by separate teams
  • Create a quick Chat production-level server

Features

  • Well-known TransportUses the well-known websocket transport to both send and receive messages
  • Publish to '/send/<message_type> 'Publishers send text or JSON messages to /send/<message_type> e.g. /send/foo
  • Receive All Message Type on '/receive'Subscribers can receive messages of all types from the route /receive/
  • Receive Single Message Type on '/receive/<message_type>'Subscribers receive messages of a specific type from the route /receive/<message_type> e.g. receive/foo
  • Non-persistent By DefaultMessages are not perisisted by default. This means that they are lost as soon as they are published. This ensures every subscriber to a given message type is in sync with the others.
  • Optional Persistence (since v0.0.1-alpha.2)To ensure subscribers recover in case they disconnect, one can enable persistence using the CLI argument '--persistent' (or '-p') set to 'true'. An optional '--ttl' flag can also be set to determine the time a given message can be retrievable if missed before it is deleted.
  • Opt-in SecurityFor security, there is an option to pass an optional '--authurl' arg or a to the CLI sepecifying a URL where the provided username and password data can be channeled to authenticate the connection as JSON. When the response from 'authurl' is of status code 200, the connection is accepted. When '--authurl' is not provided, authentication is deactivated.
  • Run in the backgroundSince version "v0.0.1-alpha.4", it is possible to run the app in the background while logging to a file.

Quick Start

  • Download the PubSubQ app for your operating system and computer architecture (note that most computers have AMD64 architecture. If it doesn't work you can always download the ARM version)
Operating System Architecture Download
Windows AMD64 Download
Windows ARM Download
MacOS AMD64 Download
MacOS ARM64 Download
Linux AMD64 Download
Linux ARM64 Download
  • Start the app setting it to run on port 8080. Assuming you are in the folder where you downloaded your app

For Windows:

pubsubq.exe run -s 8080

For Linux:

./pubsubq run -s 8080
  • Ensure you have python +3.6 installed on your PC. Here are the instructions for installing the latest python.
  • Create a folder somewhere on your PC and enter it. For example let the folder be pubsub
mkdir pubsub
 cd pubsub
  • Create a virtual environment and activate it

For windows:

python -m venv env
 .\env\Scripts\activate

For Linux:

python3 -m venv env
 source env/bin/activate
  • Install the websockets package
pip install websockets
  • Create a new folder subscribers and enter it
mkdir subscribers
 cd subscribers
  • Create a new file subscriber_one.py and add the following code to it
"""Receives messages from only /foo i.e. from publisher_one"""
 import asyncio
 import websockets

 url = "ws://localhost:8080/receive/foo"

 async def listen():
   async with websockets.connect(url) as websocket:
     last_message = "done_one"

     while True:
       msg = await websocket.recv()
       print(msg)

       if msg == last_message:
         break

 asyncio.get_event_loop().run_until_complete(listen())
  • Run the subscriber_one.py file
python subscriber_one.py
  • You will later check what is printed on the terminal
hey
 you
 how do you do?
 done_one
  • Create a new file subscriber_two.py and add the following code to it
"""Receives all messages as it is listening to /"""
 import asyncio
 import websockets

 url = "ws://localhost:8080/receive/"

 async def listen():
     async with websockets.connect(url) as websocket:
         last_messages = ["done_one", "done_two"]

         captured_last_messages = []

         while True:
             msg = await websocket.recv()
             print(msg)

             if msg in last_messages:
                 captured_last_messages.append(msg)

             if len(set(captured_last_messages)) == 2:
                 break

 asyncio.get_event_loop().run_until_complete(listen())
  • Open another terminal (Command line) in the same folder and run the subscriber_two.py file
python subscriber_two.py
  • You will later check what is printed on the terminal. It might look something like
hey
 you
 how do you do?
 1
 done_one
 3
 78
 done_two
  • Open another terminal (Command line) in the same folder subscribers and go to the parent folder of subscribers
cd ../
  • Create a folder publishers and enter it
mkdir publishers
 cd publishers
  • Create a file publisher_one.py and add the following code
"""loops through a list of words and publishes them to the pubsubq instance"""
 import asyncio
 import websockets

 words = ["hey", "you", "how do you do?"]
 url = "ws://localhost:8080/send/foo"

 async def publish():
     async with websockets.connect(url) as websocket:
         for word in words:
             await websocket.send(word)
             await asyncio.sleep(1)

         await websocket.send("done_one")

 asyncio.get_event_loop().run_until_complete(publish())
  • Run the file
python publisher_one.py
  • Create another file publisher_two.py and add the following code
"""Loops through numbers, publishing them to the pubsubq instance"""
 import asyncio
 import websockets

 numbers = [1, 3, 78]
 url = "ws://localhost:8080/send/bar"

 async def publish():
     async with websockets.connect(url) as websocket:
         for number in numbers:
             await websocket.send(f'{number}')
             await asyncio.sleep(1)

         await websocket.send("done_two")

 asyncio.get_event_loop().run_until_complete(publish())
  • Open another terminal (Command line) in the same folder publishers and run the publisher_two.py file
python publisher_two.py
  • You can now go and check the outputs in the terminals of the subscribers

  • You can also run pubsubq as a daemon by passing it a --daemon (or -d) flag. By default, it will log to a pubsubq.log file in the same folder as the running app. To change the log path, the --log (or -l) flag is provided with an absolute path file

  • To see more details of how this app is used, pass the --help (or -h) flag to run e.g.

For linux and MacOS

pubsubq run --help

or for windows

pubsubq.exe run --help

Downloads