I built a chatbot
1664928000

Hopefully I'll make it to the homepage of Product Hunt this time... go and check it out: https://pubot.chat/


PuBot

PuBot is a single-page application (SPA) that brings all of these technical concepts, that I discuss in detail below, to life. The idea here is that the entire viewport is the chat app and it’s automatically invoked on the homepage


I built the app following the mobile-first principals and it does have a responsive design. I have used this design before, like on the App Evolved homepage, because the way I built the JavaScript UI navigation logic makes this design very easy to reuse


Using natural language processing allows for near infinite use-cases. You can speak to the app just like you would a human—of course it’s not turing-complete but you know what I mean. I’m building this natural language processing around a codec layer that in itself can interact with an infinite number of data APIs (Google Places API as an example) and translate the data back into messages for the user


I’m quite literally creating a text or chat version of Siri and I’m happy with having a handful of users


The app is currently in BETA and not accepting any more users but you are welcome to provide your email address and I’ll notify you when the app goes live


WebSocket server

Chat applications generally want to be real-time and bidirectional communication protocols like WebSockets achieve this very well (i.e.: the server can invoke the messages) 


I’m not familiar with HTTP/2 server push but I do have experience with WebSockets so using them makes the most sense


WebSockets can be seen as persistent HTTP connections that have sent through the connection upgrade headers (indicating that they are wanting to be a long-running connection)


PHP

I decided to write the app in PHP because it's the language I’m most familiar with and PHP makes it really easy to prototype ideas pretty fast. PHP is a very misunderstood language but I feel I can create interesting and useful web applications with it (and couple this with JavaScript on the frontend)


I modelled the objects on what they are (classical inheritance) and I have laid-out the app architecture in a sensible way. The design pattern is MCV, only without the M and V (no database ORM or frontend view components, I only need the controller logic)


The PHP CLI invokes a single instance of the app which in turn starts-up the WebSocket server and maintains applications state. This single instance knows about all the currently connected clients and I store them in a private associative array in the messengerServer type. The WebSocket server needs to be long-running and bind to a TCP port so it’s not invoked by FPM or mod_php


Of course PHP wants to be stateless (i.e.: no server state and it wants to adopt all the properties of the HTTP protocol) but it’s important to remember that the server component of the app is invoked by PHP on the command line and the PHP CLI never ends–the app implements the use of an event-loop that I describe the greater detail below


WebSocket client connections don’t terminate up against this PHP server directly but instead are terminated up against the Apache webserver and are proxied into the TCP port, using ws_tunnel, that the PHP WebSocket server is listening on. This proxying occurs when the client sends the HTTP WebSocket connection upgrade headers. All non-upgraded HTTP requests will be sent to the front controller (which, at the moment, just sets the 200 HTTP response code and returns no content)


The mod_rewrite rules are described in .htaccess and they travel with the git repository so it’s very easy to make changes here because it’s not baked into the web servers global configuration


RewriteEngine on
Options +FollowSymlinks

# Process WebSocket connections
RewriteCond %{HTTP:Upgrade} websocket [NC]
RewriteCond %{HTTP:Connection} upgrade [NC]
RewriteRule ^/?(.*) "ws://127.0.0.1:8080/messengerServer" [P,L]

# Redirect all requests through the front controller
RewriteRule ^(.+)/?$ index.php [L,QSA]


Bloatless

I really don’t have a good idea of how TCP or UDP sockets work so I’d prefer to be abstracted away from the low(er)-level implementation details in this area


Yes, I get the idea that TCP is connection-oriented and UDP is not and I get the idea that you first bind the server socket and get it to listen on a port and then repeatedly read into this server socket which will return event-driven messages from connected socket clients but I really don’t want to have to do this myself (I’m also trying to avoid reading through all of the protocol RFCs but I will read through them one day)


I came across a framework by the name of Bloatless–or the greater project is by this name. The framework provides an easy way to get WebSocket servers up and running. It’s as simple as extending a base abstract class, which implements an interface that defines the mandatory public methods I’d need to implement, and registering a singleton of the extending type


interface ApplicationInterface
{
    public function onConnect(Connection $connection): void;
    public function onDisconnect(Connection $connection): void;
    public function onData(string $data, Connection $client): void;
    public static function getInstance(): ApplicationInterface;
}



abstract class Application implements ApplicationInterface {
	// handles the singleton for me
}



class messengerServer extends Application {
	// the type where I implement the interface/mandatory public methods
}


Each connection (represented by the Connection type on the methods above) has a unique identifier and is wrapped around a type that has methods like send(), close(), getClientId(). These methods make interacting with the WebSocket connections really easy


The heart of this framework is an event-loop (pretty much a while loop that is constantly reading into the server socket and parsing event-driven messages from the client) and the idea here is to only perform non-blocking calls or calls that don’t have to wait for a response for too long anywhere in the application (the processing occurs in series). If the call I perform is blocking/synchronous and the response is delayed, PHP will wait for the response before continuing the processing and there will be a noticeable delay for other either connected WebSocket clients or clients that are wanting to be connected


Scalability

The app can only scale vertically and not horizontally like PHP wants to (stateless PHP backends scale horizontally very easily but I have not adopted this concept here–it’s actually not possible). This is because the app maintains state or never ends/halts and, as far as I’m aware, there is no way to share memory like this across different instances of PHP (this is also by design because PHP is single-threaded and it purposefully makes sharing memory between threads difficult)


Scaling vertically is acceptable to some degree because the server only needs to be able to communicate with WebSocket client connections and the client connections do not care which WebSocket server they have connected to (clients never have to speak to other clients). So, having multiple instances of the server app is perfectly fine


Google Cloud run scales the containers for me and will start x amount of them when needed (x is a value between 0 and 100, minimum and maximum container instances)


Multitenancy

I tried really hard to avoid having separate WebSocket servers for each chat application I have (there is a chat app on my website, another one in the admin console and now with PuBot)


Bloatless has built this concept in:


public function registerApplication(string $key, ApplicationInterface $application): void


I can invoke registerApplication and provide a unique identifier of the application as the first parameter and the singleton as the second parameter


…but, I want to take this concept one step deeper and have a single instance of my messengerServer handle this. The WebSocket proxying (mod_rewrite rules) sends all the WebSocket clients to the same application endpoint (/messengerServer) and I read an “applicationName” key on the JSON. This key is used to indicate the application the client wants to be associated with


{"action":"registerApplication","applicationName":"aiAssistant"}


There is a 1:1 mapping between the value of applicationName and the actual PHP user defined type in the app. I then store this in a private array in the messengerServer type and read into it when the WebSocket client sends in an action


WebSocket client

Most browsers support the WebSocket API and using it is super simple. All I do is provide the endpoint to the constructor and I’m off to the races


The data interchange format used between the WebSocket client and server is JSON (key/values wrapped around curly braces, O(1) lookup time, serialised representation of a JavaScript object and it’s not a train wreck like XML is). Every JSON message that is sent or received contains an “action” key


When the server sends a message, the WebSocket message event will be emitted and I have an event-listener configured to handle this event and provide it into a callback function which will parse the incoming JSON message. I then dynamically build a block-level HTML element, with the innerHTML or textContent set to that of the incoming message, and I append this new HTML element as a child of a parent that has a fixed height using the DOM API


I create the scroll by setting the overflow property of the first child element to scroll and this works well


Require.js

Require.js is an asynchronous module definition (AMD) module loader that automatically handles the asynchronous fetching of all the libraries the app is making use of


It’s really easy to define AMD modules (which can be seen as a complex type wrapped around other complex types) and each module I define returns an object with its relevant methods


I have a module for handling the UI navigation and for handling the WebSocket client. It’s also handles the scoping correctly (no more global constructor functions like I had to deal with in my previous hell-hole of a job)


Zero-cost asynchronous worker

The first invocation of the PHP app will become the WebSocket server (starts/binds the server socket and invokes the event-loop) and subsequent invocations of the PHP app will become a WebSocket client. This client will read into a queuing mechanism I wrote (push to the end, read from the front) and process the entries in the queue


My custom Docker entry point enables this to happen. The app is containerised using Docker and I’ve written a custom entry point (in BASH) that forks the services start-up script (coproc entrypoint …) and then enters the while iteration below:


#!/bin/bash

# Run 'entrypoint' with these parameters in another instance of BASH and continue running
# the rest of this shell script
coproc entrypoint 7.3 ap-websockets no_wait

# Initialise the PHP WebSocket server
php7.3 /app/index.php &

# zero-cost asynchronous worker that runs exclusively for the duration of the running container
while true; do sleep 10; php7.3 /app/index.php; done


The Dockerfile is fairly simple too:


ADD ./dockerEntrypoint.sh /dockerEntrypoint.sh
RUN chmod +x /dockerEntrypoint.sh

EXPOSE 80
ENTRYPOINT /dockerEntrypoint.sh


This means that this mechanism—it can be seen as a backend worker doing all the asynchronous processing, is completely isolated within itself and it does not need any external service. This results in reduced financial and technical cost


Natural language processing

The idea here is I use natural language processing to determine the users intent (i.e.: what are they trying to achieve or what information are they looking for)


Google Dialogflow

Dialogflow from Google makes identifying intent from a paragraph of text very easy. It will also train on all the variations of the defined text without having to be very precise


Google is also extremely good with maintaining all their client libraries and they support PHP through Composer. The Composer package google/cloud-dialogflow ships all the types I need so I don’t have to do the direct API integration myself


$queryInput = new QueryInput();
$queryInput->setText($textInput);

$response = $dialogFlowSession->detectIntent($userSession, $queryInput);


Other technical components

I use Google Cloud DNS as authoritative nameservers for the domain (pubot.chat) because I don’t want to run any more domains myself. Google Container Registry (GCR) hosts the Docker container images—with the vulnerability scanning turned off of course


CI/CD

I use SemaphoreCI and the pipeline is described in YAML and travels with the git repository. The pipeline only contains a deployment stage that contains a YAML array of commands and one of these commands is executing a shell script (doing this makes the logic much more flexible and easier to manage)


version: v1.0
name: Messaging - WSS
agent:
  machine:
    type: e1-standard-2
    os_image: ubuntu1804
blocks:
  - name: Deployment
    task:
      jobs:
        - name: Deploy-cloudrun.sh
          commands:
            - checkout
            - chmod +x deploy-cloudrun.sh
            - bash ./deploy-cloudrun.sh


The shell script (deploy-cloudrun.sh) basically runs through the installation of gcloud and authenticates it to my Google account using a IAM server accounts JSON wrapped private key. Once it’s authenticated, it will deploy a new revision of the Cloud Run instance (I can build the container here as well but for now, I build the container locally)


Future considerations

I’m probably going to need to do some re-writing here


The WebSocket server app has 1527 lines of PHP and I have already forgotten most of them


cat `find ./ -name '*.php' -type f -not -path './/vendor/*'` | wc -l
1527


The first commit I pushed was back in March 2021 so go figure


commit a39a9a9b7ebb9674ecc1d86d4a123a2dce10c6ba
Author: Bruce Blacklaws 
Date:   Sun Mar 21 09:09:18 2021 +0200

Initial commit


Using a more mainline framework like Laravel Broadcasting and Echo

I don’t feel there is anything wrong with how I have built PuBot or the concepts I have implemented in the app however, there are more mainline WebSocket frameworks for PHP out there


I don’t think I’ll port the app over from PHP to anything else (I know JavaScript on the backend with Node.js is very popular for WebSocket servers) because PHP is perfectly fine and you can clearly see it working here


Cloud Run does not want to be long-running it seems

Serverless is generally not for long-running processing. I don’t want to have an unexpected bill either so I do set a maximum processing time of 10 minutes


Working with containers just makes everything easier to manage but maybe my WebSocket server needs to be deployed into traditional IaaS (EC2 or Compute Engine)