Tuesday, August 2, 2011

Project: Basic Web Application List with Real-time Updating (Part 4)

In this post, we will start working with the Channel API. We will be using the Channel API for updating the displayed list of items. The goal is that every time a new list item is added, all of the users viewing the list will see the new item inserted to the list of items they are viewing as soon as possible. This will be achieved without the use of polling but rather with the use of the Channel API which allows the server to send messages to its clients.

Unique Identifier per Client

A requirement to be able to use the Channel API is that the server must have a unique identifier for each of its clients. In actual web applications, this unique identifier can be based on the login credentials of the user. But for our purposes, we will simplify this requirement. Rather than requiring the user to login, our web application will simply generate a unique identifier per browser using the Python uuid library. This unique identifier would be stored in the clients browser through the use of cookies so that a browser refresh would not necessitate the generation of a new unique identifier.

For our application to be able to easily set cookies, we would be copying some code from this App Engine web application, https://github.com/facebook/runwithfriends/.


class BaseHandler(webapp.RequestHandler):
    # copied from runwithfriends application
    # https://github.com/facebook/runwithfriends/
    def set_cookie(self, name, value, expires=None):
        """Set a cookie"""
        if value is None:
            value = 'deleted'
            expires = datetime.timedelta(minutes=-50000)
        jar = Cookie.SimpleCookie()
        jar[name] = value
        jar[name]['path'] = u'/'
        if expires:
            if isinstance(expires, datetime.timedelta):
                expires = datetime.datetime.now() + expires
            if isinstance(expires, datetime.datetime):
                expires = expires.strftime('%a, %d %b %Y %H:%M:%S')
            jar[name]['expires'] = expires
        self.response.headers.add_header(*jar.output().split(u': ', 1))


The above copied code simply extends the webapp.RequestHandler to allow us to easily set cookies. It is not that complex. Actually, we could have just implemented something similar to the code above but its easier to copy. Add the above code to 'main.py' and also add the following import statements.

import datetime
import Cookie

Now modify the MainHandler class in 'main.py' as follows.

class MainHandler(BaseHandler):
    def get(self):
        uniqueid = ''
        if not self.request.cookies.get('uid'):
            uniqueid = str(uuid.uuid4())
            self.set_cookie('uid', uniqueid)
        path = os.path.join(os.path.dirname(__file__), 'templates/main.html')
        self.response.out.write(template.render(path, {}))

The logic in the MainHandler get method now becomes a little more complicated. Now when a client requests the main page of the application, a check if there already exist a unique identifier assigned to the client which is stored as a cookie. In the case that there isn't, a unique identifier is generated using the Python uuid library and is set as a cookie.

The following import statement is necessary to import the uuid library.

import uuid

Initializing the Channel API Connection

To initialize the Channel API Connection, we need to write some code in both the client and server side.

In the server side, we need to generate a token that would be used by the client to initialize its connection with the server. This token would be based on the client's unique identifier. This token would be passed to the client browser through a client GET request. To handle this GET request, add the following request handler to 'main.py'

class GetTokenHandler(webapp.RequestHandler):
    def get(self):
        uniqueid = self.request.cookies.get('uid')
        token = channel.create_channel(uniqueid)
        
        cm = ClientManager()        
        cm.add(uniqueid)
        
        self.response.headers['Content-Type'] = 'text/json'
        self.response.out.write(json.dumps(dict(token=token)))

Also add these import statements somewhere at the top of 'main.py'.

from google.appengine.api import channel
from clientmanager import *

The above code is quite self-explanatory, except for the following statements.

cm = ClientManager()        
cm.add(uniqueid)

We will discuss this later in this post.

To make this handler accessible, modify the WSGI application initialization.

    application = webapp.WSGIApplication([('/', MainHandler),
                                         ('/save', SaveHandler),
                                         ('/gettoken', GetTokenHandler),
                                         ('/list', ListHandler),],
                                         debug=True)

Let us now move to the client side.

In the client side, we would be utilizing a Javascript library for the Channel API. To add this Javascript library modify the 'base.html' template.

<!doctype html>
<html>
<head>
  <meta charset="utf-8"/>
  <title></title>

  <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0"/>
  <link type="text/css" href="/style/smoothness/jquery-ui-1.8.14.custom.css" rel="stylesheet" />
  <link type="text/css" href="/style/main.css" rel="stylesheet" />
</head>

<body>
  <header>
  {% block header %}{% endblock %}
  </header>

  <div id="contentwrapper">
  {% block content %}{% endblock %}
  </div>

  <footer>
  {% block footer %}{% endblock %}
  </footer>

  <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.6.2/jquery.min.js"></script>
  <script src="/js/jquery.evently.js"></script>
  <script src="/js/jquery.mustache.js"></script>
  <script src="/js/jquery-ui-1.8.14.custom.min.js"></script>
  <script src="/_ah/channel/jsapi"></script>
  {% block addscript %}{% endblock %}
</body>

</html>

The highlighted line is the only change.

Inside the main function in 'main.js' add the following code.

var setupChannel = function () {
$.getJSON('/gettoken', function (data) {
var channel = new goog.appengine.Channel(data['token']);
var socket = channel.open();
socket.onopen = function () {
console.info("connection opened");
};
socket.onclose = function () {
console.info("connection closed");
};
socket.onerror = function () {
console.info("connection error");
};
socket.onmessage = function (message) {
};
});
};

setupChannel();

The code initializes the Channel API connection to the server. It starts by requesting a token from the 'gettoken' request handler, which we just added. It then uses the token it receives to open a instantiate a channel object and open a connection to the server. The socket object above has four callback methods.


  • onopen - called when the connection is successfully opened
  • onclose - called when the connection is closed
  • onerror - called when an error occurs
  • onmessage - called when a message from the server is received, this is the most important method.
The Client Manager

In the previous section we left these statements unexplained.

cm = ClientManager()        
cm.add(uniqueid)

The above statements deals with the client manager.

In order to keep track of all the clients connected to the server, we create a simple client management system. Put the following code into a new file,  'clientmanager.py'.

from google.appengine.ext import db
from google.appengine.ext import webapp

import logging

class ClientId(db.Model):
    clientid = db.StringProperty()
    createdate = db.DateTimeProperty(auto_now_add=True)
    
class ClientManager(object):
    """
    manager for client channel ids
    """
    
    def add(self, clientid):
        ct_k = db.Key.from_path('ClientId', clientid)
        if not db.get(ct_k):
            ct = ClientId(key_name=clientid, clientid=clientid)
            ct.put()
            
    def remove(self, clientid):
        ct_k = db.Key.from_path('ClientId', clientid)
        ct = db.get(ct_k)
        if ct:
            ct.delete()
            
    def clientids(self):
        ct = ClientId().all()
        return [o.clientid for o in ct]

The 'clientmanager.py' file should be located in the same directory as 'main.py'.

The client manager is accessed through an instance of the ClientManager class. The client manager allows us to add a client identifier, remove a client identifier, and get a list of all client identifiers. The client identifiers are kept in the App Engine datastore.

So going back to the two statements.

cm = ClientManager()        
cm.add(uniqueid)

These simply means that we are adding a client identifier to the client manager.

Sending Channel API Messages

Now that we have initialized the Channel API connection and kept track of client connections. We are now ready to send messages.

Our application sends messages every time a new list item is added. We need to send messages in order to update all the lists being viewed. To do this, we modify the 'SaveHandler' in 'main.py'.

class SaveHandler(webapp.RequestHandler):
    def post(self):
        itemtitle = self.request.get('itemtitle')
        itemtext = self.request.get('itemtext')
        
        self.response.headers['Content-Type'] = 'text/json'

        cm = ClientManager()
        
        if itemtext and itemtitle:
            item = Item(title=itemtitle, bodytext=itemtext)
            item.put()

            jsonstr = json.dumps({'rows': [{'title': itemtitle, 'bodytext': itemtext},]})
            for clientid in cm.clientids():
                channel.send_message(clientid, jsonstr)

            self.response.out.write(json.dumps({'result': 'success'}))
        else:
            self.response.out.write(json.dumps({'result': 'failure'}))

Highlighted above are the changes. First we need to instantiate a ClientManager instance to allow as to  get a list of client identifers. Next, we loop through the client identifiers and send a JSON encoded message to each client.

Receving Channel API Messages

On the client side, every time a message is sent by the server, the 'onmessage' method of the Channel API socket object is called. The message is passed as an argument of the callback method. Thus we modify the onmessage callback method as follows to receive the message sent by the server.

var setupChannel = function () {
$.getJSON('/gettoken', function (data) {
var channel = new goog.appengine.Channel(data['token']);
var socket = channel.open();
socket.onopen = function () {
console.info("connection opened");
};
socket.onclose = function () {
console.info("connection closed");
};
socket.onerror = function () {
console.info("connection error");
};
socket.onmessage = function (message) {
console.info("message received " + message.data);
var d = $.parseJSON(message.data);
$("#mainlist").trigger('prependitem', [d]);
};
});
};

Highlighted above are the new lines we added. The first line simply parses the JSON string the server sent. The second line triggers the 'prependitem' event of the Evently widget to add the new list item.

In our previous post, to cover up for the lack of the Channel API connection, we added a list item to the list every time a new item is saved.

$("#itemform").dialog({
  autoOpen: false,
  modal: true,
  width: 450,
  buttons: {
   "Save": function() {
    var itemtitle = $("#itemformmain input[name='itemtitle']").val();
    var itemtext = $("#itemformmain textarea[name='itemtext']").val();
    var that = this;
    $.post('/save',
           {itemtitle: itemtitle, itemtext: itemtext},
           function () {
            $("#mainlist").trigger('prependitem', {rows:[{title: itemtitle, bodytext: itemtext}]});
            $(that).dialog("close");
           }, 'json');
   }, 
   "Cancel": function() { 
    $(this).dialog("close"); 
   } 
  }
 });

This is no longer needed as the updating of the list is now handled through the Channel API connection, so we should delete the above line.

Now the basic functionality is done. You could see this work by opening two different browsers and pointing them both to 'http://localhost:8080/'. Adding a list item to one browser would result in the both browsers being updated. If you have more than one browser installed, then you could try more browsers.

Removing Inactive Clients from the Client Manager

Notice that we only add client identifiers to the client manager and never remove any. So what happens if a client disconnects? Do we still keep on sending them messages? This would be inefficient. Fortunately, there is a way of knowing when a client disconnects. A POST request is made to  '/_ah/channel/disconnected/'. To remove disconnected client identifiers from the client manager, we need to bind to the above path a request handler. The request handler is as follows.

class ClientDisconnectHandler(webapp.RequestHandler):
    def post(self):
        clientid = self.request.get('from')
        
        logging.info("Client %s disconnected"%clientid)
        
        cm = ClientManager()
        cm.remove(clientid)

Add the above code to the 'clientmanager.py' file. To bind the above request handler to '/_ah/channel/disconnected/', modify the WSGI application initialization in 'main.py' as follows.

    application = webapp.WSGIApplication([('/', MainHandler),
                                         ('/save', SaveHandler),
                                         ('/gettoken', GetTokenHandler),
                                         ('/_ah/channel/disconnected/', ClientDisconnectHandler),
                                         ('/list', ListHandler),],
                                         debug=True)

Next Post

In the next post, we will improve on our application by making it more robust.

No comments:

Post a Comment