KillrChat, a scalable chat with Cassandra, AngularJS & Spring Boot

This post is the first of a serie on KillrChat, a scalable chat application I developed as a hands-on exercise for Cassandra data modeling.

I What is KillrChat ?

KillrChat has been developed with horizontal scalability in mind. It means that today, with a dozen of users, the chat can run on a small box or even a VM but if the users count increases in the future, the design can scale by adding new hardware and without breaking the code to re-design from scratch.

II Why KillrChat ?

After 4 months giving talks and hands-on sessions on Cassandra, I just realised that most of hands-on exercises are mere pieces of real word examples or some HelloWord/TODO List application. Those examples are not really consistent and can not be re-used by the attendees. Thus the idea to provide a real and usable application, to make people design the data model in Cassandra and code the data access part.

This project was also an opportunity for me to strengthen some of my front-end development skills and learn AngularJS in depth.

In fact, as a technical evangelist, most of my time is spent on public talks. Such a development project keeps me in touch with the development, which is always a good thing.

III The technology stack

For the database, the choice was obviously Apache Cassandra™. To take care of the repository layer, instead of using the raw Datastax Java Driver, I used Achilles as an object mapper to make my life easier. Furthermore, Achilles provides some nice support for TDD.

For the front-end, I choose AngularJS for several reasons:

  • it is easy to learn
  • it is very extensible thanks to the directive system
  • the framework focuses on developer productivity, with a declarative (instead of imperative) programming style
  • technical resources and tutorial are quite abundant on the web
  • last but not least, there is a nice integration module for Twitter Bootstrap. Since my web design skills are so lame, I’m more than happy having Twitter Bootstrap at hand. At worst, my front-end UI would look standard and common, but never really ugly (how can you create an ugly interface with Bootstrap anyway …)

As already mentioned, I pulled an integration of AngularJS with Twitter Bootstrap, called UI-Bootstrap.

For the glue between the front-end interface and the database, I opted for a classical Spring stack.

  • Spring Boot with an embedded Jetty server for the application skeleton
  • Spring Boot Security to take care of the security part (at least account creation + login/logout)
  • Spring Boot Web for the HTTP REST communication with the front-end
  • Spring Boot Messaging as the broker for the Web Socket server-side

Some interesting notes about the Web Socket feature. I spent many time playing with Atmosphere but there is so much boilerplate code to make it integrate with Spring that I just gave up. Instead, Spring Boot proposed a very nicely packaged Web Socket architecture with SockJS on the client-side and Spring Messaging at the back-end.

Indeed, a Web Socket can be seen as an abstraction of a publish/subscribe broker. Many clients can subscribe to the same chat room to receive updates on new chat messages. The Spring team did a good job of extracting the broker part from their very mature Spring Integration project to create a more lightweight Spring Messaging module.

So in a nutshell, from top to bottom, below is the technology stack:

IV The architecture

A Local mode

On a single node, the architecture would resemble this:

Single Node Architecture

In this mode, on the back-end side, everything can be embedded in the Tomcat/Jetty server, including an in-memory broker for the Web Socket and an embedded Cassandra.

B Scale-out mode

On a fulll-fledged production deployment, the architecture would be:

Production Architecture

To cope with the high traffic, the number of instances of Tomcat/Jetty servers can be increased. Since the back-end application is stateless (there will be still some session-stickiness handled by the load balancer), we can throw in new boxes to support the load.

On the broker side (necessary for the Web Socket publish/subscribe pattern), we need to use a dedicated infrastructure. A solution like RabbitMQ/ZeroMQ can be a good fit. If you need extreme scalability/fast response time, Apache Kafka could be also a good choice. All you’ll need to implement is a connector between this broker and the Spring Messaging abstraction on each back-end server

For the database, a full Cassandra cluster with horizontal scaling-out is sufficient to cope with any increase in term of traffic.

V The application

A Where to get it ?

KillrChat is an open-source application. You can get it there.

Installation instructions

Please ensure that you have the following dependencies installed before-hand:

You’ll need to install Git and clone the GitHub repository locally to run KillrChat

> git clone https://github.com/doanduyhai/killrchat.git

Running in development mode

Go to the Git repository you just cloned before.

To run KillrChat in development mode

killrchat> mvn clean test
killrchat> mvn spring-boot:run -Pdev

When running the application in dev mode, Achilles will start an embedded Cassandra server and create the following data folders:

  • /tmp/killrchat_cassandra/data
  • /tmp/killrchat_cassandra/commitlog
  • /tmp/killrchat_cassandra/saved_caches

You can change those default values in the src/main/resources/config/application.yml file.

Then connect to the chat by opening your browser at http://localhost:8080/killrchat/index.html.

Running in production mode

You’ll need to have a Cassandra 2.1 running somewhere so that KillrChat can connect to. If you’re deploying on multiple back-end servers, do not forget to configure properly the broker and not re-used the in-memory version.

To run KillrChat in production mode:

killrchat> mvn spring-boot:run -Pprod

When running the application in prod mode, Achilles will connect to an existing Cassandra server using the server host and port in the the src/main/resources/config/application.yml file. By default Achilles will execute the src/main/resources/cassandra/schema_creation.cql script to provision the killrchat keyspace and appropriate tables.

Then connect to the chat by opening your browser at http://your_back-end_ip:8080/killrchat/index.html.

To be continued …

7 Comments

  1. Jim

    Why not use the spring-data-cassandra lib instead of Achilles (in keeping with the spring-boot theme)?
    Just wondering if there was a performance trade-off or at the time spring-data-cassandra was in its infancy…

    Thanks,
    -Jim

    http://docs.spring.io/spring-data/cassandra/docs/1.0.4.RELEASE/reference/html/cassandra.core.html

    Reply
    1. doanduyhai (Post author)

      Several reasons for not using spring-data-cassandra:

      1. It does not offer any support for JUnit test
      2. It does not offer any support to start embedded Cassandra server
      3. It does not offer any significant extra feature compared to the default mapper module of the Java driver, and it requires to pull all the infrastructure code of Spring Data.

      In general, hosting under the same code infrastructure many NoSQL databases that do not have much in common is a bad idea. The only thing you can offer is very simple CRUD operations (INSERT, DELETE, SELECT BY ID)

      Reply
  2. Nimbuslin

    Cann’t I leave two bugs here? or what is genius Sir Doanduyhai’s mail address for asking questions?

    Reply
    1. doanduyhai (Post author)

      This is a blog, not a support site. The code is open source so do whatever you want but don’t expect any commitment or support from me

      Reply
  3. Nimbuslin

    Hi, Dear genius sir Doan,

    My computer environment is listed as before, and how to solved this error in sin10+OracleJDK1.8.0_202?

    I run command: mvn spring-boot:run -Pprod
    compiler (1.8.0_202) Bug as below:
    java.lang.IllegalStateException: endPosTable already set
    at com.sun.tools.javac.util.DiagnosticSource.setEndPosTable(DiagnosticSource.java:136)

    Reply
  4. Nimbuslin

    Dear genius Sir Doan,

    A good news is that your program run with cassandra2.1.2, but when it work with cassandra3.11.3, it throw the 1st error of

    Caused by: org.springframework.beans.factory.BeanCreationException:
    Could not autowire method: public void org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration.setFilterChainProxySecurityConfigurer(
    org.springframework.security.config.annotation.ObjectPostProcessor,java.util.List)
    throws java.lang.Exception; nested exception is org.springframework.beans.factory.BeanExpressionException:
    Expression parsing failed; nested exception is org.springframework.beans.factory.BeanCreationException:
    Error creating bean with name ‘securityConfiguration’: Injection of autowired dependencies failed;
    nested exception is org.springframework.beans.factory.BeanCreationException:
    Could not autowire field: private com.datastax.demo.killrchat.security.service.CustomUserDetailsService
    com.datastax.demo.killrchat.configuration.SecurityConfiguration.userDetailsService;
    nested exception is org.springframework.beans.factory.BeanCreationException:
    Error creating bean with name ‘customUserDetailsService’: Injection of autowired dependencies failed;
    nested exception is org.springframework.beans.factory.BeanCreationException:
    Could not autowire field: private com.datastax.demo.killrchat.service.UserService
    com.datastax.demo.killrchat.security.service.CustomUserDetailsService.userService;
    nested exception is org.springframework.beans.factory.BeanCreationException:
    Error creating bean with name ‘userService’: Injection of autowired dependencies failed;
    nested exception is org.springframework.beans.factory.BeanCreationException:
    Could not autowire field: info.archinnov.achilles.persistence.PersistenceManager
    com.datastax.demo.killrchat.service.UserService.manager;
    nested exception is org.springframework.beans.factory.BeanCreationException:
    Error creating bean with name ‘achillesConfiguration’: Injection of autowired dependencies failed;
    nested exception is org.springframework.beans.factory.BeanCreationException:
    Could not autowire field: private com.datastax.driver.core.Cluster
    com.datastax.demo.killrchat.configuration.AchillesConfiguration.cluster;
    nested exception is org.springframework.beans.factory.BeanCreationException:
    Error creating bean with name ‘cassandraNativeClusterProduction’ defined in class path resource
    [com/datastax/demo/killrchat/configuration/CassandraConfiguration.class]:
    Bean instantiation via factory method failed;
    nested exception is org.springframework.beans.BeanInstantiationException:
    Failed to instantiate [com.datastax.driver.core.Cluster]:
    Factory method ‘cassandraNativeClusterProduction’ threw exception;
    nested exception is com.datastax.driver.core.exceptions.NoHostAvailableException:
    All host(s) tried for query failed (tried: localhost/127.0.0.1:9042
    (com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured table schema_keyspaces))

    where is your table schema_keyspaces?
    I do a full search in your programs, but cann’t find.

    Or would you like to tell me what is the newest cassandra version that is supported by your chat program with JDK 1.8.0_202?

    Reply
  5. Nimbuslin

    Dear Sir Doan,

    I have found a bug in your project:
    your this demo run well with -Pdev, but when run with prou ahead, it will throw this exception:
    is noThe entity ‘PersistentToken{series … is not in ‘managed’ state.

    And would you like to tell me how to debug your project by spring-boot-maven-plugin:1.2.0.RELEASE:run in Eclipse?

    P.S.: I have import as a maven project in, but still have several Java problems, such as:
    The serializable class UserNotFoundException does not declare a static final serialVersionUID field of type long UserNotFoundException.java

    Reply

Leave a Comment

Your email address will not be published. Required fields are marked *