Making a full-stack publish/subscribe server in 1 week

I am back on my blog with another super helpful how-to post. This time is about making a full-stack publish-subscribe system with a browser-based client and managment that is backed by cloud-based data storage and subscription management services. This project is the warmup project for the class I am taking right now COMP 410: Software Engineering Methodology

Our team has 10 people working together on 3 system blocks frontend, backend and database. I was working on the backend side using ASP.NET (C#) and SignalR but I also helped with designing the database models and developing the frontend. Here, you can view the full source code and a youtube video demo of our product

Introduction

Introduction of the application:

This is a publish-subscribe system which can support messaging between people in the same group. This is the basic idea to make a chat application. We have 3 main functionalities: create a room, send message to a room and subscribe to a room to see all the chat history.

Use Case Diagram:

Developing a software or application will start with making a complete use case diagram. This is a graphical depiction of a user’s possible interactions with a system. Our use case diagram helps us visualize all the user’s actions within the system. Down here you can also see the different blocks of our system frontend, backend and database. The arrow’s directions show how each part of the system interacts and sends data to each other

Sequence Diagram:

We also need to design the user flow within the application so we need to also make a sequence diagram to show how the users progress through the different parts of the system

Database Models

To make a full-stack application, we definitely need to design the models for the database. This means that we need to define different fields and their corresponding types beforehand so that you can understand what type of data you are dealing with. If you make extra database objects then you also need to define their types here. Below is an example of our database models diagram

Tech stack:

• For DevOps, we used Azure DevOps to keep track of our repositories including 1 for frontend and 1 for backend. We deployed the frontend and backend separately for maximum scalability
• For the frontend, we used React for keeping track of the message and rooms in real-time. We deployed the frontend using Azure Static Web Apps
• For the backend, I made all the methods to communicate between the server and the client for the messaging functionalities using ASP.NET (C#) and SignalR. We have also deployed our backend using Azure Cloud Service
• For the database, we used MongoDB in Azure CosmosDB for efficient document reading

Design Criteria Philosophical Standpoint

This is a very important part of designing an application. Here, we need to consider the design goals and how does the design address these philosophical standpoint

• Flexibility: How is the system able to handle many different situations?
• Extensibility: How easily can the system be extended to handle new situations? This includes scaling issues.
• Adapt to changes: Be sure to address how the system will handle new platforms and platform changes. This is particularly important if only one platform is demonstrated.
• Robustness: How resiliant is the system to improper usage both accidental and deliberate?
• Correctness: How does the system ensure that it is attempting and achieving the proper operations?

UX Design/Mock ups on Figma and Glossary

• After completing the use case and sequence diagrams, it should be sufficient to make a UX design and mock up of the application on Figma. Using Figma, we can design the components of each part of the application so that the frontend team can work based on that. After that, we also need to make a glossary of the commonly used terms in our application. In this case, we used the terms users, rooms, channels and subscribe very common so we made a document about it.

• You can checkout our complete frontend documentation

API endpoints and writing backend documentations

We need to define all of our methods and API endpoints so that it matches the use-case diagram. Example of defining API endpoints can be seen below. We need to define the parameters of the methods, the return types and a description of the purpose of the method. You can see our full backend documentation here

Frontend

In order to connect to the SignalR in the backend, we used the HubConnectionBuilder from the ‘@microsoft/signalr’ library from React. In the URL, we put in our deployed backend server (will be explained in the next section).

1
2
3
4
5
6
7
8
9
10
11
12
13

const initialConnection = new HubConnectionBuilder()
  .withUrl("https://warmup-b-signarlrservice.azurewebsites.net/chatHub") //TODO: Replace with deployed backend url
  .configureLogging(LogLevel.Information)
  .withAutomaticReconnect()
  .build();
initialConnection
  .start()
  .then((result) => {
    console.log("Connected!");
    setInitialLoad(true);
    // getRooms();
  })
  .catch((e) => console.log("Connection failed: ", e));

We also listen to the ReceiveSendMessage function from the server to get all the messages in the current room. This helps us auto-update the current message history in a room and display all the messages to everyone who just joined the room

1
2
3
4
5
6
7
8
9

const sortedMessages = Object.values(updatedMessages)
  .map((message) => ({ ...message, time: new Date(message.timestampUTCms) }))
  .sort((a, b) => (a.time > b.time ? 1 : -1))
  .map((message) => ({ ...message, time: message.time.toUTCString() }));
messageApi.open({
  type: "success",
  content: "New message in " + room + "!",
});
setAllMessages({ ...allMessagesRef.current, [room]: sortedMessages });

Important Note: We used a strategy mapping of the data type to the corresponding way to display that type of data to the frontend. This will enable our application to scale and extend easily because if I want to display a different type of data like images, I can just add another key-value pair to this mapping

1
2
3
4
5

const catToComponent = {
  Text: TextMessage,
  Number: NumberMessage,
  UserProfile: UserProfileMessage,
};

Backend

For the backend, we used C# (ASP.NET) and SignalR to write all of our functions including

The backend server is deployed here on Azure Cloud Service

• The first function is JoinRoom, which is a user will join an available room

1
2
3
4

public async Task JoinRoom(string room, string user){
   await Groups.AddToGroupAsync(Context.ConnectionId, room);
   ...
}

• The second function is AddRoom. User can add a new room with a category (a data type such as Text, Number, UserProfile)

1
2
3
4
5
6
7
8
9

public async Task AddRoom(string room, string category){
     RoomInDB? createdRoom = DatabaseManager.tryAddRoom(room, category);
       if (createdRoom is not null)
       {
           Console.WriteLine("Adding room successful");
           List<RoomInDB> rooms = new List<RoomInDB>();
           rooms.Add(createdRoom);
       }
}

• The third function is getRooms. User can automatically get all the current available rooms

1
2
3
4
5

public async Task getRooms()
{
    List<RoomInDB> roomNames = DatabaseManager.getAllRooms();
    await Clients.Caller.SendAsync("ReceiveRooms", roomNames);
}

• The last function is GetMessages. User can get all the message history of a room they just subscribed to

1
2
3
4
5

private async Task GetMessages(string room)
{
    List<MessageInDB> messages = Database.DatabaseManager.getAllMessages(room);
    await Clients.Caller.SendAsync("ReceiveGetMessages", messages, room);
}

• Also for the backend using SignalR, you need to add the CORS policy handling so that the client will not be blocked by CORS policy. We will also need to configure the services that will communicate with the database. Thus, we will need this in our Program.cs file

1
2
3
4
5
6
7
8
9
10
11
12
13
14

builder.Services.Configure<DbSettings>(
builder.Configuration.GetSection("Database"));

builder.Services.AddSingleton<DatabaseManager>();
builder.Services.AddCors(options =>
{
    options.AddPolicy(name: "CorsPolicyAllHosts",
        builder =>
        {
            builder.AllowAnyHeader()
                   .AllowAnyMethod()
                   .AllowAnyOrigin();
        });
});

You can view the deployed application here

Database

This will be a description of how we have designed the database. Conceptually, each channel has three parts: its name, the type of data being published in that channel (the category), and the list of data that has been published in that channel. Each piece of published data has three parts itself, two metadata (the timestamp of when it was published and the name of the user that published it) and the actual contents being published.

Now that we’ve established how the database will store the data, we need to specify what format the data will come in.

• Name of channel: String
• Data category of channel: String. This is not necessarily String, it can be anything as long as unique values are used for different categories, and all parts of the application agree on what the unique values mean. We chose string for ease of developer readability.
• Piece of data: - Timestamp: Long integer, representing milliseconds since Jan 1, 1970. Chosen for developer ease of working with time. - Publishing User’s Name: String - Content of data: String. We decided to convert all content into one C# type to reduce work, and we felt String was sufficiently general that encodings and decodings to and from String were writable for any type of data we chose to support, including compound types.
  Therefore our database is structured as following:
• 1 Collection: Mapping channel names to data categories
  • Unique name “roomToData”, defined in SignalRChat.Database.DatabaseManager.roomToDataCategoryName
  • Documents: Mapping a single channel name to a single data category. 1 Document per created channel. Represented in C# as RoomInDB.
    • String roomName
    • String roomDataCategory
• 1 Collection per publish/subscribe channel:
  • Name is the name of the channel
  • Documents: 1 Document per piece of data published in this channel. Represented in C# as MessageInDB.
    • Long Integer timestampUTCms
    • String sender
    • String message

Here is a screenshot of what the development database looks like

Conclusion

I have discussed all the steps that I went through to make this full-stack application. It is crazy to thing that we were able to set all of this up in just 1 week. I hope you understand some important steps in developing an end-to-end software. I have a lot of new content that I want to make soon so I look forward to it.

• ← Previous Post
• Next Post →