GSoC 2025: MetaBrainz Notification System

Introduction

Hello, my name is Shaik Junaid (IRC nick fettuccinae and fettuccinae on GitHub). I’m an undergrad computer science student from MGIT, Hyderabad, India. My project focused on adding a central notification system for MetaBrainz.

Project Overview

This project’s idea was suggested to me by mentor @ruaok (AKA mayhem on IRC). I submitted my proposal on the MetaBrainz Forum and got it reviewed by @kartikohri13 (AKA lucifer on IRC), and finally got selected for GSoC 2025 .

A centralized notification management system will various MetaBrainz projects send notifications to users without rewriting boilerplate code. It will also keep users informed about the latest events and new features across projects. This is a goal bigger than the scope of a single GSoC project. To keep it reasonable, my project focused on implementing REST APIs, hosted on metabrainz.org, to manage notifications and user preferences for notifications. Additionally, I integrated the system with ListenBrainz to demonstrate its functionality.

The project spec sheet can be found here.

Pre-Community Bonding Period

Coding Period

Phase 1

1. I started my coding period by creating a notification table and a user_preference table and their respective ORMs.

Schema for notification table:
    id                        INTEGER GENERATED BY DEFAULT AS IDENTITY
    musicbrainz_row_id        INTEGER NOT NULL,
    project                   notification_project_type NOT NULL,
    read                      BOOLEAN DEFAULT FALSE,
    created                   TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    expire_age                SMALLINT NOT NULL, -- in days.
    important                 BOOLEAN DEFAULT FALSE,
    email_id                  TEXT UNIQUE,
    subject                   TEXT,
    body                      TEXT,
    template_id               TEXT, 
    template_params           JSONB,
    notification_sent         BOOLEAN DEFAULT FALSE

Schema for user_preference table.
    id                  INTEGER GENERATED BY DEFAULT AS IDENTITY
    musicbrainz_row_id  INTEGER UNIQUE, 
    user_email          TEXT UNIQUE,
    digest              BOOLEAN DEFAULT FALSE,
    digest_age          SMALLINT -- in days.</code></pre>

2. DB functions: I worked on adding database functions for these tables.

def fetch_notifications(user_id: int, projects: Optional[Tuple[str, ...]]=None, count: Optional[int]=None, offset: Optional[int]=None,
                        until_ts: Optional[datetime]=None, unread_only: Optional[bool]=False ) -> List[dict]:

def mark_read_unread(user_id: int, read_ids: Tuple[int, ...], unread_ids: Tuple[int, ...]) -> int:

def delete_notifications(user_id: int, delete_ids: Tuple[int, ...]):

def insert_notifications(notifications: List[dict]) -> List[tuple[int]]:

Next, I wrote tests for these functions and found some edge case bugs. For example: in mark_read_unread() , If the unread_ids tuple was empty, the function would raise an SQLException for passing in None instead of an empty tuple. Even though, I was incredibly slow in writing tests, I have found a new appreciation for Test-Driven Development.

3. Views: I worked on adding endpoints for these notification functions, I looked into MeB repo and LB repo to align my coding style for these endpoints. I added the following endpoints.

I spent quite some time reading up on how to use requests_mock to mock HTTP requests, other testing techniques and added tests for these endpoints.

4. Authentication: I had very little idea about OAuth2 implementation. I had both metabrainz and listenbrainz running locally. Thankfully, the network tab in the browser’s inspect tools was useful. It took me a while but I was finally able to understand how various endpoints need to be secured. Then, I coded a decorator which uses client credentials OAuth2 flow to obtain access tokens on the client side (in this case another server) to properly authorize with the central notification APIs.

I wanted to test invalid tokens in one place and found a very hacky way to do so.

    def test_invalid_tokens(self, mock_requests):
        endpoints = [
            {"url": "notification/1/fetch", "method": self.client.get},
            {
                "url": "notification/1/mark-read",
                "method": self.client.post,
                "data": {"read": [1, 2]},
            },
            {"url": "notification/1/delete",                                                               '            "method": self.client.post,                                                          '             "data": [1]},
            {
                "url": "notification/send",
                "method": self.client.post,
                "data": [{"test_data": 1}],
            },
            {
                "url": "notification/1/digest-preference",
                "method": self.client.post,
                "data": {"digest": True, "digest_age": 19},
            },
        ]
        headers = {"Authorization": "Bearer token"}
        for e in endpoints:
            url = e["url"]
            method = e["method"]
            json = e.get("data")
            result = method(url, json=json)

5. Sending Notifications: Now, for the final piece of the puzzle, I used the existing implementation to send mails through brainzutils (a python library with common utilities for various MetaBrainz projects). A shortcoming of BrainzUtils is that it only supports sending plain e-mails and not HTML e-mails.

I added a NotificationSender class to immediately send important notifications directly to the user. Non-important emails respect the digest preferences of the user. Once sent, the notification records in the database are marked as sent. I also used Redis cache to store the notifications which failed to deliver.

6. Cron Jobs: I looked into the documentation of runit to understand how cron jobs were scheduled in the LB repo. I added cron jobs that would fire every day to delete expired notifications and send notifications in digest. I tested these cronjobs by running production image locally and was really happy when I saw that “hello 123 123” in cron logs.

Phase 1 PRs can be found here

Phase 2:

7. Integration into ListenBrainz : I passed the mid-term evaluation and moved into ListenBrainz repo to integrate the new notification system.

I created a notification sender function in ListenBrainz to create a notification using the endpoints created in the first phase. The function also automatically obtains an access token from the MeB OAuth2 provider, caches it in Redis and refreshes it if expired. This token is sent in the Authorization header of the HTTP requests to create the notifications. I then added tests for these functions.

I replaced the instances where e-mails were sent out using BrainzUtils to use send_notification instead.

As I had MeB and LB docker containers running side by side, I felt very happy when I sent a notification from my local LB instance and it correctly generated a token from MeB, and then the notification showed up in the MeB container logs.

8. Notification Settings: The only frontend component in this project (Finally! I can add some images 🙂 ).

I added a page where users can set their preferences for receiving notifications.

I added respective endpoints which initially fetch the digest data from MeB.org, and if the user changes their preference, it sets the preference by sending a POST request to MeB.org’s <user_id>/digest-preference endpoint.

Phase 2 PRs can be found here

Current State

Currently, all of the features mentioned are implemented and (almost) merged. This project is in metabrainz-notification branch of both repositories. It can be deployed to production after user data is migrated to metabrainz.org (PR by @kartikohri13 here) and we have the “user” table for foreign key and user_emails.

Future

Although the project has met all of the expected outcomes, there are still a lot of features I’d like to work on. Some of them are :

• Integration into remaining MetaBrainz projects
• Integrating MB-mail into the notification system to send HTML e-mails.
• A /notifications endpoint in MeB.org for users who need non-important e-mails sent to them immediately.
• Creating templates for non-important Notifications and sending them to users instead of just rendering them on User feed.

Conclusion

It was a really great experience participating in GSoC 2025 with MetaBrainz. I have learned so many new things this summer while working on my project — mainly understanding how web development works in a large org with a lot of moving pieces, docker containers , github ci-cd, REST framework, Authorization Framework and the essence of open source software.

Im thankful to my mentor Robert Kaye @ruaok for guiding me through this project and for his insightful reviews. I am also thankful to @kartikohri13 (AKA lucifer) and @saliconpetiller (AKA monkey) for your incredible support.

It’s been a pleasure working with you all! Thank you for an incredible summer.