Hi everyone!
I am Eric Deng, known as ericd on MB element server and ericd23 on Github. From May through August, I worked on adding various syndication feeds (Atom feeds) to ListenBrainz, a project that was mentored by Mayhem.
Project Overview
ListenBrainz is a powerful platform that allows users to keep track of their listening history, share music tastes, and discover new music. The primary goal of my project was to extend ListenBrainz’s functionality by adding syndication feeds, specifically Atom feeds. This feature enables users to subscribe to various feeds, allowing them to access their data in a more unified (through a feed reader) and customized (with various options each feed offers) manner. This feature is an addition that builds on ListenBrainz’s existing strengths and open data nature.
The goals of this project includes:
- Basic feed generation infrastructure
- Various feed endpoints: user latest activity, user listens, user statistics, playlists, recommendations (weekly jam and exploration), fresh releases (user and site-wide) and statistic art creator.
- UI for accessing feed URLs
What I Did
The primary focus of my project was to implement syndication feeds in ListenBrainz using the Atom format. This feature allows users to subscribe to various feeds with the help of interfaces to copy the URLs of the feed. Here’s a breakdown of what I accomplished during the project:
Feed endpoints
To implement the syndication feeds for ListenBrainz, I used Flask for routing and the feedgen library to create Atom feeds. The process of generating each feed follows a structured approach to ensure consistency across various endpoints.
The feed generation begins with setting up a route in Flask, such as the listens feed accessible via /user/<user_name>/listens. The route handler retrieves relevant data based on query parameters like time ranges, which are then structured into an Atom feed using feedgen. A utility function _init_feed standardizes the initialization of the feed across different endpoints, ensuring consistency.
def _init_feed(id, title, self_url, alternate_url):
fg = FeedGenerator()
fg.id(id)
fg.title(title)
fg.author({"name": "ListenBrainz"})
fg.link(href=alternate_url, rel="alternate")
fg.link(href=self_url, rel="self")
return fg
This function sets up the basic elements of a feed, including its ID, title, author, and links. It is reused across various feed types to maintain consistency.
For the listens feed, the endpoint provides users with a real-time view of their recent listening activity. This feed accepts a minutes parameter that defines the time window for the data. For example, choosing ’30 minutes’ will include listens from the last 30 minutes. Then a user can set the feed reader’s refresh interval to match this time range for optimal updates. The back end fetches the listens from the database using start and end timestamp using minutes, and each listen is added as an entry in the feed.
listens, _, _ = timescale_connection._ts.fetch_listens(
user, from_ts=from_ts, to_ts=to_ts, limit=MAX_ITEMS_PER_GET
)
The fresh releases feed is another type that highlights new music releases, both site-wide and user-specific. The site-wide feed pulls data from a central database. It accepts a days parameter that defines how many days of past releases to include in the feed. The feed generation involves sorting releases by date and creating entries for each.
db_releases, _ = get_sitewide_fresh_releases(
ts_conn, date.today(), days, "release_date", True, False
)
For user statistics, the stats feed provides a summary of data such as top artists, albums, and tracks. This feed supports various time ranges like weekly, monthly, or all-time. I utilized a helper function _get_entity_stats to fetch and format the relevant statistics from couchdb before adding them to the feed.
entity_list, to_ts, last_updated = _get_entity_stats(
user["id"], "artists", range, count
)
The playlist feed keeps users updated on changes to their playlists, such as new tracks added. This feed is dynamically generated by fetching the playlist’s metadata and tracks, then structuring this information into Atom entries.
fetch_playlist_recording_metadata(playlist)
For music recommendations, the recommendations feed allows users to receive personalized suggestions. This feed includes entries generated by ListenBrainz’s recommendation algorithms, handling different types such as “Weekly Jams” and “Weekly Exploration.”
playlists = db_playlist.get_recommendation_playlists_for_user(
db_conn, ts_conn, user["id"]
)
The stats art feed offers visual representations of user statistics, such as album cover art grids. The feed allows users to customize dimensions, layouts, and time ranges, making it a flexible tool for generating personalized stats art.
Lastly, the user events feed aggregates various activities like follows, listens, and recording pins, providing a comprehensive view of a user’s interactions on the platform.
To avoid repetitive code and ensure maintainability, I created several abstraction layers. For instance, the _generate_event_title function generates descriptive titles for different types of user events, ensuring consistency across the user events feed.
def _generate_event_title(event):
if event.event_type == UserTimelineEventType.RECORDING_RECOMMENDATION:
return f"{event.user_name} recommended {track_name} by {artist_name}"
# Additional conditions for other event types...
These utility functions play a crucial role in streamlining the feed generation process and reducing the likelihood of errors.
On the front end, I developed a user-friendly interface to complement these back end feed endpoints. The SyndicationFeedModal component is a React modal that allows users to configure their feed subscriptions with ease.
It provides a nice and simple abstraction to add feed button to various places. Pass in the props and the modal will many useful information for the users.
export type SyndicationFeedModalProps = {
feedTitle: string;
options: {
label: string;
key: string;
type: HTMLInputTypeAttribute;
values: { id: string; value: string; displayValue?: string }[];
min?: number;
max?: number;
defaultIndex?: number;
tooltip?: string;
}[];
baseUrl: string;
};
As users adjust these settings, the feed URL is dynamically updated and displayed.
const buildLink = () => {
const queryParams = new URLSearchParams(selectedOptions).toString();
return `${baseUrl}?${queryParams}`;
};
The modal uses React’s state management to track user selections and regenerates the feed URL in real-time, allowing users to immediately see the impact of their choices.
The UI is both functional and informative, with tooltips providing additional context for each option to ensure that users understand what each setting does. Once users finalize their choices, they can easily copy the generated feed URL with a single click.
This integration between back end and front end ensures that users can take full advantage of the new syndication feeds in ListenBrainz, making the feature both powerful and accessible and user-friendly.
The Current State
As of now, several of the feed implementations have been merged into the master branch and deployed to the production server. The remaining feed endpoints and the feed UI are nearing completion and are expected to be stable soon. The user latest activity feed is dependent on the new MusicBrainz OAuth integration in ListenBrainz, which is in progress. Once this dependency is in place, the latest activity feed will be ready for merging. Most of the features described in this post will be released in the coming weeks, but as already mentioned the latest activity feed cannot be merged until later this year.
What’s Left
While many feed endpoints and the corresponding UIs are implemented, there is always room for improvement. Additional features within ListenBrainz might benefit from having their own syndication feeds—features that I may not have considered during the project. Fortunately, the infrastructure I’ve built makes it straightforward to add new feeds, as the process for generating a feed is standardized, and the feed UI modal is also abstracted away.
Additionally, the feed UI could be refined to further improve the user experience. For instance, providing more detailed explanations for each feed option could help users better understand the available customizations. Since interacting with feeds isn’t as straightforward as using a web application—requiring users to copy the URL into a feed reader, refresh it, and navigate through potentially large lists of entries—it’s important to ensure that users have a clear understanding of what each option does upfront. This would make the process more intuitive and help users get the most out of the syndication feeds.
Final Thoughts
As a music lover, it’s incredible to work with MetaBrainz and on ListenBrainz, of which I’m a heavy user. It’s also a great learning experience. I gained valuable insights into web development and I’m grateful for the support of Mayhem and the ListenBrainz community.
Looking forward, I’m excited to see how users interact with the new syndication feeds and hope that these features enhance their experience with ListenBrainz. I also look forward to continuing my contributions to ListenBrainz.
I hope everyone enjoy using the new syndication feeds in ListenBrainz!