Creating a blogroll page from Mastodon bookmarks
Why did I want a bookmarks page?
A thing that has been on my ideas list for quite some time is to create a links type page from the Toots I bookmark on Mastodon. The mastodon bookmarks are really handy for items that I want to go back to, to read more or as a prompt to get me to do something. Having them actually on my website makes it more like a blogroll of old and here is my bookmarks page.
Mastodon API
Mastodon has an API that you can pull many interesting things out of and this was what got me thinking about this idea in the first place. It has been on the ideas list for a few years because I can't really write Javascript. I can look through it and make some good assumptions to whats happening but coding this is still a dark art to me. The Mastodon API docs are available to peruse and I had a glance at one point.
Down the rabbit hole of code I go
So I turned to AI. Of course I did, and if you think less of me for it then that's just fine with me and no doubt you will give me some grief on Mastodon. Off you toddle.
Meanwhile for those of you that are still interested I went through a fair bit of prompting and trial and error to get this working in the first place and then how I wanted it to work. It wasn't 5 mins time of ChatGPT and there it was. No, it was a week of spare moments pulling out hair, trying to understand things and then trying again. In the end I got there and I did learn a lot from it.
With the Mastodon API you can pull out 40 bookmarks in one go - you can pull out more but then you have to call the API again asking for pagination and I have not had time to work out how to do that properly yet. I may not ever bother as 40 takes me back quite a while as I don't bookmark that often, though I might do more of it now!
The environment file
First I had to set up a .env file with my Instance and Access token at root level in my 11ty folder so that the script to make the authenticated call. In this file I added the following two lines (I have not filled in the instance or token in this example!):
MASTODON_INSTANCE=https://your.instance MASTODON_ACCESS_TOKEN=your_access_token
My instance being the Mastodon server I am on. The Mastodon Access Token I needed to create by going to my Mastodon Preferences > Development to create New application. I give the Application a name and then at the top of the page I found my access token - it's a long string. Next I choose the Scope - read:bookmarks is all that I needed for this and then I saved the application down at the bottom with the Save Changes button.
I added those two values to my .env file.
The bookmarks Script
Next I created a bookmarks.js file in my _data folder and this is my current script:
// Load environment variables from the .env file
require("dotenv").config();
// Import EleventyFetch for fetching remote data and caching it
const EleventyFetch = require("@11ty/eleventy-fetch");
// Get Mastodon instance URL and access token from environment variables
const MASTODON_INSTANCE = process.env.MASTODON_INSTANCE;
const MASTODON_ACCESS_TOKEN = process.env.MASTODON_ACCESS_TOKEN;
// Helper function to check if an image URL is valid (it will try to fetch only the header)
async function checkImageExists(url) {
try {
// Send a HEAD request to check if the image exists without downloading it
const response = await fetch(url, { method: "HEAD" });
return response.ok; // If the response status is 2xx, the image exists
} catch (error) {
// If there's an error (network issue, or image doesn't exist), return false
return false;
}
}
// Helper function to extract the 'next' page URL from the Link header (used for pagination)
function getNextPage(linkHeader) {
// If the link header is missing, there's no next page
if (!linkHeader) return null;
// Regular expression to extract the 'next' link from the pagination header
const match = linkHeader.match(/<([^>]+)>;\s*rel="next"/);
return match ? match[1] : null; // Return the next page URL or null if not found
}
// Main function to fetch and process Mastodon bookmarks
module.exports = async function () {
// Check if the required environment variables are set, otherwise log an error
if (!MASTODON_INSTANCE || !MASTODON_ACCESS_TOKEN) {
console.error("Missing Mastodon credentials in .env file");
return []; // Return an empty array if credentials are missing
}
// Construct the URL to fetch bookmarks from Mastodon API
const url = `${MASTODON_INSTANCE}/api/v1/bookmarks?limit=40`; // Fetch up to 40 bookmarks
// Add the authorization header with the Mastodon access token
const headers = { Authorization: `Bearer ${MASTODON_ACCESS_TOKEN}` };
try {
// Use EleventyFetch to fetch the bookmarks with caching for 12 hours
const data = await EleventyFetch(url, {
duration: "12h", // Cache the result for 12 hours
type: "json", // Expect JSON data from the API
fetchOptions: { headers } // Include the headers (with the access token)
});
// Process the fetched data (map over each bookmark and handle the media)
return await Promise.all(
data.map(async bookmark => {
// Check for images in the media_attachments array and validate them
const images = await Promise.all(
bookmark.media_attachments
.filter(attachment => attachment.type === "image") // Filter only image attachments
.map(async attachment => {
// For each image, check if it exists by using the checkImageExists function
if (await checkImageExists(attachment.url)) {
return attachment.url; // Return the image URL if valid
}
return null; // Return null if the image doesn't exist
})
);
// Return a simplified structure for each bookmark with the relevant data
return {
id: bookmark.id, // Bookmark ID
content: bookmark.content, // Bookmark content (text)
url: bookmark.url, // URL of the bookmarked post
created_at: bookmark.created_at, // Timestamp when the bookmark was created
account: {
username: bookmark.account.username, // Account username
url: bookmark.account.url, // Account URL
avatar: bookmark.account.avatar // Account avatar URL
},
images: images.filter(img => img !== null) // Remove null image entries
};
})
);
} catch (error) {
// If there was an error fetching data, log it and return an empty array
console.error("Failed to fetch Mastodon bookmarks:", error);
return [];
}
};
Page code
To display the output I set up the following. I have the styles inline at the moment as I am still mucking around with it so they may well end up in the site css.
{% for bookmark in bookmarks %}
<div style="border: 1px solid #ccc; padding: 10px; margin: 20px auto; border-radius: 20px;max-width:360px;overflow-wrap:anywhere;">
<img style="width:40px;margin:0" src="{{ bookmark.account.avatar }}" alt="{{ bookmark.account.username }}" width="40" height="40">
<p>from {{ bookmark.account.username }}</p>
<p>{{ bookmark.content | safe }}</p>
{% if bookmark.images.length > 0 %}
<div class="images">
{% for image in bookmark.images %}
<img src="{{ image }}" alt="Mastodon image" />
{% endfor %}
</div>
{% endif %}
<small><a href="{{ bookmark.url }}">Link to the the Toot</a> – Saved on {{ bookmark.created_at }}</small>
</div>
{% endfor %}
Process of adding the bookmarks
I did not want this process to be client side as it would have taken too long to load the page and there was a bigger risk of exposing my token. So I have this setup to build out the page when I run my build command locally. the html page gets build and the images added from cache to the img folder. That is then submitted to Github and deployed to the server.
The disadvantage with this is that I need to go through a build process to get the latest 40 bookmarks onto the page. I am always tinkering with this site though so that's not a problem for me!
What next?
Hopefully some others will find this useful and create something similar if not a lot better and I await for you to post on Mastodon so that I can bookmark them!
Next post: Recovery from a Bing de-indexing
Previous post: A Winters Day Trip To Rye
