I had a phone on hand to send them. To save us all a lot of headaches I started setting up the phone for him before sending it. I go about with the usual suspects: Gmail, Facebook, and a tracker for the price of gold, possibly more important than the former two apps.

My dad has been bedridden for about a year now since the revolving door of medical care that started with a case of sepsis. He uses his phone a lot and every time I visit it’s more or less the same. I appreciate that I’m trusted with his digital life and helping him with whatever odd setting may have changed or apps disappeared.

Upon logging into Facebook on his new phone I found an eerie ghost town of AI generated videos, shock content, and engagement bait. A talking bowl of cereal. A lion about to pounce on someone. A vintage photo of a tough guy overlaid with the text “a real man always takes care of his family. Do you agree?”

A few months ago, I went through his feed and unfollowed a bunch of pages and downvoted a bunch of this type of content, which seemed to help in the moment. I’ve asked dad about it before and he doesn't want to see this stuff, but it sucks him in enough to tell the algorithm to show him more.

Following the posts back to the groups they came from, I found quite strange content. Things like environmentalism in South Africa, but the posts were videos of people in waist-deep water picking up plastic bottles, or textile factories in Southeast Asia, or nostalgia-slop content with images of celebrities with captions like, “There’ll never be another Chuck Norris. Rest in peace.”

As I was "leaving" these groups, sometimes a message would pop up with a message like: “Check this box so people can’t add you to this group again.” So... someone is adding him to these groups? My dad lived a full life, but he was not traveling the world collecting strangers on Facebook. As long as he has had Facebook, I have known pretty much everyone in his life. I started rooting around his friends list and many of these accounts were definitely not people he knew. They fell into a few big categories: hot woman as the profile picture, names in Cyrillic characters, or names from Southeast Asia or Eastern Europe.

They accounts appeared in clusters: an account would have several mutual friends, and those mutual friends were usually connected to one another as well. It may be due to Facebook’s setting limiting friend requests to friends of friends. Some of these accounts only had five or ten friends. My theory is that after you are friends with one they tell these other accounts to add you too. Once they are connected, they start adding you to these Facebook groups, and your feed fills up with group slop. Being in the groups also seems to make it easier to pull you into spam group chats.

During this exercise, his friends list shrunk from ~320 to ~120.

Being close to my dad and knowing him as well as I do, I can make pretty good guesses about who he actually knows. If all their posts are in another language, he probably does not know them. If they say they live in some faraway country, he probably does not know them. If all they post are ads for cheap Ray-Bans or promises of free tokens for some online game, he probably does not know them. Also, none of these accounts ever had direct messages or signs of interaction with my dad.

I hope it means his feed will have content that’s more enriching and less outraging… but who knows. This is the labor of digital caregiving that many people are now obliged to do. Unfortunately, many more people probably have no one to do it for them.

It's part of why I believe people need systemic protections. My dad could have been just a few messages away from being duped into sending his life savings through a crypto ATM to someone on the other side of the world. With my urbanist friends, I often voice concern for exposing local housing markets to global finance investors. Echoing that sentiment, we should do our best to avoid exposing local aging populations to global fraud networks.

Example profile: name in Cyrillic, photos of a gambling game, posts on wall all in another language.

I used data from OpenStreetMap for thrift stores (labeled as such, or as selling secondhand goods) and mapped it against income data in San Francisco. Somewhat surprisingly, no strong spatial patterns emerged as thrift stores are in areas of many income levels rather than clustering in high or low income areas as one might expect. Perhaps thrift stores are more centered around high foot traffic areas in the city. Beyond the analysis, I also took the opportunity to contribute back to the community by updating several outdated thrift store entries in OpenStreetMap.

Final Map

Data sources

• Income data was from: B19013 Median Household Income in the Past 12 Months (in 2023 Inflation-Adjusted Dollars) by Census Block Group in San Francisco County from here

• 2024 TIGER/Line Shapefiles: Block Groups from here

Bulk adding information to Wikidata by hand is a pain. I have an idea for creating a map of MVNOs (mobile virtual network operators) and their ownership (AT&T owns Cricket, Verizon owns Visible and many brands, T-Mobile owns Mint), like this map of food brands. I want it to be powered by Wikidata so it will update automatically and group together all the MVNOs based on who owns them and what network they run on. Since I'm focusing on the U.S., I needed to add the country field to every MVNO that didn't have it it.

I used Gemini (in Google Sheets) to extract the country name from the description of about 20 MVNOs that I queried from Wikidata. I wanted to add them using QuickStatements, a tool to bulk update Wikidata, but I couldn't get the string of the country to convert to a proper QID. I could do it manually but I wanted to get better at using Quickstatments. It may have supported functionality long ago, but doesn’t anymore, likely because there is always a chance of incorrect matches when using just the string. Georgia the state and Georgia the country are a classic example of why a bit more context is needed for accuracy.

Thankfully, my friend Drini showed me Wikipedia and Wikidata Tools, a Google Sheets extension that lets you do this query as simply as =WIKIDATAQID("en:Spain"). It works decently, but giving a tool access to your account, readings the docs, and being dependent on Google aren't ideal.

To solve this, I created a Wikidata string to QID tool (source). Paste in a list of strings and get a list QIDs. It also shows the description so you can feel more confident in the match and go about your merry way editing Wikidata.

I also went ahead and updated the Wikidata QuickStatements page with some information about this, since it was not very easy to find.

To help people find this on the web, here are some of the ways I searched for this:

• How can I turn text into a Wikidata QID?
• How to add a string as the value of a property using QuickStatements for Wikidata?
• Resolving a string to the corresponding Wikidata ID

In fact, this most recent bus even had 250 MB of free WiFi. I didn’t bother using it but I found it to be an amusingly small amount for a 10 hour trip. I think it’s the first time I saw WiFi on a FlixBus. According to Reddit, they also block video streaming. It’s powered by RebelRoam, which is quite proud to provide "Network Optimization" by blocking video.

Blocking video doesn't seem that bad for the context but it's pretty lame that they get away with advertising "free WiFi" that rarely exists and offers so little data.

Not sure how long they've been doing this but I found a Tweet about it from 2023.

I'm not sure if it's on every route but I was traveling on bus N61 going from Amsterdam to Nuremberg on October 3rd from 10:30 pm to 8:10 am.

They also sold beer on the bus, another new one for me. 🍺

A few screenshots:

At the end of the day, the process is relatively simple. However, despite having worked with all of these tools on existing projects, I hadn’t really set up new projects with them and didn’t find a complete guide so here goes.

Tools Involved

• uv - the replacement for pip that has been skyrocketing in popularity. It takes away a lot of the pain points.
• fastapi or flask - two popular web frameworks for Python. Flask is very barebones and time tested. fastapi is more full featured with batteries included.
• coolify - a very simple and easy tool for self hosting apps. Similar to Heroku or Vercel. (This tutorial assumes you already have coolify setup)
• nixpacks - this automatically generates docker images for you based on your repo’s structure.
• GitHub - hosted version control. Mentioning this here because we’ll have a push-to-deploy setup.

Steps to Deploy Your App

1. Initialize your uv project: This command sets up your Python project structure, creating essential files like pyproject.toml to manage dependencies.

   uv init hello-world && cd hello-world
   

2. Add your web framework dependency: Use uv add to install either fastapi (with optional extras like standard for uvicorn and pydantic) or the classic flask.

   # For FastAPI
   uv add "fastapi[standard]"
   
   # Or for Flask
   uv add flask
   

3. Create your application code: Add starter code to main.py. Here's the example from the FastAPI documentation:

   # main.py
   from typing import Union
   from fastapi import FastAPI
   
   app = FastAPI()
   
   @app.get("/")
   def read_root():
       return {"Hello": "World"}
   
   @app.get("/items/{item_id}")
   def read_item(item_id: int, q: Union[str, None] = None):
       return {"item_id": item_id, "q": q}
   

4. Configure the start command for Nixpacks: Create a nixpacks.toml file. This tells Nixpacks (and therefore Coolify) how to run your application inside the container.

   touch nixpacks.toml
   

   Add the following content. We use port 3000 as it's the default Coolify expects.

   For FastAPI (using uvicorn):

   # nixpacks.toml
   [start]
   cmd = "fastapi run main.py --port 3000"
   

   For Flask (using gunicorn):

   # nixpacks.toml
   [start]
   cmd = "gunicorn -w 4 -b 0.0.0.0:3000 main:app"
   

5. Commit your code:

   git add .
   git commit -m "Initial project setup"
   # Push to your GitHub repository
   git push origin main
   

6. Configure the application in Coolify:

   • Go to your Coolify dashboard and create a new application.
   • Select "GitHub" as the source and choose the repository you just pushed.
   • Coolify should automatically detect Nixpacks as the build pack and 3000 as the port based on your nixpacks.toml.

7. Deploy:

   • Click the "Deploy" button.
   • After the initial deployment, Coolify will automatically redeploy your application every time you push changes to your configured GitHub branch.

Final Notes

If you’re not using GitHub you can also configure webhooks in Coolify.

You probably want to add a .gitignore file for the __pycache__ directory

This is written for Coolify v4. Since v5 was just announced by the time you read this things on the Coolify side may have changed.

The source code for the Flask version is available here.

Thanks for reading!

Because I was preparing for a move at the time, I submitted a pre-recorded video instead of presenting live. This turned out to be fortunate, as I encountered some technical difficulties during the live Q&A portion – Jitsi indicated my microphone was active when it was actually muted!

Despite that small hiccup, I feel the presentation was successful. I was able to answer several questions and hopefully encouraged some attendees to check out Open Library.

You can watch the recording here.

Oh right, back to the conference. After waking up at 5:30 and driving two hours, I arrived just in time to sleep through the second half of the keynote. The talk wasn't boring, but rather that I was exhausted and needed to build some energy for socializing. I ended up seeing four talks, including Diane’s – which was so fun (and her first time speaking at a conference!) – and Lizzie’s about Cloudflare Durable Objects, where I learned something completely new.

The crowd at the conference was friendly and nerdy (in the good way). I chatted briefly with a few strangers, but not being deeply immersed in the tech world, I felt a little shy about approaching people (the lack of sleep wasn't helping either). One person, however, talked with me quite a bit about government and technology. I told him about the street experiment where someone turned a parking spot into a temporary picnic area, which resulted in the police coming to remove the picnic table. His main response was, “Good! That’s for cars. We wouldn’t let a car take space in a park, so we shouldn’t allow a picnic table in a parking spot.” At that moment, it hit me that I was in a place where nearly everyone relies on cars and seems happy to keep it that way. And now, as I write this, I realize this difference in perspective is exactly what the Lab of Thought is trying to address: opening people’s minds to alternatives. That’s relatively easy to do in progressive cities or with fellow urbanists, but I was woefully unprepared for even the mild resistance my anecdote faced there.

Alas, Orlando Code Camp was a great experience. I made a new friend, celebrated an old friend, and reignited some excitement for both software and urbanism.

PS: The banner photo is of the ending ceremony where they gave away prizes

Nextcloud is an open-source alternative to Google Drive. I use it because I value control over my data and appreciate its use of open standards. While many people complain about Nextcloud, if they invested that time in reporting issues or contributing fixes, the platform would be much better.

Earlier this year, while working on my thesis, I began using Nextcloud Tasks extensively. I encountered some minor jankiness and decided to do something other than complain. I reported three bugs and even fixed one of them. Here's a summary of my contributions:

1. Navigation Menu (sidebar) CSS transition is janky with sharp corners

• Reported on March 3, 2024

• My fix was merged on March 20, 2024

2. Three-dots menu button on task details sidebar always looks selected

• Reported on March 7, 2024

• Fixed on March 20, 2024

3. Remove extra CSS margin for URL in task title

• Initially reported on March 1, 2024

• Reported on a different repo on March 20, 2024

• Fixed on May 4, 2024

If you want open-source software to be good, help make it better. Sometimes, it's as simple as filing a bug report.

In the near future, I want to do a deep-dive comparison of Nextcloud Tasks and TickTick to see how we can polish the UI for Tasks and make it feel smoother overall.

Seeking feedback, I turned to Reddit, where a question about the necessity of inline dynamic imports set me on the right path. I realized we weren't using dynamic imports at all. I had initially enabled them as required for IIFE mode, which I had chosen for its compatibility with older browsers.

There's a better method for supporting legacy browsers while maintaining modern functionality. The solution is to use the legacy plugin and create two separate builds: a modern version and a legacy fallback. It looks like this for the client:

<script type="module" src="{modern_build}"></script>
<!-- Fallback for legacy browsers -->
<script nomodule src="{polyfills_build}" defer></script>
<script nomodule src="{legacy_build}" defer></script>

This approach offers several advantages:

1. It produces smaller file sizes.
2. It extends support to an even broader range of older browsers.
3. Modern browsers use the optimized build, while legacy browsers automatically fall back to the compatible version.

With this improved setup, we can now utilize multiple inputs and outputs in our build process, eliminating the need to run the build tool separately for each component.

As a result, our vite.config.js file has been simplified:

const buildInput = {};
componentNames.forEach(name => { buildInput[name] = getTemporaryVueInputPath(name) });

export default defineConfig({
    plugins: [
        vue({ customElement: true }),
        legacy({ targets: ['defaults', 'not IE 11'] })
    ],
    build: {
        outDir: join(BUILD_DIR, '/production'),
        rollupOptions: {
            input: buildInput,
            output: {
                entryFileNames: 'ol-[name].js',
                format: 'es'
            },
        },
    },
});

While my issue reports suggesting improvements to the documentation remain unaddressed, I hope these blog posts can assist others facing similar challenges.

Found this helpful? Consider joining our Open Library community as a dev, librarian, designer, or in any other way you want to help.

I want to be better at reflecting and want to remember these wonderful years. So here I am, at the end of January, writing my 2024 in review. I was inspired by Steve and Shreyans’ blog posts and actually finishing the YearCompass first.

This year I lived in Amsterdam 🇳🇱, Oviedo (Spain) 🇪🇸, Thailand (just 3 weeks) 🇹🇭, and Bogota 🇨🇴.

🎓 I completed my MSc in Urban Studies (4CITIES). I learned a lot and now have well informed opinions on a few things now. The few months of focused writing on the thesis were hell but I’m pretty happy with the result: The Uneasy Emergence of the Edible Insect Industry in the Netherlands. Still figuring out how to bridge urbanism and tech for a career.

FOSDEM in Brussels was fun, volunteering at MozFest Amsterdam was a blast, interning with the Lab of Thought was a bit lonely but I made some great connections, I loved the “Dance to Share” class I took at CREA, making dumplings with a big group for Chinese New Year was chaotic bliss, visiting the Internet Archive in SF turned so many little faces from Zoom into real people, attending the 50 years of Ciclovia celebration got me fired up for making change, and Novenas and Christmas in Bogota gave me a great appreciation for how family makes the holidays special.

I had five intentions last year.

1. Open Source Contributions: Made lots of contributions, shared it in blogs, and I’m generally happy about it. Open Library has been a great community to be part of.

2. Cooking: Improved significantly, mastered dishes like creamy chicken pasta, and cooked (more) frequently, especially towards the end of the year.

3. Spanish Language Learning: Spent a solid part of the year working on it every day (loved Dreaming Spanish videos) and was able to hold my own during the holidays in Bogota. Still not where I’d like to be though.

4. Social Life: Maintained an active social life, especially in Amsterdam, with regular dinners and quality time with friends. After Amsterdam, not so much thanks to the thesis.

5. Reading & Writing: I wrote 11 blog posts, pretty good, but fell short on reading often (outside of thesis). I’ll do better this year.

Overall, I feel I did alright on the intentions, which were deeply disrupted by the thesis months. I think I’ll carry a few of them forward.

Finally, I’ll say it has been a pretty great year. I thoroughly enjoyed my time in Amsterdam and deeply appreciate the friends I made there. My year was also made oh so much better by the friends who visited including Alex, Sam, Sam & Cliff, Zoe, Kevin, and more. All that being said, I can’t imagine how this year would have been if I didn’t have my wonderful partner Maria Paula by my side through it all.

Lets make 2025 even better!

Selected Photos

Here are some photos, the bring cozy memories to me. Notably, excluding photos with people because there are too many good ones (and something about AI).

The beautiful Amsterdam apartment where we hosted many dinners and friends.

The view of the University of Amsterdam I enjoyed so many days.

Enjoying some beetle beer on the canal in Amsterdam.

The ominous datacenter I saw during a tour for MozFest.

A beautiful beach gathering in the North of Spain (beach is behind me).

A countryside bike tour in Chiang Mai, Thailand.

The locals in Colombia wondering what a gringo is doing way out here.

How I felt after weeks of celebrating Novenas, Christmas, and New Years, (also a photo of Olivia the cat).

TL;DR: Migrating Vue 2 Web Components to Vue 3 isn't as straightforward as the docs suggest. We tackled three main challenges: setting up Vite builds, managing multiple components, and handling plugins.

You can see the final pull request here.

This is the story of how a seemingly simple migration turned into a 20-hour adventure through the world of Vite and Vue 3. If you're considering a similar migration or just curious about the state of Vue 3 Web Components, this is for you.

Understanding Vue and Web Components

In contrast to typical Single Page Applications (SPAs), where Vue renders the entire application, Open Library leverages Vue for specific interactive elements within traditional HTML pages. This is accomplished by integrating custom HTML tags, or Web Components, which are then transformed into Vue components.

For example, when you look at the source of an Open Library page, you might see an <ol-barcode-scanner> tag. This custom tag, along with its JavaScript, allows Vue to render just that specific element rather than managing the entire page. It's like having mini Vue apps scattered throughout traditional HTML pages.

Vue 3 Migration Challenges

While Vue 3 is largely compatible with Vue 2, and there are many resources available for migrating from Vue 2 to Vue 3, our challenge lay in building web components in Vue 3. The Vue 3 documentation covers Web Components, but the approach was not directly applicable to our use case, which involves building one component per page with isolated scripts.

Vue 2 Build Setup

The process of building Web Components for Vue 2 was very straightforward. We would simply run vue-cli-service build BarcodeScanner.vue, and a JS file would be generated that we could use on any page.

Challenges with Vue 3 Web Components

Let me break down our journey into three main challenges. While each challenge has a straightforward solution in hindsight, finding these solutions took considerable exploration.

Building one Web Component with Vite

The deprecated Vue CLI we used to build Vue 2 Web Components does not support Vue 3 Web Components and has not been updated in years. As a result, I investigated the latest and greatest build tool: Vite. There's one big difference with Vite: it doesn't accept .vue files (Single File Components) like Vue CLI did. Instead, Vite requires a configuration file as its input.

This posed a challenge because I was 1) disbelieving that it couldn't take .vue files and instead required two new files for every component, 2) trying to get it to create all the components without so many new files, and 3) attempting to get a Vue plugin working (more on that later). That being said, the docs are fairly straightforward on how to do this for just one component. I just needed two new files:

// vite.config.js
export default defineConfig({
    plugins: [vue({ customElement: true })], // Because all of our Vue components are customElements
    build: {
        outDir: 'PRODUCTION_DIR',
        emptyOutDir: false, // Preserve existing files since we build components individually
        target: 'es2015', // The oldest browsers Vite supports out of the box
        rollupOptions: {
            input: join(BUILD_DIR, `BarcodeScanner.js`),
            output: {
                entryFileNames: `ol-barcode-scanner.js`,
                inlineDynamicImports: true, // needed for components to work with just one js file
                format: 'iife' // use iife to support old browsers without type="module"
            },
        },
    },
});

// BarcodeScanner.js
import { defineCustomElement } from 'vue';
import ele from './BarcodeScanner.vue';
customElements.define('ol-barcode-scanner', defineCustomElement(ele));

Building multiple Web Components

At this point, we have a Vite config file that can only have one input, which is a JavaScript file, and it produces a single output, the desired JavaScript file. This limitation poses a challenge since we have numerous components to build. While Vite config files support multiple input and output setups, this functionality is not available when using the inlineDynamicImports option, which we need for the components to work in isolation.

The problem we face is that we need a single configuration file to be mapped to one JavaScript file, which in turn is mapped to one Vue file. This setup seems overly complicated. Three files now for each component instead of just one .vue file. The two new files are basically identical for each component. There has to be a way around this, right?

The left half is the old Vue CLI setup, the right half is the new vite setup.

Trial and Error

Through reading docs, trial and error, and ChatGPT, we painfully accepted that we indeed need a Vue file, a JavaScript file that points to the Vue file, and a config file that points to the JavaScript file (please someone come and show me how I'm wrong). If we were to approach this in a traditional manner, we would require a Vue file, a JavaScript file, and a Vite config file for each component. Currently, we have five components, which means we would end up adding ten new files that are essentially identical. There has to be a way around this, right?

Solution

It seems too hacky for some of the best webdev tools (Vue and Vite), but I found a solution that actually worked without adding 10 new files to the codebase. It comes in two parts.

First, set an environment variable COMPONENT_NAME that vite.config.js reads so that we don't need one config file for every component:

const COMPONENT_NAME = process.env.COMPONENT;
export default defineConfig({
    build: {
        /// shorted to show the change
        rollupOptions: {
            input: join(BUILD_DIR, `vue-tmp-${COMPONENT_NAME}.js`),
            output: { entryFileNames: `ol-${COMPONENT_NAME}.js`, },
        },
    },
});

Second, we need to deal with these pesky input files. So we just generate them from a string... in the config file itself.

generateViteEntryFile(COMPONENT_NAME);

export default defineConfig({...})

function generateViteEntryFile(componentName) {
    const template = `
import { defineCustomElement } from 'vue';
import ele from './${componentName}.vue';
customElements.define('${kebabCase(componentName)}', defineCustomElement(ele));
`;

    try {
        writeFileSync(join(BUILD_DIR, `vue-tmp-${componentName}.js`), template);
    } catch (error) {
        // eslint-disable-next-line no-console
        console.error(`Failed to generate Vite entry file: ${error.message}`);
        process.exit(1);
    }
}

All things considered, this solution is simple and keeps us from having many nearly identical files to keep in sync. That being said, I really can't believe these hacks (environment variables and generated input files) are the best way.

Plugin Support

Everything was functioning smoothly for all of our components, except for one that utilizes a Vue plugin: vue-async-computed. We previously considered removing this plugin since it didn't support Vue 3, but we ultimately decided against it because it helps maintain a clean structure in our code. Drini actually made the PR to add Vue 3 support years ago. Given the lack of updates to that plugin, maybe it's worth reconsidering switching to newer alternatives like computedAsync, but dang it, I'm on a mission to upgrade to Vue 3. Besides, Drini made a good point that there are nice plugins we should consider using after the upgrade.

I explored various approaches to integrate the plugin into our Vue component. The guides I found often went into Vue's internals, particularly regarding how to incorporate the plugin for the component. Our team does not consist of Vue experts; we simply want a little more interactivity on parts of the site.

Ultimately, I decided to use the vue-web-component-wrapper library. To be fair, I saw it earlier, but resisted adding another dependency for something that should be easy. Anyway, the library simplifies the process by allowing us to pass in the desired plugin along with our component. Short and simple, and I even added a conditional so the plugin is only added for the one component that needs it.

Summary

I faced many challenges while migrating from Vue 2 Web Components to Vue 3 Web Components. However, they don't seem too hard now that we have solutions.

I am certain that there are technical reasons for the difficulties encountered during this migration. However, if I had a magic wand, here are the changes I would like:

1. Vite (rollup under the hood) should allow for multiple inputs and outputs while utilizing inlineDynamicImports. This feature alone would have greatly simplified the entire process. If this cannot be implemented, then the Web Components guide should be updated to include a reasonable workaround. Related issue here.

2. The Web Component guide should be updated to include information about plugins. It would be helpful to explain how to add them without introducing another dependency, or at the very least, to point to vue-web-component-wrapper as a viable solution. Issue opened here.

3. Vite should be able to accept .vue files in the vite.config.js.

Overall, this migration process required approximately 20 hours of my effort (not counting the attempts by other folks in previous years). Throughout this process, I did not find a single example that addressed all of these challenges. I hope that this account will be helpful to those poor souls who find themselves migrating from Vue 2 to Vue 3 in 2025 and beyond. Perhaps the Vue and Vite teams will take note of this and improve the official guide and possibly even the tooling. Happy open sourcing!

Found this helpful? Consider joining our Open Library community as a dev, librarian, designer, or any other way you want to help.

PS: Huge thanks to Drini for reviewing and merging this pull request!

Context

I recently shipped a feature for Open Library that displays Wikidata (a Wikipedia sister project) information on author pages, including Wikipedia links. This enhancement automatically shows Wikipedia links in the correct language and updates when new Wikipedia pages are created about an author - as long as we have their Wikidata ID.

The Problem

A long time ago, Open Library had a field on authors specifically for Wikipedia links. While users can no longer add or edit these legacy Wikipedia links through the UI (they now use a generic links field instead), the old links are still visible. Simply moving these links to the generic field would work, but we'd lose the benefits of language-aware links and other Wikidata integration features we're planning. That and a bunch of the links were broken anyway.

The Solution

Replace the old Wikipedia field entries with their corresponding Wikidata IDs.

Implementation Steps:

1. Download the Open Library author data dump

2. Query the dump to find authors with Wikipedia links

3. Use the Open Library API to add Wikidata IDs and remove Wikipedia links

Here's the query I used to extract authors and their Wikipedia links: SELECT column1 as key, json_extract(column4, '$.wikipedia') as wikipedia FROM read_csv('~/Downloads/ol_dump_2024-07-31/ol_dump_authors_2024-07-31.txt.gz') where wikipedia is not null limit 1;

The dump revealed 1,786 authors with Wikipedia links - a tad too many to manage in spreadsheets. So, with the help of AI, I built a small tool to help review them quickly.

The Tool

I built the review tool using Vue.js, Tailwind, and PocketBase. I chose this stack primarily to experiment with PocketBase, which I'm considering for future projects. It's an excellent solution for simple database hosting. I also wanted to try PocketHost, though they've since removed their free tier (fortunately, I'm grandfathered in).

For importing data into PocketBase, I used pocketbase-import. The initial import was slow, so I requested batch import capability from the creator, which they added soon after.

While using the tool, I discovered many authors already had correct Wikidata IDs and just needed their redundant Wikipedia links removed. I wrote a script to extract Wikidata IDs from Wikipedia links, which helped identify cases where we could simply remove the Wikipedia link without manual review.

This automation reduced the manual review cases to 209 authors - much more manageable! During the review process, I also identified and merged duplicate author records using the Open Library QuickStatements tool I've been developing.

After processing these records, I found a small oversight in my query logic. I had been comparing QID != QID_from_wikipedia, but needed to manually review cases where both fields were null.

To ensure completeness, I wrote a verification script to check the live data for any missed Wikipedia links. This caught a few redirects from merged authors, which I fixed.

Now Open Library is a little easier to use and the data is a little cleaner!

Video walkthrough: https://youtu.be/QQRKMWFK5yE

Next Steps:

• Run the query again after the next data dump to catch any new Wikipedia links

• Submit a PR to remove the old Wikipedia logic from Open Library

• Remove Wikipedia field from the Open Library schema here

The aftermath of breakfast lingers on our 3D animal placemats - well-used plates and a crumpled napkin keeping company with two ceramic coasters, gifted from a friend after her visit to Spain.

There are pink roses that I got for Mapy before she returned from visiting the doctor's office about her knee. They sit in a one-liter kettle, the only thing we have that can accommodate them.

There is also an incense holder with a few ashes and a small bag of napkins that forever slides around the table. The table and chairs are plastic patio furniture. Like almost everything here, they came with the house.

Looking beyond the table, we see the window that I love to look out. The best part is the dogs across the way running in the field, playing.

This particular park seems to be the one for dog families. The park a few hundred meters in the opposite direction seems to be one for kid families.

The worst part is the two withering trees in the center. Not because they’re drying, because they form the informal public urinal for all of the gig workers, taxi drivers, or anyone in a rush.

Between the trees and the park lies a large and, usually, dry canal. Just on the other side of the canal is a street that few people walk down, but cars frequently pass, and a long queue builds every rush hour.

However, the street doesn't go unused; most days there are a few recicladores that set up camp on the grass with their dogs. They're part of the informal waste system of Bogota, collecting and sorting cardboard and other waste to be sold.

They often have a small bonfire to stay warm and cook a snack. A quiet group, they always seem to be working. Even though the mess they leave behind is displeasing to the eye, there is a pleasure from observing this family, or group of friends, working and playing, and making the best with what they have.

Through the doorway there is a wind-beaten turquoise tarp. It is a visual reminder of the primary sound I hear throughout the day, which is the work from the construction crew that put up that tarp and are putting together a new building. So far, it seems rather well put together, unlike some of the shoddily constructed houses around. They start work early with their hammers, drills, and trucks. I don’t mind it much… I guess it is the sound of progress.

Between the construction site and my apartment is another building with an enclosed driveway housing nine perritos (little dogs). When the sun shines bright, I love to sit just outside the doorway, playing a daily game with my unseen neighbors - sending a "woof woof" floating across the air, guaranteed to trigger a chorus of excited yips and barks.

This is the view I have seen and enjoyed the most these past few years of travel. I have much peace, pleasure, and respect for this quaint community I have observed and become a part of.

Photo taken on January 8, 2025 in Cedritos, Bogota, Colombia.

While listening to Crying in H Mart, my hackNY Secret Santa gift, I used a little JS to scrape ~100 faculty profiles from EC Scholar. I manually verified each profile against Wikidata and created entries for faculty members who weren't yet represented. This manual process enabled me to quickly add LinkedIn profiles and other identifiers via the Wikidata for Web extension. Using QuickStatements, I updated each entry with the following attributes: human, gender, English speaker, Eckerd College employment (including the source URL), and a described-at URL.

The experience was both straightforward and enjoyable. Although I had been aware of QuickStatements for years, this was my first practical use of the tool. Essentially, I created this spreadsheet containing the IDs and fields I wanted to add and then pasted it into the tool. Most individuals did not yet have Wikidata entries, and I'm glad I could add them, as it should increase the discoverability of their research. It was also fun to learn more about their research interests and academic contributions.

Additionally, while working on this, I improved the web page title extract pattern field for several properties, enabling the Wikidata for Web extension to scrape the names for more properties.

A Trip Down Memory Lane

Back when the original Google Home was released, I was excited by its promise. As a broke high school student, I hunted on Craigslist for a cheap one, eventually making an hour-long journey near Tampa with my mom to pick one up. The excitement was palpable – this was supposed to be the future.

However, as many of us experienced, these smart speakers didn't quite live up to their promise of getting better over time. In some ways, they regressed. Google Keep integration for shopping lists was replaced with a less capable Google Lists service. Features I hoped for, like Google Voice integration for messaging, never materialized.

The Past Event Problem

One persistent limitation across Google's voice assistants was the inability to create calendar events in the past. While this feature briefly appeared in Google Assistant on phones at some point, it wasn't consistently available. Even when Gemini launched on Android as Assistant's replacement, this functionality was absent.

But now, finally, Gemini can create past calendar events on Android. (see cover image)

Why Create Past Events?

Years ago, I came across a yearly planning notebook filled with detailed daily schedules – meals, appointments, everything. Initially, I wondered, "Who plans their life in such detail?" But I later learned that planners often serve as historical records, not just future planning tools. People retrospectively document their activities to maintain a history of their days.

I've long used my calendar this way, adding events after they occur to maintain a record of my activities. It's gratifying to see Google Gemini finally supporting this use case.

Looking Forward: AI and Task Management

Speaking of AI assistants and task management, I'm wondering about the current state of AI integration in to-do list applications. While creating new tasks through AI/voice is relatively straightforward, I haven't encountered any apps that allow natural language commands for marking tasks complete or updating task descriptions. Given the current capabilities of AI technology, this functionality seems feasible, yet I haven't come across any to-do list applications that implement it. Let me know if you find any solutions that offer this type of AI integration.

A video tutorial demonstrating this process is available here.

Introduction

Brand presets help OpenStreetMap editors like iD, Every Door, and StreetComplete maintain consistent tagging and spelling across the map. These presets appear as suggestions when adding businesses and points of interest. This guide explains how to contribute new brand presets to OpenStreetMap.

The image above shows a brand preset for Warby Parker appearing in the iD editor's search interface.

Background: Name Suggestion Index (NSI)

The Name Suggestion Index (NSI) is:

• A comprehensive database of brand information and identifiers

• Connected to WikiData for retrieving logos and brand details

• Separate from but used by the OpenStreetMap community

• Integrated into multiple OSM editing tools

Using NSI Guide

Checking Existing Brands

1. Visit NSI Guide

2. Search for your brand of interest

3. Review the results to see:

   • WikiData information

   • Geographic coverage

   • Existing preset details

Regional Availability

Brand presets are region-specific. For example, Warby Parker presets only appear when editing in the United States since that's where their locations are.

Contributing New Brands

Prerequisites

1. Verify Brand Notability

   • The brand should have at least 30 physical locations

   • This requirement may be flexible in some cases

   • Use official sources, Wikipedia, or OSM queries.

2. WikiData Entry Required

   • Brand must have a WikiData entry

   • You can create one if it doesn't exist

How to Contribute

For developers:

• Follow the technical guidelines in the CONTRIBUTING.md

For non-technical contributors:

1. Check Existing Entries

   • Search NSI Guide

   • If a brand appears in NSI but not your editor:

     • Verify the brand's configured regions match your editing location

     • Check when the brand was added - recent additions may not be in your editor yet

2. Submit via GitHub Issues

   • Go to the NSI GitHub repository

   • Create a new issue including:

     • Brand name

     • WikiData link

     • Number of locations

     • Supporting documentation (Wikipedia links, official sources)

Review and Integration Timeline

• Maintainer review typically only take a few days

• Complex cases may require community discussion

• Editor integration depends on their release schedule

Conclusion

You now know how to contribute brand presets to OpenStreetMap. Whether adding new brands or updating existing ones, your contributions help improve mapping consistency for everyone.

PS: I created this blog post as a companion to the video tutorial, focusing on searchability rather than traditional blog formatting. After repeatedly struggling to find the Name Suggestion Index documentation, I wanted to create an easily discoverable resource for others looking to add brand presets to OpenStreetMap.

Update Nov 19, 2024: If you have a JSOM preset and you want to merge it into NSI someone shared a script to do so here.

It's a powerful documentary because it doesn't seek an objective truth about the situation. It merely tells the story of people as they go through the chaos.

A brief summary: In Brooklyn's Industry City, a large developer purchased the industrial complex and began renovating it, converting upper floors to offices. They then sought to rezone the lower floors from mandatory manufacturing use to allow retail use. This sparked intense debate in the community.

Many residents oppose it because they're worried:

1. The good blue-collar manufacturing jobs will go away

2. They don't want the low-quality service jobs

3. It will lead to greatly increased rent costs and ultimately displacement

In short, they believe the changes won't benefit the existing community.

The counterargument by Adrian Kimball, the CEO of Industry City, is that it's disingenuous to say local residents' children won't be able to take the new office jobs. However, this argument seems flawed since many families would be priced out of the neighborhood before their children are old enough for such positions.

This battle plays out over many years with UPROSE, a local environmental justice organization, leading the opposition. Throughout the process, stakeholders grapple with questions about what development scenarios are possible, probable, and truly beneficial for the community rather than just the landowners.

As a friend put it: "the movie made democracy look messy." If you're interested in changing your city, making it greener, making it more bike-friendly, or understanding how these changes happen, this is the film for you. Check out the movie's website for upcoming screenings. If it's not showing near you, look at Kelly Anderson's other documentaries about the city. I haven't watched any yet, but they look promising.

Urbanist musings

• The community participation seemed to be token at best. Industry City didn't do a good job gathering feedback and seemed surprised by the questions and demands of people. Of course, the film didn't show all the meetings, but it seemed like the CEO was speaking much more than listening. If they were serious, they should have offered free childcare for these meetings (one woman in the documentary mentioned the struggle of staying late with kids at home). Given all the office space they control, you'd think they could have organized some venues that weren't overheated and overcrowded.

• In urban redevelopments, one way to reduce displacement is to offer fixed rents for 10 or 20 years. Industry City couldn't implement this solution because they don't own the nearby residential properties.

• The idea that manufacturing should stay to help build green infrastructure for the future is noble. But it's not clear that the manufacturing companies there would be likely to actually take on those green tasks. This reflects a form of the local trap - assuming local production is inherently better for the community.

• "We are the makers and the box is empty. This is where we fill it," declares Industry City's promotional video - while they work to empty manufacturing spaces and fill them with retail.

• Adrian Kimball sounds like he's full of (sh)it when he says things like they don't know how much profit could be made from rezoning and that nearby housing prices won't go up from the rezoning and development.

Cover photo is a remix of Industry City mural.jpg by Vivian Toy and Industry City Dec 4, 2018 07.jpg by Epicgenius. Licensed under CC BY-SA 4.0

On a side note, during my recent trip to San Francisco, I missed the in-person meetup at the Internet Archive by just a day, but I still had the chance to meet the team, including Mek, Drini, Scott, and Rebecca, who graciously showed me around. Experiencing the Internet Archive firsthand was truly inspiring, and I highly recommend visiting one of their public events if you get the chance.

My Open Library Contributions

Here is my single slide from the presentation:

Mentoring Volunteers

This year, I dedicated significant energy to mentoring volunteers. Many individuals express interest in contributing to the project, and I am happy to provide guidance and respond to their questions in a timely manner, even hopping on occasional calls. I enjoy this and hope it allows the staff to focus on other important tasks (such as reviewing our pull requests 😅). I strive to foster a welcoming and accessible community, which is a rewarding experience that makes me proud to be part of the team.

Open Syllabus Project Integration

I chose to highlight two projects: the Open Syllabus Project integration and the Author Wikidata integration.

The Open Syllabus Project provides data on how often books are referenced in syllabi, which is now available in Open Library's Solr database.

The integration of the Open Syllabus Project data offers opportunities to enhance search functionality by highlighting books frequently referenced in syllabi. Additionally, displaying this information on book pages can serve as a valuable indicator of a book's relevance within a particular field. While these features are not yet implemented, I hope they'll be coming in 2025.

Author Wikidata Integration

This project has been on my back burner for years. Finally, in the past year, Drini and I worked together closely to get it out the door.

Wikidata is a rich, crowdsourced database of information on various topics, similar to Wikipedia (and run by the same folks). It is easy to query and has contributions from people all over the world, not just book nerds like those at Open Library.

Now, Open Library downloads information for any author with a Wikidata ID. This information is updated when an ID is added to an author and every 30 days thereafter to ensure the data stays fresh.

Currently, as shown in the screenshot of my slide with the red circle, we are only using Wikidata in one place: to display the short description of the author on their page. However, now that the data is integrated into the system, it will be straightforward to add new features.

Soon, we will implement language-aware Wikipedia links. If you are browsing the Spanish version of Open Library, you will see the Spanish link to Wikipedia if it exists; if not, it will display a link in another language. Additionally, I hope we will start importing authors’ photos from Wikimedia Commons. These are two relatively small enhancements that I am excited to see come to fruition. In the long term, we envision creating a map showing the origins of all the authors you have read and more.

Finally, I want to emphasize that this is a great example of utilizing Wikidata while also encouraging people to learn about and contribute to it. When volunteers add information about authors, they contribute to a dataset for the world, not just the Open Library website.

Closing

I've been contributing to Open Library for a few years now and it just keeps getting better. I'm happy I found this community and hope it inspires people to build similarly welcoming and collaborative projects. If you'd like to contribute, contact me or visit the volunteer page. Whether you're a budding librarian eager to contribute by adding missing Wikidata IDs, a designer ready to enhance our author pages and search results, or an engineer interested in building visualizations and improving search functionality, we welcome your skills and passion.

This blog post was not written by AI, but it was transcribed by AI, as I mostly dictated it to my phone.

Here’s a short blog post about how and why.

Why did I pick this topic?

1. I lived in Hawai'i for about a year so I have an affinity for the topic. (I also maintain a list of documentaries about Hawai’i)

2. Since the kingdom no longer exists, there shouldn’t be too many updates needed for the page.

3. The kingdom was around recently enough that people alive then probably published some books.

How did I do it?

I queried Wikidata for all people with:

1. An Open Library author ID

2. Their place of birth or country of citizenship set to the Kingdom of Hawai’i

It took a bit of tinkering to get the query to work because of the place of birth can have the country encoded in different ways.

I used that data to generate the collection page. The whole script is here.

Results

The result are... a bit lackluster because there are only a handful of authors and most of their works don’t have covers. However, it’s a good start to see how we can use Wikidata to power more interesting pages on Open Library.

On the plus side, there really wasn't much information readily searchable about authors in the Kingdom of Hawai'i so maybe this will be useful to someone one day.

PS: My pull request to augment the Open Library author pages with information from Wikidata was recently merged so more to come soon!