Mapping libraries and archives on Mastodon

I’ve enjoyed being on Mastodon for the past year. It reminds me of how Twitter was in the early days. But Mastodon’s decentralized structure means that I find it hard to wrap my head around the entirety of the fediverse. For better or worse, I only see the parts that are adjacent to my instance.

To better understand this ecosystem, I wanted to see where the library and archives communities are on Mastodon. What instances do librarians and archivists use? Where are the conversations happening?

The Mastodon API lets you search your instance’s public timeline, including federated posts, for hashtags. This means that I can programmatically search for hashtags like #library, #archives, #archivist, etc. This allows me to collect some rough data about what instances are posting about libraries and archives.

The obvious downside to this is that this search is limited to the perspective of my local instance. To mitigate this, I created accounts on five instances in order to get a broader picture of the fediverse. These accounts don’t really do much – they don’t toot or follow or anything – but they periodically count known posts about libraries and archives. And because they are spread out over five instances, they are federated with a much larger number of Mastodon instances. Because Mastodon is not that big, relatively speaking, I can run the counts once a day and easily capture all of the relevant posts that these five instances see.

Here’s what I found after a couple of weeks of data gathering:

There are some obvious omissions here, like and, both of which may be too small to be picked up by my search. Hopefully these results are interesting nonetheless.

Posted in api, archives, libraries, mastodon | Comments closed

The fictograph

APIs are useful for librarians. There are many things we can do with API data to benefit our libraries. But we don’t have to only make practical tools; we can also make things that are a bit silly. For example, I made a tool I’ve called the Fictograph, which lets you plot the awesomeness of an author’s works chronologically. You select an author, and the Fictograph queries the Goodreads API to get the average star rating for each of their books, then charts the results on the web. Go ahead and graph your favorite authors! The code is here. Feedback is appreciated.

Posted in api, goodreads | Comments closed

A reason to like broken Python

Python developers often want their code to be “Pythonic”, usually meaning that they want what they write to be particularly suited to the idiom of the language. To the beginner, this usually seems like an obviously good idea, but so vague as to be hard to put into practice. To be honest, I often opt for other, more painfully obvious criteria to evaluate my code, like: “Does it run without crashing?” This hasn’t always had the best outcomes for me.

Since these different criteria capture different aspects about what’s good about a piece of Python, it’s worth putting them into a table:

Table comparing Pythonic code with code that runs

As I filled in this table, I realized that I think that Pythonic code that doesn’t run is often better than non-Pythonic code that does. If it seems implausible that broken code could be better, hear me out. Except for the most disposable, throwaway scripts, it’s important to consider how much work it will take to fix or improve a piece code. Understanding Pythonic code will be mostly painless, so you’ll be well on your way to improving it. Understanding non-Pythonic code will be much more difficult, and even if it runs right now, it’s going to break as soon as you start messing with it.

So now I’m really motivated to have a more Pythonic coding practice. FWIW, I found this course by Michael Kennedy to be a helpful step in the right direction.

Posted in language, python | Comments closed

Other ways forward

Learning new technologies is empowering in a very immediate way, but moving the needle with technology over time requires constant learning. However, I’m not a fan of the metaphor of “keeping up”; it makes me a bit uncomfortable. I’m less interested in the growth of technology, and more interested in growing as a learner.

It’s important to me that when my self-teaching hits an uninteresting patch – which it invariably does sometimes – that I have other ways forward. Even switching media makes a big difference. Having other ways to learn is my way of unblocking my progress toward my learning goals. It also helps that it makes learning more enjoyable.

Posted in learning | Comments closed

In praise of tech meetups

I’ve said before that communities of people matter in programming. But of course that’s not the only factor worth considering. Having the place and time to code is essential too. For those of us who aren’t professional developers, those hours are often hard to find in our rather full schedules. That’s one reason why I liked the (now sadly discontinued) Learn Python NYC weekend meetups so much: showing up meant I could focus on code for 3 hours every weekend. This meant that when I arrived back at work on Monday morning, I was significantly further ahead on my projects.

Since these weekend Python meetups ended a few months ago, I’ve tried to push myself to code on my own every weekend. This hasn’t been nearly as effective. Without a group, it’s more difficult for me to create the time and space I need to work on my projects. I’d like to find another programming meetup to keep me on track.

Thankfully, New York City has a vibrant tech meetup culture. These meetups can be great opportunities for us to improve as programmers. Of course, it’s important to find a welcoming, supportive, non-toxic group. Fortunately, there are many choices in NYC. While certainly has plenty of options, there are many other ways to find a good group; for example, the Graduate Center’s Python Users’ Group comes to mind. Making time and space for learning will help us all improve as programmers.

Posted in learning, meetup | Comments closed

On indexing

Before working at CUNY, I occasionally made back-of-the-book indexes for books in religious studies, anthropology and gender studies. Indexing is fun, though very time-consuming work. It doesn’t make much money, but it’s gratifying and interesting.

I feel that indexing is a field with a lot of potential. Building conceptual maps of book-length texts is, in my opinion, very useful work. But book indexing is a profession that has its problems. Many of these have to do with technology. Most indexers rely on niche software tools that are easy to use but are often de-skilling. Our dependence on these tools contributes to the perception that indexing is low-value work.

Rather than rely on vendors, indexers could build their own tools to support their indexing. We could write and open source code to move the profession forward. To make this point using a concrete example, I built a sample index visualization tool:

Screen shot of the index visualizer
Click the image to try it out.

The point is not that this tool is particularly sophisticated (it’s not), nor that it’s particularly novel (also not). Instead, its rhetorical agenda is that it is made by a (former) indexer and is open sourced for other indexers. It’s a very modest effort, but my point is to suggest that if indexers prioritized open source work, we’d be much further along in developing our tools and our skill sets.

Posted in indexing, open source | Comments closed

Building usability testing tools

My colleague Carlos and I have been doing some usability testing recently, and have built some of our own tools to make it happen.

We created a testing interface using JavaScript, which has largely been a success. Our interface gives students tasks to complete, and workspaces to complete them. Reassuringly, students participating in our study seem to understand this interface intuitively. They are generally able to work through the tests with very little guidance. Developing this tool has been a positive experience, allowing me to level up my JavaScript skills and teaching me about some of the subtleties of JS.

We also needed to track the data produced by the tests. After a failed attempt to use Hotjar, a subscription-based analytics tool, we decided to do this ourselves too. I made a Flask application that would process and save parameters passed in the url when a user completed a test.

To do this, the Flask application would serve up a “Congratulations, you’ve completed this task” page, and record a bunch of data from the url about how the user got there. Ultimately the data was appended a CSV file and saved. Analyzing this CSV is still going to require a few more steps, but we’ll get there.

My lack of patience with existing tools led me to put far more time into building these solutions than I initially expected. Nonetheless, because we built them ourselves, they met our needs very well. Besides, building them has been a good learning experience.

Posted in javascript, python, usability | Comments closed

Programming language matters

While it is probably true that you can learn to code in any programming language, lately I’ve felt that language choice is nonetheless important. The languages we learn affect the kind of work we end up doing a bit further down the road. I’ve recently begun to notice how leaning to code in Python has subtly shaped the kind of projects I work on. Things could have easily turned out differently.

There are probably a couple of reasons why language choice matters. Here are two from my experience:

  • In my programming journey, I’ve enjoyed the weekend afternoons I’ve spent in the company of the Learn Python NYC meetup group. That IRL community has been formative for me. People really matter.
  • Tools also matter. Python has excellent data science libraries, so I’m not surprised I headed further down the scientific Python rabbit hole. It makes sense to follow the most interesting parts of a language. Those interesting parts both make the language unique and shape our development.

So perhaps language choice sorts us in some way. Python has been interesting for me, but I’m also curious about what’s next. Language will influence the journey ahead.

Posted in language, python | Comments closed

Making an annotation tool at CodexHack

I spent last weekend at CODEX Hackathon working on a tool called LitRen. LitRen is meant to make ebooks editable and annotatable. The idea behind this project was that editable ebooks would help people who write fan fiction: fanfic authors could insert their ideas and stories into ebooks, or even modify the existing text as they like. With public domain books, entirely reimagined versions of the story could be created by modifying and expanding on the original text.

Making this annotation tool required some heavy lifting in javascript. There are some existing javascript annotation tools that we adapted to work with ebooks. As much of the javascript was over my head, my small contribution was to watch others code, and to build a responsive landing page with Bootstrap.

By the end of the weekend, the basic annotation/editing tool was working, though a lot of supporting functionality remained unbuilt. That was fine; our goal was to have a proof of concept. Anyhow, it was a fun project and a great weekend!

Posted in ebooks, hackathon | Comments closed

Archiving with TCAT

For quite some time now, our library has been archiving tweets about our college using twarc. This has been fine, so I hadn’t really dug any deeper into the world of archiving bots until earlier this week when my colleague Shawna Brandle approached me about using TCAT, the Twitter Capture and Analysis Toolset.

TCAT has a lot more packaging than twarc: from installation scripts, to a GUI, to extensive reports. Once it’s set up, it is a lot more user friendly than twarc. If you want a web-based twitter archiving tool that doesn’t require any command line knowledge of its users, TCAT is a good choice. The biggest technical hurdle is getting TCAT running on a linux (virtual) server. I set it up on AWS, with help from the good installation documentation.

TCAT offers a great opportunity to give your colleagues a tool to archive tweets. Beyond that, it provides a lot of ways to analyze and export collections of tweets. It’s got a lot more overhead — as well as more user-focused functionality — than twarc.

Posted in archives, tcat, twarc, twitter | Comments closed
Need help with the Commons? Visit our
help page
Send us a message
Skip to toolbar