After the storm

For a number of years, all I wanted to spend my time on was code. This is apparently a common occurrence among new programmers. It is easy to get obsessed, over-focused and maniacal about learning programming. This can be very productive; it is also often a little bit unhealthy.

Anyhow, I think I am recovering a bit. I am beginning to be more balanced about my interests, rather than only thinking about code all the time. I’ve been reading books about economics, for example. While I still very much enjoy programming, I now do it in more measured doses.

Where does this lead? I’m not sure. I hope my technical skills aren’t slipping. I’m trying to strike a new balance. We’ll see how this goes.

Posted in learning | Comments closed

One year of Open Journal Matcher

Today marks one year since I first launched the Open Journal Matcher. Since that time, it has averaged about 5.9K hits per month (although of course some of those are bots). It has also been included on various LibGuides here and there. It is nice that librarians out there value this work.

One thing that has struck me as peculiar about this journey is that a surprising amount of traffic is coming from a small number of universities. I am curious about why this is the case. Is the OJM being included in workshops, or some such thing? Is the traffic from one very enthusiastic librarian, student or faculty member? Who knows! Maybe some day I will find out.

Another thing that has struck me is that this project remains, to my knowledge, the only open source option in this space. I am curious about this, too. Sure, there is overhead to run a project like this (I have been fortunate to have grants), but I don’t think the cost is so prohibitive as to discourage other entrants. I admit I am a bit mystified as to why this is the only one.

Posted in journal recommender | Comments closed

What’s next

My next programming project, which I’ve already started working on, is a tool that would help authors make indexes for their books. If this sounds rather niche, well, it really is. While I don’t want to give away my plans too much, I do want to say that I’ve been deeply dissatisfied with existing indexing software. It’s too specialist-oriented, not visually compelling, and generally old-timey feeling. I’m going to try to do better.

Indexing is a very small field, and indexing software is a niche within that niche. But there’s definitely room for new tools that are useful to authors, not just to professional indexers.

Posted in indexing, software | Comments closed

Jquery, bootstrap, vue

When I started at Kingsborough, I built a library homepage that I’ll call “the jquery site”. I was a bit behind the curve on web technologies, and the site very quickly seemed rather dated. The jquery site lingered on far too long, becoming increasingly problematic with each revision, until it was clear that it should have been killed quite some time ago.

Anyhow, now I am replacing it with what I’ll call “the bootstrap site”. I’ll concede that I’m still probably behind the curve on web technologies, but the bootstrap site is so much of an improvement over the jquery site that I think everyone will be greatly relieved, at least for a little while.

I don’t know what’s next, but I think I’ve dug far enough into LibGuides CMS that I’ll be able to make it work with a more sophisticated web framework. As I mentioned before, I am working on learning Vue. Perhaps the next incarnation will be “the vue site”?

Posted in bootstrap, jquery, vue | Comments closed

A return to Twarc

Our library started archiving tweets about our college in 2015. At first, our archive ran an archiving tool called Twarc, on SDF. SDF is a hobbyist programming community; it’s a dynamic place, full of enthusiastic tinkerers and notoriously unreliable infrastructure.

The archive trundled along that way until 2017, when I switched it over to TCAT, the Twitter Capture and Analysis Toolset. This switch was possible because the Amazon Web Services Educate program provided me with ongoing funds to run a server in the AWS cloud. Moreover, the helpful people at the Digital Methods Initiative offer a version of TCAT that is easy to spin up on an AWS server (I used an ec2 nano). This setup was much more reliable than SDF.

There were still some bumps along the way. Twitter’s switch from 140 character tweets to 280 character tweets in 2017 required an overhaul of the archive’s database. The end of long term support for Ubuntu 14.04 also resulted in some disruption as we switched over to Ubuntu 18.04. Nonetheless, the overall trend was toward increased uptime.

But recently we’ve come back around to Twarc. The release of the Twitter v2 API means that our archiving strategy needs to be revisited once again. Twarc has already made great strides in adapting to the v2 API. It is now a pretty impressive piece of software.

So in its latest incarnation, our archive is running Twarc on PythonAnywhere. PA is a cloud service for running Python applications. It allows “always-on tasks”, without the need to run a dedicated server. This is perfect for Twarc. Given that I also quite happily run other unrelated projects on PA, this platform was an obvious choice.

One thing I’ve learned since 2015 is that having a Twitter archive means that you’re going to have to regularly overhaul your archiving technologies. Maybe every two years, at best. Things definitely do not stand still.

Posted in archives, pythonanywhere, tcat, twarc, twitter | Comments closed

The impostor and the RC

Over the past few weeks, I’ve been working on an application to participate in a programming retreat at the Recurse Center. Yet I haven’t been able to bring myself submit the application. It provokes too much anxiety.

So today I sat myself down and thought it through. When I’m being most honest about it, I recognize that this application is causing me an incapacitating amount impostor syndrome. I recognize that it really shouldn’t be like this. I’ve met quite a number of Recursers, and they’re all very kind, giving, and inclusive with their time and knowledge. There’s really nothing to be intimidated about. And yet.

My feelings about it are: maybe I can’t cut it technically, maybe I can’t live up to the standard of kindness among Recursers, maybe my project ideas aren’t good enough, maybe I’m too old. Maybe all of these; I’m not sure. But I did discover that when I finally decided not to apply, I suddenly felt a big sense of relief.

Posted in imposter syndrome, recurse center | Comments closed

The latest abstracts

Right now, as I type, I’m downloading the latest English-language abstracts from the Directory of Open Access Journals API. This is something I do periodically, to refresh the data used by the Open Journal Matcher. I do this regularly in part because new journal articles are published all the time, and it’s obviously valuable to have the latest scholarship available via the OJM.

But this isn’t the only reason. Link rot is also an issue. Some ISSNs that previously resolved using the DOAJ API no longer point to any meaningful data. Presumably this is because the DOAJ is keeping their data up to date, while the OJM, further downstream, is constantly playing catch-up.

Anyhow, when it’s complete, the refreshed data will yield improved results for users of the OJM. Big thanks to the Directory of Open Access Journals for letting me harvest all of this data en masse!

Posted in api, doaj, journal recommender | Comments closed

Open Journal Matcher gets further funding

I just found out that I was awarded a PSC-CUNY Research Grant to fund the Open Journal Matcher! The grant will pay the bills for the OJM for a while, and will also allow me to develop it further. Previously, the project had been funded by Google.

I have two specific goals for this grant. The first is improving the OJM code. My goal is to move most of the text processing out of Google Cloud Functions. Instead, my hope is that I can pre-process much of it, and do the remaining bits in my Flask application. If it works out, this will greatly reduce the computing power needed to run the project, and ultimately reduce the cost.

The second goal is to finish a paper I am writing about this project. Basically, I need to buckle down and write.

Posted in funding, journal recommender | Comments closed

More on the strangeness of JavaScript

I am amazed at how JavaScript can do really strange things, and JavaScript developers just seem to be totally fine with it. Maybe it’s my own misperceptions, coming from Python (with my Pythonic assumptions) that are causing my bafflement. Or maybe JavaScript developers put up with these things because, hey, it’s the language of the web, and you just have to deal with the guff if you want to play the game.

But slowly I’m beginning to appreciate its quirks. Not because it makes writing JavaScript easier for me (it doesn’t), but because I appreciate eccentricities; in people, and apparently in programming languages too.

Posted in javascript | Comments closed


I was doing two things earlier this week: sitting in on a webinar about a neat library tool called Unsub; and thinking about digging further into Vue, the JavaScript framework.

Anyhow, I was so impressed with the UI of Unsub that I looked up what I could about their tech stack (it’s on GitHub), and of course it turns out that they use Vue.

This was all very fortuitous, and now I feel very motivated to try to learn Vue again. I have heard good things about VueSchool, so I will give it a go this weekend.

I hope to report back soon!

Posted in javascript, vue | Comments closed
Need help with the Commons? Visit our
help page
Send us a message
Skip to toolbar