Dawn of the Dead Packages

Zombie hand

If you used npmjs.com website to look for NPM packages in the past 3-4 years you might be having abandoned, outdated and unmaintained "zombie" packages in your package.json. Read the full story below. It will be updated as events unfold.

Update: NPM seems to have fixed the issue described below.

A curious case of a missing package

I'm an author of a React animation library called React Reveal. Like many such projects I'm publishing it on Github and NPM. It has over 400 stars on Github and hundreds of daily downloads on NPM. I've been doing this on and off for several years now.

Perhaps like many package maintainers I'm became addicted on checking download counter on my NPM page. I'd often go to npmjs.org website type "react reveal" in the search box and after a few clicks I would be able to finally see the counter. I suppose it is one of these little guilty pleasures.

Then, one day I couldn't find my package at all. Strange, I thought. Did I offend NPM somehow? Then, I remembered that I've added a few keywords to my package.json (animation effects name) in my last release. Was it too much for npm and I'm considered a spammer now?

Things are starting to get interesting

I quickly reverted the changes and, lo and behold, I'm back on top of search results. I guess you learn something new every day. But something still felt wrong. What could it be? Oh yeah, when I type exact name of my package react-reveal into the search bar, the top hit is some old unrelated package that was abandoned a year ago and I'm not even sure what it does.Screenshot of react-reveal search suggestions

In fact, my package was nowhere to be found in the list of suggestions. ( Update: Curiously it made a come back just before publishing time). All the suggestions were for some kind of old unrelated packages that seemed to be dead for a long time. Kind of a digital package graveyard. Then, things grew even more stranger. Even as I was typing the exact name of my package into the search bar and pressing enter, sometimes, I would still be taken to some abandoned package. Was I haunted by ghosts of dead packages? Time to investigate.

The Aha! moment

After a couple of tries it became clear that if I'd hovered anywhere over the auto suggestion box and then moved my mouse outside the suggesting box (the highlight would disappear) and even clicked back on the input and pressed enter (or the search button) it would take me to the package that I was hovering last. It would take to this dead package page no matter what was in the search box.

Well, tough luck being a small project. NPM doesn't care about you, I reckoned. Perhaps, just to feel even more miserable, I did a search on some well known packages to see what suggestions I would be offered. Surely, it will work for, say, NPM itself.Screenshot of npm search suggestions

Express, React, Gulp, npm itself - all suggestions were for old and dead packages. They were abandoned 2, 3, 4 years ago, sometimes completely unrelated. Even worse, if you flick your mouse over any of the suggestions in a disbelief that, say, npm can't find itself, you would be taken straight to the dead package page even if you didn't click any of the suggestions, clicked back on the input and pressed enter.

Well, npmjs users are a smart lot, aren't they? They wouldn't fall for such thing and would never confuse a dead package with a real thing. On the contrary, the download counters of these old packages were huge. Pretty much every regularly maintained package is being haunted by a ghost. Clearly, a lot of folks are falling for this. Lets have a look at a few cases.

Case Of jquery-ui-npm

The top suggestion for npm on it's own site is a package called jquery-ui-npm. It is sort of difficult to confuse npm with jquery-ui-npm as it even doesn't have a readme and was last released in july 2016. Perhaps, it is being confused with jquery-ui npm package which seems to be a maintained package and the official release of jquery-ui. But looking at the graph below you can clearly see that starting in October of 2016 things have picked up steam for jquery-ui-npm.

jquery-ui-npm downloads

Case Of png-async

png-async had miserable download stats while it was maintained but things have markedly improved since its final release in September 2016. Perhaps, it has something to do that it's a top suggestion for async search term.png-async downloads

Case Of jquery.cookie

jquery.cookie is no longer maintained, superseded by js-cookie about three years ago. It's latest release is in May 2015. Against all odds, it's doing great. It's either jquery is becoming massively popular again or, more likely, that it is because jquery.cookie is a top suggestion for js-cookie ( a maintained package ).jquery.cookie downloads

Case Of gulp-batch-replace

The top suggestion for both gulp-replace and gulp itself is the gulp-batch-replace. It is a fork of a gulp plugin called gulp-replace. gulp-replace is a very useful plugin. For example, react-reveal is using it. The gulp-batch-replace for all intents and purposes is a dead fork. It has had one release in February, 2014. Issues aren't answered, pull requests are left untouched and it hasn't been updated since the publishing date ( unlike gulp-replace ). But as far as downloads are concerned it is doing really great. Have a look a the following statistics:

gulp-batch-replace downloads

In Conclusion

I think, it is quite clear by now that a lot of of projects are having old, abandoned packages in their dependencies. Packages, that would never receive any update (even for critical vulnerabilities), costing both NPM and its users wasted bandwidth, lost productivity. Kind of a digital zombies, if you will. They were alive at some point, then were put to rest by their authors and finally were resurrected from their digital grave by the NPM autosuggest.

These living dead packages could be quietly hiding in the dark corners of package.json. As they never receive any updates and named confusingly similar to the living packages they never attract any attention to themselves.

It seems that the root cause of this unfortunate situation is the following chain of flaws on npmjs.com website:

  • Search field is not autofocused. Therefore, users have to take one hand off the keyboard and place it on the mouse to select the search field and then awkwardly start typing search term with the other hand
  • Autosuggestion algorithm has a strange tendency to favour packages that are no longer receiving updates
  • The final flaw is that you'd only need to hover the wrong suggestion to be taken to its page. It is quite natural to hover as you have one hand on the mouse already. This seems to create a feedback loop for the training algorithm that amplifies the initial erroneous guess.

I think, it is a massive issue for such a critical piece of internet infrastructure that NPM has become. Please share, repost, tweet, spread the word about this story so it would get fixed as soon as possible. Longer term, zombie packages should be identified and NPM users should be made aware of this issue (especially if they have zombie packages in their dependencies). It might be a gargantuan task but a necessary one for the NPM ecosystem to remain healthy.