“It was the best of times,
it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of light, it was the season of darkness, it was the spring of hope, it was the winter of despair.”
― Charles Dickens, A Tale of Two Cities
2020 will certainly be remembered as the Year Nobody Had Seen Coming, even though it had been sitting on everyone’s calendar, awaiting its turn like 2019 had just before.
While the pandemic is far from over at the time of writing, there already are a few learnings to be taken. This report will try to walk you through what we had planned and achieved, and what we had not planned and yet achieved nonetheless.
Until February or March offline access to educational resources seemed to almost everyone like an edge case: the world was increasingly connected, and it was only a matter of time before the solution would become irrelevant. Until 250 million children were sent home nearly overnight and keeping education going suddenly became very, very relevant.
This is a report about being ready, and being prepared.
Users, users everywhere!
Measuring offline audiences is notoriously hard.
Besides direct server connections, we asked every single organisation we work with to tell us how many people they served. Lateral distribution (people sharing Kiwix with their neighbours and friends) is therefore not accounted for.
Countries & Territories
Generating ZIM files off Wikipedia has always presented us with two issues:
- Size, of course, as archives can be pretty big (thus making direct download a challenge).
- Language. How do you make a landing page that is meaningful and useful for users whose language you can’t speak or write?
The first issue we had started solving in 2017, with the first release of Wikimed: for students in Health Sciences, it made sense to have a medical encyclopaedia focused on medical content. But nothing, in fact, prevented us from releasing similar selections dedicated to History, Geography… or Basketball.
We then realised that in order to make the first interaction with these new selections more interesting it would be wise to look at… Wikidata. The sister project neatly compiles all of Wikipedia’s entries and all the relevant facts about them – starting with their spelling in each language and which image(s) can be used to illustrate those entries.
The next step was to simply compile selections and take the 100 most frequently viewed articles and use the Wikidata-selected images to build a tiled landing page. Each selection now offers a sleek, compelling landing page, with a list of articles whose title is in the correct language. No more clumsy Google translations, no more guessing what to share.
New selections released this year included Geography, History, Maths, Physics, Chemistry, Molecular Biology, Football, Basketball, and Cricket in more than a dozen languages. The largest Wikipedias also saw the release of a “Top” selection with the 50,000 most read articles.
Updating ZIM files used to be a real labour of love: each scraper had to be manually started, and the resulting file transferred to the Kiwix library. This was already hard to manage when we had several hundred zim files in our catalogue, but with several thousands it simply had become a daunting -and tedious- task.
Things changed for the better in 2019 with the advent of the ZIMfarm. To be honest, it resembles more a ZIMfactory than a farm: almost a dozen servers located around the world and running 24/7, all year round to automatically update every single one of the more than 4,000 files we provide every day to millions of users.
Recipes (scripts used to copy a given type of website structure, e.g. mediawiki pr Youtube) are easy to clone, making it much easier to aggregate new content: this is the logical next step that will allow us to grow our catalogue ever more quickly and offer more choice to everyone, in their own language.
The ZIMfarm now manages almost 1,200 recipes for as many websites (some of them come in various flavours, e.g.. with or without images), producing nearly 5,000 updates every month or so.
And because we are committed to openness, all farm operations are viewable here:https://farm.openzim.org/
More 2019 milestones
A much-needed update to our code base, with an almost complete upgrade to Kotlin which resulted in performance improvement and a slightly improved UI. Android phones can also now work as hotspots, meaning that users can share content with their friends via Kiwix.
Get it here.
Kiwix-desktop 2.x (beta)
2019 might have signalled the end of the tunnel for this project. A series of beta versions were released for both Linux and Windows throughout the year so as to allow users to replace the ageing and deprecated Kiwix for desktop 0.9 (there’s never been a version 1.0). Newer features will be rolled out with the final version in 2020.
The first step towards building a Kiwix hotspot that could scale and reach more users consisted in sharing an easy to use installer. People with zero knowledge of command lines should be able to assemble the content they want and create a hotspot for themselves, their families, or entire schools. Get access here.
This year’s Hackathon was held in Stockholm and took place next to Wikimania, the Wikimedia Movement’s annual gathering. Participants came from Switzerland, France, India, Mali, the UK and the US and met for 9 days of non-stop coding.
The full summary of goals and achievements can be found here.
Kiwix-JS for Windows
Another volunteer-led project, Kiwix-JS initially started as a browser extension so that people could directly open and read ZIM files from Firefox or Chrome. The technology was adapted as a standalone app for Windows. Kiwix-JS can actually run on older 32-bit computers that are still largely used in many countries.
Looking out to deliver these in 2020 and beyond.
Supported by the Mozilla Foundation, ZIMit will allow us to convert almost any website and make it go offline. Released planned for Q4/2020.
More custom apps
One size does not always fit all. How about your very own, very portable, very offline dictionary?
We’re planning an easy sign-up interface to the cardshop, as well as hotspot images to go. Stay tuned.