Here is a technical status report that was sent by Emmanuel Engelhart, our CTO, to the Kiwix Board of Directors in June 2021. Minor edits for clarity.
Dear Board members,
I wanted to share a new general technology update before our next meeting. As a reminder, you can always find an overview of our software stack here.
Overall I consider the project is healthy, even if the situation is slightly degraded in comparison to last year. This is visible if you look at https://metrics.kiwix.org/. Reasons are multiple: significant grant reduction due to COViD which led to tighter limitations over tech expenses, as well as important delays/rescheduling on the libzim/libkiwix side (which I will discuss below).
That said, the size of our software portfolio remains pretty stable and so does the corresponding code base quality. We made significant progress in a couple of areas but a few sticky points remain (or even appeared). The strategic pivot we initiated last year -from focusing on Kiwix as a reader to focusing on scrapers/content and the hotspot- is clearly gaining momentum.
From now on, I will always try to differentiate between three parts in our software portfolio:
- Core: libzim, libkiwix and Kiwix ports (desktop, Android, …)
Mostly older/mature projects. If one of them is buggy then immediate problems are experienced by users. There is no real business model around these (maybe with the exception of Kiwix-Android), but they are central to our products and mission.
- Content: scrapers, Zimfarm, Zimit, CMS, etc. – everything that allows us to make/publish ZIM files.
We do business with them, but most of the time on a project base (e.g. Stack Overflow offline, sponsored by Stack Overflow; or Khan Academy, paid for by the Orange Foundation).
- Strategic assets: mainly the Kiwix Hotspot and Cardshop.
The long-term vision here is to be able to build an offline Raspberry Pi library in the Cloud. We also think this is our best shot at generating recurring income.
Core
Since last Fall we have continued investing massively in libzim (the reference library for the ZIM file format). The goal was first to remove the namespace from the ZIM file URL displayed. Then the Zimit project showed the weaknesses in our concept. We are now almost at the end of the project but also 6 months behind schedule. This is worth it. Libkiwix (the code shared by all Kiwix ports), because of domino effects, suffers of a similar problem,
but the revamping of the library mgmt is on its way and promising. We plan to release the long expected libzim7 and libkiwix10 in the first half of the Summer.
Kiwix ports are the real victims (in our portfolio) at the moment. In general they are OK, but no significant update was possible because of a lack of resources and the libkiwix delay. I am still concerned with the lack of quality of
Kiwix Android and Kiwix Desktop (particularly with regard to library management). We have (ongoing) plans to improve this but money is tight. We will not have miracles, and it will take time.
For the near future, we want to center our effort around library management in libkiwix (a few months’ worth of additional work) and then all of this should be in maintenance mode: no new feature, focus on bug fixes and usability (in particular content libraries).
Content
Here we have made significant progress. Our scrapers are better than ever before and there are no major concern regarding their maintainability. Thanks to multiple innovations, it is always easier/quicker to build new quality scrapers.
The Zimit project has been completed and even if the support of Zimit-based ZIM files is still not fully there yet this is a promising tool. We will monitor Zimit’s stability and usage over the next few months so as to decide what to do next.
My concerns of last Fall around MWoffliner (Mediawiki/Wikipedia scraper) are still a bit there, but the situation is not as dire thanks to a few volunteers’ hard work.
The strategy is to be better in content. The trend is good. Work on a new CMS (a new ZIM publishing platform) development has begun. We will recruit a bit to move more quickly.
Strategic assets
The situation has improved a lot since last Fall. The Kiwix-Hotspot solution has been stabilised. The Cardshop itself has been adapted to allow B2C business via www.kiwix.org. From January on, anybody can order their custom offline library online at a very affordable price.
We have already plans to launch a dedicated shop later this year called offspot.it and this will be the next big iteration. The goal is to have a fulltime developer soon to support this strategic project.
Kiwix-Hotspot is already a unique software solution but there are a few other projects offering an alternative approach. Thing is, they sometimes are as much competitors as they are partners. I believe however that we are now very close to be the best solution (technically speaking) in the field of offline libraries. This, in turn, would do a lot to help the business part of the challenge.
Conclusion
Kiwix is in a good shape overall and on a good path. Things are always slower than expected but we are definitely making progress in the direction we chose. 2020 was a bit difficult, but 2021 is promising and our agenda is pretty full with (sponsored) projects. It was not possible to handle them properly anymore without a tracking tool, so I started a Gantt (see attachment).
Long time investments start to pay off with the launch of the cardshop gateway. We are now focused on scaling, and on moving from a few dozen monthly orders to several hundreds.
Emmanuel