Trove Data Guide status

The Trove Data Guide was released in June 2024, but I always imagined it as a living resource that would continue to expand and develop. I was working on new sections about maps and images when, in February 2025, the NLA cancelled my API access and work ground to a halt. I haven’t been able to make any significant updates since then.

The Trove Data Guide is built using Jupyter Book and the contents are contained in Jupyter notebooks. These notebooks embed many calls to the Trove API which render as tables and visualisations when a new version of the Guide is built. Without an API key the build process fails. In addition, the ‘overview’ sections often make use of pre-prepared data harvests which I now can’t update. And, of course, without an API key I can’t maintain or develop any of the tools or code examples that underpin the Guide.

The only way I can change the content of the Guide is by editing the static HTML produced by the last build process, but that’s of limited usefulness and not very sustainable.

That said, there’s still a lot of useful content in the Trove Data Guide. As the ‘Who is this for?' section says:

The Trove Data Guide aims to help researchers understand, access, and use data from Trove. But just because it’s about ‘data’ doesn’t mean you need to be able to code. To understand Trove data and its possibilities for research, you first need to understand Trove itself – its history, its structure, its assumptions, and its limits. This knowledge is useful to any Trove user.

For example, all Trove users would benefit from knowing more about works and versions, or how to use the ‘simple’ search box for complex queries. There’s also an introduction to what’s in (and not in) the digitised newspapers, and similar overviews for other digitised content such as books, parliamentary papers, and oral histories.

Most of this documentation isn’t available through Trove’s own help system.

I’ve just moved the Guide to a new server and made some minor updates. I’ve added a warning on the home page about the restrictions that the NLA imposed on data access last year. Because of these new limits on API use, researchers might need to negotiate an exemption to the terms of use before making use of the tools and examples in the Guide. I’ve also added a note to the ‘About’ page explaining the lack of updates.

I’m not hopeful that the gatekeepers at the NLA will change their attitude. However, there is one way that the Trove Data Guide can remain a living resource. The Guide embeds support for Hypothes.is, a web annotation tool. If you set up a free account in Hypothes.is, you can highlight any of the text in the Guide and add your own comments. If these annotations are set to ‘public’ they’ll be visible to all users. So feel free to make corrections or additions and help keep the content up-to-date!

I’m also planning to copy some of the non-API content into a new resource focused more broadly on online resources for research in Australian history. I have a domain name, but nothing else as yet! If you’re interested let me know and that might prompt me to get moving.